Make sure you drive the forward assist roll pin in from the top of the receiver.
Thanks! On the lower I had driven in the bolt release from the front which was a mistake. I got it in without marring the receiver, but man it was a pain getting it in far enough to use a vise to do it the rest of the way. By the time I realized I was going the wrong way I couldn't back it out.