this happened during my drop/reload mag drills. Id put one round in a mag, fire, drop and put new mag in. I cant recall now if it was a specific mag that was doing it or if it was randomly doing it. So I'll have to wait till next outing to experiment.
I usually rest my left thumb on the takedown lever.
Fire it left handed a few times and see what that does. You won't have those fingers interfering then.