The best thing you can do is pick them all off carfully and see it out, if the bananas open your plant will start to produce seeds and stop the production of bud, I dont know how far you are into flowering but if your near the end there is no point chopping her down just check every day for bananas and cut them off!
Alot of plants can turn hermie if they get disturbed lighting cycles mulitple times aswell as a load of other stuff there is alot more information online and what can cause your plants to herme.
I wish you the best of luck and hopefully i gave you some insight to whats going on or what to do,