I am sure I have some fungi in my dirt as I add compost tea made with great white every few weeks but No mushrooms or any thing growing out of the pot tho. No bugs or anything on the leaves or any of the stems.Check for aphids. The ants may be farming aphids on your plants for their sugar. Aphids look a bit like scales on the stems and leaves.
Ants nest close to food, or their farms. If you kill the queen, you kill the colony.
Ants also cultivate fungi. Do you have mushrooms growing in your pots? Fungi under ground will look like very fine threads, like loose cotton balls.