The swelling of the gland usually is due to 'war' between your immune system and some foreign organism fungus, bacteria, virus etc (that is why its a flag for doctors).
The alternative explanation would be that it might have been elicited by the yeast (fungus) but you never noticed it at the time. (It can remain swollen for a while even after treatment.)
You must however understand the basic medical logic, it never swells up, unless your body is fighting or fought a 'war' with a foreign organism (virus, fungus, bacteria or even parasite).
The glands in the groin cover the genitals and all the way to the feet. (The correlation with STD was mostly because of your history of un-P-sex)
Infections from feet and leg are usually obvious and the glands are invariably painful.
Painless glands usually suggest STD. (lymphogranuloma venerum caused by a type of chlamydia)
Other causes worthy of mention include:
chancroid (STD, but causes sores on the penis)