If any of the components of the cake had been contaminated unknowingly, it is possible for any guest who ate a piece of cake to have food poisoning.
We cannot be sure which component of the cake had been contaminated and much more, we cannot be sure when did the contamination happened.
Possibly, the vanilla custard had been contaminated. It can also be the eggwhites. It can be from any ingredients. Once any food component had been contaminated, refrigerating it thereafter won't get rid of the contamination. Any food component that has been contaminated, cooking it will not get rid of the toxins and can still be potentially harmful. Another is that leaving food outside for hours allows the bacteria to grow and it is always not safe to assume that foods left outside for hours are always safe to eat.
Sibreeze, we cannot be give a straightforward answer if the food poisoning is from the custard filling or frosting but for sure, it is possible to have poisoning from any contaminated food. One thing that can give an idea is the time food poisoning symptoms appear after the food ingestion. If the symptoms appear 1-6 hours after eating, you can suggest it is caused by toxins. Symptoms can appear longer with ingestion of live bacteria.
To have poisoning, there is something called an infectious dose. Assuming that any of the cake ingredients had been contaminated, if you ate any of them after putting them on the cake, chances are, the amount of bacteria that time is not sufficient enough to cause food poisoning. But for a time later, the bacteria were able to grow and increase in amount significantly which then can cause food poisoning. Another thing is that, most bacteria causing food poisoning re greatly reduced in number with the effects of stomach acids. Thus, reaction to foods can vary individually. That is the reason why there are times that when sharing a food, one person can get food poisoning and the others not.
Just being curious, is the cake the only food in that occasion? Or was it suggested that those who got food poisoning are only those who took share of the cake?
Ideally, there is something called the "two-hour" rule which is also applicable to the length of time the food can be safely left out at room temperature. If the temperature is hotter, the time allowed can be lesser.