Under most circumstances, & in many horse owner situations, horses are left out 24/7. Ideally, they will have good shelter from inclement weather that is easily accessable. I have had many horses ( right now 16 ), both in boarding facilities, & now on my on farm. Only if they are sick, old, or being treated in some way will they be stabled overnight. Otherwise, they are free to choose. In your situation, it is more important to know what you were told to expect in the care of your horse, & if there was a specific reason for keeping Simon out for the 3 days. Perhaps some work was being done in the barn.
For the most part, I believe it is healthier for horses to be outdoors. Simon's agitation may have come from his desire to remain outdoors, perhaps with a buddy he had become attached to. Perhaps something had changed in the barn while he was out. That is why it is important to contact the facility to ask why Simon stayed out.
If Simon calmed down after 20 minutes, to me this suggests he was more interested in remaining out than coming in. But if you notice this happening again, then it is important that Simon maintain a strict regimen. Some horses need a very regular structure to their day, including turnouts, return to stall, & regular feeding schedules. Your boarding facility needs to know this to make sure this does not happen again. While low, there is some concern that the stress of irregular care can leave a horse more prone to illness, including colic.
Does this make sense?