I'm guessing that both of you were caught off guard and didn't quite know what to do - otherwise he easily could've said something to you too, yes? If it continues to bother you, there would be nothing wrong with looking him up and telling him that you were surprised to see him and didn't quite know what to say or how.
And, it doesn't surprise me that you're having a little anxiety about it. We're usually so busy living in the present and worrying about the future, that being unexpectedly jolted into our pasts can be disorienting. You've just been taken back to a time when you were more vulnerable and all the feelings were newer. So, it's not unusual to feel something (whether that be anxiety, melancholy, excitement, etc, depends on the person). I wouldn't worry about it - just let the memories come and see how you feel after some time has passed.