Well, if you are feeling worse, or weaker -- or have other symptoms such as numbness in the left arm and leg, etc... then it would be worth going back.
It certainly could be something minor, as your doctor has mentioned.
If you are still up and walking around - then any leg weakness you have is relatively minor.
But certainly, for symptoms that persist, or worsen - further testing is helpful, and in general the first step after a neurologic exam is imaging of the head, which may be a CT or MRI of the brain.
So, if the symptoms are persistent, and concerning you, then it would be helpful to go back and get it reevaluated.