For now I agree with your doctor.
The TSH is the key here, not the t4.
The TSH being slightly low MIGHT indicate a HIGH FUNCTIONING thyroid, however, the T4 being low argues against this.
A high functioning thyroid is called HYPERTHYROIDISM, click on the link to learn about it.
The botXXXXX XXXXXne is that labs, especially with thyroid, are helpful but in this case not diagnostic of any problem. The TSH level, as well as the T4 level, can fluctuate greatly due to stress, illness, certain medications, and seasonal changes, so I think the wait and recheck approach is very reasonable.
Read through the article and see if you have the symptoms of hyperthyroidism. I imagine you might find one or two that may be going on, but this too, would not indicate a problem with an overactive thyroid.
I hope this is reassuring to you. I certainly do not think you need to see an endocrinologist at this point. If your levels continue to be 'off' on the next draw AND you have some symptoms of overactive thyroid then I think at that point a referral to endocrinology might be warranted.
If you have questions about my answer please do ask them in the box below.
Positive feedback is always appreciated, and should you need my assistance in the future simply start your question with 'for 911Doc only'.