Hello I am Dr. Joey. Thanks for trusting me to help you and your pet today. I am a licensed veterinarian with over 16 years of experience. I look forward to working with you. I apologize no expert has jumped in to answer your question until now.
I am not sure I would link anxiety with lethargy, especially for a labrador. Most of the time anxiety manifests with almost hyperactive behavior (pacing, panting), possibly drooling, and possibly destruction of things.
My concern would be there is something else going on. Has he developed any further issues?
Since he only is a bit down/lethargic right now, you could monitor for a few hours and see if he develops any further symptoms. If he develops any pain, vomit/diarrhea (especially if bloody), etc then I recommend you have him examined today.
I am at a point I need to know what questions you have. I hope that the information I provided has been helpful.
Please let me know if for any reason you need further clarification, have more questions, or were expecting a different type of answer.
If you received all the information you needed, then kindly submit a rating.