Hello XXXXXXXXXX,
I believe that your mouse is simply behaving like a mouse. What I suggest is to continue handling him. Offer him a sprig of millet. I know it's considered bird food but mice love the seeds. Work with him often and eventually he will settle down to a routine. I also suggest that you wash your hands prior to handling him as he may smell something that doesn't agree with him.
Hope this helps. Let me know if you need anything else.
animal c&ontrol officer
18 years experience with animal control and vets