I am happy to help you today.
Could he refuse to write a letter for more time off??
The answer is yes.
Usually though the doctor will try to go by the patients wishes if you have a legitimate reason.
Since he has already given you 3 weeks you obviously needed the time off so more than likely he will give you more time.
I am not sure about where you work but you might have to go into leave of absence since you are off so much.
This could be different if this is workmans comp.
Your doctor might be able to suggest some things too to help you make the transition back to work.
If you tell him your concerns and that you really are physically ready yet I would think the doctor would want to stay on the sage side and let you be off 2 more weeks.