It is more common for men to keep on their tee shirts rather than their underwear. It is usually a sign of embarrassment about physique. One approach is to make love with a small candle burning which allows the intimacy, but avoids the embarrassment. I would also suggest you talk to your lover about it. Communications is so important in good sex which most people overlook. Best wishes.
If you require further clarification or more information, feel free to ask!
Please click the green "ACCEPT" button on this page if you are satisfied with my answer so that I get credit for my work. Your positive feedback is most important and bonuses are always welcomed. If this is a mental health, or medical emergency, please call emergency services (911 in the USA), or go to your nearest Emergency Room. The opinion provided does not replace an evaluation by an appropriate professional. Use of this website DOES NOT CREATE a provider-client relation and this information is for educational purposes only.
Talking, sufficient foreplay and ultimately erectile meds like Viagra or Cialysis should solve the problem.