There is no truth in every joke. Sometimes people just say things that they don't mean and end up regretting. He may have not even though about what was coming out of his mouth then and at times men find some things funny while women do not.
As far as being overly sensitive, that is something you'd have to weight against other times when comments not only by him had affected you days later. In that case, there is an indication that you internalize the comment and ruminate over it instead of letting it go. That would indicate that you're critical of yourself and comments like that erode at your self esteem. That can make one sensitive mistakenly perceiving the sensitivity to be a defense (by justifying what you think and how you feel)
Thank you. I think his self esteem is also a factor in all of this. We truly love each other and also have our "issues". One of our core issues is this very dynamic. Hopefully over time we can learn which buttons not to push in each other and create a relationship built on even more mutual appreciation and trust. That is the goal!