You may want to confront him on the lack of sensitivity at home. Sometimes people are really not aware how their words and actions affect others. If he brushes it off, write him a letter outlining how his words affect your self esteem.etc. See if your mother can vouch for you. It is unfortunate that he is doing this and it can be quite hurtful.