I know it hurts. The problem is, if one person tells you something, then you may think its possible its a fluke or it may not be true, but when more people tell you the same thing, then you know there may be some truth to it. The best indicator of future behavior is past behavior.....the fact that his other girlfriends complained about it as well means that it's not personal towards you, but a problem he's had for a long time.
If he has never seen a therapist or had counseling, this might be a good time for him to think about it. Perhaps if he knows how badly this is affecting you he might consider it. There are many men (and women) who are not affectionate, whether its because of the way they were raised or because they have been hurt, but if someone truly cares about you they should be willing to compromise.
If he is willing, perhaps you can try some touching exercises. Hand massages, foot massages, full body massages (that don't necessarily lead to sex) are all good ways to help him get used to touching and being touched. Ask him if he can remember to hug you or touch you at least twice a day. Once he starts doing it, it can become more habitual and he may get over his fears or whatever it is that's stopping him. If he can't remember, he can put two colored rubber bands on his wrist, and when he sees the rubber bands, it can remind him, or you can put a note on the fridge "don't forget my hugs" and this can help him remember. I know it might seem silly, but when it works, it might not seem so. :)
I welcome your thoughts, let me know if you want to talk more.