Once you start sleeping with someone, it can be difficult to figure out where things are going because the emotions are so high and the passion gets in the way. Although he thinks that you don't open up to him or hide into yourself, I think that you should stay that way unless he gets into a relationship with you, because those are things you save for you significant other not someone you are sleeping with. It seems like he wants the best of both worlds....he wants your body and your mind, but without the work and commitment, and thats not fair. He says that your going backwards, but sleeping together before deciding what kind of relationship this is, is going backwards to. I know that you really care for him, and you probably want to stay involved, even if it means just physically, but you'll want to hold back on your feelings unless he's willing to talk about his feelings as well.
I welcome your thoughts, let me know if you want to talk more