I think the fear of him leaving again has you torn and a relationship should not be fearful but happy and joyous if you are not ready to sleep with him or be intimate then don't rush it and if he doesn't understand and gets angry and leaves then that will tell you that he hasn't changed and it may be time to move on and finalize the relationship. You need someone there that is going to be there through thick and thin even if things don't go his way, since he has a history of leaving for weeks, months and years at a time that fear is always going to be there that it will happen again. You just can't put your life on hold waiting for him to yet again come back to you and your child together. Life is too short to waste it waiting for someone that doesn't respect you or take into consideration your feelings.
The only thing that may help this unhealthy relationship is Couples counseling so that he can figure out why he can never stay when things get rough, he may have seen that in his family or has had bad relationship where the woman has left and he decided he would leave before they did from now on, whatever the reason he needs to figure that out and then find ways to stay and fight for the relationship especially if he loves and wants to fight for you. As long as that threat of leaving is there you will never have a happy and healthy and trusting relationship because you will be thinking "Is he going to leave?" or "Will this be the day he leaves and doesn't come back for years?" That is why counseling is so important to teach you both to better communicate without it always being a fight. They can also get to the root of his problem of leaving when things don't go his way or he will be running the rest of his life.
What you have to do is sit him down and tell him that you need for him to go at your pace and not pressure you and that you both have alot of issues to work through before you feel comfortable totally giving yourself to him again and if he cannot accept that then it really isn't worth fighting because you both have to want to fight. You have done things his way now it's time for him to try to do things on your terms so that you can feel comfortable with giving him another chance and giving your heart to him again. He has to realize that relationship is 50/50 not 80/20 the sooner he realizes this the better your relationship can become.