The fact that you still do things together is a good sign. Use this time to build good memories and bonding with your baby.
If your behavior over time shows him that you are no longer making the same choices you did when you were single, he may slowly begin to trust you again.
However, there are no guarantees. If the things that bother him are things that happened before you even met him, he may have control issues. If you already know he has control issues, please be very cautious. The fact that he wants to live with you, but is still saying you're not a couple may be a sign that he wants the control, but not the responsibility and commitment.
I would advise you to not talk about getting back together, but just go about your life with him and the baby. If he is just being cautious, spending time together and learning to trust you may help him commit to the relationship.
If, however, no matter how devoted to him and the child you are now, he still holds things from the past over your head like a punishment, you may want to think about whether you want to spend the rest of your life apologizing, or whether it's better to move on and find someone who is more accepting.