Do you have a job or some other focus for your life besides your child? Could you ask him to plan a good vacation for you as a family when he has some holidays?
It seems like the only thing he has that you're not sure he's going to give you is his time. That's why I'm suggesting you "apply" for a complete dedication of day-in day-out as well as some expense to go somewhere really attractive for you and your child. THAT could serve as his visible commitment to having a satisfying life together with you. Then you can ask him to get his other woman to move out of the house right by his parents so she is NOT a part of his life or your family's neighborhood. Even if finances are tight, that is NOT too much to ask. His reaction to that will tell you quite a bit about whether he's willing to recommit to you or not. You should not have her right there on his doorstep waiting for your relationship to tank again so she can jump in with him.
Since he hasn't married either you or the other woman, you're probably not going to have the peace of mind you need unless he makes arrangements for the other woman to move far enough away that there's no chance of rubbing shoulders without very definite intention to intrude. If he won't do that, then I'd feel the same way you do: threatened, on probation. But in fact he's also on probation with you, that is to prove that he will not jump ship again, even if you get into a rough passage.
And one more thing. What's keeping him from marrying you? I don't know how your customs about marriage are going in UK, but as long as you're not married he hasn't done everything he can to treat you right. Are you afraid of what would happen if you ask that? [I'm not suggesting you do that right away. The other two steps could come first--tho you'll know better than I about that.] How do his parents feel about you? Don't they have to take a stand too? But don't leave it up to them, cuz it's your move first.