This is indeed very confusing, which is why you need some face to face support. He is a mix of very odd tendencies and even from a psychological standpoint, much of his current behavior is odd. So, to deal with this, best practices are in order. Here are a few:
Do not blame yourself for his actions. Not having sex is not an excuse to cheat. He could easily have worked any feelings or issues out in therapy with you or in a dozen other ways. He is to blame here, not you.
Sometimes men want both situations; two women or families. They may find it exciting, dangerous, enhancing.
He obviously gets something from being with you and the feelings like he is now showing, this double minded thing, may be confusing...but for many males who wonder outside of marriage the reasons are unfinished issues with their families of origin.
I have no doubt that he is an angry, terrified and scared man who was controlled by a parent, probably mom, and never dealt with it. As a result he is destructive, yet begs forgiveness. He is acting like an angry young boy/teen in all truth, and the issues probably go back that far in his history.
He will chase you. You are the strong one in his eyes; he the weaker one. He needs you, his parental (mother really) figure to tell him he is okay. But he is not. He can get better, but he needs to step it up and show it through actions. You cannot save him and should not. Right now he has no rights in this relationship. He ruined that. Now, you set the tone and the actions. And, until you see reason to not be protective of yourself, do so...And try not to see this in any way as you. It is him, and his unresolved issues from the past that are being taken out on you. Protect you and your family. Let him act and show you through behavior that he is different and accept no excuses for lack of action.
Thank you for letting me help you today. If i have helped at all, please rate me positively. Steven