Are you open to giving him a chance but still struggling with what he did? What is he willing to do hence forth?
He's broken your trust even if they did not start an intimate relationship and it does take time to heal and for you to gain back your trust.
It is possible that he's made a mistake and is regretting the fact that he almost lost his family because of that.
As far as how you deal and react to what happened would also depend on what he's going to do and whether or not you think he's being sincere. Right now because you were betrayed, you may find yourself more suspicious of his motives or intent.
If you have some desire to give him a chance and he's going to do anything to 1/address this issue 2/ claim responsibility for what he did 3/apologize and 4/come up with a plan about how to move on, you could try giving him another chance. It would take time as this is going to be a process of establishing trust, working on having an open communication and in both of you addressing those things that need to be mended in the marriage. Using the kids as an excuse to being married to you is quite lame and something you'd be talking to him about. Do you think this was some sort of mid life crisis on his part?
It is possible that he loves you and that he talked to her because it was opportunistic. He really would have to step up and take responsibility for what he did without trying to say it is because you did this or did not do that. You do not read minds and if he's unhappy about something, he has to tell you and the two of you as partners to work on it.
You do have a reason to be suspicious because often past behavior predicts future behavior, unless he decides to work on changing. Then, you'd have to figure out if you're going to be willing to wait to see some progress on his end. Being in love is a stage that takes place at the beginning of a new relationship as people learn about one another and try to impress each other. As they get comfortable and complacent through the relationship, the in love stage can progress to something more solid- a partnership based on love and trust. Of course, he can become in love. That depends largely on what he tells himself about you and how he treats you. He can do romantic things and the two of you can always reconnect emotionally. Just because there were problems and he at one point felt "out of love" does not mean that it has to stay that way.
This situation has to be addressed. It would be preferable that initially the two of you work with a marriage counselor if he's open to it. It would be helpful to have someone objective assist both of you during this period. The two of you can also discuss what are some of the things you want to change, redefine your relationship goals and come up with a plan about what to do next to start repairing your trust and addressing his dissatisfaction or crisis mode.