I didn't imply it would be simple to change jobs...as I know full well it isn't.
From your original note, I had envisioned the two of you running into each other in an office all day. Since that isn't the case, it may be possible to get through this without giving up a good job.
You'll need a therapist: --someone whose only concern is your emotional well-being and has no other connection in your life. You have suffered the loss of the other important male in your life, your dad, which will make giving up this man even more difficult. It might even be a good idea to try a male therapist given your particular situation.
Some other thoughts: What kind of man chooses to live a sham life rather than be with the one he loves? Hiding behind his children is a cowardly move. (Unless one or more of the children are medically fragile) Children survive divorce--they often survive it healthier than they do growing up in a home where there is no affection. If he thinks his children don't know what's going on, he's wrong. Children can sense play-acting, false smiles,etc. quicker than adults.
A relationship begun on cheating--and he is, no matter what he says about the marriage--will suffer several handicaps even if he were to leave his wife. In my experience, the new relationship rarely survives. Because the woman knows full well how good the man is at hiding his activities, sooner or later, usually after about a year, suspicion creeps in. Look at how long the two of you have been together on the side. He knows he can get away with it, he knows how to hide his tracks. And before you protest that he would never do that to you...soon you would be the predictable one worrying about bills, fixing the house, etc., and he will be looking for a new fantasy woman.
I am actually amazed that you allow him to talk about your affair openly, standing at the bar at company get-togethers. This seems to me very disrespectful: In a way he is claiming you so no one else at the company dare approach you, but he doesn't have the courage of his convictions.
And think about it. If this is such a loveless match he has at home, why would his wife get vindictive? I think there is a very good chance that she has no idea he's unhappy, that they continue to live as man and wife in every sense of the word. He can spin his story to his full advantage, knowing that you have no possible way of fact-checking.
How many women do you know that would listen to a husband say he doesn't love you, and go on as before? And I will tell you that the "I have no money because the wife spends it all, she doesn't like sex anymore, and she doesn't care about anything but money, and if it weren't for the children, I'd leave" is such a classic script of the cheating married man, I'm convinced that somewhere on the internet is a site telling them what to say. If you lived in the US I could introduce you to no fewer than ten women who have believed what you believe, and lived to regret it.
If you think that being 49 makes it difficult in the job market, wait until you find out what the dating scene is like for women over 50. Please don't waste too many more years with this going nowhere situation, as the prospects for you finding a new, healthy relationship get fewer with each passing year.
The best indication of future behavior is past behavior. Think about what daily life with this man as your partner would be like. The physical attraction will fade with familiarity-- that is almost guaranteed.There will no longer be the aphrodisiac of clandestine meetings, and being ill-fated lovers. It sounds like you are both struggling financially--it will be worse when he's paying alimony and child support. It would be hard not to resent his continued ties to his ex as she calls when the children are acting up, she needs more money, the roof is leaking, etc. And then there will be the fear --is he really where he says he is when he's traveling?
What I have tried to do here is to give you negative things to dwell on to help support your decision to end this matter. The point is not if I got a few details wrong about your particular man...the point is that, like it or not, you are in a classic situation that many other women have found themselves in, and the scenario rarely changes. It is a win-win for the man: live goes on at home as always, and he has a woman who thinks he's wonderful waiting for him at work. If the affair ends, his life doesn't change a bit. The woman is left wondering why she squandered years of her life with nothing but a few memories to show for it. Of course he won't let you alone--why would he give up a good thing? He won't be the one with the devastated, lonely life when this ends. He is being selfish. If he truly loved you, he would encourage you to go live your life without him, as your happiness would be paramount to him. As it is, you've said you wanted to end it, and he has advised you he'll get at you no matter what. That's not love. That's selfishness. He has nothing to offer you for your future, but wants to keep it for himself.
Please get a good therapist that you feel comfortable with to support you through this transition.
I wish you all the best,