Thank you for writing to Just Answer.
It can be hard when a friend changes the way yours has. We remember what we used to have with them, and try to re-create it--usually unsuccessfully.
I think this situation has the potential to end badly, and it would be better to end it sooner rather than later. Even though this is strictly a friendship, you also acknowledge a sexual tension between the two of you.
Even though you did the right thing by leaving at that moment, it sounds as if your friend does not have such strong boundaries as you do. Her comment about having to stop playing these games means she is still feeling that sexual attraction, most likely.
The fact that her husband no longer approves of the friendship adds to that possibility.
Do you really need all this drama in your life? It seems it would be better to put all this emotional energy someplace more positive--like your marriage. You are lucky enough to have a very understanding wife--many women would have resented nightly phone calls and this woman's jealousy over you.
Emotional infidelity is a situation where one partner is putting more energy into another relationship than they are into the marriage. The language you use---"driving us apart" "letting her go" sounds dangerously close to this...and the fact that things have progressed enough that her husband no longer supports your friendship.
I would advise you to walk away from this friendship before it causes real problems in other areas of your life.