Hi, I'd like to help you with your question.
First, I am sorry this is happening to you. Anytime a spouse strays outside the marriage it can cause strain to the marriage and undermine trust, which is essential to a good marriage.
It is up to you if you decide to work this out and try to save your marriage. If it is always on your mind, then it is triggering your feelings about yourself. Also, if it is unresolved, it is going to bother you. If you do decide that you want to work on your marriage, then there needs to be some things that happen first:
One, is your husband sorry for what he has done? This is important, because if he does not take responsibility, recovering your marriage will be difficult.
Two, has your husband stopped all contact with the other women? He must do so as a first step to regaining your trust.
Three, has he done anything to start repairing the marriage?
Four, are the two of you talking about what happened? Your husband needs to be open and honest about what he did and let you ask any questions you need to. He broke the marriage vows and dragged you into another relationship. He needs to own up to it.
These are some of the most important issues you both need to be working on. Seeing a therapist would help a lot. Talk with your doctor about a referral. If you attend church, talk with your pastor. Pastors are often very good marriage counselors. Also, if you have problems affording therapy, try your local community mental health center. They can offer therapy on a sliding scale fee system.
The main issue is to rebuild trust. It can take a while and a lot of work, but you can do it as long as you both are motivated. But if he keeps having affairs and will not stop, then it may be time to end the marriage. If that is the case, then seeing a therapist for you to help you recover would be helpful.
There are some books that may help you.
Should I Stay or Should I Go?: A Guide to Knowing if Your Relationship Can--and
Should--be Saved by Lundy
Bancroft and JAC Patrissi
Infidelity: A Survival Guide by Don-David Lusterman.
Not "Just Friends": Rebuilding Trust and Recovering Your Sanity After Infidelity by Shirley P. Glass and Jean Coppock Staeheli.
You can find these both on Amazon.com or your local library may have them.
You can also work on your self esteem. Along with therapy and working on your marriage, you can also read more about self esteem issues to help you learn ways to address how you feel. Here are some resources to help:
Self-Esteem: A Proven Program of Cognitive Techniques for Assessing, Improving, and Maintaining Your Self-Esteem by Matthew McKay and Patrick Fanning
The Self-Esteem Workbook by Glenn R. Schiraldi
How to Raise Your Self-Esteem: The Proven Action-Oriented Approach to Greater Self-Respect and Self-Confidence by Nathaniel Branden
I hope this has helped you,