Hi, I'd like to help you with your problem.
It sounds like you have done a lot of self reflection and do not like what you see. I understand that looking at all you have gone through is difficult, but judging yourself by looking into the past is only going to cause you to feel shame and pain.
People tend to do what they feel is the best thing at the time. Everyone goes through periods where they make mistakes and have regrets about choices. That is ok.
Are you looking to try and repair what happened, or are you looking to move forward and just deal with what you are feeling? You can do either, or both. If you feel going back and trying to repair your relationships would help, you could try talking with your first wife and letting her know how you feel now about what you did. You could also speak with your children. I think that unless there is a lot of hurt and anger on their parts, they would welcome you trying to make amends.
Or, if you want to move on and try to work on a better future, you can start by forgiving yourself. Keep in mind that many people do not even realize that they make mistakes in their lives and therefore continue with the same behavior. You are not doing that. You are taking a look at yourself and trying to improve what you see. This is a good thing. It shows you can grow as a person.
Since you have tried therapy, you may or may not want to try again. Therapy is usually a good step to take. It can help you with gaining more insight into your issues and guide you to the point you feel you can work on this yourself. But if you chose to not go back to therapy, you can work on this yourself. Try educating yourself on feelings of shame and regret. Here are some books that may help you get started:
Shame & Guilt: Masters of Disguise by Jane Middelton-Moz
Letting Go of Shame: Understanding How Shame Affects Your Life by Ronald Potter-Efron and Patricia Potter-Efron
Healing Your Emotional Self: A Powerful Program to Help You Raise Your Self-Esteem, Quiet Your Inner Critic, and Overcome Your Shame by Beverly Engel
You can find these on Amazon.com or your local library may have them available.
Also, surround yourself with support. Talk with friends and family about how you feel. Let them give you some insight into how you can move forward. Although sometimes it is hard to hear others tell us what we need to do, it can help in the long run to give you the motivation and insight you need to live without regrets.
I hope this has helped you,