It's understandable to still have feeling for your husband you have a history with him and two beautiful children with him, what you are going to have to do is find out who it is you want to be with and then find closure with those unresolved feelings you have for the other and then be with the man you want to be with and feel will treat you better. It's not always strange to find yourself reminiscing about an ex especially if you for the most part had a nice relationship and caring but if you felt neglected by your husband and that is why you cheated then there was something missing in your marriage that you may have had to find some where else but in your husband's eyes that doesn't excuse what you did, he feel as though you broke up a happy marriage to be with the person you had the affair with so if you choose to try and get your ex husband back then you are going to have to find closure in the relationship you're in now and then prove to your ex husband that what happened before won't happen again, if you choose to stay with the current boyfriend then the same goes for your feelings for the ex husband.
A marriage has a significant impact on our lives, a marriage brings two people together but there are times a breakup happens, leading to both going separate ways. Patience, understanding and trust are key factors in a long lasting marriage and sometimes it just doesn't work out that way whether if one cheats on the other or one or both are not happy they often go outside the marriage looking for that happiness but often find out later that they aren't finding what they want in the other person either. You should not have thought about trying to find someone new or going with the person you cheated with right after a breakup. Hasty decisions made at that point will not work out for you. Your mind is still filled with hurt and anger during this period. You should give yourself some time and space to make up your mind what it is that you want - live by yourself for a few weeks. Take some time to straighten out your thoughts and find out who it is that you truly want to be with.
To be honest if you choose your ex he will not take you back if you are living with the other man, you will have to make a clean break from him so that your ex will see that you are no longer with the other man. Maybe a counselor will help you figure out what it is you want and who you want to spend your life with, it will basically come down to who you want more and who will be better for you and make you happy. If you were unhappy in your marriage then you will have to figure out what about your marriage made you unhappy and if things could be different the second time around.