All relationships/marriages go through stages. It is possible that the stressors that you were exposed to made you feel burned out. Couples also start to feel complacent and follow a routine (versus do spontaneous things that they may have enjoyed while dating and prior to having children) That for some people feels like they have fallen out of love where in fact, they are no longer in the initial in-love/infatuation stage. Their relationship had matured to a different level.
As you take in more responsibilities in your life- being a mother, working, parenting, being a wife, etc. you start to feel insignificant as a lover. You start to focus on other priorities. If you partner does not help you or offer emotional support, if he stops being romantic and responds to your needs, you start to feel lonely and to doubt your feelings for him.
After you have had the affair, he's forgiven you. That is something that should not be taken for granted.
Also, he would not be able to change if you do not show him the areas of the marriage where you are not satisfied/happy. The two of you would have to talk and make the changes together.
Couple's counseling is always an option (depending on what you and him want out of the marriage) Why give up before trying first?
A workbook suggestion-
The Couple's Survival Workbook: What You Can Do to Reconnect with Your Partner and Make Your Marriage Work by David C. Olsen (Paperback)