You have to do what you feel is right - and that doesn't mean what he feels is right. You have the right to a happy home. If you feel too much damage has been done then you should end the relationship so that you can emotionally move on. If you feel that the relationship is something you would work on then find a marriage counselor in your state. Sometimes people stay too long and they don't realize that. You should have to make your decisions based on what you think your future holds. If you feel he is abusing the relationship then end it. Don't stay out of obligation. You should go with your instinct because you know him and his ways.
Think clearly about what you want and follow through. If your husband cannot respect the relationship then show him that you want one that will make you happy