Thank you for using Justanswer for your concern.
It sounds like he may have the problem. Has he been willing to go to counseling?
You seem absolutely willing to look at yourself and improve yourself.
You may need to stand up to him and ask him to stop making these statements in from of the children, and affirm that you are not crazy.
But I realize that the situation is more complex than that. I really think it sounds as if he needs to attend couples counseling. Problems in a relationship are rarely just one person's problem. They are shared.
It is basically verbally abusive to call you "crazy". That is about as politcally incorrect as saying the "N" word would be to a person.
Let me know what other recommendations or help I can give you.
Edit - I'll be sending this to the relationship category, as per Justanswer policy. Good luck!