I'm sorry, I'm about to give you an answer that you don't like.
I don't agree with your local tax instructor that tax law supersedes a Marital Settlement Agreement (MSA). If an MSA or divorce decree says one parent is entitled to take a child as a deduction, than the MSA or decree controls
However, I do not agree that father's failure to visit with children and failure to provide support means that mother, who is the only custodial parent, is entitled to take both children as a deduction. If the MSA or decree expressly states that if father breaches the MSA or decree, mother is entitled to deduct both children, then I agree that it would enable mother to take both deductions.
If you look at your MSA or decree, you should have a provision that states that breach of one provision is not a waiver of all provisions of the agreement.
In other words, you are not entitled to breach the MSA or disobey a Court's order because husband did.
I don't think it's equitable to you at all, of course - but the legal way to resolve that issue would be to ask the Court to modify the MSA or decree.
Please don't "shoot the messenger!"
I think that your situation is incredibly unfair, and I certainly understand why you did what you did.
However, you need to let the judge make the decision about what is appropriate in your circumstances, since a judge is the one who told you what do (either by a MSA or decree) in the first place. Failing to comply with a Court's order is contempt.
That being said, take your ex back to Court and ask for him to be held in contempt for failing to pay child support. He should actually be required to pay more because he doesn't even have them at all.