It sounds like you don't have a well-practiced way to express your anger to your husband. Are you afraid he'll blow up at you if you criticize him?
I see you're not online now. Why don't you write down what you would say to him in a note, and assume you're not going to give him the note, so you can try as many ways as you want to try to say it. Then you could put the one or two ways you think would work best here online, and after each one tell me how you think he'd react. I can then advise you on how to alter what you've written. If he does typically overreact to disagreements, then you could email your comments to him and start with PLEASE DON"T REACT TO THIS BEFORE YOU'VE THOUGHT ABOUT IT.
If I were him I'd think: "I'm not disrespecting either you or your daughter. I'm $%&*#$& at your parents for (what? NOT doing what with your daughter--and isn't she HIS daughter too?)
And if you don't think he'll be able to respect what you're telling him, then tell him you'll need to bring your dispute to a skilled and neutral person, that is a couple counselor, because that will help BOTH of you keep from igniting each other, and insist that you contain your anger when it starts to get out of control. But this second version is starting out better--that's what happens when you let the heftiest steam off first!
So I would guess that you're used to your parents blowing up at you, or at least one of them. It also sounds like you may have married each other (on an unconscious level) because you both had somewhat similar difficult relations with your parents (like my wife had a narcissistic screamer-mother and a saintly early-dead father--11-, and I had a scary-temper narcissistic father and a depressed fairly early-dead --23--mother), and you are angrily (in keeping with your parents' style) insistent that he's not allowed to interfere with YOUR relationship with your parents by inserting HIS reaction to them. I agree that he needs to keep it away from the 3yr old.
But I think you'd benefit quite a bit from clarifying how to deal with your respective families by talking over your experiences with a couple therapist. You seem to be convinced that you MUST be able to be warm and close (or at least close) with your parents, while he chose to divorce himself more completely from them than you did--so now you're fighting over who did the right thing with whose parents.
Being out of work is pretty destabilizing for manymen.