I don't really believe that not wanting to bring another child into the world speaks about whether he loves you or not. The two of you have four children between you already. It takes a LOT of money to raise a child today, not to mention the strain on the parents and the children you already have. He may just be thinking about that. Four children may be all that he feels he can rear successfully. That shows a great level of maturity and love for you all.
Remember also that there are many couples who marry never intending to have children at all. That doesn't mean that they don't love one another.