Excellent question, and one I think many middle-aged married men struggle with. When clients present with this very issue, there is almost invariably one (or both) of two issues involved:
(1) You are at an age where testosterone naturally drops off. Around 45, your sex drive is almost inevitably going to drop because of hormonal shifts that accompany middle age. This drop affects some men more and some less, but almost none are totally exempt from some effect (slight weight gain is also a side-effect of dropping testosterone). In fact, several physicians offer shots of testosterone for men whose levels drop below an acceptable level.
(2) Given that you went out of your way to write, ",,,especially with my wife," I am guessing that there is some issue within the relationship that has not been overtly addressed. Built up resentment is almost exclusively thought of as affecting the sex drive of wives, but the same issue can also have a deleterious effect on the sex drive of husbands. If you are finding your wife less physically attractive or are harboring building anger toward her, you will be far less inclined to be intimate with her. Couples therapy may be warranted to determine the future direction of the marriage.
I wish you well. This problem can leave men confused and irritated. Best of luck. If you are satisfied with the response, please hit "Accept." This is the only way I can receive credit for my answer. Thanks-