Since you want to keep this on target of wifely duties........the fact is you are not his wife and thus have no duties for him.
Now with that being said.........no it is not your duty to be thin and pretty nor is it any womens duty to be what her husband wants. A women is who she is and a man is either attracted to that or he isn't. When he says things like he said basically he is making the statement that it would be easier to end the relationship if he was not attracted to you. Does this mean he has no feelings other then sexual? No......it simply means that he knows what he is doing is wrong, he knows he should end it but he finds himself attracted to you and wants you in a physical way.
You say he does not want to leave his wife, sounds like he loves his wife or simply loves the convince of his wife and wishes he had what he has with you in her. Is his marriage normal........no. Though what is normal for one couple may be preposterous for another. If he was truly unhappy he would have left. I am not saying he is lying but the reality is he is being untruthful with his wife....so why would you trust that he is being truthful with you?
While in a perfect world he would walk away from her and be with you the reality is he wants to be with his wife and it is unlikely he will ever leave her for you. More often then not when a man leaves his wife for the mistress he tends to end up resenting the new women and often cheats on her out of anger for what has happened. Forcing him to choose is not going to turn out well regardless of what he chooses.
I am going to skip the lecture as you are smart enough to know what you are doing is wrong. I do not like to use the word Karma but think about it.......you are having a affair with another womens husband and hoping for your own miracle. Taking someones husband is not the answer. I know it is not easy to find true love......but you will never find it with a married man. What you had before can never be recreated no matter how hard you try.
I would suggest letting him know that it is obvious that he really does not want to be in this relationship and instead of wishing for you to grow fat and ugly you will give him a out by leaving. Then do so. If he really loves you and wants to be with you then he will do so by leaving his wife. Otherwise it is time to move on.