I can understand the frustration that you are experiencing. You are correct about the issues creating animosity. In fact according to John Gottman there are four key predictors for relationship dissolution; defensiveness, contempt, criticism, and stonewalling. I can tell you from personal experience that what you are experiencing will cause you to grow to resent him and cause further problems. He specifically states the following as antidotes for each area:
"When you attempt to defend yourself from a perceived attack with a counter complaint you are being defensive. Another way to be defensive is to whine like an innocent victim. Unfortunately, defensiveness keeps partners from taking responsibility for problems and escalates negative communication. Even if your partner is criticizing you, defensiveness is not the way to go. It will only fuel a bad exchange. The antidote to defensiveness is to try to hear your partner’s complaint and to take some responsibility for the problem."
"When you criticize your partner you are basically implying that there is something wrong with them. You have taken a problem between you and put it inside your partner’s body. Using the words: “You always” or “you never” are common ways to criticize. Your partner is most likely to feel under attack and to respond defensively. This is a dangerous pattern to get into because neither person feels heard and both may begin to feel bad about themselves in the presence of the other. The antidote to criticism is to make a direct complaint that is not a global attack on your partner’s personality."
"Contempt is any statement or nonverbal behavior that puts yourself on a higher ground than your partner. Mocking your partner, calling them names, rolling your eyes and sneering in disgust are all examples of contempt. Of all the horsemen, contempt is the most serious. Couples have to realize that these types of put downs will destroy the fondness and admiration between them. The antidote to contempt is to lower your tolerance for contemptuous statements and behaviors and to actively work on building a culture of appreciation in the relationship."
"Stonewalling happens when the listener withdraws from the conversation. The stonewaller might actually physically leave or they might just stop tracking the conversation and appear to shut down. The Stonewaller may look like he doesn’t care (80% are men) but that usually isn’t the case. Typically they are overwhelmed and are trying to calm themselves. Unfortunately, this seldom works because the partner, especially if a woman, is likely to assume they don’t care enough about the problem to talk about it. It can be a vicious circle with one person demanding to talk and the other looking for escape. The antidote is to learn to identify the signs that you or your partner is starting to feel emotionally overwhelmed and to agree together to take a break. If the problem still needs to be discussed then pick it up when you are calmer."
So that is Gottman's approach. Now, you have attempted to communicate with him about what is bothering you and how you feel, but have you specifically mentioned that your needs are unmet? That you feel that you are not a priority to him? You may be in need to share with him exactly what you are missing and you can do this through discovering your love language....
I will share more about that towards the end and send a link for you!
It sounds like you may be dealing with some personal insecurity, but he also may feel like his "hands are tied" in the aspect of his ex..... but for you.....Insecurity can be really hard to overcome, but it doesn’t necessarily mean that this has to be a part of your relationship. You can focus on how you can use your insecurity to create intimacy, but you have to do this through honesty and vulnerability, rather than focusing on your anger. Sometimes we are reactive when we feel insecure. We really want to understand, but instead we react. There is a difference in responding and reacting. Make sure he understands that you wish the relationship seemed to come first for him also.
I am going to send some tips on healthy conflict and how it can lead to you guys becoming closer. (yes, more communication stuff)
There is no need to fear fighting, it is healthy— it opens communication and leads to resolution. Often conflict shows us where we can or need to grow.
When fighting, focus on the issue— not the person. you cannot attack someone’s character or go after their past mistakes to make them feel bad. You have to remain open to their statements and how it can be solution focused. It’s enough to deal with the problem without adding the new problem of hurting each other’s feelings.
Also, listen to the other person when they are communicating. Be aware of your body language and tone when communicating. Sometimes we don’t realize that we begin to assert ourselves loudly and that escalates the situation. So talk softly and make eye contact. Be honest and vulnerable. His own vulnerability with you leads me to recognize he truly trusts you....
It is also easy to be defensive, and it would seem natural to want to justify your feelings. Try to be curious about what they are saying, but not with sarcasm. Be sincere.
Fighting ends when cooperation begins. Ask for options, ask what you can do to adjust your role and show empathy? Offering alternatives of your own shows that you also are willing to try something different and work toward a collaboration with him.
Take ownership of your role too, make concessions to him about your fears and feelings. If you give a little, it makes room for the other person to make concessions too. This isn’t about scorekeeping. It’s about finding a solution that is workable for both of you.
Be vulnerable and remember that anger is a secondary emotion that protects us when we are hurting, disappointed or scared…. Try to be honest with him and recognize how much is your issue to address without blaming him.
The other thing that I would encourage you to try is understanding your love language and his. Gary Chapman found that there are 5 patterns of emotional connections and how we demonstrate them. He calls these our "love languages." The Love languages are words of affirmation, gifts, physical touch, quality time, and acts of service. I will explain a bit of these. Words of affirmation are based on demonstrating appreciation, approval, recognizing efforts made by you or your partner- on the flip side if you recognize that you are sensitive to criticism or perceived criticism this can be a sign that your love language may be words of affirmation. Gifts are about little tokens or offerings that suggest that you are thinking of someone. It can be simple or elaborate, but it's the thought that counts. As far as acts of service, a quote comes to mind, ”Actions speak louder than words.” Acts of service may include mowing the yard, helping with laundry, starting the dishes, making the bed, helping load the car or unload the car of groceries. It could even be something that helps someone else in their career— anything that offers support someone through an action. Quality Time is based on providing undivided attention to your partner. Turning off the TV, making eye contact, turning toward them as they communicate with you— it is about focusing on your partner without allowing distractions to occur. Finally, physical touch, there is power in tenderness. This does not have to include sexual touch, but more or less, gentleness. It can be holding hands, a stroke of their hair, a massage, or a simple hug. There is encouragement in touch.....This can help you meet your partner's needs, but also have yours met. He may need to hear about what he is "doing right" so that he feels encouraged to try to meet you halfway.... Does that make sense? It isnt about talking about the problem always, but also talking about what works and thinking from a solution focused place-- like, how you both can make amends and change the patterns....
Here is the link for the love languages if you would like to try it!