Hi, I'd like to help you with your question.
From what you are saying, it sounds like you do not feel important to your boyfriend/partner. You want to feel like you are a priority to him, which you should be.
You said that you've been together for four years and that he still puts you second. There could be a few reasons that I can see. One, he is putting you second because he is not as devoted to the relationship. Two, he may be devoted but feels resentful or upset about something in your relationship. Three, he may have a background that taught him to put his relationship with a girlfriend/wife behind other things in his life. For example, his parents could have had the same type of relationship and therefore he learned that is how relationships work. Although he should be putting you first, sometimes our feelings about our relationships blocks our ability to express ourselves. Regardless of his reasons, anytime we feel this way in a relationship it can hurt and cause much resentment, which is usually expressed through anger. That is completely understandable. However, for the relationship to heal, both people need to come together and forgive.
Communication is key here. You said that you already gave him an ultimatum. I'm not sure that will produce the results you are looking for, unless you feel that you are really ready to move on. What really needs to happen is the two of you have a serious sit down discussion about the state of your relationship and where each person feels it needs to go. There must be rules to this discussion, however. One, each person uses "I" statements. No accusing here. Two, the tone must stay neutral. No yelling or fighting. If that occurs, take a break and come back together when everyone is calm. Third, each person must be allowed to say how they feel without being criticized. These tips will help you get started.
There is also a site that might help you. It's at http://www.5lovelanguages.com/. This site is helpful to gain more insight into each other's needs.
If all else fails, consider couples therapy. If he won't go, go yourself. You may need the assistance of a neutral party to help you work out the issues and develop a good, solid foundation for your relationship.
If this relationship is what you really want, don't give up. There are numerous books, resources and help out there to keep you both together.
I hope this has helped you,