Welcome to the site.
I am Dr. Kaushik and i believe i can help you with your problem.
First of all i am sorry to learn about your predicament as it is always immensely difficult and emotionally draining for one to deal with borderline personality individuals as such people tend to become dependent on you leaving no space to breathe or act freely.
Now coming to your problem, well you see it is about time for you to draw and demarcate boundaries within your friendship for your friend to know , acknowledge and accept them ( provided he chooses still to be friends with you as this will be like walking on fire and it may jeopardize your friendship with him), so basically no more ignoring of issues which violate your right to happiness, because if you look at this relationship your friendship in some ways reflect a parasitic relationship where your friend has been feeding on your time , emotions , free will , happiness without giving much in return.
So by sitting him down and politely yet firmly telling him to obey the boundaries which you are going to outline within the relationship so that the friendship lasts long and does not become a sorry state of affairs where one of you out of desperation has to end it once and for all.
So in simple words if you put it , yes you got to tell him not to take you for granted or push you around like the way he has been doing for a while, there shall be some clearly noticeable boundaries which either of you shall follow, explicitly delve into things which he does and which turn you off, which irk you immensely so that he does not repeat them in future. After doing this much there will always an option for him to accept this and move on with you as good friends towards future or else call it quits.
You see by ignoring his disruptive behavior will only become more aggressive and decompensating for you , so kindly do the needful which has been avoided by you so far. But yes do take a subtle and polite approach while addressing these core issues.
I hope this helps.
Wish you all the best.
Please kindly leave a Positive rating if you are satisfied with the answer.
If there is any further assistance needed please feel free to ask using the reply button.