I'd like to help you with your question.
From what you wrote, it seems that there is not much of a relationship between the two of you. His coming in and out of your life suggests that he may have other relationships that he is involved in or that he is not interested in a relationship at this time. In any event, this is not about you but about him....about what he wants and doesn't want.
Sometimes people are just not honest about their feelings and that leads to hurt feelings.
I am sorry this happened to you.
If you would like to chat more, I would be happy to talk more.
Perhaps he is conflicted or indecisive. If he is not over an ex then certainly that could be a reason not to move too fast into another relationship. Having a 3rd person - meaning her - in a new relationship is a difficult situation. A job loss can hurt one's self-esteem in a big way...and it tends to drain away our feelings of worth. So that's another possibility.
My best advice is to not give up. Give him some space, check in with a text here and there just to say "thinking of you" and see what happens.
Maybe he is just not in a good place right now.