He still is the one to decide about how to respond. But, with things being busy for both of you, you can just send another message or phone and say that you're just following up to see what would be a good day and time to meet because you do not like to leave things like this over the phone/email and would rather talk in person. At that point, if he follows up with making the arrangements or does not, will let you know how he wants to deal with the situation- either head on or by avoidance.
Try not to over interpret it unless he absolutely follows up with saying something else to confirm your fears. Men are not always very expressive (his response is typical) and may even indicate his hurt (and he's replying like that as a way to avoid further painful exchange of info vial the telephone messaging)