It is common to feel torn about such an issue and I can appreciate your divided feelings. On one side you want him to feel better but are also concerned that you may not be able to handle a relationship with him. You made a promise to him and feel obligated.
Is there any reason why a decision to maintain this relationship has to be made immediately? You can still fulfill your promise and then make a decision about your relationship...No one can really make this choice but you but it seems there is a middle ground that is not being fully explored.