What specific question would you like feedback on?
If in fact his behavior is driven by guilt, you can continue to reassure him about where you stand. He has to work through this on his own. You can not speed it up for him even when you're supportive. He has to accept his mistake (everyone makes them), accept you've forgiven him, and then forgive himself/deal with the shame and move on.
Another possibility when someone acts like this way is that they are still struggling with doing what they know is expected and may have regressed but not told you about it. But, if you've never had this issue before, that may be what is making it difficult for him to get over it. If he is willing to talk to someone objective, you may ask him if he's open to talk to a counselor. There are also online support groups.
Otherwise, be there for him to listen when he vents, reassure him and do not spend a lot of time on your end talking about the issue. Tell him you've found your peace and wish he does try to find his.