I always caution women when men are focusing and hiding these behaviors that there could possibly be a sexual addiction. If that is the case, it is difficult to get his behavior under control even when they want to. it is similar to drugs because the symptoms can be an inability to eliminate sexual material or behavior, hiding these things, compulsively doing these things, ignorant of consequences, and excessive use of porn, sex lines, prostittutes etc.
With that sad, it is very hurtful to your relationship and could explain why he keeps getting caught. You have to come together as a couple to eliminate this and it will take his total commitment. It also will take some ability to avoid such behavior such as not going to the 7-11 for instance if he buys dirty mags there. I know it may sound juvenile but it is that simple
Also he needs to work on rebuilding trust because that is crucial. You need to maintain strong boundaries that put limits on his behavior. You also need to set limits and stick to them. Set a consequence but be willing to back it up. Only then will he know you are serious.