Thank you for using JustAnswer.
I'm sorry to hear about your situation. You certainly should call Capital One, and probably call the authorities. The reason is that that would be consistent with what you would do if you were innocent. Keeping it from the credit card company / authorities would make you seem "more guilty" and would be more consistent with someone who was trying to engage in identity theft. So absolutely contact Capital One. There's no doubt that you should do so. Also contact the credit bureaus and "freeze" your credit so that you'll be protected should he try something with yours. Here's a guide on how to do that: http://www.clarkhoward.com/credit-freeze-and-thaw-guide. Contacting the authorities is not as necessary, in that it's not necessarily a "crime". The point is that you can contact them, explain the situation, and then ask what you should do. The reason for this is more about creating a "paper trail", so that you can prove that you really didn't have anything to do with it.
As far as destroying the letter, you should be in contact with Capital One to see what you should do. Document everything that they say (who you talk to, when, etc...) and follow their instructions. If you unknowingly have any accounts open with them, close them (unless you want to keep them open).
If the account was opened in his name, they might change the address rather than close it, but they also might close it if they can't get in contact with him. And you should certainly tell them that someone is sending mail to your address and it's not for you. Again, the reason is more to create a paper trail and show that you took actions that were consistent with being innocent in the matter. If you were trying to "steal his identity" you would not contact the credit report and say that this was a mistake.
Hope that clears things up a bit. If you have any other questions, please let me know. If not, and you have not yet, please rate my answer AND press the "submit" button, if applicable. Please note that I don't get any credit for the time and effort that I spent on this answer unless and until you rate it positively (3 or more stars). Thank you, ***** ***** luck to you!