Hello scott e,
There is a Federal Law which prohibits the "Obstruction of correspondence" - 18 United States Code Section 1702. That statute provides:
Whoever takes any letter, postal card, or package out of any post office or any authorized depository for mail matter, or from any letter or mail carrier, or which has been in any post office or authorized depository, or in the custody of any letter or mail carrier, before it has been delivered to the person to whom it was directed, with design to obstruct the correspondence, or to pry into the business or secrets of another, or opens, secretes, embezzles, or destroys the same, shall be fined under this title or imprisoned not more than five years, or both.
So, it is illegal for someone to take a letter out of your mailbox. But the law above also requires that the person doing so must have a "design to obstruct correspondence ..." In other words, there must have been an intent to prevent the letter from getting to you.
So, in your case since the person actually delivered the letter to you without opening it, I think it would doubtful if the person could be prosecuted under the above law.
On the other hand, if the mailbox was on your property, the person could be subject to local trespass laws.
I hope this answers your question.