If you don't sell the property as per the agreement, the buyer can sue for breach of contract and "specific performance" to force you to transfer title. If the buyer is successful, you would also have to pay the buyer's legal expenses, which may be quite large ($25,000 or more).
You would have to prove that the buyer promised that you would have no tax liability on the transaction. If that promise was oral, then the written contract supersedes the oral promise, and you would lose that claim.
In sum, and based only on the very limited information that I have about your transaction, I do not see any easy means by which you could back out of the contract.
Doesn't seem worth it to me. Probably easier to just sell the property and move on -- unless you have some "smoking gun" evidence of the buyer's fraudulent representations that you could use to leverage your way out of the deal.
I realize that my answer may not be exactly what you were hoping to read. However, under the circumstances, the best that I can do is to explain what the law is and is not, so that you can avoid expending valuable resources looking for answers that do not exist, and concentrate on the options that are actually available.
I hope I've answered your question. Please let me know if you require further clarification. And, please provide a positive feedback rating for my answer (click 3, 4 or 5 stars) -- otherwise, I receive nothing for my efforts in your behalf.
Thanks again for using Justanswer!