The US tax law requires US citizens - regardless where they live - to file the tax return and report all worldwide income. These law provisions are in effect for very long time. However recently they draw more public attention because of 2009 and 2011 Offshore Voluntary Disclosure Initiatives offered by the IRS.
The fact that Canada's finance minister is aggressively speaking out is another evidence of the growing public attention. That is also an evidence that more and more US citizens living in Canada become aware of IRS initiatives and less US citizens ignore these initiatives.
As an US citizen you have certain tax responsibilities with US taxing authorities. These responsibilities are separate from your Canadian responsibilities. First of all, you have same filing requirements as all other US citizens regardless if you have any tax liabilities or not. Thus - you are required to file the US tax return and report all worldwide income.Most likely - you do not owe any US taxes because (1) you are eligible to claim foreign earned income exclusion (2) for income which is taxable in the US and in Canada - you may claim a foreign tax credit.Then - you are a subject of FBAR reporting if a foreign account value is above $10,000.The form to use is TD F 90-22.1 - this form is mailed separately.
I suggest filing your tax returns and clearing the situation. Generally - the IRS expects you to file tax returns at least for past six years. Note that if the foreign activity started more than six years ago, the IRS may also have the right to examine additional years. If you do not owe any taxes - it is unlikely you will pay any penalty.
Seems as your thoughts about renouncing US citizenship because of tax issues is overreacting, but you may do so. A person wishing to renounce his or her U.S. citizenship must voluntarily and with intent to relinquish U.S. citizenship:
See for reference - http://travel.state.gov/law/citizenship/citizenship_776.html
Also - if you do not owe any taxes - that is most likely - you may file tax returns on your own with minimal costs for you. Even if decide to use any tax preparation service - the cost would not be 2000 per return as you expected.
Let me know if you need any help or clarification.