If you have the contact information for the person you talked to at the IRS, try to call them back and discuss the matter with them. If you do not have the phone number, call them and re-initiate your contact with the IRS. Explain your situation to them and what the prior IRS agent communicated to you and see what they say. If you have the conversation documented it is definitely more advantageous. Always take down there ID number and any other information communicated. This is where I would start and see how it plays out.
If they do not reduced the penalties and interest there are are other avenues that you can utilize such as a payment plan or an offer in compromise. However, the IRS is not giving these out so freely since the recession and the government's lack of funds. You will have to prove that you are in a financial hardship to qualify.
If you have any further questions, please do not hesitate to contact me.