Hi and welcome to Just Answer!
If the property was rented for less than a fair market value - that is no-for-profit rental.
In this case:
- no depreciation is allowed
- no rental losses may be deducted against other taxable income
If that was done incorrectly - your mother would generally need to amend her past tax returns
and might be liable for additional taxes and penalties.
If depreciation was deducted correctly - it should be recaptured and added to their other income. Assuming the depreciation basis was $20,000 - total depreciation was $20,000 / 27.5 * 11 = $8000 - that amount should be recaptured.
Your wrote - she converted the second floor to an apartment - that actually means - there are two separate dwelling units - and capital gain should be calculated separately for each unit. If that is correct - only one unit may be treated as a primary home, The other unit would be treated as a rental property and the full capital gain will be taxable.
Unfortunately - original incorrect reporting triggered issues for following years.
There is no simple solution.
Let me know if you need any clarification.