Thank you for using justanswer. Whenever you sell stock, you net the cost of the stock (plus any fees if they are not netted out of the gross proceeds) from the proceeds and you will pay tax only on the difference between your cost and the sale price. If I understand your question correctly, you took the money you made on the sale of the rental and invested it in the stock market. If that's true, then the $ amount you paid tax on is your cost basis.
If this transaction results in a negative amount (loss) you may claim up to $3000/year of that loss over & above your cost basis per year until the $ amount is used up.
Please see below for more in depth information
Frequently Asked Questions - Keyword: Stock Sale
Tax Topics - Topic 409 Capital Gains and Losses
I hope this helps you
Positive feedback and bonuses are always appreciated