Hello, and welcome. I am a licensed attorney and happy to assist.
It sounds like you are concerned about the law of adverse possession which, under certain circumstances, allows a non-owner to take ownership of someone else's land. This can occur when someone occupies the property of someone else, under a claim of ownership openly, notoriously, and against the interests of the actual owner (i.e., without permission) Paying taxes is not enough. All of these other elements must also be present.
In a case of co-ownership, as with you and your brother, you would have three years from the date the brother advises you that he does not intend to honor your ownership interest, to bring a lawsuit to assert your rights. If you do not do so within three years of the notice date, the brother could obtain sole legal title to the property. If your brother has not yet given you notice that he claims or intends to claim the property as his alone, the three year statute of limitations, within which you must assert your rights in court, has not begun to run.
I hope this helps. If you need additional information or clarification, just let me know and I'll continue to provide assistance. If I have addressed your issue and/or pointed you in a positive direction, please let me know that, as well. Also, please remember to leave a positive rating when prompted, as that is the only way attorneys on this site are credited for the help we provide.