I am an attorney in Texas. It is always difficult to provide certainty in any opinion because a final decision is made by a judge or a jury.
However, property lines can be altered by adverse possession. It seems when the house was built in the 1960's that it was the owner's intent to claim ownership of the underlying land.
There are some facts that are important, but it is my assumption that you have been taxed for the land underneath your home for the past 20 years and that the prior owner's were also taxed for the land underlying your home.
With those basic concepts, you have an extremely strong claim to ownership of the land underneath your home and inside the "yard" that you have used in connection with the home. This claim is based on the 3, 5 and 10 year statutes of limitation. A successful claim by you under any of those statutes would result in a court judgment eliminating any claim of your neighbor
The above has assumed there is any merit to the neighbor's statement about your home being 12 feet over the property lines described in the deeds to your respective properties. You could obtain a survey to determine whether your neighbor's statement reflects how the property line is described in those deeds or not.
Then there are other questions as to whether there is a mistake only in your neighbor's deed or not, for example.
So, in short, you should consult an attorney in your area who may very well recommend that you start by obtaining a survey. Secondly, please understand you have, again, an extremely strong case favorable to you based on the fact that the home has been on the underlying real estate
for 50 years or so.
If your ownership is either confirmed by the survey or through a lawsuit based on the statute of limitations, then you can follow through with your plans
Please let me know what additional questions you have. I want to fully respond to your question so that I may be paid.