Unfortunately, they are "allowed" to withhold your money even though it is their mistake. This kind of thing happens all the time, unfortunately.
There are really no lawyers that will be much help in this although you can certainly hire one if you don't think you can handle the appeal yourself although a SS appeal is really just filling out forms. You can also as that they spread the reimbursement out over a longer period of time. They will usually do this if the request is made but they dont' always do it very promptly.
The best thing to do is fill out the appeals forms. You can ask for them to waive the overpayment, let you pay it back in smaller sums, etc.
The request to waive the overpayment form amd to request a lower payment is at https://www.ssa.gov/forms/ssa-632.html
You can also get a lot of help by contacting your local US Congressman's office and ask for their help. They will usually do you more good than anyone else. DOn't try to talk to the Congressman, just work with their staff.
If your question has been answered then I'd offer my best wishes to you and ask that you please not forget to leave a Positive Rating so I receive credit for my work.
Of course, please feel free to ask any follow up questions in this thread. I want to be sure that all of your questions are answered. In addition, once you issue your Positive Rating the question will lock open and no longer time out so you can come back to it anytime in the future if you think of any follow ups.