FIRST some background info
This is, in essence, a contract law issue.
And I want to start by saying that, at the end of the day, it is quite possible you are forced to pay back some, if not all of the money, even if you win the contract case under the legal theory of "unjust enrichment" (more on that below)
But lets talk about how this can possibly play out.
Under contract law, both sides to the contact have a duty to comply with their obligations under the contract. So, for example, if you promised to work for a period of time and do not, that could be grounds for a lawsuit to seek damages (the money the company lost by your not complying with your contract obligations).
At the same time, the company has a duty to act "in good faith" during the contract period. So, for example, if they did not provide you with safe working environment? That can be grounds for you to declare the company in breach of the contract and the contract void.
Prior to declaring the contract void, you also have a duty to act in good faith. And this would require you to give the company notice of the safety issues AND the chance to correct the problem.
If you give the company notice of the safety issues and the company refuses to correct the issue, that can be grounds for you to void the contract. If you do not give the company notice and a chance to fix, or if there is not a safety issues that impacts your personal safety (chance of death or injury), then you would not have grounds to void the contract.
THe best way to give notice to the company would be in writing (so there is no issue as to what you said and when you said it)
What you describe? It sounds like you gave notice, but not in writing. That is a potential problem. Since you have to prove what you said and when you said it. The other potential problem is you have to prove a clear threat to your safety other than what is expected in the normal working environmnet.
Assuming you can prove both of the above? Then you would have a basis to void the contract.
But then there is the concept of "unjust enrichment". They gave you money. You used it...and then you did not work for them. So even if they had safety issues, there is still the issue of unjust enrichment.
In law, unjust enrichment is where one person is unjustly or by chance enriched at the expense of another, and an obligation to make restitution
arises, regardless of liability for wrongdoing. A common example is when a party contracts to provide a service, but the contract is terminated prematurely due to a breach, and the contractor unjustly receives no compensation for partial services rendered.
SO it could be you go to court on this, fight and win the contract issue and still have to give back some of the money.
SECOND...on to your question
What you describe, there is a collections company trying to get money from you. They can attempt to collect. And you can refuse to pay. And life goes on...
They can not force you to pay unless they go to court.
If they go to court, you could raise the issue of contract breach...and it may be you win this (and do not have to pay any money) or that you win but still have to pay back some money under unjust enrichment
EIther way, if you go to court you will want to use a lawyer to assist you.
I suspect they are reporting this on your credit. They can. And you can dispute it...and if they continue to report, you can hire a lawyer and sue them under the Fair Credit Reporting Act. The FCRA is federal law that helps consumers fight improper negative credit reporting.
But what you describe, this would be a somewhat complex case...it would require you to prove they breached the contract and you gave them fair warning AND that there was not unjust enrichment. It is not a "slam dunk" case on your part. It could be you win...or it could be they win.
Still you have that option...you can sue them to force them to stop reporting this on your credit
Or...if you are not worried about your credit, you can ignore this unless/until they try and sue
Finally, you can try and negotiate a settlement with them...perhaps they will take a portion of the money they are demanding to close this case (and remove the negative credit reporting). If you do negotiate a settlement make sure to get it in writing