Different contributor here. Please permit me to assist.
First, California has two different bad check laws. A civil law, and a criminal law. The civil law provides penalties for failure to satisfy the insufficient funds check in a timely manner. The law is very complex. See this link for a detailed explanation.
However, you don't really need a detailed explanation. The botXXXXX XXXXXne is that if you don't pay the check, then your penalties will start to mount up and you will get stuck in the mud with a huge debt. So, you need to get the money from TD Ameritrade, and pay the check cashing service so you can get rid of that part of the problem.
Concerning the service reporting you to the police, Penal Code 476a provides that it may be prosecuted as a felony to pass a check in an amount of $450 or more. However, to prevail, the district attorney must prove that you intended to pass the check with insufficient funds, and since you presumably could prove that it was a problem of the investment institution, then that would avoid the criminal conviction.
But, it won't stop the police from arresting you and charging you with the crime, and that means you will have to go through the criminal justice system which would cost a lot of unnecessary time and money.
So, once again, you need to get the money and pay the check cashing service.
Once you have done that, you can demand that TD Ameritrade pay all of your penalty fees that you had to pay the check cashing service.
If they refuse, then you will have to demand arbitration (because TD Ameritrade customer agreements have an arbitration clause, so you cannot sue in court). If you can show that the account had sufficient funds, and the check was stopped due to an error by TD Ameritrade, then the arbitrator will order TD Ameritrade to pay you -- and to pay your arbitration costs. Hopefully, that would be the end of the matter.
Alternatively, you can file a complaint with FINRA. If the error is as cut and dried as you describe, FINRA can force TD Ameritrade to pay you, or risk losing its brokerage license (which would be ridiculous, given the small amount of money at stake).
Those are the options.
Hope this helps.