If that isn't on the record then the Courts of Appeals won't consider it. The best thing to have done was 1) not to speak with them at all without your lawyer present and 2) to have questioned them about that in court.
Nothing, other than that remark which isn't really useful under the facts you describe, will point toward them being prejudiced against you. The same type of thing happens to fathers all the time and it's more a matter of them being prejudiced against mean than your race, or at least that's what the statistics will say.
You can appeal the temporary decision but realistically it is better not to do so and to concentrate on the next hearing. If you can afford to hire an attorney, do so. Some court appointed attorneys are great and some are not but the disadvantage of getting one is that you never know what type you will have.
I'm not sure why they are using a guardianship through probate court rather than trying for conservatorship through the family court. You may want to discuss this with whatever lawyer you end up with.
You can also discuss asking the judge to issue an order for drug testing with it to be done immediately. The lawyer will explain how that works because you don't want her to have enough time to get "clean".
There is really no way to file a complaint against the judge in the judicial process, you have to go through the judicial complaint process and you probably don't want to do that, if you do it at all, until the case is over because it is something that you would carry with you even if you get this judge struck, the next judge will know you filed the complaint and may hold a grudge.
If your question has been answered then I'd offer my best wishes to you and ask that you please not forget to leave a Positive Rating so I receive credit for my work. Of course, please feel free to ask any follow up questions in this thread. I want to be sure that all of your questions are answered.