His behaviour might amount to criminal stalking, in which case you should refer it to the police. Indeed, doing so may prompt them to contact him and warn him off such behavior even if presently his behavior isn't quite serious enough to warrant a prosecution or Domestic Violence Order. If his behaviour is actually menacing such that you have a concern he may become violent or similar, you can apply for a domestic violence order against him to restrain him from such conuct, and if he then breaches the DVO, he will be committing a serious offence and will likely face a term of imprisonment (DVO's don't physically stop people from misbehaving, but they greatly enhance the consequences of such behavior and are therefore a generally effective deterent). You can read about stalking and the possibility of obtaining a DVO at the following links:
If his behavior falls short of actionable stalking, you should also consider reapplying to the family court for new custodial orders, and this ultimately is the only way you can actually terminate or reduce his access to the children, and whilst you do need to attempt mediation before the court will impose new orders, once you have done so, or if you have done so the court has the power to grant orders terminating or reducing the father' access.
That said the courts make their orders based on what is in the best interests of the child and there is a strong, but rebuttable presumption that it is generally in the best interests of a child to have and maintain a good relationship with both parents and it is only in cases where it is clear to the court that such a relationship is or will be damaging to the child that custody/access will generally be blocked by the court. Typically such orders may arise in cases where there is compelling evidence of violence, abuse (which can include verbal abuse), mental illness, drug abuse or serious criminal behaviour sufficient to convince the court that the parent is or will be a damaging invluence on the child or children.
Given what you describe, if one of your children is seeing a counselor and apparently is doing so because of the behaviour of the father, it would seem likely that were you to make an application to the court, the court would have to arrange for your children to be assessed by a court appointed psychologist/counsellor to ascertain how the children are feeling about their parents and what issues they are having. Children generally are not permitted to give direct evidence before the court so they are assessed outside the court context and a report is then provided by the court appointed expert for the court's consideration. This procedure is to avoid placing children in the position of having to testify against a parent.
Depending on that report the court may form a negative view of the father's role, but given the young age of the children, it can be difficult to predict with confidence how such you children will present when assessed, and this can make the outcome of such reports uncertain. It is always better if you can also support such a report with other evidence of the abusive or disturbing behavior of the other parent and for this reason it is really essential that if you wish to pursue such a case that you engage a lawyer to properly represent you and to ensure all the available evidence is found to support your case because the case has to compelling to convince a court to terminate a parents access to their child.
As your children get older the court will likely give more weight to their preferences, but until they reach their teen years the children's preferences are usually not treated as a major factor by the court, but that is something to think about in the future if you cannot get the orders you want at present.
As to moving to NSW fro Queensland, that may invite the father to seek orders that your return the children. At present there are orders in place that grant him regular access and the mere fact you want to move to NSW is not of itself a legal basis for not making the children available, and without powerful justification the court may well order you to return. If you are considering such a move then I think you would be better served applying to the court for permission to move and at the same time raising all these issues with the court as well as any other advantages for the children of such a move. In some ways though, that may undermine your case since it may seem to the court that you may be making the allegations against the father to justify the move, rather than the move being caused by the desire to get away from the father. Such an application is not a simple process and I would strongly advise that you engage a family lawyer to assist you.
If you need help locating a suitable family lawyer, contact the the Law Society of Queensland as they can refer you to suitable lawyers in your area.
I trust the above has given you some ideas as to how you can proceed.
Good luck and please rate my answer.