Yes, you should definitely continue on. I am glad you are retaining a lawyer, as this is not a hopeless case. She lost a lot of credibility in claiming "neglect" if she has waited this long to bring up the unfortunate incident you reference. Most Judges will rather question her continuing to let you have your child as you were, if she was truly concerned that you were a neglectful parent.
More than likely, someone has told her she is "entitled" to something, or she has tried to apply for state benefits, and was told that you'd have to be paying support for her to qualify for them.
You should expect to be asked what you think would be a reasonable schedule for visitation
. Your best tactic will be to ask the Judge to continue visitation and custody as has been occurring because your child is used to it. The Judge must consider above all what is in the best interest of the child
. Your argument will be that keeping the schedule as is better for the child than what your ex has proposed, which is highly disruptive.
You may be asked about the incident you reference. Tell the truth, and be honest, but also remind the Judge that it hasn't happened since, and at the time your ex was not concerned about it and made no fuss.