Family Law Questions? Ask a Family Lawyer Online.
You will have to file a motion to modify/cancel his visitation rights based on the fact that you and your daughter fear being harmed if left alone with him.
The court will consider the wants of the child, together with other factors such as any history of abuse, threats, etc. It may be that the court allows visitation, but limits it to supervised, etc.