You will want to petition the courts for custody based on the best interest of the child. I would advise you to hire a lawyer as they can get a emergency hearing in quickly and see if the judge will grant you custody based on the father not spending adequate time with the children. As far as his responsibility's these will not change with a full custody order. He will still be required to pay child support and such. The visitation may change some but he will still be granted visitation as well.
If this is not a possibility to hire a lawyer you will need to fill out the forms at the following link: Divorce Source: State Specific - Modification Forms for Custody ... Fill them out completely and submit them to your local clerk of courts office with the filing fee's. The clerk will help you get the paperwork needed to serve the father and will set up a hearing for a custody order.
I wish you the best,
DISCLAIMER: Answers from Experts on JustAnswer are not substitutes for the advice of an attorney. JustAnswer is a public forum and questions and responses are not private or confidential or protected by the attorney-client privilege. The Expert above is not your attorney, and the response above is not legal advice. You should not read this response to propose specific action or address specific circumstances, but only to give you a sense of general principles of law that might affect the situation you describe. Application of these general principles to particular circumstances must be done by a lawyer who has spoken with you in confidence, learned all relevant information, and explored various options. Before acting on these general principles, you should hire a lawyer licensed to practice law in the jurisdiction to which your question pertains.
The responses above are from individual Experts, not JustAnswer. The site and services are provided “as is”. To view the verified credential of an Expert, click on the “Verified” symbol in the Expert’s profile. This site is not for emergency questions which should be directed immediately by telephone or in-person to qualified professionals. Please carefully read the Terms of Service (last updated February 8, 2012).