I certainly understand the situation and your concern. You would need to proceed through the court and seek to terminate his parental rights. Based upon what you shared above, it may be easy to show, since he has basically abandoned the child. I have provided for you below what the court will look at and what must be done. Once the parental rights of the father are terminated, then you could adopt the child. I 100% think you need to retain an attorney to help, to make sure this is done right, as well. Doing this will prevent him from coming back and doing anything to break your family apart.
27-20-44. Termination of parental rights. 1. The court by order may terminate the parental rights of a parent with respect to the parent's child if: a. The parent has abandoned the child; b. The child is subjected to aggravated circumstances as defined under subsection 3 of section 27-20-02; c. The child is a deprived child and the court finds: (1) The conditions and causes of the deprivation are likely to continue or will not be remedied and that by reason thereof the child is suffering or will probably suffer serious physical, mental, moral, or emotional harm; or (2) The child has been in foster care, in the care, custody, and control of the department, or a county social service board, or, in cases arising out of an adjudication by the juvenile court that a child is an unruly child, the division of juvenile services, for at least four hundred fifty out of the previous six hundred sixty nights; or d. The written consent of the parent acknowledged before the court has been given. 2. If the court does not make an order of termination of parental rights, it may grant an order under section 27-20-30 if the court finds from clear and convincing evidence that the child is a deprived child.
27-20-45. Proceeding for termination of parental rights. 1. The petition must comply with section 27-20-21 and state clearly that an order for termination of parental rights is requested and that the effect will be as stated in section 27-20-46.
2. If both of the natural parents of the child are not named in the petition either as petitioner or as respondent, the court shall cause inquiry to be made of the petitioner and other appropriate persons in an effort to identify an unnamed parent. The inquiry must include, to the extent necessary and appropriate, all of the following: a. Whether any man is presumed to be the father of the child under chapter 14-20. b. Whether the natural mother of the child was cohabiting with a man at the time of conception or birth of the child. c. Whether the natural mother of the child has received from any man support payments or promises of support with respect to the child or in connection with her pregnancy. d. Whether any person has formally or informally acknowledged or declared that person's possible parentage of the child. e. Whether any person claims any right to custody of the child. 3. The court shall add as respondent to the petition and cause to be served with a summons any person identified by the court as an unnamed parent, unless the person has relinquished parental rights, or parental rights have been previously terminated by a court. 4. If the court, after inquiry, is unable to identify an unnamed parent and no person has appeared in the proceeding claiming to be an unnamed parent of the child or to have any right of custody of the child, the court shall enter an order terminating all parental rights of the unnamed parent with reference to the child and the parent and child relationship. 5. If a petition for termination of parental rights is made by a parent of the child under this section or if a parent consents to termination of parental rights under section 27-20-44, that parent is entitled under section 27-20-26 to legal counsel during all stages of a proceeding to terminate the parent and child relationship. 6. Subject to the disposition of an appeal, upon the expiration of thirty days after an order terminating parental rights is issued under this section, the order cannot be questioned by any person, including the petitioner, in any manner, or upon any ground, including fraud, misrepresentation, failure to give any required notice, or lack of jurisdiction of the parties or of the subject matter, unless the person retained custody of the child. 7. At least ten days before the petition is heard, the clerk of district court or juvenile court shall provide a copy of the petition and summons, if any, to the department of human services.