I'm sorry to hear of your dilemma.
You do not have necessary facts with which to convince a court that both parents should have their parental rights terminated--and you say that you do not want to do this, so it is a non-issue.
The only thing that you could do at this point as a proactive measure, would be to apply to the court to be appointed your grandson's Legal Guardian, if you are not already in that position.
You say that you are the custodial parent, however, if there is no court order to that effect, either parent could take your grandchild without notice and you would be hard pressed to prevent it. The status of Legal Guardian would give you the protection you are looking for. Otherwise, all you can do is to respond through the court system if he is taken by your son or by the natural mother.
I urge to you act right away. While the action is pending in the court, the court can order that the child remain with you.
I wish you and your family the best.
Thank you very much for having allowed me to assist you. It would be greatly appreciated if you would click the green Accept icon so that I can receive credit for having assisted you.