Thank you for using JustAnswer.
No, it's not legal. Now as to what could be done about it, assuming that the court order governing custody is clear and states specific times and methods for the return of the child, the custodial parent
can potentially call the police, and this could be prosecuted as "custodial interference". Now sometimes the police will say that it's a "civil matter" and that the custodial parent will need to contact the court that handled the court order regarding visitation / custody.
This is certainly a "contempt" action, in that noncompliance with the court order without a very good reason to do so is a violation of that order. The custodial parent can go to court, seek a contempt order, and the if other parent does not respond, the judge can actually revoke any possessory interest of the noncustodial parent (meaning that the continued possession f the child becomes "kidnapping" and can land the noncustodial parent in jail).
So in short, yes, it would absolutely be illegal.
Hope that clears things up a bit. If you have any other questions, please let me know. If not, and you have not yet, please rate my answer AND press the "submit" button, if applicable. Please note that I don't get any credit for my answer unless and until you rate it a 3, 4, 5 (good or better). Thank you, ***** ***** luck to you!