That's a little tricky question b/c there is no right or wrong answer...this is grey area. I do think your primary loyalty remains to your son; also, he is an adult so ask him his opinion on this.
You don't pick your family, and it is very tough when there is family conflict. I have told many patients that if their family is "toxic" and not supportive, that maybe they should keep distance or cut complete ties. Most people struggle with feeling guilty over this (I sense that with you); if you cut ties or even keep distance, your family may not understand and blame you...but that's there issue. If they can't accept your son, they have to accept the consequences of a psychologically immature & discriminating decision.
I'm sorry I'm not giving you an exact, concrete answer...I think you have to a) keep distance, or b) cut ties completely. If it's very hard to decide, you could consider seeing a live therapist to help you figure this out. Otherwise talk to your son and any other loved ones you trust for counsel. Good luck & take care.
_____________________________________________________________________________Please let me know if you need more information. We can continue the dialog until you are satisfied. Once your are happy with the answers don't forget to hit the ACCEPT button & provide expert feedback. Also remember that this is not an official doctor patient relationship and not a substitute for a full live psychiatric evaluation. Thank you for using justanswers.com