You're welcome, Helen!
It is understandable that he fears losing his privileges with his children, but he should be working that out with his attorney. No reasonable judge would take his children away unless their were serious issues and it does not sound like there is any, so that may not be a valid reason. It seems like there is more to this than he is telling you. Plus the fact that you have only a few very brief interactions with his children tells you that he is still firmly committed to his wife and what she wants.
Your expectations are definitely not too high. He won't leave his wife for some reason but it is probably not for the ones he told you. Does he expect you to wait until the kids are of age? That would be a good question to ask him. That is many years from now and not worth waiting for. He basically is asking you to put your life and expectations of a normal relationship on hold while he does what he needs to with his wife and kids. And that is not fair to you. He is either in or out of your relationship. Unfortunately, you may have to make that choice for him and move on if you want a more committed relationship from someone.