Small children make fast uncertain movements and have high pitched squeals which can either scare a dog or bring out the prey drive in a dog. This is why it is never safe to leave small children alone with a dog no matter how good natured the dog is.
If your dog is not used to being around small children then this will further the apprehension.
What you want to do is, first always supervise your dog when near children, and second, have the dog view children in a better light so he actually gets excited to see children.
Take some human food (cut up hot dogs, cheese chunks) ,something that the dog is not used to getting so it is considered special. Bring the dog to a public area where you may run into children and have the children call the dog's name and drop a piece of the food in front of him. You do not at this point want the children near the dog as you are reconditioning the dog's view of children. Once they toss the food they should keep walking. Gradually as yous see the dogs behavior change as children near him then , and only with permission, can the children try to pet him. And they should toss the food first, once the dog has eaten it, then they should pet under the chin not on top of the head. If the dog barks as they approach then he is not ready and don't push it.
This may be difficult as the children are so young so if you have friends or neighbors that have small children do it in your home first where the dog is more comfortable.
I would also recommend serious obedience training so the dog is 100% obeying commands when asked only once. If you have a dog that you have to ask to sit over and over again before the dog sits then this is not an obedient dog. I am going to direct you to a site for training that you can do at home. It is based on reward for the behavior you want. I would suggest doing this and doing it out in public as well.