Terri asked me to see if I could help you with this issue you are having. My first recommendation would be starting house training all over from the beginning and doing crate training. I'm not sure why you decided to stop crate training, but I do know many people stop because a dog barks or whines until the owner gets tired of it and lets them out. When an owner does that, it reinforces the barking to be let out behavior. So we recommend that a dog never be let out of a crate untless they are calm and quiet. I'm going to give you a site that goes over proper crate training.
You will want to clean any areas that the dog has eliminated on with a good enzymatic odor remover. Other cleaners are good enough to fool our noses, but not a dog's nose which is more sensitive than ours. Dogs will naturally go to areas where they have gone previously to eliminate again especially males. So if the odor is still present, it encourages the dog to eliminate there again.
I also recommend a medical checkup. Pugs are prone to a few conditions of the back such as disc issues and spina bifida and hemivertebra which might contribute to him not being able to control his bowels or urinary functions all the time. You can read about these here:
At night, he might be further from the dog door and unable to reach it in time. The same may be the cause when you are not home. But a medical checkup looking for this type of condition can help rule that out. A fecal check should also be done to rule out parasites which can also make it hard to control their bowels.
It might also be that your dog is not yet mature enough to want to leave the safety of the house when you are asleep or it is dark out or you are not home. He may feel he needs to stay inside to "watch over" the house (territory) and you. Many dogs are reluctant to go out at night without their owners.
The fastest solution to the problem would be to crate him at night and when you were not home. I know some clients that have put the crate with the door open up against the doggy door and secured it when they left which forced the dog to use the doggy door and the dogs learned to go out when the owners weren't home. This is usually only an option at night time or when there are more than one entrance to the home.
I hope this information is helpful to you and one of the suggestions helps solve the issues.