My name's Kel.
The exhaust could definitely be flowing via the outlet.
Turn off the power. Remove the plate. Unscrew the outlet. Pull it out. Caulk all holes. Caulk around the perimeter to seal between box and block. Do both outlets.
You should also check the weatherstripping on the door to the house. Is the door pulling up tight to it? Is it cracked? Are their gaps?
How's the seal between the bottom of the door and threshold? It may need to be adjusted because of wear or replaced.
Is there a ceiling with an access to an attic? The hatch will need to be weatherstripped.
Have I answered your question?