You are exactly right on that. I just wanted to be sure we are both talking about the same noise, this is what I thought you would be dealing with. VERY common on this body ES and also the RX.
Open the driver door and get on your knees with a flashlight, behind the brake pedal area you will see where the steering shaft comes down to exit the vehicle.....the shaft passes through a big black rubber boot. Your issue here is that the boot is contracting with cold temperature and the lubricant between the shaft and the boot has dried up, this causes the boot to "drag" on the shaft when you turn it and make noise....it's like shoes on a basketball court, annoying.
What I recommend you do is use some glass cleaner or rubbing alcohol to clean out this area a little, stick your finger in the boot and clean around the shaft as best as possible, it's a tight fit but the boot does have some flex. Once it's cleaned out you need to stick a blob of silicone grease on your finger and work it down in this area, rub it all around the shaft where it touches the boot/etc....it helps to start the car and turn the steering around some while you do this also to work the lubricant in.
If you don't want this hassle sometimes you can short cut all this by spraking some wd-40 or spray lubricant in this are and work the steering around but usually this will not last.
Let me know if you have further questions, I hope this helps!