Does he use the potty pads?
When he goes inside what sort of surface does he choose?
When you take him out is he out on a leash or loose in the yard?
What sort of conditions was he kept in before you got him?
Does he respond to food treats?
If you are using a collar with him for leash time try going to a properly fitted harness.
A harness is usually less terrifying to a dog than the feel of the collar around its neck being pulled on by the leash.
He has missed critical socialization time and may have never been outside before geting lost. With no early handling its harder for him to bond to humans.
It sounds like he is more used to a hard surface under his feet for potty purposes.
I'd suggest keeping track of the timing of when he goes so you can get him outside before he typically needs to go.
When he goes in the house take the clean up out to the yard. Rinse off wet paper towels or the pee pad onto the grass to help scent it so he can recognize the area as OK to go on.
Set up a chair outside if you can. Go out with him on a harness and leash and give him space to get to either a hard surface like cement or tar or gravel and grass. Being near a wall may help him too. If you can get a bit of vinyl flooring to put in the yard that might help him too.
Bring a book or music for you and sit down and pay no direct attention to him but give him plenty of leash. Be boring and not stressed and let him work on checking the outside out. A longer than usual leash may help if he needs to be away from you to go.
You can put down some paper plates or plastic container lids with a small but yummy and aromatic treat on them for him to find in different spots.
Do this as often as you can. The idea is for him to get comfortable being out side, to find outside interesting and not terrifying and to eventually realize he needs to potty and go out there instead of inside. If going inside is his big wish let him go in once he potties. Be prepared for a long wait. He may not be able to go the first times out there even with the scent down for him. In that case bring him in but keep him controlled.
Inside the house put a 6' leash on him and tether him to you so he cannot sneak away and potty. This also gets him used to having to follow your lead.
Bring him out any time he has had a meal, a big drink, has woken from a nap, has played hard etc.
With this sort of situation the dog may never be a normal pet. If you can handle another dog one thing that could help him is to be with a friendly, well socialized housebroken dog to help him learn what should be done.
With a dog this poorly socialized and trained, as a rescuer, I might spend one to two years in rehabilitation work before considering him suitable to go to a new home and then I'd likely only place him where there is another dog.
You may be able to get him better than he is now. Training through positive reinforcement methods should help, even if you have to toss him a treat instead of handing him one, but he is going to be a project to get him to where you'd like him to be.
This site has some clicker training videos on the bottom below the ads which may help you work with him
http://www.clickertrainusa.com/clicker-training-videos.htm
and this site goes over helping shy dogs
http://kimmurphy.net/shy-k9s-faq.html
Hope this helps you!
Dog Expert:Rescue, Train,Breed,Care
30+ yrs dog home vet care & nursing, rescue, behavior&training, responsible show breeding, genetics