Dogs will eat/taste the strangest things, and believe it or not, eating drywall is a concern I've actually heard several times in the clinic I work in!
The school of thought at our clinic is that dogs eat drywall for one of two reasons: some kind of deficiency or plain old boredom.
First off, the deficiency. It could be that your dog is lacking some sort of nutrient in his diet. Of course, if you're feeding him a high-quality diet (think Science Diet, Eukanuba or Iams) then there shouldn't be a deficiency since these foods are loaded with 'extras' like vitamins and such.
The other reason is behavioral. Your dog is bored or doesn't have enough to do when you're not available to interact with him. If he's eating the drywall when you aren't home, then you're going to have to crate him or do something to keep him away from the walls. Although in theory drywall isn't toxic to ingest, it does have some chemicals in it and there's always the chance of blockage.
You can try spraying something called "Bitter Apple" (available at all petstores) on the walls to keep him from licking them. And if he's doing it when you're not home, maybe you can find a dogwalker to take him for a stroll during the day to break up the monotony...or better yet, doggy daycare!
I hope this helps.