I hope this message finds you well. I am a licensed attorney with over a dozen years of practice experience handling matters of a similar nature. It is a pleasure to assist you today.
Your questions is interesting and well taken. In reality (the way the states view this issue), driving may be considered a right - not merely a privilege, in that your ability (and freedom) to travel shall not be blocked or infringed upon in this nation.
Having said that, just as the right to own a gun, just because it is a right does not necessarily prohibit government regulation. In the vast majority of circumstances, even though you have the right to own a gun, you will have to subject yourself to a background check to purchase one. In many areas around the country, even though you can legally own a gun, you must have a permit provided by the government in order to carry that gun in public areas.
The same is true of your ability to drive. It is undeniable that a person does not have to have a driver license to drive on private property. However, government has the ability regulate the right to drive on public thoroughfares. The regulation comes under the protection of the common good (so they say) in assuring that only those that can be trusted to drive on common roadways are allowed to do so.
Your thoughts and arguments are valid. However, in the end, government protects itself and you will have a very hard time winning an argument that you are not required to have a driver license if you are stopped on a public road. As crazy and perverted as it sounds, government can even regulate our rights for the common good.
Let me know if you have any other questions or comments. Please also rate my answer positively (THREE OR MORE STARS) on the ratings bar on your end so I can receive credit for my response from the site.
Thank you and best wishes!