This sounds like a hit and run because if you damaged the pole which you hit, that is government property. If you're involved in an accident that causes property damage, you're required to notify the police. The fact that you didn't know that is a good reason for not doing it. Unfortunately, however, it does not rise to the level of a legal defense, because as judges and prosecutors are fond of saying, ignorance of the law is no excuse. All of us are deemed to know all of the laws of our land, even though that's not true. Otherwise, all we'd ever have to say was "But I didn't know it was illegal. . . " and we could never be accountable for anything.
I cannot tell you what you're looking at in terms of a penalty, because it varies from state to state and I don't know where you are. But if you are charged with this crime, it's generally a misdemeaor charge, and the worst you'd be looking at, unless you have prior criminal offenses of this kind is typically a fine and a suspension of your license. However, if you have a lawyer with you in court and if you have a clean record, a lawyer may be able to get the state to bargain this charge down to a traffic infraction. In that way, you'd avoid having a criminal conviction on your record, and you may also avoid a suspension and be able to pay a smaller fine.
If the police haven't contacted you yet, what you don't want to do is to talk to then about the incident without a lawyer being present.