Nope I would not worry about this at all.
Sounds to me that you are a fit and healthy man - keep up the good work!
PS - please get back to me if you want to discuss this some more.
There are lots of factors that affect resting heart rate - anxiety levels caffeine intake etc.
For sure athletes tend to have lower than average heart rates but for the rest of us ,who simply do regular exercise, it is unlikely to make any difference.
To be honest its not worth recording your resting heart rate - doing it when exercising is a good idea to check that you are having suitably hard workout.