Great question. I like that your son has a general feeling of comfort around you and that he does not feel shame or embarrassment about his body. Your boyfriend may have a point to consider, though.
Boundaries are important to think about when raising kids, particularly if you are a single parent. Often times, single parents (we'll call them "moms" in this case!) do get very close to their children following divorce or death or abandonment, which can skew the kiddos understanding of healthy boundaries. For example, if your son grew up with dad around full time, then there would have been a natural hierarchy in the home - dad is the leader of the pack and son falls into his role as the child. Too, you would be invested in keeping better boundaries because you would be a mother and a wife, rather than focusing full time on being a mom. But here's the point: if your son feels too close to you, he will have a hard time moving away from you once he reaches an age where romantic relationships and "leaving the nest" are supposed to happen... and this would be harmful to him.
Of course, keeping the better boundaries will be difficult for both of you initially because you have been operating like this for so long. It is somewhat the equivalent of moms (or dads... parents) who allow their kids to sleep in the parental bed for too long. After while, it is just not appropriate anymore, and something has to be done. But the kids are entrenched in sleeping in the parents' bed, and do not want to leave. Your situation is not as odd as those, but it hits upon the same concept - kids need to establish their own space and keep to it so they know better how to protect themselves and stay intact once in other intimate relationships.
And I will add this - if there is a history of sexual abuse or molestation in either yours or your son's history, then boundaries are even more important to attend. Making sure everyone has their well-defined roles and that body/nudity boundaries are adhered is even more important.
So have your kiddo get his own towel and coach him to wrap a towel around his waist from the bathroom to his bedroom. In the long run, he'll be better off!
If you are satisfied with the response, please hit "Accept." That is the only way I receive credit for my answer. Thanks-