Great question. And yes, "Transvestic Fetishism" is indeed a disorder listed in the DSM (the "cookbook" of psychiatric diagnoses). But here is the deal with that: Fetishes are categorized as "diagnostic," or in need of psychiatric care, when they interfere with (a) intimacy (i.e., because of the fetish, the bearer cannot engage in a healthy, emotionally and sexually intimate relationship with a partner), and/or (b) sexual functioning (i.e., the bearer cannot function sexually without the object of their fetish being involved). So, based upon your description, I would not right now categorize your issue as being in need of psychiatric care.
With just about any psychiatric diagnosis, the issue will move from normal to abnormal based upon how much distress/interference it causes. In other words, just about everybody gets depressed from time to time - but depression is only "diagnosed" when it interferes with a person's functioning. If a person feels blue but gets along with their life, they will eventually get better and move ahead. The sexual continuum is about the same: most everyone has one or two "quirks" to their sexual repertoire that, when used occasionally in fun or used as a part of their relationship with their partner, all is well.
I hope that made sense. Using the women's attire to masturbate occasionally and enjoying the feeling of the silk or lace on your skin is not in and of itself diagnostic, despite that it is statistically "abnormal." Your anxiety regarding getting caught is also grounded some is reality - there would probably be consequences if you got caught, say, at work... so you can minimize the anxiety by being more selective about when you engage in the cross-dressing and fantasy.
For now, enjoy... but keep your eye on the habit. If you ever cross the line I indicated above, then perhaps a psychiatric intervention would be helpful. If you are satisfied with the response, please hit "Accept." That is the only way I can receive credit for my answer. Thanks-