It doesn't sound to me like the touching you experienced could be the source of this problem.
I suspect that your peripheral nerve disorder may be the most likely culprit. If you don't receive the normal signals from your genital area, you won't have the usual response.
I am no expert on peripheral nerve disorders. I would discuss this with a neurologist.
I don't think that you're doing anything wrong. Many women have a problem with reaching orgasm. For the average woman, it's mostly a matter of exploring what feels good and what feels sexual and is a turn on.
If you try exploring and can't find anything that really feels good and sexual and exciting, the two possibilities are that there is a block, either locally or in your brain.
The block in your brain would usually come from some bad past experience, such as sexual abuse. If you think there might be something like that, the best way to figure it out is usually by talking to a therapist.
If the block is due to poor signals from the peripheral nerves, perhaps a neurologist would have some answers for you. Best wishes.