Yes it does help. It sounds like it's happening quite often while at church. What has happened is that yes, it is pleasurable to you to look at the women and over time this has simply become an unconscious habit. But obviously noone likes to be looked at, stared at, etc. and you want to be more in control of yourself as well.
Like any habit, you need practice and commitment to break it. You say that you look, and then you realize after what you have done. So, the key is to be conscious and to be aware of this habit before looking. You need to hold it in mind, that you will be changing behavior, before becoming triggered. The attractive (I'm assuming) women is the trigger, Once triggered, you respond out of habit.
So, before you go to the church or place where there are triggers, you need to stay mindful of your goal - TO NOT LOOK! You can place something in your hand as a reminder, such as a stone to hold, etc. That stone is your reminder of your goal. Tell yourself that you will look at the stone, every few minutes to hold it in mind.
If not something in your hand, then focus on something in the church that is your reminder, such as something on the wall, a step in the church, etc. and refocus your attention to that every few minutes.
You do this, until the habit is broken. You are replacing the habit with a new behavior, in order to gain mindfulness. Then once you are just more present and more aware, you can stop altogether.
You will need to set small goals, along the way and reward yourself along the way too, with something else that is pleasurable but more appropriate.
Also, practicing deep breathing, meditation and anything you can do, exercise, etc. to help you feel more grounded and aware of yourself will be helpful.
Try this for a couple of times and see how it goes... feel free to write to me again here to let me know.
Please click ACCEPT button for this info, otherwise I'm not credited for my help today.
Thank you. Warm regards,