If the area is cool to the touch it indicates a potential blood flow issue and this is something that needs to be evaluated in the office. The best course is to make an appt with your primary care physician to see what the cause is. This can be caused by a blood clot, physical injury to the blood vessels in the area, etc.
I'm not sure what you are looking for. You asked if you should be concerned with what you are experiencing and I explained what can occur. I also stated that you should be concerned and it is important to get into your regular doctor since this can indicate some serious issues that need to be ruled out.