That is ok. Thank you for the clarification.
It sounds like you need more affection and love in the relationship and your husband is going the opposite direction. The issue here is why is he going in the opposite direction.
The incident you witnessed between your husband and this woman sounds very suspicious. Although he may not have taken it any further, he also is giving you a good reason to not trust him. Trust is vital in the relationship to make it work.
You need to sit down with him and talk this out. There needs to be some decisions made about your relationship.
First, start by talking with him in a neutral and gentle way. Do not accuse or demand anything. Let him know that you feel the relationship needs work. That both of you could use help and recommend counseling. You can find a counselor through your doctor or if you attend church, your pastor could help. Also, you can search on line at http://www.find-a-therapist.com/.
Gage your husband's reaction. If he seems to be thoughtful of what you said and/willing to work with you, that is a good sign. In that case, call a truce. Agree to not bring up problems until you get to see the counselor. For his part, he needs to agree to cut off any and all relationships and contact he has with the woman or anyone else.
If your husband does not seem that he is interested in fixing this problem, back off for a few days and try again. If he still seems uninterested, then this one is going to unfortunately be up to you to decide.
You may want to see the counselor yourself if he will not go, before you make any decisions about the relationship. Sometimes, time away from one another helps. It depends on what you feel would work for you. But if he is unwilling to work on this, he leaves you little option. You either have to decide if you are willing to live with his behavior, or if you want to leave the relationship.
Here are some resources that may help give you more ideas and insight into your situation. One is called http://www.marriagetoday.com/. Another is http://www.5lovelanguages.com/.
Some books that may help you are I Don't Want a Divorce: A 90 Day Guide to Saving Your Marriage by Dr. ***** *****e and Dr. William G. Clarke, The Seven Principles for Making Marriage Work: A Practical Guide from the Country's Foremost Relationship Expert by John M. Gottman and Nan Silver and Not "Just Friends": Rebuilding Trust and Recovering Your Sanity After Infidelity by Shirley P. Glass and Jean Coppock Staeheli. You can find these on Amazon.com or your local library may have them available.
I hope this helped you,