Thank you for trusting your question to JA today. I am a licensed attorney with over a decade of law practice and over 20 years of experience in the legal field. I’m happy to be of assistance.
Simply put, you can't avoid him suing you if he wishes. If you told me your company name right now, I could sue you on Monday. That's all I'd need. Now, I wouldn't have a valid claim and it would be dismissed, but you can't avoid my suing you.
So, your focus needs to be on establishing the facts here so that if you are sued, it isn't successful. Just put, in writing what happened. I don't think you need to (or should) try to classify it as sexual harassment. There is no need (or gain) to your doing so. Just stick with the facts and note those as the basis for termination.
In employment law, unless a person has an employment contract that states they can only be terminated for cause, they can legally be terminated at any time, with or without cause. This is called at will employment. So, you would already be protected from any legitimate suit by that fact. Add to that these facts, and you have more than sufficient basis to protect against any allegation that you terminated for an illegal purpose.
If you have any further questions, please let me know. I invite follow up questions, so use REPLY for those. If you have no further questions then good luck going forward and please do not forget to rate my service with a three, a four or preferably a five star rating so that I receive credit for working with you today. Please rate me based on my service and not on your satisfaction with the law, which I am not in control of and I am just reporting to you. Also, feel free to request me in the future, if you have questions concerning a different matter.