This relationship may be crossing your boundaries and that is making you insecure. It actually would be surprising if you weren't a little insecure. You just have to make your wishes known and be reasonable in how you deliver your message. If he is a supportive partner he will find a compromise that works for both of you. You can tell him that his promise seems very transparent considering that situations will continue to arise. You have to find as a couple firm boundaries to deal with this situation and not feel like you are overstepping your boundaries. It is okay to need him to compromise. This situational should be as comfortable for you as it can be. He may also be hiding awkwardness on his end but doesn't want to admit it. This situation may be exacerbating your other issues. I would step up the communication that you have as a couple. This may mean finding new resources that help you avoid communication pitfalls. Use this situation to accomplish that and then everything else will be trivial.