Normally therapists are not very direct in giving specific advice, but this is a clear question with a clear course of action.
These are young adults. They have decided to act outside of that role and are taking advantage of you. For their own sake as well as yours there needs to be established rules. If they do not wish to live by them, they have a choice. They can leave and there is nothing wrong with having adult expectations of adults. Truly, they are not disabled, impaired or limited.
Most parents establish chore and rule lists as early as 7years. This prepares children to accept the basic structure of life, earn and give back.
In this case that process is not in place but take heart. It is not too late to start.
I would suggest a couple of things. You mention that you do not like confrontation. Most people do not and rule enforcement is a difficult and sometimes unpleasant, but a necessary task.
However, before you begin to establish rules and process in your home I would like you to see a therapist. Why? You need a sounding board, someone to help you look at the nature of why you avoid confrontation on essential issues and how to regain your confidence and authority. Once this is done you will be able to successfully place limits and enforce them in a healthy and beneficial way. This will not take long but is important, even essential in this case, as you are dealing with adults, adults who can reason and talk back and minimize your efforts. You want to be successful and this will maximize your chances.
This is resolvable and will some brief guidance I can say that your household will be very different in a few months. Take heart! This is something that we, as parents, are never taught yet expected to simply know. That isn't reasonable either and I think you will do well once given the back up and support that you need.
Edited by Steven Olsen on 9/9/2010 at 12:17 PM EST