Have Parenting Questions? Ask a Parenting Counselor for Answers
You're certainly correct that boundaries need to happen and an agreement between you and your partner need to make the first agreement that unless your partner is there then there is no visitation from this daughter.
If she has a key then a new lock needs to find it's way to your doors via your partner's support.
Frankly ,you are in danger if you don't get an agreement between you and your partnerand you want to seriously consider other,safer living arrangements if this person does not want to sit down with you,your partner and the father of the baby and peacefully commit to new ways immediately.
Hope this helps!