The rules governing your case would be found in any orders the Judge has handed down. These are individualized for each case.
So if you have a temporary order
stating that you are the primary parent, etc, and a visitation schedule has been started, then your child's father would be in violation of the court order, and a Judge could give the police the authority to go get your daughter back.
Some courts issue what are called "standing orders" that are orders that are always attached to any case. Usually, in these orders, it tells each parent that they are not allowed to interfere with the other parent's time of custody.
Of course, if you have no orders in place, then the default is that each parent has the right to their child equally. Essentially, you will need to read very carefully any order the Judge has handed down and signed, and that is what is the controlling rule in your case.
I would immediately speak with an attorney, as the longer you wait, the longer your child is kept from you.
Legally, the father could be charged with Custodial Interference in the First Degree, which is a class C misdemeanor, if he is interfering with your custody. He can also be held in contempt of court and jailed or fined for violating the court order.