Hi, I'd like to help you with your question.
I agree with you, what your brother is doing is inappropriate and out of line. It appears that he never quite resolved whatever caused him to be abusive towards you originally, and now is taking his feelings out on you and your daughter in another way.
It is up to you to restrict your contact with your brother. It is probably a good idea to not allow him around your daughter without constant supervision. Since he abused you and has not dealt with his issues, he may attempt to abuse your daughter. Since he is saying inappropriate sexually based things to her already, then it is a possibility that he would attempt to take it a step further.
It would be a good idea if you can not see him at all. However, if this is not possible, try to only see him when others are around. That should help him keep his inappropriate comments to himself.
I also highly recommend that if you have not done so, seek counseling for yourself to help you deal with what happened and to give you support about dealing with your brother now.
I hope this helped you,