OK, I think you answered that you signed a buyer's broker agreement.
That being the case - you are bound by the term of the agreement.
However, if the agent hasn't used due diligence or acted promptly or done as it states they should within the buyer broker agreement they would - then you can contact the broker by formal letter (sending it by certified mail, return receipt requested) and ask that they formally consent to termination of the agreement immediately.
You should also state that if they refuse you will contact the office manager and demand that they terminate the agreement effectively immediately or you will file a complaint with the real estate
licensing commission and will pursue any and all legal remedies available including but not limited to the filing a breach of contract action against the broker.
That should elicit an immediately consent to terminating the buyer's broker agreement - they don't want a complaint filed with the real estate licensing commission against them.
Kindly do not forget to positively rate my answers so I can be compensated for assisting you.
If you have any questions, about this or anything else, please ask for me, Law Pro, directly in the question and I will try to assist you as best I can.
For example, you would state, "This question is for Law Pro . . . (then on with your question).