Unless you donated the monies with the specific notation and limitation that the money could only be used for the purchase of the building and nothing else - and to be used for such within a specific period of time - then your stuck and can't get the money back.
Once a gift is made - the gift is now the property of the donee.
For a valid gift or donation to be made - there has to be:
1) intent to make a gift
2) delivery of the gift to the donee
3) acceptance of the gift by the donor
Once those three events occur - then it's a valide gift and the donor can't get the money or gift back.
I'm sorry, but it didn't seem that you made the gift with a condition that it could only be used for the purchase of the building - that you just made a gift with the idea that it was going to be used for the purchase of the building. However, without that express written condition - then they can use the gift for any purpose they desire and don't have to return it.
Please click on the ACCEPT button for my answer so that I receive credit for assisting you. You may continue to ask follow-up questions after accepting. If the information is helpful, I would very much appreciate positive feedback. Bonuses are also appreciated.