From your question, I understand that you have a property, consisting of a garden, which is surrounded by other properties, on which building is going on, to the effect that your access to the garden from the street with a trailer. Or did I get it wrong? Is there a house (yours) between your street and your garden?
In any case, here is the law:
1. Any person must have access to their property from the street. This right is given by Section 917 of the German Civil Code. This access must be tolerated for any kind of normal use of the land (i.e. if a garden, for taking the trash out). However, it must also be reasonable, i.e. not with a trailer bigger than what is needed for the garden.
2. In exchange for this right, you must compensate the neighbors inasmuch as you use their land and for the entire duration of your use, with periodic payments, in advance. If you do not agree with them, they may ask for them in court and the court will decide on the sum. Viceversa, if they do not allow you to use their property, you may ask for this right of access.
So all in all, you may still access the garden, but only inasmuch as it is impossible from another way or by another reasonable means for the intended purpose, and if you use the neighbors' land, you must pay in advance and periodically.
I hope this answer was useful and look forward to your rating, which is essential to my activity.
Dr I L Vlad