Because it is not in writing it is your word against his.
But you will have to ten ti to court to get an order that he be removed.
The following is of relevance to you
he owner may apply to court for your tenancy to end on the grounds that they would suffer hardship if they had to terminate the tenancy under any other section of the Residential Tenancies Act 1987 (WA) (s. 74).
If the court gives the owner the order to end the tenancy, the court can decide when the tenancy is to end and order compensation to you for any loss caused by the tenancy being ended (s.74 (2)(b)).
It is important you go to court to discuss your situation if the owner makes this type of application. The Residential Tenancies Act 1987 (WA) has no provision allowing the tenant to terminate the tenancy due to hardship. Contact Tenants Advice Service for more information.
Here is the link it came from.
It is not going to be easy. If they fail to pay rent or breach the lease in any other way then you can terminate and have them evicted.
I hope this makes sense and is of assistance. if there is nothing further
thank you for using my services.
If I have missed anything, or you have any further questions please let me know
If there is anything else in the future please do not hesitate to ask.
Please do not forget to leave positive feedback.