Since you are on quite high doses, this would require a physician's supervision for the tapering down of the doses. The withdrawal, sometime can precipitate seizures, so one has to be very cautious. A tapering of about 1/4 mg every 15 day would be the norm initially. Your physician may consider covering you with phenobarbital substitution. This will cover up for the side effects and prevent adverse effects like seizure.
Hope this helps.
Please feel free for your follow up questions.