Hello and thank you for using just answer. You do not have to remove the instrument panel on this car to service that. I have replaced several of these, you do have to remove the glove box and all the connectors/ecu's etc... that are infront of the evaporator case. Here are some illustrations for you:
You will also need to replace the reciever drier once the system has been opened.
The blower unit is held in place by 3 screws and 1 nut, once pulled out from under the dash the evaporator case is exposed, you will now have easy access to replace these components.
The suggested labor time is 1.8 hours to replace the expansion valve to give you an idea of how long and in depth this repair is.
Sorry, those are the best illustrations I have until monday, there are 3 large phillips screws that hold the blower housing to the a/c evaporator case and there is 1 nut up top near the cross pillar, if you follow the seam of the case you should be able to find the 3 screws easily (black in color and they also have a 10mm bolt head around the phillips head), the nut is a little harder to locate but can still be pulled from under dash. Take another look, you did the right thing by removing the blower motor to make it easier to move the case around but until you remove the case you will not have access to the a/c case.
Sorry for the dealy, it takes a while to upload the documents, here are the instructions for you Directly from the Toyota manual- (not haynes/etc...the TOYOTA manual)
Download 3 -download 3 is only needed to refrence glove box removal, also don't forget the reciever drier will need to be replaced once reassembled to ensure proper a/c operation