The symptoms you describe are likely the result of a failing control module (amplifier) assembly. There is a technical service bulletin from Toyota detailing a revised design amplifier assembly to correct similiar complaints:
December 3, 1999
AUTO A/C SYSTEM IMPROVEMENT
IntroductionA production change has been made for the 1998 - 1999 Avalon A/C Amplifier. The internal logic has been corrected to eliminate excessive Air Inlet Servo Motor travel, which may affect the motor and/or the servo-link.
NOTE: A/C system may restore to normal condition upon cycling the ignition switch.
Applicable Vehicles All 1998 Model Year Avalon XLS produced before VIN: 4T1BF18B*WU273780
Remove and replace the Air Inlet Servo Motor.
Inspect the condition of the Servo Link (Damper-plate) for damage as illustrated. Replace if necessary
I hope this is helpful; if so, please click my accept button! Thanks!