The axle shaft nut does not need to come off to remove the rotors on the 2004 camry. These rotors are designed to slide off the hub. If it doesnt, then there is corrosion between the rotor and the hub face keeping it from sliding off.
Look between the lug studs on the face of the rotor and you will see 2 threaded holes. Use 8m x 1.25 x 2" long bolts to thread into each hole. Alternate tightening each bolt a couple of turns at a time. The bolt ends will hit the face of the hub and press the rotor off the hub as you continue to tighten the bolts: