Removing the ABS fuse will disable the ABS system by removing power from the ABS control unit. This may hide the symptom of teh light coming on, but with the power removed from teh controller the ABS is not working and will not activate in a panic stop when needed. The ABS light comes on to warn you that something is wrong in the system; when the controller sees something such as a wheel speed sensor signal becoming erratic or a solenoid not responding to commands, it turns the warning light on and should store an ABS service code in memory to give a technician some starting place to begin diagnosis. Although a failed modulator would be pretty expensive, this is rare on these vehicles, it is more likely to be something else such as a speed sensor failure or a poor ground connection possibly. I would suggest taking the vehicle in to a technician equipped to diagnose the ABS system, and at least find out what is wrong that is causing teh light to come on. Removing the fuse to make the light go away diesnt make teh underlying problem disappear...
By "cv joints going bad" I asssume you are getting a loud clicking noise on turns; this is the usual symptom of a worn outer CV joint. Usually, a reman axle is the way to go for repair of this problem; a complete remanufactured axle is actually much cheaper than one single CV joint, and much easier to install. You can buy reman axles for most front wheel drive vehicles for less than $100 pretty much anywhere these days. A worn CV joint will not cause the ABS light to come on, unless there is a damaged tone ring that is part of an axle assembly. If one of the teeth on a tone ring is damaged, it will result in an erratic speed signal from that paarticular wheel.
The other odd electrical problems you are describing sound remarkably like a poor ground connection somewhere in the vehicle, possibly in the dash area or in the rear of the vehicle. With the fuse out of the ABS system, teh ABS light should never come on; the fact that it still flickers during braking would seem to indicate that some other circuit active durig braking may have a poor electrical ground and is seeking a ground path elsewhere through some instrument panel circuit. The same with the radio illumination dimming during braking; this could be related to a ground problem.
Try taking a look at your breake lights while someone sits in teh car and holds the pedal down. You may find a brake light not working, or a corroded ground connection or lamp socket in teh rear of the vehicle allowing the brake light circuit to seek a ground path back through the tail lamp wiring through the instrument panel illumination circuitry. Try removing the tail lamp bulbs and see if the electrical issues you are experiencing when pressing the brake pedal go away; if so, you know that you are on the right track. Try locating and cleaning any ground connections you can find, especially in the rear of the vehicle.