Unfortunately you are very correct on this - the front bumper cover has to be removed to replace the headlight assemblies as some of the headlight fasteners are well hidden behind the bumper. This is standard on all Hondas/Acuras since about 2000.
Front bumper removal directions:
Am I just removing the plastic cover or is there going to be major parts involved ?
You are removing the front bumper cover(plastic part). It is the piece labeled "A" in first diagram, and also removing the top cover piece over the grille and radiator support shown in diagram 2.
It takes about 30-45 minutes to remove the bumper cover by yourself on the MDX