Here is information on how to replace the mirror. The assist handle does not need to be removed.
SIDE VIEW MIRRORREMOVAL
No it is number 1 in the picture and just pry up on it. I was just saying if you wanted to remove the piece with the handle on it that is how but doesn't need to be removed