There are fasteners holding it in place. See the 1st image above for the locations. CLick "View Image" to get a nice, close-up view.
Actually they are threading out, they just do not come out of the trim panel. Once they are all loose, carefully pull it off. There are clips that hold it in as well. Start from a corner, and work your way around.
Hey - I got it..... the inserts just needed a small straight screwdriver to "help" them unthread then popped the rest out... then started on one side and worked way around and popped it out... I was just real gun shy since 2 yrs ago when removing the passenger side taillight assy, I broke one of the retaining clips (plastic crap) trying to get that off to replace the brake lamp...
all better now. thanks!!
Glad I could help! Have a great day!