Hi, thank you for contacting JustAnswer.com. My name is***** will do my best to provide the right answer to your question.
I understand your safe to be a Sentry Fire Safe.
The yellow LED does indeed mean, on this model, that the batteries need replacing.
As for replacing the batteries:
1. if the batteries are internally placed on this model (the manual is not clear about this) then try opening the safe with just the key - I don't think it will work, but it can be tried.
2. in case the battery case is external, place both hands upon the keypad, then pull it upward to unlock it from the door, then pull it toward you to access the battery compartment. (This applies to some models with keypads. For other models it might be a somewhat different method, so please tell me your model #, and I will tell you whether this is indeed appropriate for your safe model.)
Also look for a pinhole in the facade of the lock and keypad area; if there is one, push a sharp object or a straightened paperclip into it, to unlock and release the battery case.
Note that if the old batteries have leaked brown corrosion onto things, cleaning will have to be done, of the battery compartment terminals at least, before the safe will have a chance of working again. This is best done with cotton (or rag cloth) moistened with rubbing alcohol. Once the terminals are shiny again, let them dry for 5+ minutes, then put in the batteries - with proper orientation with regard to + and - poles - and try the safe's combination again.
Let me know how this goes... I will advise further as needed.