Does this happen with both keys or just one?
This is actually a fairly common problem. It will not hurt your car to drive like this, don't worry.
If you look at your key you'll see it has 2 small indentations on it's sides. When you insert the key into the slot these fit into two small plastic catches inside the key hole that retain the key and prevent it from coming out. This problem is caused because the plastic retaining tabs inside the slot become damaged preventing them from working as they should. It doesn't take much damage to stop them working properly. The key slot usually has to be replaced to rectify the problem which is quite a simple repair and your dealer will likely have the part in stock because as I said it's quite a common problem.
BMW has a service bulletin out for this because it is so common. This should be covered by your warranty if you are still in your warranty period.
Please let me know if you have any further questions.
I hope that helps. If so, please press the green ACCEPT button below so that I am paid for my time and effort. I will continue to answer any follow up questions should they arise. I do not receive any credit for my work until you press accept.
Thanks and good luck.
FYI, here is a copy of the bulletin:
NEW designates changes to this revision
MODELE90 (3 Series)
NEW E60/E61 (5 Series)
NEW E63/E64 (6 Series)
SITUATIONThe customer complains that the key is not adequately retained in the key holder and sometimes falls out. There are also situations where the key gets stuck in the key holder and cannot easily be removed.
CAUSEFaulty key holder.
CORRECTIONReplace the key holder with a part with an improved mechanism for gripping the key. Improved parts were introduced in series production September 2005.