Silica or Silicon Dioxide is said to have some side effects when inhaled in small particles overtime which include bronchitis, silicosis, and some form of cancer.
When ingested in small pieces, it is said to be harmless.
There are some reports however, that the substance can possibly cause different undesirable effects though some of the theories are not scientifically based.
This links has more information:
As a simple answer, ingestion of low doses is said to be harmless but over exposure may have harmful effects but is said to be reversible. Inhaling the substance for longer period is where the danger is.
Silicon generally can be harmful to health if it is in high levels. At low levels, it generally does not cause significant undesirable effects. In fact silicon is found in many foods we eat and water we may also contai silicon.
I hope that helps.