The semi-hard objects you are removing are likely tonsil stones (tonsilloliths) within cryptic tonsils (appear as a hole or cavity in the back of the throat). The mal-odorous off-white collections of bacteria and food particles are a frequent cause of halitosis ("bad breath") and are best treated with regular (at least three times a day) gargling of salt-water or a non-alcohol non-sugar based mouthwash. You should have the spots examined by a physician as well, especially if they don't resolve with the gargling. Manual extraction with a cotton swab is an option but should first be done with a physician to ensure good technique and decrease the risk of introducing infection/trauma.
I hope this helps,