It sounds like these are tapeworm segments. They will usually be found around the anus but sometimes you just see them in the litter box and on the carpet where they were sitting. The medication of choice for tapeworms is Droncit or Drontal. This will have to be purchased from your vet as the over the counter dewormers do not usually treat tapes, only rounds and hooks.
I would also get all the cats on flea preventative as usually the tapeworms are obtained by ingesting fleas. Even if you're not seeing any fleas, does not mean that there are not larval and egg stages in the environment. You can start them all on Frontline flea prevention- it is over the counter, effective and safe.
I hope this helps, let me know if you have any other questions.