People can file lawsuits, even if the lawsuits don't have merit. However, a Court isn't going to grant injunctive relief to enforce a provision of a contract that is invalid and unenforceable in California.
With respect to a "fine" of $100,000, a "fine" is criminal in nature. Sometimes contracts have something called "liquidated damages", which is a pre-set monetary remedy for a breach of contract. Liquidated damages can be recovered if there is a breach of an enforceable contract. However, since the attorney is using the term "fine", he or she is doing so to scare you.
By the way, your current employer may have insurance available that would provide a defense to you if you are sued by your former employer. You may want to check with your employer to see if this is available to you.
An attorney could write a letter for you, explaining that covenants not to compete are unenforceable in California -- and the attorney can demand that your former employer and his or her attorney stop threatening you. The attorney could also include a warning that if the employer's attorney insists on filing litigation, the attorney could be subject to sanctions under Code of Civil Procedure Section 128.5, 128.6 and 128.7. That's basically a warning from an attorney to an attorney that their action does not have merit; and that if the attorney does file the action, the attorney could have monetary liability for filing the action.