It sounds like the cyst has gotten infected. Or if it is not a cyst, it could be an inflammatory nodule as art of the infection of the skin.
The usual treatment for a skin infection would be antibiotics. The best that can be acquired over the counter is a topical antibiotic, such as Neosporin. If the infection is more severe, then oral antibiotics would be necessary, but that would require seeing your doctor.
There also are some interventions that can be done to sooth the area. Certainly, it is fine to use any over the counter pain medicine, such as acetaminophen, ibuprofen, or naproxen. It also would be fine to apply cold compresses to sooth the area. Applying salt water compresses also would be soothing.
So, at present, it would be appropriate to apply Neosporin and the soothing interventions along with pain medicines. If it improves with these interventions, then it does not need further interventions, but if the problems persist, it would be appropriate to consider oral antibiotics, so you would need to be seen.
If I can provide any additional information, please let me know.