Gorilla Glue falls in a general class of polyurethane glues. I hate to be the bearer of bad news, but according to the Elmers Gorilla Glue website, GORILLA GLUE , in answer to exactly your specific question,
"How do I remove Gorilla Glue?",
the reply is:
"There are no known solvents for Gorilla Glue, so please use caution when gluing. While the Glue is still wet (usually for the first 15 minutes or so), you can use a standard paint thinner to remove it. Always use solvents with care, and with protective clothing/gloves. In most cases, once the Glue is dry, it can only be removed by a chisel, scraper or sandpaper."
In case you are able to scrape, chisel or sand off the unsightly glue, I have always had good luck using cyanoacrylate "crazy" glues for crockery repair. Elmers Gorilla glue is intended mostly for wood, and the wood should be moist for best results.
Using extreme care, a high-speed Dremel drill with an appropriate tiny sander or router bit may be useful for the removal of the glue that shows. My Dremel drill (which I have used to remove grout) reminds me of a dentist's drill. They are sold in places like Home Depot for about $75. It probably isn't worth the effort and investment unless your crockery has high real or sentimental value. Practice on something else first; you are likely to scratch or break the crockery if you make the slightest slip.
There are extremely powerful solvents used to remove paint and varnish that are sort of like a jelly that you brush on and allow to set for about 5-10 minutes, and then wipe off. They are extremely corrosive to the skin and may not work in your case. They will also remove any paint on the crockery that has not been glazed over. It may be worth a try in a small area, to soften the glue before trying to remove it by mechanical means.
Here is one thing I'd try if I were you that is easy, cheap, and won't cause further damage. Soak the pottery in paint thinner for about an hour. It wont dissolve the glue, but may soften it. Then, wearing gloves, use a single-edge razor blade to scrape off the excess glue that shows on the outside surface. It just might work.
Let me know how it goes! Good luck.