It could be either a rectal or anal prolapse, one of which is worse than the other. If you notice it after a movement but it returns to normal, or your cat has worms which can irritate her, then it could be a less serious Anal Prolapse. If it is more pronounced and seems to be protruding up to a few inches, or does not return to normal throughout the day, your baby may be in some trouble with a rectal prolapse.
Here is a link
to help you determine which it is. If you are unsure a trip to the vet is highly recommended in order to be safe. You'd know it if it was the rectal prolapse as it would look like her intestine is coming out.
Since this is likely due to constipation, there are a few things you can try, but since she seems to be pretty uncomfortable, I would try to have her seen tonight if you have the means. If you want to try a home remedy and she is eating, try some mineral oil in her food to help lubricate and hopefully help her pass it.
I wish I was there to see her but I am not so I do highly suggest having her seen tonight, especially if she starts crying, hiding, hissing or stops eating or becomes lethargic. Since she is skittish, that tells me she is pretty uncomfortable.
Please let me know how this goes, I'll be thinking of you both!