Staying because you feel pressured is not a healthy environment for anyone. Does your children's mother know how you feel? Have you together gone to talk to anyone about your situation? You could ask for a time to sit down and tell her how you feel by making a list of things you are willing to do to support the child you have together and the one on the way. In the meantime it is not advisable that you seek out a new relationship until you have resolved the issues here. Demonstrate that you will be a good father by buying the items your 3 year old needs and helping with the cost of the prenatal care for your child on the way. If you wish further conversation, let me know as my availability is limited today.
I would recommend the professional help for you personally to sort out what you want for yourself before you decide what is best in the relationship. If finances are a problem, then check into family counseling centers which have a sliding scale or go to www. helpstartshere.org to look for a social worker in your area that has a sliding scale. Let me know if further advice is wanted.
Emotional roller coasters are normal for pregnancy. I would inquire of the irrational behavior is just during pregnancy or at other times too?