This can be a herniation or a cracked rib as you describe. In either case we would want to manage your pain before you go into labor, especially if this affects your breathing and ability to strain... this is important to have evaluated soon, especially since you are so close to the end of your pregnancy! Unfortunately because you are pregnant, xrays would not be advised and magnagement of this problem would be difficult... it is likely you will be prescribed an anti inflammatory and be monitored. Additionally, in the future when you cough you need to practice splinting techniques (hug yourself around with a body pillow or in a self-bear hug to brace your sides) to try to prevent the injury from getting worse or from a new one from developing. The swelling and discomfort, albeit that it is described as getting better, needs to be evaluated sooner than later and I would suggest that you call your doctor at this time. They may go ahead and tell you to take tylenol at home, but they may want you to come in and be seen. I know if you were coming to the clinic that I work in, we would want to have heard from you before your appointment! Not because this is likely to be deterimental to the pregnancy, but simply because you are uncomfortable enough at this stage of your pregnancy without adding this and suffering needlessly : ) I hope this is helpful to you, please remember to press accept and feel free to contact me if you have anything to add.
** note: I noticed you mentioned using a heating pad. Remember that this is NOT recommended on your abdomen and you should not be heating your tummy.