Puppies absolutely can get a rectal prolapse -- some might refer to this as hemorrhoids. This is where the soft pink tissue of the inside of the rectum (the rectal mucosa) starts to protrude to the outside of the anus where you can see it bulging. This occurs due to repeat straining to have a bowel movement. The most common cause of this is diarrhea. It is VERY common for puppies to have diarrhea, most likely from intestinal parasites. You need to get a good fecal parasite screen done, correct any diarrhea, and use some topical steroid cream on the rectal prolapse to reduce the swelling and allow it to go back into place. In worst case scenarios, that prolapse can be pushed back into place then held there w/ what's called a purse-string suture.
Hope this helps!
Most prolapses will resolve 100% once the diarrhea issue is resolved. If you are willing to spend some money to explore fecal tests and a good steroid cream (prep H is pretty wimpy -- you need to get some stronger prescription strength medicine), then this can be treated. Really it depends on how attached you are to this pup and how much money are you willing to spend initially on his problem. You may spend anywhere from $75 to $200 (or more if purse-string sutures are required). Once this is dealt with, though, I would not expect this to be a recurring issue in his life.