There are several common problems which can cause pain around the elbow - and it's impossible without examining you or asking more questions to be sure which might be causing your problem. Only your doctor can give you the correct diagnosis.
In a nutshell, the pain may be coming from the ligaments, the bones, the bursae (fluid-filled sacs around the joint) or the joint itself. Or it may be what's known as referred pain - that's pain caused by disease in a nearby structure, in this case the shoulder.
It may have been triggered by a small local injury that you hardly noticed at the time or have long forgotten about. Another common trigger of aches and pains in the bones, muscles and joints is a change in activity so that you're suddenly putting into use muscles and ligaments which haven't had to work so hard before (this can be a very subtle change, which gives you a not-so-subtle pain).
Arthritis is another common cause of bone and joint pain; especially inflammatory arthritis, such as rheumatoid, or that linked to psoriasis. Finally there are much rarer causes of bone, joint or muscle pain - such as infections and tumours.
You don't have to play tennis to get tennis elbow! This condition is an inflammation of a tendon where it attaches to one of the bony lumps at the outer side of the elbow. Similarly, golfer's elbow is not a preserve of people who like to whack small balls with an iron stick; it's simply inflammation of the tendon which attaches on the inner side of the elbow.
Treatment involves regular painkillers. This is usually enough to settle the pain with time, but anti-inflammatory drugs (from your doctor) may help to speed this process. Rest is important, and local strapping or devices to take the pressure off the tendon can also help.
If pain persists, your doctor may be able to give you an injection of steroids into the tendon which can bring rapid pain relief.
No sport here, but a good book can cause you to lean overlong on your elbows, causing a traumatic bursitis or inflammation of the bursa behind the elbow. This bursa or sac of fluid becomes swollen and painful. Infection and gout can have the same effect.
Treatment involves draining out the fluid and, if it's a traumatic bursitis, a steroid injection.
Hope that helps!
Please let me know if you have more questions.