It sounds like your Finch has what is called "Twirling/Star Gazing". This happens to finches occasionally. Nobody knows what causes it. I have my opinions. My opinion is that the star-gazing bird's metabolism is faster than the other finches' metabolisms so the star-gazing bird requires more minerals. The star-gazing bird requires much more calcium and other minerals; so much that they eventually become depleted of these minerals if they can't ingest them frequently daily. In the wild these birds would have access to many varieties of plants, soils, and gravels to ingest which provides them with very high levels of minerals to ward-off neurological pathology. When they are confined inside a cage they are limited to the foods and minerals that we provide for them. This is why it is important to always supply these birds with sufficient mineral blocks especially calcium blocks and cuttlebones, a good high-quality charcoal and mineral grit, a variety of fresh seeds, and greens e.g. romaine lettuce leaf, fresh spinach leaf, carrot tops, mustard greens, etc.
For some reason, occasionally a finch will require extra amounts of various minerals and after awhile, when the bird hasn't ingested the appropriate amounts and varieties of minerals for his specific metabolism, then he will start to display neurological symptoms such as difficulty holding his neck up, or some finches will have difficulty with their equilibriums and they will turn around in a circle trying to get their bearings so they can fly toward a perch.
I've had 1 beautiful silver Mannikin that had twirling disease. He lived with it for several years because I hand-fed him extra calcium and kept him and his mate in a warm cage (90 degrees). He had a pretty good life for 8 years. Occasionally his neck would contort and he would gaze upwards uncontrollably. Otherwise, he was fine. He ate good and he had a mate.
Give your Finch additional Calcium. http://item.express.ebay.com/__Pet-Supplies_Mineral-Block-Medium-Calcium-Source-for-Parrots-Birds_W0QQitemZ7744230505QQihZ018QQptdnZPetQ20SuppliesQQcmdZExpressItem
When you hand-feed him the Exact, add some calcium to it. This is how: take a calcium block (purchase at Petco) then take a butter knife and scrape-off about 1/2 teaspoonful of the calcium block into the Exact that you're going to hand-feed the Finch. Mix it up. Then, withdraw the mixture into your bird-feeding syringe and slowly inject it into his beak with the tip of the syringe (without needle) down into the right side of his beak. If you don't have calcium block then scrape-off some cuttlebone into his food. Only feed him when he is awake and looking at you so he will swallow the food.
Keep him warm at all times.
Feed him warmed (105 degree F) Exact with calcium.
You've followed all of the steps that I've told you previously. This should help him.
I forgot to tell you to check his vent (where the poop is eliminated) to be sure that he doesn't have a poop stuck.
Take a very warm (but not hot) wet Q-tip and very gently swipe the cotton end of the Q-tip on his vent to remove any impacted birdy-poop.
Please keep me posted regarding how he's doing. No additional charge. Just reply to this message whenever you have an update or any additional questions about your finch.
Dr. Hanson, DVM, PhD, Ornithologist and Aviculturist