I am Pete, and I'll be happy to assist with your Kindle question.
When you restarted, did you press and hold down the power button down for a full 20 seconds and lett it completely reboot itself? This should clear out any temporary data that may be causing the footnotes to appear.
In addition to restarting, I'd suggest removing the Bible from the Kindle and then download it again - Amazon will have remembered the purchase do you do not need to pay for the Bible twice.
Please reply back and let me know if that works, or if you still need further assistance...
Have a Happy Easter!