The problem is the transmission mount. Sometimes it becomes loose where it bolts under the transmission pulley. It's easier to see from underneath. Also check the the four bolts at the axle.
What's happening is the transmission is loose and probably sitting at an angle. Don't just tighten the loose bolts take them out and look for signs of stripping. If they aren't stripped, reinstall with loctite.
The problem is internal within the transmission then.
These transmissions are easy to disassemble. Sounds like the bearing on the input shaft has failed. See if you can move the pulley back and forth any, you're going to need to remove the belt anyway so do that now by holding the idler back and slipping the belt off the big pulley.
There is also a possibility that the bracket that holds the gear shift could be loose. But it really sounds like the problem is inside the transmission.
Let me know what you find.