The box you are describing is a fan limit switch. I am assuming if you have a moncrief this is not exactly a new furnace and you probably live in or around the NE US.
Question time -
Has the furnace blower motor ever cycled since you have lived in your home?
Have you recently changed the wall thermostat in you home?
Is there an "on" "auto" switch on your wall thermostat? Which is it set at?
I am assuming that you do not have central cooling is this correct?
I thought you in Ohio but I didn't want to creep you out by me saying right off that I knew that! I am also assuming you are North of Columbus likely in the Cleveland area since most of these units stayed pretty close to their maker.
The fan switch is what is causing you constant on. Now, it could be bad or it was wired or set to be always on by the previous owner or installer. I would remove the cover of the fan switch and make sure the dial is not set to the lowest "fan on" setting causing it to be on constantly. Mark the setting location and feel free to rotate it carefully to see if the fan on can go off. If it doesn't you will need a new switch. Make sure to make yourself a good drawing of the wiring. Remove it and compare it to a new unit at your local heating and cooling or appliance parts store. These generally (depending on the length of the rod in the furnace) will run between $40 and $90.