Welcome to Just Answer. I am here to help you resolve your tax and finance concerns. Please feel free to ask anytime you need extra help.
You are mostly correct. The interest on the bonds will be taxable income for his estate and will require a filing. Any other income he received after his date of death is taxable along with this. It all gets reported on form 1041. There will probably be no tax (based just on what you report) but the return should be filed. I would also suggest that you consider applying to Social Security for his death benefit. It is not much (about $250) but why let it go? Also remember that the interest income on US Savings bonds, while taxable at the federal level, is not taxable at the state level.