The only time that this would become a problem is if the IRS were to determine that your former employment termination was intended for the sole purpose of obtaining a distribution of your retirement benefits, and that it was accomplished with the assistance of your employer. That would be tax evasion, in violation of IRC 7201. However, if that wasn't your intent, then there is no tax evasion.
Moreover, as long as the funds remain in an IRA, you have not actually evaded any tax, so once again, no evasion would occur.
I don't see any problem with your return to your former employer.
Please let me know if my answer is helpful. And, thanks for using justanswer.com!