To answer your question , yes the idler pulley could be making that noise ,especially after changing belts & the noise went down
1-best way is one you changed your belts , you spin all the pulleys by hand and see which one is making noise or feel hard to turn .
2-in the morning it will make noise ,until the bearing inside of it gets hot and the noise goes away .
Let me know if you need anything else on it ,if you are good , Please Kindly rate me.
Hope I was helpful to you
A Positive Rating is Much Appreciated
This Post Will Stay Open for You To Come back and Ask me Any Follow Up Question You May have on this issue , even after you rate me
ASE Certified Technician