Ok, that is good added info about the tank being empty. you are thinking right on this problem. on the washer pump itself, there is a check valve, looks like a round black valve. this valve has a rubber check valve assembly. this will sometimes will stick and not allow washer fluid thru. this check valve is part of the washer pump. so yes, you have made all the right checks and a replacement assembly is in order.
If there is anything you dont understand, please feel free to ask me to explain