Based on what you described, it doesn't sound like you did anything wrong. It would have helped had there been an exact time understood between the parties, but it sounds like you were reasonable. If when he called, you said you were about 20 minutes away, that again was reasonable.
Whether you can call and make a complaint would be up to you. You certainly can make whatever call that you want. Folks do it all the time and report any number of things--I'm not sure how effective it would be though. While his actions (in front of his son) were certainly inappropriate, they may not be a violation of any law, court order, or parenting plan that he is to follow. However, if you are in legitimate fear that he may have taken out his frustrations on his son (or do so later tonight), perhaps it is best to call the sheriff's office and make a report. Provide the information, and let law enforcement decide how to proceed.
If CPS still has an open investigation, I'm sure they would like to know the details of this incident too. If he has a history of violence or anger issues, it's possible he may take it out on the boy.
Furthermore, if he threatened you or touched you in any way, you may be able to file assault charges, but you did not mention any of that in your account.