Thanks for the reply.
Unfortunately, there may be very little that he can do. Upon conviction, he had the right to appeal but a notice that he intended to exercise that right had to be filed within 10 days of the judgment. If he missed that deadline and could show good cause as to why, he would have up to a year from the judgment to file a lat notice of appeal. If he's been in for 5 years, then he either appealed his case already and lost it, or he is time barred from exercising it now except in a small handful of circumstances, most of which are technical.
At this point, old cases can only be reviewed again if there when there is newly discovered evidence of the sort that would have resulted in an acquittal but which nobody could have known about at the time or if, for example, the law under which he was convicted was overturned as unconstitutional, or it was discovered that the court lacked jurisdiction over the offense. Without something like that, it's too late to get the matter reviewed.
That means about the only thing he may be able to do is to apply to the governor of his state for a commutation of his sentence (clemency). The governor has the power to do that to prevent an injustice. I am attaching the application form for him to fill out. Clemency is a longshot, but so is every other post-conviction remedy.
As you are not all that clear about what transpired, there may be more that he can do. You may want to contact a local post-conviction (appellate) criminal lawyer and have him speak to your brother and get the full background of his case to see if there is anything else still available to him. On the basis of what you've told me, there doesn't appear to be but neither you nor I know enough about what happened to be 100% sure of that.