Public defenders have heavy caseloads and spend all of their days in court. Unlike private attorneys who can pick and choose the cases they take on and have plenty of time for office visits, public defenders generally meet with their clients on their court dates. The exception is that when they are on trial, they devote all their time to the trial case and other lawyers in the office pick up their other cases so that they are free to get the trial all the attention it deserves.
All criminal defendants, whether they have public defenders or private attorneys, feel that they are being pushed through ths system. There is a great deal of pressure from the judge on down to keep the cases moving and to press for a speedy disposition. If you're not accustomed to that, it is very intimidating to criminal defendants. However, you did apparently tell your public defender that you are not guilty, and if you aren't interested in resolving this matter with a plea agreement, then the case will have to move towards trial. They are the only two real choices that any criminal defendant has.
Public defenders are highly experienced lawyers because of the volume of cases that they handle, and they know how to try and win cases. So even though you can't make an appointment with your public defender, that doesn't mean he or she is not on top of the issues and can't provide you with a good defense. However, if when you talk to him or her you can't get the answers you need to feel that your lawyer is looking out for your best interests, you can tell the judge that your lawyer is not willing to communicate with you, ducks your phone calls, won't set up a time and place to discuss your case and that you want your lawyer to be removed from the case and a new public defender appointed for you.
Prior to doing that, however, you can call your public defender's office and ask to talk to your lawyer's supervisor. A complaint to a PD's supervisor generally goes a long way to solving a problem. He can get on your lawyer's case and require him to return your calls and/or to make more time for you and to give you the representation you deserve.