In order for your fiance to come to the UK to live permanently, he will need to apply for a fiance visa under Rule 290 of the Immigration rules. This application will need to be made in the USA at the embassy on form VAF4A which is for settlement. The current fee is £750 but this will be increasing on 6 April to £810 payable in local currency. Your fiance will also need to complete the English Language Requirement exemption form on the basis that he is from an English speaking country. The application form can be completed online and payment taken online. Your fiance will also need to have his biometrics (fingerprints) taken if he has not already done so. You will need to show that you are both free to marry and intend to do so, that you intend to live together as husband and wife and that you can afford to maintain and accommodate yourselves without recourse to public funds. You will need to provide evidence of your relationship by way of photographs and evidence of continued contact would be beneficial.
The process can take up to 12 weeks, sometimes even longer depending on how busy the embassy is. It is possible to pay and extra $300 and get a decision within 3 weeks by opting for the settlement priority option. This is only available from certain embassies and so it would be wise to check the World Bridge website before you pay the money.
The fact your fiance previously overstayed should not stop him getting his visa to come here as the re-entry ban under Rule 320C does not apply to people making applications on the basis of family. The Home Office will probably be aware of it even though they did not say anything when he left and so the best thing to do would be to do a letter to go with the application addressing the fact that he overstayed, explain the reason, explain that he has never breached immigration law in the past and until that point had a good immigration history record. Providing you include all other necessary evidence to prove that the other rules are met, this should not prevent him getting his visa to come to the UK.
If you are planning for your fiance to live here permanently then coming on another tourist visa and just hoping for the best at the airport is no good anyway. He is better off applying for a settlement visa if that is in fact his actual intention and proving that he can meet the rules. If granted the fiance visa will be valid for 6 months and allow you to marry in the UK . You will then need to submit an application for a spouses visa which can be done from within the UK.
I hope this answers your question in which case please accept. If you require any further information, please ask.