Hi there and welcome
I would recommend the Sony ICD-UX533 Digital Flash Voice Recorder
It is very simple and straight forward to use, and automatically records at the best quality.
To rewind and re-record, you can just pause it with one button then rewind then unpause and continue recording, very much like the older analogue tape based records.
It is also simple to recharge (via USB) and transfer the content to your computer.
It is available online here
If you would like any additional information or assistance, please do not hesitate to let me know.
Else if you can take a second to rate my service by clicking one of the stars at the top of the screen then submit, that would be greatly appreciated.