If you have an emotional attachment to this doctor, you might find my advice a little blunt, but there is an obvious solution...

1. Do NOT talk to his wife, unless she is a personal friend. Your committment does not extend outside your normal business hours and location.
2. Depending on your country, there is a medical association or licensing organization that permits doctors to practice.  They also have a branch or department that handles problems like malpractice, sexual harassment, etc.  Contact them immediately.  Ask for anonymity if you like. I'm sure they'll respect that.
3. They will do their own investigation... probably interviewing the doctor and a cross-section of his patients.
4. Assuming what you've reported here is correct, they will likely recommend that he retire from medicine to save face. Otherwise, they'll take away his license, which often leads to public disgrace.
5. Begin your search for a new job immediately.