You need to leave him. I know guys like this. Once a cheater always a cheater. He is playing both you and his wife because he does not want to have to give up on either one of you.