You got two very good answers from people who have "been there, done that". As a customer, I would feel very uneasy with someone recording my phone conversation with them. It seems adversarial, perhaps over a simple misunderstanding. If some prior contact leads you to believe that the customer is avoiding the bill or is going to be abusive or threatening, by all means tell them that you prefer to have audio documentation. If I'm told the other party is recording, I become wary that the comments they make may have double meaning, an attempt to get me to say something that is not true or can be taken out of context to that party's benefit. Companies that announce that "your call is being recorded for training or quality assurance" are probably telling you a plausible untruth. What you say, or don't say, can be held against you, the caller. I'd decline to allow recording, as the caller, and resort to doing things in writing. It's too bad that this society is so litigious. It makes everyone suspicious and suit happy.
If the customer calls and clearly is at odds with you, you can announce to them that future communication will have to be in writing. Then send them "notes" alluding to the content and nature of the phone call in order to document what happened and restate your case.
On the other hand, take the customer's point of view. Assume they did not misunderstand the arrangement. How does a customer safeguard that the work will get done, and to their satisfaction? One way is to provide part payment, with the balance when the job is inspected. Picture what too frequently happens: the homeowner goes to work, leaving payment for the entire amount agreed to. The client discovers that the job is incomplete, poorly done, or worse, damage was done while the worker was in the house. What recourse does the homeowner have? The worker has the money, the work is unsatisfactory, and the homeowner is at the disadvantage.
I hope there was a misunderstanding, that they thought the rate was only $10 per hour. Or perhaps the $50 was to show good faith on their part, but the balance would be paid after the job was done as agreed to. In any case, hopefully the adversarial atmosphere doesn't really have a basis in fact.