I remember a discussion on this subject many years ago; I don't believe a conclusive answer was ever arrived at. But, here is the theory that most people accepted:
Many years ago, only upper-crust people had fancy fasteners (like buttons) on their clothes; common folk usually had pull-on garments or fasteners like cloth ties. Men tended to dress themselves, so the buttons were attached to enable his own right hand.
However, high-class ladies were helped into their clothing by a servant, so their buttons were attached to enable the maid's right hand.
Or maybe the ladies' buttons were positioned to accommodate her beau's right hand, when undressing her...???