Inappropriate is inappropriate, thats the bottom line. You are right and there needs to be lesson taught. He seems to be one of those men who needs to be clearly shown where the line is, sexual abuse if rife, and sometimes its not as clear as day, but you are right in confronting him and putting him in his place. Stand firm.