This is a definite red flag. Why did he take his ex-girlfriend out at all? Did you agree to this? Telling him to do what he wants is not going to make your situation less stressful even though it appears you are trying to avoid a fight. It will cause you to silence your resentment until one day it explodes. And this is not a healthy way to live. I would suggest sitting down with him in a non-threatening environment and speaking to him about your concerns.