Can you accept a third party in your relationship? Where is the commitment? How would he feel if you also had a partner other than him where you work? I think the answers should be obvious.