A good way to help yourself find the right answer, is to weigh out your options.

Ask yourself, "What is the very worst thing that could happen to your wife and your children if you leave?"

With that picture still in your mind, Then ask yourself, "ok, if that did happen, would it be really worth it?  Ask "What would I do, if that really did happen and the worst possible scenario came true, then what??"

Then compare your answers with your other option,

What is the very worst thing that could happen if I chose another person that I've known for ___ (years, months, weeks)  over my wife & kids, whom I've known for ___ years?? Then ask, "What is the very worst thing my new lover could do, what is the worst possible thing to have happen by choosing my "soul mate"?  i.e. The worst case scenario,

What that picture in mind, what would you do? 

Then ask yourself, "What is the very worst possible thing that could happen if I chose my wife & children over "her", when the new wears off and you're in it with her all the way and life is down to the every day-to-day mundane stuff, and you discover something so shocking about "her" that it leads you to question your decision, and have doubts, is there anything she could ever do or anything you could find out later, that  could make you be sorry about later it on? Are you 110% positive? 

what would it take doing, to have your wife & kids become the best thing that ever happened to you?

Because the truth is.... you DON'T know what you've got and how great it is, until AFTER it's gone and you loose it.  Many people have got caught in this trap and fell into temptations and paid for it dearly, for years afterwards.  Their children turned against them, they found out that "they" themself were the one who was wrong ,  That "THEY" were actually the one who had "issues", who was wrong, and begged endlessly for forgiveness, and for another chance that they'll never get, 

choices like this, can take YEARS of trying to UNdo, to make right again, what was already right to begin with.  Sometimes the problem is only on  the inside, and can eassily be fixed.  Sometimes we just need to open our eyes a little wider from the other person's point of view. Or it could merely be a few things we've been responsible for creating, that put us in this place to begin with.  It could be test. Only you and God knows the truth.

But regardless who you decide to follow & accept, you can't accept one, without rejecting the other,

when all else fails, if you don't remember anything else, just know, that it's the truth that will make you free.