My husband and I split up and he was very angry....didn't want the divorce. So when the kids went to visit him, he called me and told me that he wasn't going to bring them back. I was told by my lawyer that until I filed the divorce and requested custody, that he did not have to give them back until the judge ordered him to do so. I know its hard, but it will truly look much worse for him because he is doing this. The judge will not look on his taking the kids as a good thing unless there are extenuating circumstances, such as if you are not taking proper care of the kids. If he has no reason to be doing this, it will make him look very selfish and definitely not as a caring father. Now, please know that this was a loooong time ago. But in my situation, I just filed for the divorce, and then he brought them back the next day. Hopefully your situation will end as pleasantly.
Best wishes....