Listen to what you're saying. You don't like him, you spend most of the time not talking to him. There seems to be quite a bit of abuse in your relationships. Is this really the man you would want to become the father of your children?
It seems to me that your relationship now are not going where it should, and that you're keeping up with this only because you're already in. I think you want out, and just need the extra push to get there. I don't know you, and you should probably consult an expert, like a therapist or a counselor or something of the sort. I wouldn't advise anyone to marry someone they can't stand to look at.