Men are like that. Even the toughest hardest looking men have a soft side. Does sound like he wants you to go back to his for sex (95% of men have one thing on there mind when they ask you to go back to their place) Also what men tell you and what they mean are two different things. Make him wait for sex, then you'll know if he's serious or not.