this depends where you both are in relation to the ball, if you are stopping him from moving in the direction of the ball and he had a chance of getting to it that would be a let and you take the point again, if you stop him from being able to hit the ball or if hitting the ball would be dangerous because of where you are then that is a stoke (the person in the way loses the point)  with the let though, the player does have to make an effort to get to the ball so if they could move very slightly and still get there they cannot have a let, as oytherwise they could just run into you everytime and just keep calling lets.