When you ask him to stop right now, does he just stop or does he turn and look at you?
If he turns to you, you're already in right direction. I would start by walking toward him like you do now, but start by being few feet away and take your lunge whip / end of line or whatever and swing it toward to get him to move. Just start increasing the distance until you don't have to move toward him at all. Usually all I have to do is switch the lunge whip to my other hand and move it a little so the horse knows to go the opposite direction.