At first I would have said Scotland were curling was first invented back in 1541. Looking at the world curling ranking I would say Canada sine they are highly ranked in the men and women as well.
For more stuff check:
http://www.worldcurling.org/
