It's probably because of menthol, which is the main aroma of spearmint and peppermint. Menthol has a cooling effect -- as you seemingly found out -- which is accentuated by cold water. I don't understand much about the specifics, but the basic mechanism is that menthol irritates skin's cold receptors and makes you feel cold. Quite similar to what causes the hot taste of chili.
See this blog entry for more info and pointers.
