One's nutrtion and activity level have a definite effect on cold sensitivity.  If you suffer from this, you should explore several options with your doctor, to rule out any chemical or hormonal imbalance: get blood tests that check your Thyroid function, and also check for baseline levels of iron and Vitamin B12. If your blood is weak, you will feel more cold because oxygen is not getting transported as efficiently as it could be.

Furthermore, those who are sedentary are more likely to have issues with the cold, because they are negelcting the circulatory and metabolism aspect of their health.