Well, there are two sides of this as I see it.
If you are volunteering to make yourself feel good about yourself by the reasonable use of your skills, then I am in complete agreement with you.
If one is volunteering because an organization is in need of man power and one wants to help this organization, then one would be expected to serve in the place that has the most need.
I do not have any advice for you other than to suggest that you examine your motives. I do not disagree with either stance that I presented here.
Best wishes.
Jay