I don't know if you should try to lose the last five pounds. Your body seems to think it is important to keep it.
If you work out an hour a day, you should be pretty fit now. Probably more than 99% of us. Does five pounds actually show?
Keep seeking opinions on this, but I think your problem may be perception, not fat.