The kind of magnet needed to damage a hard drive would be a large and powered electromagnet, with an oscillating field. It would also have to be fairly close to the laptop's bare hard drive, within a couple of inches. A simple refrigerator magnet, or one used to close cabinets, would never be capable of interfering with modern hard drives.
To give you an example, try taking an old hard drive apart, preferably one that's already broken. You'll find a ridiculously powerful pair of magnets ALREADY in that drive! Hard drive manufacturers use balanced magnetic fields from high-gauss rare-earth magnets for the drive's servo arm. Rarely in the consumer world will you find magnets that powerful, with several pounds of pulling force, in such a small area.