A pony is characterized not only by its height but also by having a large head, short legs, thick coats, tiny ears, and very thick, fuzzy manes and tails. They also tend to have a very stubborn, "naughty" pony which many people call the "Pony Attitude". I even used to ride a full-sized horse (a warmblood no less!) who had the "Pony Attitude". Man, he could really be a booger.
A miniature horse is just that, a horse in miniature size. Their heads and legs are proportionate to their bodies and they tend to lack the "Pony Attitude". However, they also seem to have a unique attitude all their own. They're very intelligent, quiet, happy to be where they are, and adapt to change ridiculously well.