Even the most expensive hardware is going to fail. Hard drives come to a stand still. CPU's overheat. A trillion things can go wrong.
If you have a small web-site (or a few small sites), just put yourself together a basic server and run with it. Larger sites require more planning.
I researched this a lot, and have come to the conclusion that it is far cheaper to create a redundant cluster than spending the money on expensive servers. I am an Apache/MySQL/PHP fan and firmly believe that creating a cluster is the way to go (especially if you are using linux) when using these tools.
To build a cluster you probably want 5 cheap servers (<$1000). Note that $1000 can buy you a REALLY fast server (multi core; raid; gigabit ethernet). This is roughly $5000 in components for 5. Most high end servers cost $2500. So for the same price you can buy two servers.
Don't forget to include backup batteries when you are estimating the cost. My guess is that your biggest cost (time and/or money) is going to be setting up and maintaining the servers - whichever route you take; oh yeah, and bandwidth.