I am assuming that you are referring to the 2004 Summer Olympics. The winner of this "award" goes to Greece. However, given the fact that that the question has to do with the Olympics, the answer is not that straight forward.
Greek sprinters and Olympic favorites Kostas Kenteris and Ekaterini Thanou, withdrew from their team and from competition because they failed to take drug tests the Friday before the games began. The explanation was that they had been involved in a mysterious motorcycle accident, that no one ever saw. In the end, all drug charges were dropped, but there were criminal charges that were brought against them due to the motorcycle crash.
Greek weightlifter and bronze medalist Leonidas Sampanis, failed a drug test just 48 hours after Greece's Kenteris and Thanou announced that they were pulling out of the Games. He tested positive for twice the allowable amount of testosterone. He was stripped of his bronze medal and expelled from the Games. He became the first competitor at the Athens Games to be stripped of a medal for a doping infraction.
Lastly, there were 2 Greek baseball players, Andrew Brack and Derek Nicholson, who tested positive for banned substances. Although they are currently Greek citizens, they are both of Greek-American descent, and were members of the Greek baseball team.