Chapter 7: Searching for the Right Words
This kind of intelligent search technology was what the Google founders were building when I first met them in Boston, and it is why we keep coming back to them. It turns out that Web sites, like memory, are only as good as the search mechanisms that enable them to be retrieved. An amnesic has no use for her memory, because she cannot retrieve it (assuming it is still there). And a Web site is irrelevant—as a neuron, a meme, or a vehicle for e-commerce—if it is not retrievable. In the chaos of the Internet (it is fair to say that 800 percent growth and 100 million Web sites count as chaos), order comes in the form of a search and rescue mission. (For that matter, we may as well call the World Wide Web the Wild Wild West. We can keep the WWW.) The great land grab on the Internet was always about search, and virtually every leading company—including AOL, Yahoo!, Microsoft, Amazon.com, and Google—tried to join the footrace.
It turns out that Google has already won the race. Google does more searches every day than all other Internet search engines combined, including Yahoo!, Ask Jeeves, MSN, and any other one you can think of. As the others lost the battle, they joined Google’s fray by trying to build Web sites that ranked high in the very search engine that they were competing against. They couldn’t beat ’em, so they joined ’em, as countless companies began building their sites around Google’s algorithms and partnering with it for advertising. This level of coverage is what has made Google both dominant and frightening (and made its employees rich beyond belief—more than a thousand Google millionaires and counting, including the in-house masseuse).
You may say that Google is a human-made, rational enterprise. But Google also has an evolutionary imperative: there are many search engines, after all, and all of them are competing to deliver the most content-rich material to their consumers. A search engine that rewards users with lots of information will be used again. One whose searches are wrong, inconsistent, or filled with nutrientpoor spam will fail. Natural selection is at work at this level.
So it is no wonder that search engines craft their spiders well: in the ability of the algorithms to discern rich Web site content from poor Web site content lies the ability of the search engine itself to survive. At the top of the algorithmic food chain is Google. Surprisingly, however, even though people perform more searches on Google than on all the other search engines combined, Google searches represent only 2 percent of time spent online. This is because Google does its job very well: users search, find what they are looking for, and leave. Compare that with Yahoo!, which has roughly one-tenth of the search traffic but holds its users hostage for six times as long as does Google.