Desperately Seeking Susan OR Suzie NOT SushiIf the World Wide Web ever adopted a theme song, it could do worse than picking "I Still Haven't Found What I'm Looking For." Searching the Web is the most popular online activity -- and often the most frustrating. In June, more than half of the top 10 most-visited domains were Web search sites, according to an Internet metering service, Media Metrix. But how many of the people visiting those sites found what they were looking for right away?
Not most, according to Karin Rex, whose Pennsylvania-based company Computer Ease conducts Internet search classes. "Most people type in words and get a bazillion hits," Ms. Rex said. "Some of the ones on the first page may pertain to what they're looking for, but most of them won't."
Sal DiMarco Jr. for The New York Times
A TOUR GUIDE - Karin Rex teaches classes on how to conduct searches on the Internet.
On the surface, it ought to be simple. You're looking for Lincoln's Gettysburg Address, you enter those three words, and assuming it's somewhere on the Internet (and that's a pretty safe assumption), the search site gives you a list of relevant Web pages. Right? Not so, Ms. Rex said.
"You'll get sites about the Lincoln Continental and vacations in Gettysburg, and real-estate sites listing addresses," she said, "but often, nothing about Lincoln's Gettysburg Address."
Danny Sullivan, editor of the Search Engine Watch newsletter, agrees. His publication and Web site monitor the world of Web searching, and despite improvements over the past two years, he said, he still sees problems.
To search the Web successfully, pick the right engine and learn how to use it.
"They've gotten better, faster and easier to use, but search engines have got a long way to go," Sullivan said. "They're poor for people who are doing really basic searches. Enter 'Disney' or 'travel,' and it's a crapshoot whether they'll get the Disney site or any good travel sites."
One search site could provide 10 top results of pure gold, while another serves up either nothing or dross. Why is there a difference in results? Because there are three basic components of all search engines, and while there is often a lot of overlap, no two engines are exactly the same. One element is the index of Web sites or Web pages that your search roots through; each search site collects its information and updates its database differently. Each site's search function works differently, too, and the order in which the results are sorted is usually based on a proprietary algorithm that no company would be willing to share.
To make things harder, search sites generally do not do a good job of explaining how they work. Few people understand, for example, that Yahoo is fundamentally different from search sites like Hotbot, Alta Vista and Infoseek. Yahoo is not really a search engine but rather a Web directory, compiled by humans who classify Web sites under headings. The others are Web search engines, which use software agents called crawlers or spiders to index the contents of individual Web pages, then follow links to other pages.
Web directories like Yahoo and Web search engines may look the same, but each type of site is good for finding different types of information.
The first step in creating more effective searches is picking the right search site for the job. "If people are doing a general search," Sullivan said, "they should start off at Yahoo or a Yahoo-like directory like Snap or Look Smart." A directory-style search provides two ways to research broad topics: dive through a list of broad topics by clicking on the appropriate links or fill out a search box to find listings.
But directory searches are less effective when looking for specific information -- things like the author of a book, the complete text of the Declaration of Independence or research on drug treatments for a medical condition. For this kind of information, search engines like Hotbot and Alta Vista are the way to go. Because they search an index of keywords drawn by spiders from millions of Web pages, the chances are greater that they will find obscure terms in obscure Web pages.
There's a third kind of search site, one that includes popular sites like Metacrawler, Ask Jeeves and Dogpile. These sites -- also called metasearch tools -- don't maintain any kind of index of their own but instead issue search requests to fistfuls of other Web search sites. When Yahoo, Hotbot, Alta Vista and the like return their results, the metasearch site collects them onto a single Web page for display.
Because no two search sites index exactly the same set of Web pages, metasearch tools give you a wider scope of results -- but it's worth remembering that more does not necessarily equal better. What really counts is relevant results that are sorted in a relevant order. And that's the rub.
Simply picking one of two or three types of sites to search from is no guarantee of good results.
Brad Hill, the author of "World Wide Web Searching for Dummies" (IDG), says most search sites deliver too much information. "Search engines do a good job on indexing," he said. But because of that, they deliver more than you want.
So when you're faced with several hundred thousand results over dozens of pages, what should you do? "Don't go past the first page of results," Hill said. "If it doesn't have something of interest, you've probably entered the wrong search string."
Most people could get much more relevant results with a few simple tricks for constructing a search "string" -- the words you enter in the search box. The most obvious is to type in several relevant words instead of just one or two. In general, the fewer words you enter, the more general your results will be.
But not every search engine returns the most relevant results first -- which leads to lots of pages about Lincoln Continentals instead of Lincoln's most famous speech. To give a search engine more instructions, it helps to master a site's instructions, which search techies call "operators."
"Search operators tell a search engine how to interpret your key words,"
Hill said. "Words like 'and' or 'not,' and quotation marks can really narrow down search results."
And it's narrowing the results -- giving fewer, better pages -- that really counts.
"The simplest techniques, like using quotes around a phrase, help the most people," Ms. Rex said. The result of slapping quotation marks around two or more words is remarkable. Type in "Gettysburg Address," with quotation marks, and you tell the search engine to look for a phrase instead of two separate words -- knocking irrelevant vacation sites and real-estate listings out of your top 20 results.
That trick works in many search sites, including Yahoo, Alta Vista, Hotbot, Excite and Infoseek.
Not all search sites use the same rules for making better searches. Most will let you exclude some terms from your results -- which is great if you're trying to search for, say, the gross national product of Jordan and keep getting sports sites about Michael Jordan. Exclude the word Michael, and you'll trim a few hundred thousand irrelevant results right away.
But how you exclude words from a search depends on the search site. In the regular search forms at Yahoo, Excite and Alta Vista, for example, you put a minus sign before the word (-Michael). But in Hotbot, you click on the More Search Options button and select Must Not Contain in the Word Filter section.
It is hardly surprising that many people find Web searching confusing and inefficient. So how are you supposed to know which rules apply to which search site? Karin Rex includes a simple piece of advice in lesson one of her Internet search class.
"Read the instructions," she said. "The only way to learn the inner workings of each site is to read the help files or frequently asked questions document. Most people don't even realize there are help files, so they'll never be able to take advantage of advanced features."
Sage advice though this is, search sites tend to use jargon that's not easily understood by the uninitiated. A single mention of Boolean operators is enough to send many would-be searchers into a tailspin. (Named for a 19th-century British mathematician, George Boole, Boolean operators are words like AND, OR and NOT that many advanced search sites use to make searches more precise.) But in truth, Boolean logic is not hard to learn -- and in many cases, search sites label it with easy-to-understand phrasing like "search for ANY of these terms" or "search for these terms as a phrase."
There are options for sufferers of Boolean anxiety, though. One site with a novel approach is Ask Jeeves. When you enter a vague query, Ask Jeeves will throw back a series of questions. From the single-word query "travel," for example, it comes back with 10 possible interpretations of what you might be looking for, including "Where can I rent a cellular phone in a foreign country?" and "Where can I get tourist information about foreign countries?"
Another approach is to reduce the scope of your search. Searching the entire Web for a highly specialized piece of information isn't always the best way. For one thing, many Web search engines index only Web pages in HTML format, and many Web pages are generated from databases that search engine spiders can't penetrate. To uncover information from these databases, you usually need to use the search engine provided at the database's Web site. There are literally thousands of these highly specialized Web search tools across the Web.
So how can you find these specialized search tools? About 3,000 of them are listed at Internet Sleuth in Yahoo-style directories. But unlike Yahoo's directories, Internet Sleuth's include only searchable sites and include the search form to issue a query right away.
No matter what advice you get, however, you discover the best search techniques by experimenting. Danny Sullivan of Search Engine Watch uses all the major search sites frequently, and refuses to name his favorite site. His reason?
"Judging the results is subjective," Sullivan said. "If your friend raves about a site and you don't like it, try another. Use whatever you find gives you the answers."
Following are links to the external Web sites mentioned in this article. These sites are not part of The New York Times on the Web, and The Times has no control over their content or availability. When you have finished visiting any of these sites, you will be able to return to this page by clicking on your Web browser's "Back" button or icon until this page reappears.
Copyright 1998 The New York Times Company
Copyright 1998 The New York Times Company