Lesson 6

Representative search sites in each of the major categories of information retrieval services have been discussed in this module. In this module, you have viewed some of the types of information stored in web sites. You have toured topical authority in each of the major categories of information retrieval services. You have learned about some of the challenges to finding information on the Web, and you have seen typical search results pages from both a directory and a search engine.

The term "Representative Search Sites" does not correspond to a specific, well-documented entity or event in the context of information retrieval services that I can pinpoint to a particular point in time when they ceased to exist. However, based on the broader context of information retrieval and the evolution of search services, here's what can be inferred:

• Historical Context: Information retrieval services, particularly in the context of search engines and directories, have evolved significantly since the 1990s. Initially, there were services like Yahoo! Directory, which could be considered "representative" of how information was manually curated and categorized for users. These were more prevalent before the dominance of algorithmic search engines like Google.
• Shift in Paradigm: The transition from directory-based search services to more dynamic, algorithm-driven search engines began prominently in the late 1990s and early 2000s. By the mid-2000s, most traditional directories either lost prominence or transformed into more automated search platforms.
• Current Status: As of your query date in 2025, traditional "Representative Search Sites" in the sense of human-curated directories for broad categories of information have largely diminished or have been integrated into larger, more sophisticated search engines that use AI and machine learning for indexing and retrieval. Websites like DMOZ (which shut down in 2017) were among the last of these types, where human curation played a significant role.
• Modern Equivalents: Today, specialized search services still exist but are often niche or focus on specific types of content (like academic research with services like Google Scholar or PubMed). The general concept of "Representative Search Sites" has morphed into what we now see as advanced, AI-driven search engines across various domains, including but not limited to general web search, legal documents, medical literature, etc.

Given this evolution, it is hard to pinpoint an exact "cease to exist" moment for "Representative Search Sites" because they've gradually evolved rather than abruptly ended. However, by the late 2000s to early 2010s, the traditional model of manually curated directories had mostly transitioned to automated, algorithmically driven search paradigms.
Now that you have finished this module, you should be able to:

1. List various reasons people search the Internet
2. Describe the different categories of information retrieval services
3. Identify information on a search results page
4. Explain why you need a search strategy

Further data from the Enquiro and Eyetools eye-tracking study shows which organic results users notice when looking at a search results page (See Figure 2-6)
Figure 2-6: Visibility of natural search results

Rank	Visibility
1	100%
2	100%
3	100%
4	85%
5	60%
6	50%
7	50%
8	30%
9	30%
10	20%

Notice this data shows that the visibility of a listing in the natural results is double or more (up to six times) of the visibility of the same position in the paid results. For example, only 60% of users ever even notice the natural search result in position five, but the paid search results fare even worse, with only 10% of users noticing the result in the fifth position. With the advent of Search, the visibility of the paid search results is even further reduced. Paid search advertisers will have increasing incentive to appear in the paid results that appear above the organic results, and advertisers that do not appear there are likely to receive even less traffic.

This module introduced you to the following new terms:

1. Directory: A database containing indexed Internet Web pages and returning lists of results which match queries.
   Directories (also known as Catalogs) are normally compiled by hand, accept user submissions, and involve an editorial selection and breakdown into hierarchical categories.
2. Gopher: A menu-based text information searching tool that gives users access to various kinds of databases, such as FTP archives, and allows them to view information within them. Links are presented in hierarchical menus to the user. As users select options, they are moved to different Gopher servers on the Internet.
3. Hypertext link: The underlined words that you click in one document that take you to another document, in the same Web site or in a different Web site. Each link has a URL (Uniform Resource Locator, or Web address) stored in it.
4. Metasearch engine: A site offering a unified interface to multiple search services. Some provide a single search form that, once you have composed your query, will submit it to several different search engines. Others simply provide a list of different search engines and provide text fields with which to initiate a search through any specific engine.
5. Robot: An automated program used by search services to follow and record Web links and index documents found on Web sites. The information is stored on a database for user searching. Also called "spiders."
6. Search engine: A topical collection of information, references, and links to other Web sites, are also known as collection pages, compendium pages, and index pages. Typically, these pages are maintained by individuals; some are maintained by organizations.
7. Search query: One or more words and optional operators used by search services to match against documents in a database. Words in a search query are also known as keywords.
8. Subject page: A topical collection of information, references, and links to other Web sites, are also known as collection pages, compendium pages, and index pages. Typically, these pages are maintained by individuals; some are maintained by organizations.
9. Usenet: Another section of the Internet that contains common areas of postings (or articles) and responses, broken down by categories, on a variety of subjects; Usenet is also referred to as "Newsgroups" or "Discussion groups."
10. WAIS: Wide Area Information Server, a database on the Internet that contains indexes to documents that reside on the Internet. Using the Z39.50 query language, text files can be searched based on keywords. A directory of WAIS servers and sources is available from Thinking Machines Corporation, Cambridge, MA.

The next module will discuss the workings of directories and search engines in more detail.
You will make some simple searches and compare the results of searches in similar and different categories of search sites.

Click the Quiz link below to review some topics that we have covered in this module.
Internet Search Quiz