The IT Law Wiki


A search engine will find all web pages on the Internet with a particular word or phrase. Given the current state of search engine technology, that search will often produce a list of hundreds of web sites through which the user must sort in order to find what he or she is looking for.[1]

How it works

A search engine starts with a list of one or more websites. The engine then requests the home page from each site on its list. When a home page is retrieved that has links to yet other pages, the search engine requests a copy of each of those pages that these links point to. And if those pages in turn contain links to yet more pages, the search software requests a copy of those pages. And so on, day after day, ceaselessly.

At its most basic level, a search engine maintains a list, for every word, of all known Web pages containing that word. The collection of lists is known as an "keyword index." Search engines vary according to the size of the index, the frequency of updating the index, the search options, the speed of returning a result, the relevancy of the results, and the overall ease of use. No two search engines work the same way.

In practice, most search engines do not exhaustively cover all possible websites. In addition, some search engines pass along material for review by human editors, who rate the pages retrieved on a variety of scales — quality, appropriateness for families, and so on. The creation of such an annotated index obviously takes longer than it does to create a comparable unannotated index. Search engines are the primary means by which Internet users can find digital information. However, it must be remembered that a search engine is NOT searching the Internet as it exists at the time of the search, but is only searching the search engine's database, which may be days or weeks out of date at any given point in time.

A recent area of development is search engines that are specifically designed to build profiles of individuals based on personal data found on the Internet.


  1. Sporty’s Farm L.L.C. v. Sportsman’s Market, Inc., 202 F.3d 489, 493, 53 U.S.P.Q.2d (BNA) 1570 (2d Cir. 2000)(full-text).