Search Engines vs. Directories
Search engines, such as AltaVista, create their listings automatically.
Search engines crawl through the web. Search engines eventually find your
site and index the pages they find. Page titles, body text,
META tags (provided by
www.about-the-web.com) and other elements all play a role in what gets
indexed. People then search the index of what was found. A complete
dictionary of terms can be found at
NetLingo the Internet Dictionary :^)
on
http://www.netlingo.com.
Or access it directly using the following links as-on NetLingo:
The Online Dictionary - start
here!
The Pocket Dictionary
The Smileys Page
~:-)
Chat Acronyms & Shorthand!
ASCII Art, ooh la la
All About E-mail
International Country Codes
The Web Design Coloring Book
Color Swatch Reference Guide
HTML Code Cheat Sheet
The Interactive IQ Test
A directory such as Yahoo depends on human editors to create its
listings. You submit a description of your site to the directory for editors
to review. A good site, with good content, will be more likely to get
reviewed than a poor site. A search of Yahoo looks for matches only in the
editors descriptions.
Some search engines use a hybrid of a crawler and an associated
directory. Being included in a search engine's directory is a combination of
luck and quality. Submitting your site for review is no guarantee that it
will be included in the directory. Reviewers have the final say in who is
included in their directory. They often search the Internet on their own
looking for good sites to add to the directory.
Search Engine Elements
The three major elements or a search engines are: the spider, also called
the crawler; the index or catalog; and the search engine which displays the
results of your query in your browser.
The spider visits your web page, indexes it, and then follows links to
other pages within the site. This is sometimes referred to as being "spidered"
or "crawled." The spider returns to the site every month or two, to look for
changes.
The index is a giant database that contains a copy of every web page that
the spider finds. When a web page is changed, then this database is updated
with the new information.
Sometimes it takes a while for pages or changes to be added to the index.
Therefore, a web page may have been "spidered" but not yet "indexed." Until
it is added to the index, it is not available to searches by the search
engine.
Search engine software sifts through the millions of pages recorded in
the index to find matches to a query and ranks them in the order of what it
believes is most relevant. Different search engines often produces different
results.
Search Engine Tips
If you have a general subject in mind (like "coins"), type the word or
words in the Search box and click the Search button. Results are usually
listed in order of relevancy based on keywords and other factors.
Advanced Search Features
If you know exactly what you want, you can get better results by entering
very specific information into the Search box. Despite differences in each
search engine's tools, there are tools that many search engines have in
common.
- Searching for an Exact Phrase
- To require that an entire phrase be found in a search,enter quotes ("
") around the terms. For example, "giants
baseball" returns listings where the words "giants"
and "baseball" appear together and in that
order, either in the title, the URL of the Web site, the description,
the keywords, or the document. If no sites are found that contain both
terms, sites that contain either term will be displayed.
-
- Searching For Required Words
- You can type the plus sign (+) or
the word AND before a word to
require that it be found in all of the search results. For example,
giants +baseball (include a space
between the first word and the + symbol) or
giants AND baseball returns all listings that contain "baseball"
and "giants" but not necessarily together.
-
- Searching For Excluded Words
- Use the minus sign (-) before a
word or the word NOT to require
that it not be found in the search results. For example,
giants -baseball (include a space
between the first word and the - symbol) or
giants NOT baseball lists sites containing "giants"
but not "baseball." Some engines like
AND NOT (two words) or
ANDNOT (one word) better than just
NOT.
-
- Searching For Multiple Words
- Use the word OR to require that
one or the other term be found in the search results. For example,
giants OR baseball (include a space on
each side of the OR) lists sites containing "giants"
or "baseball." You can combine
AND, OR, AND NOT by using
parentheses. For example, to find
documents that contain the word giants but
not either the word baseball or
football type giants
NOT (baseball OR football). You could also type this
giants -(baseball OR football). Note:
You cannot begin a search with a "-" term. You must put some other search
term first.
-
- Using Wildcards
- You can use the asterisk (*)
character to indicate a wildcard search. This is useful when you are
trying to match a term that may or may not be plural or might use one of
several verb tenses. For example chemi*
will find results containing words that begin with 'chemi' (e.g.
chemical, chemistry, chemist). You must have
at least four non-wildcard characters in a word before you introduce a
wildcard. This is not necessary for plurals because a search on
cat will also return results
containing the word cats, and a search
on cats will return results containing
the word cat.
Some search engines support two wildcards. The asterisk (*)
is used to replace multiple characters and the percent (%)
symbol is used to replace only one character. For example
psych*ist will find all results which
contain words that begin and end with 'psych' and 'ist' (e.g.
psychologist, psychiatrist), and
gene%logy will return sites containing
words beginning with 'gene' and ending with 'logy,' separated by a single
letter (e.g. genealogy and geneology)
which is useful for commonly misspelled words. You can also use multiple
wildcards within a single word.
Major Search Engines
 | AOL NetFind
- An Inktomi-powered search engine used by AOL
users. Uses listings from the Open
Directory. Click here for
more
information.
|
 | AltaVista - In
terms of pages indexed, AltaVista is one of the largest search engines
on the web. It has very comprehensive coverage and a wide range of
power searching commands, which make it a particular favorite among
researchers.
Click here for more information.
|
 | Ask Jeeves -
A human-powered search service that attempts to direct you to the
exact page that answers your question. If it fails to find a match within
its own database, then it will provide web pages from various other search
engines. Results from Ask Jeeves also appear within AltaVista. Here's
more
information.
|
 | Direct Hit - A
company that works with other search engines to refine their results.
Sites that get clicked on more than others rise higher in Direct Hit's
rankings. Direct Hit's technology is also used by HotBot
and Lycos and is available as an option at LookSmart
and MSN Search. DirectHit also crawls the web and refines its database.
Direct Hit's People Powered Search Engine tracks the sites people
actually select from search results and the amount of time spent at those
sites. By analyzing the activity of millions of previous Internet
searchers, Direct Hit determines the most relevant sites for any search
request. Aimed at the general consumer seeking information through web
search, Direct Hit provides users with the most relevant results for any
Internet search. More information on the
Direct
Hit spider is here.
|
 | Excite - One
of the most popular search services on the web. Excite offers a
medium-sized index and integrates non-web material into its results, when
appropriate. Launched in late 1995, Excite quickly grew and acquired two
of its competitors, Magellan in July 1996, and
WebCrawler in November 1996. Both continue to run as separate
services.
|
 | FAST Search -
Formerly called All The Web, FAST Search aims to index the entire web.
It was the first search engine to index 200 million web pages.
|
 | Go/Infoseek - This
portal site produced by Infoseek and Disney offers the search capabilities
of the former Infoseek search service. It uses an impressive
human-compiled directory of web sites to provide quality results in
response to many general and broad searches. Here's the web page where
you
add URLs to Infoseek. You can also become a
GO Guide and help to map out the web for others.
|
 | GoTo - Unlike other
search engines (except for AltaVista), GoTo
sells its listings. Companies can pay money to be placed higher in the
search results, which GoTo feels improves relevancy. Non-paid results come
from Inktomi. Here's all
about GoTo.
|
 | Google - A
search engine that makes heavy use of link popularity as a primary way
to rank web sites. This can be especially helpful in finding good sites in
response to general searches such as "cars" and "travel," because users
across the web have in essence voted for good sites by linking to them.
More information
about Google
is here.
|
 | HotBot - Like
AltaVista, HotBot is another favorite among
researchers due to its large index of the web and many power searching
features. In most cases, HotBot's first page of results comes from the
Direct Hit service (see above), and then
secondary results come from the Inktomi search
engine, which is also used by other services. It gets its directory
information from the Open Directory
project (see below). HotBot is owned by
Wired Digital,
a Lycos Network
Member.
|
 | Inktomi - The
Inktomi index powers several services. All of them tap into the same
index, though results may be slightly different. This is because Inktomi
provides ways for its partners to use a common index yet distinguish
themselves. There is no way to query the Inktomi index directly, as
it is only made available through Inktomi's partners with whatever filters
and ranking tweaks they may apply. Here's more information
about Inktomi.
|
 | LookSmart -
Another human-compiled directory of the web. In addition to being a
stand-alone service, LookSmart provides directory results to AltaVista and
many other partners. AltaVista provides LookSmart
with search results when a search fails to find a match from among
LookSmart's reviews. Here's more information
about LookSmart.
|
 | Lycos - Their main
listings come from the Open Directory
project, and then secondary results come from spidering the web. Lycos
also features another directory of web sites called
Lycos Community Guides. Sites are automatically listed in these
guides using technology from WiseWire, a company Lycos acquired in early
1998. Lycos is one of the oldest search services, around since May 1994.
|
 | MSN Search -
Microsoft's MSN Search service is powered by Inktomi.
On the MSN portal site, other search engines are also featured, along with
directory results. Here's information on their
Advanced Search Features.
|
 | Netscape Search
- Netscape Search's results come primarily from the
Open Directory and Netscape's own
"Smart Browsing" database, which does an excellent job of listing
"official" web sites. Secondary results come from Google.
At the Netscape Netcenter portal site, other search engines are also
featured. Here's a FAQ
about searching with Netscape Navigator.
|
 | Northern Light
- Another favorite search engine among researchers. It features the
largest index of the web, along with the ability to cluster documents by
topic. Northern Light also has a set of
"special collection" documents that are not readily accessible to
search engine spiders. There are documents from thousands of sources,
including newswires, magazines and databases. Searching these documents is
free, but there is a charge of up to $4 to view them. There is no charge
to view documents on the public web -- only for those within the special
collection. Here's more information
about Northern Light.
|
 | Open Directory -
Uses volunteer
editors to catalog the web. It was acquired by Netscape in
November 1998, and the company pledged that anyone would be able to use
information from the directory through an open license arrangement.
Netscape itself was the first licensee. Lycos also
uses the information for its main service and within Lycos-owned
HotBot. Here's information
about Open Directory.
|
 | Snap - Another
human-compiled directory of web sites, supplemented by search results from
Inktomi. Like LookSmart, it aims to challenge
Yahoo as the champion of categorizing the web. Here's some more
information
about Snap.
|
 | WebCrawler -
Has the smallest index of any major search engine on the web --
think of it as Excite Lite. The small index means
WebCrawler is not the place to go when seeking obscure or unusual
material. However, some people may feel that by having indexed fewer
pages, WebCrawler provides less overwhelming results in response to
general searches.
WebCrawler Direct provides PC users an application, separate from your
browser, to search the Internet. Mac 8.5 and above users can use the
system feature called Sherlock to do the same thing.
|
 | Yahoo - The
web's most popular search service has a well-deserved reputation for
helping people find information easily. It is the largest human-compiled
guide to the web, employing about 150 editors in an effort to categorize
the web. Yahoo has over 1 million sites listed. Yahoo also supplements its
results with those from Inktomi. Yahoo is the
oldest major web site directory, having launched in late 1994. |
|