Introduction to Search Engines

Without search engines the Internet would for the most part consist simply of web sites waiting in isolation for the occasional visitor, that has seen the web site address on a business card or printed advert or has been told the address, to type the address into their Internet browser.

The primary function of a search engine is to analyse and index the content of the billions of pages making up the millions of web sites that exist on the Internet so when someone types a search term or phrase into the search engine it is able to based upon its analysis return search results that are the most relevant to the search criteria.

If a search engine does not know of the existence of your web site it will not have indexed it therefore your site will not be listed in any results that the search engine returns in response to a users search.

Some search engines rely upon being explicitly told about the existence of a web site by means of search engine submission whereby the web site address of the site and some information regarding the content of the site are submitted to the search engine. Other search engines do not require submission of a site address in order to know of a web sites existence, search engines such as Google will find your site sooner or later during the course of their spidering or crawling of the Internet.

Search engines typically comprise a web site front end where users are able to conduct searches and view search results and a back end database that stores the indexed information that the search engine has collected about the sites on the Internet which it uses to determine the most relevant search results for a users query. In addition to these components the search engine also comprise search engine spiders, crawlers or bots and it is these components whose job it is to continually traverse the Internet moving from page to page, web site to web site, gathering information about the page content and sending it back to the database for indexing and analysis.

Sometime after submitting a site to a search engine a search spider will be dispatched to the web sites address and it will begin traversing the content of the web site. If it finds links on the web site to other web sites it will follow these also, thereby building up a map of the Internet. In the case of search engines that do not require submission and rely on links between web sites to find new sites it therefore stands to reason that if a site has no incoming links or outgoing links to other sites the chances of it being found to begin with are very slim.

Different search engines work in different ways and each one has its own unique methods and algorithms which it uses to index web pages as well in determining a pages relevance to the search. Some search engines are fairly strict as to the structure and content of meta tags which it requires be included in the, normally hidden from view, source code of the page. Other search engines do not worry about meta tags and will use other methods to index the page content.