Connected: An Internet Encyclopedia
Search Engine Design

Up: Connected: An Internet Encyclopedia
Up: Project Documentation
Prev: RFC Fixup Procedure
Next: Conclusions

Search Engine Design

Search Engine Design The Encyclopedia's search engine is based on CNIDR's Isearch software, currently version 1.41. The actual search page is managed by a Perl module, Isearch.pm, designed for either standalone use with a CGI script or integrated into a web server using mod_perl.

The search engine caches the results of all its searches, so a dozen back-to-back searches for the same term will only trigger the search engine once. Also, if you start a search for a term and click Stop on the browser before the search finishes, the search will complete in background. If you later re-submit the same search term, the cached results will be displayed without waiting for a new search.

If the client browser supports HTTP's Referer: directive, which informs the server of the URL whose hypertext contained the link to the URL being requested, then the search engine script can use this information to generate URLs back to the original source. For example, assume you are browsing a copy of the encyclopedia on the fictitious site www.mirror.com. (1) The search page on this site (http://www.mirror.com/CIE/search.htm) contains the standard Encyclopedia search form, linked to the URL http://www.freesoft.org/CIE/cgi-bin/Isearch.cgi. (2) Running a search from this page will thus trigger a form submission to www.freesoft.org. (3) The search engine, detecting a Referer: directive pointing to http://www.mirror.com/CIE/search.htm, will generate URLs starting with http://www.mirror.com/CIE. (4) Thus, although the search ran on www.freesoft.org, clicking on any of the results will return the browser to www.mirror.com. (5) This technique can be used to allow mirror sites without search engines to provide search engine functionality without relaying all traffic to www.freesoft.org. In the future, this will also allow CD-ROM users to use the search engine, though this is not yet supported by the current CD-ROM.


Next: Conclusions

Connected: An Internet Encyclopedia
Search Engine Design