I\'m talking the kind of search that auto-suggests your query as you type, the w
ID: 651490 • Letter: I
Question
I'm talking the kind of search that auto-suggests your query as you type, the way Google does, the way Wikipedia does, the way Stack Exchange suggests other questions as you type the title, etc. And Wikipedia used to have an awful search implementation, for those of us who remember. It had no auto-suggest, was shut down by the slightest typo or misspelling, and rarely gave pertinent results.
As an amateur web programmer, I'm trying to figure out how I would implement an effective and high-performing search-and-auto-suggest as Wikipedia now has, and as SE has, and as so many other websites have.
How can you obtain that kind of incredibly fast performance and flexibility? Does everyone just wing it, or are there principles, guidelines, helpful libraries, or other such resources to help you get it done right in the style of the "Internet of the Future"?
Explanation / Answer
For a start you will need a full text search engine like Apache Solr or Sphinx (there are more and some databases have full text features too, but I know those two and they are free and work great). If it has facet search (like Solr) this will help a lot (for certain types of queries). This will cover the largest part of indexing and performance issues.
From there on you need to analyze your data and the search needs of your customers and come up with a good setup of fields, field types, field weights, and the text search specific issues like stemming or multiple languages, handling of special characters, splitting compound words into single words (very important for german search, we can chain words arbitrarily here).
In addition you can do a lot of statistics like counting words or phrases your users search for and compare them against statistics of your index data (for example to find out what synonyms are relevant for you)