search mobile facets autocomplete spellcheck crawler rankings weights synonyms analytics engage api customize documentation install setup technology content domains user history info home business cart chart contact email activate analyticsalt analytics autocomplete cart contact content crawling custom documentation domains email engage faceted history info install mobile person querybuilder search setup spellcheck synonyms weights engage_search_term engage_related_content engage_next_results engage_personalized_results engage_recent_results success add arrow-down arrow-left arrow-right arrow-up caret-down caret-left caret-right caret-up check close content conversions-small conversions details edit grid help small-info error live magento minus move photo pin plus preview refresh search settings small-home stat subtract text trash unpin wordpress x alert case_deflection
Search Concepts / Indexing


Indexing refers to the process of collecting website data in a structured format (i.e. search index) that is optimized for a search algorithm. When developing a search engine infrastucture, developers need to devise a system for indexing existing website content and new content as it is published.

In Swiftype

While many third-party search providers require site owners to pass website information to their search index via .xml or data feed, Swiftype indexes website content with a high performance web crawler, Swiftbot. The regularity of this indexing process varies based on your pricing plan. Free users receive partial recrawls of their website content each day, with the option to initiate full recrawls once a week. Enterprise customers can initiate full recrawls each day, with the option to enable Constant Crawl to ensure that Swiftbot immediately indexes new website content as it is published.