search mobile facets autocomplete spellcheck crawler rankings weights synonyms analytics engage api customize documentation install setup technology content domains user history info home business cart chart contact email activate analyticsalt analytics autocomplete cart contact content crawling custom documentation domains email engage faceted history info install mobile person querybuilder search setup spellcheck synonyms weights engage_search_term engage_related_content engage_next_results engage_personalized_results engage_recent_results success add arrow-down arrow-left arrow-right arrow-up caret-down caret-left caret-right caret-up check close content conversions-small conversions details edit grid help small-info error live magento minus move photo pin plus preview refresh search settings small-home stat subtract text trash unpin wordpress x alert case_deflection advanced-permissions keyword-detection predictive-ai sso
Site Search Documentation / crawler: Content Inclusion & Exclusion

Include or Exclude content from your Search Engine

Content Inclusion

In some cases you may have many items you want excluded from the page but only a few you would like included. In these scenarios you can use the content inclusion feature to whitelist elements of your pages for indexing. For example, if you have a single content section of every page you can very easily set data-swiftype-index=true on this element and our crawler will only extract text from there.

<body>

  This is content that will not be indexed by the Swiftype crawler.

  <div data-swiftype-index='true'>
    <p>
      All content under the above div tag will be indexed.
    </p>
    <p>
      Content in this paragraph tag will be included from the search index!
    </p>
  </div>

  This content will not be indexed, since it isn't surrounded by an include tag.

</body>

Content Exclusion

You may want to keep certain content from being indexed in your site search engine -- for example, your site header, footer, or menu bar. You can tell the Swiftype crawler to exclude these elements by adding the data-swiftype-index=false attribute to any element, as illustrated below.

<body>

  This is your page content, which will be indexed by the Swiftype crawler.

  <p data-swiftype-index='false'>
    Content in this paragraph tag will be excluded from the search index!
  </p>

  This content will be indexed, since it isn't surrounded by an excluded tag.

  <div id='footer' data-swiftype-index='false'>
    This footer content will be excluded as well.
  </div>

</body>

Nested Rules

Nesting the above rules will work as you might expect. If there are multiple rules present on the page, all text will inherit its behavior from the nearest parent element with a rule. This way, you will be able to include and exclude elements within each other. If there is text outside of any container elements with applied rules on the page, it will be treated opposite of the first-appearing rule on said page. For example, if the first rule is data-swiftype-index=false applied to a child element, any text outside that element and any other element with an inclusion rule will be indexed to the page's document record.

<body>

  This is content that will not be indexed since the first rule is true.

  <div data-swiftype-index='true'>
    <p>
      All content under the above div tag will be indexed.
    </p>
    <p>
      Content in this paragraph tag will be included from the search index!
    </p>
    <p data-swiftype-index='false'>
      Content in this paragraph will be excluded because of the nested rule.
    </p>
    <span data-swiftype-index="false">
      <p> 
        Content in this paragraph will be excluded because the parent span is false.
      </p>
      <p data-swiftype-index="true">
        Content in this paragraph will be INCLUDED because the parent container is true.
      </p>
    </span>
  </div>

</body>