search mobile facets autocomplete spellcheck crawler rankings weights synonyms analytics engage api customize documentation install setup technology content domains user history info home business cart chart contact email activate analyticsalt analytics autocomplete cart contact content crawling custom documentation domains email engage faceted history info install mobile person querybuilder search setup spellcheck synonyms weights engage_search_term engage_related_content engage_next_results engage_personalized_results engage_recent_results success add arrow-down arrow-left arrow-right arrow-up caret-down caret-left caret-right caret-up check close content conversions-small conversions details edit grid help small-info error live magento minus move photo pin plus preview refresh search settings small-home stat subtract text trash unpin wordpress x alert case_deflection
Swiftype Documentation / api: Indexing

Indexing

Engines

Search engines are the top-level container for the objects you wish to search, and most sites will have a single engine. The engines themselves contain one or more document types, each of which contain the documents themselves.

GET /api/v1/engines.json
POST /api/v1/engines.json
GET /api/v1/engines/{engine_id}.json
DELETE /api/v1/engines/{engine_id}.json
Example - List every Engine in your account
curl -X GET 'https://api.swiftype.com/api/v1/engines.json?auth_token=YOUR_API_KEY'
  
Example - Create an Engine called bookstore
curl -X POST 'https://api.swiftype.com/api/v1/engines.json' \
-H 'Content-Type: application/json' \
-d '{
      "auth_token": "YOUR_API_KEY",
      "engine": {"name": "bookstore"}
    }'
  
Example - Create an Engine called bookstore
curl -X POST 'https://api.swiftype.com/api/v1/engines.json' \
-H 'Content-Type: application/json' \
-d '{
      "auth_token": "YOUR_API_KEY",
      "engine": {"name": "bookstore"}
    }'
  
Example - Get an Engine called bookstore
curl -X GET 'https://api.swiftype.com/api/v1/engines/bookstore.json?auth_token=YOUR_API_KEY'
  
Example - 'Delete an Engine called bookstore
curl -X DELETE 'https://api.swiftype.com/api/v1/engines/bookstore?auth_token=YOUR_API_KEY'
  

DocumentTypes

DocumentTypes specify the structure of a set of documents in the Engine and are the entry point for searches. There are multiple types of fields on a DocumentType — string, text, enum, integer, float, date, and location.

GET /api/v1/engines/{engine_id}/document_types.json
POST /api/v1/engines/{engine_id}/document_types.json
GET /api/v1/engines/{engine_id}/document_types/{document_type_id}.json
DELETE /api/v1/engines/{engine_id}/document_types/{document_type_id}.json
Example - Get every DocumentType in an Engine
curl -X GET 'https://api.swiftype.com/api/v1/engines/bookstore/document_types.json?auth_token=YOUR_API_KEY'
  
Example - Create a DocumentType called books in the bookstore Engine
curl -X POST 'https://api.swiftype.com/api/v1/engines/bookstore/document_types.json' \
  -H 'Content-Type: application/json' \
  -d '{
        "auth_token": "YOUR_API_KEY",
        "document_type": {"name": "books"}
      }'
  
Example - Get a specific DocumentType called books in the bookstore Engine
curl -X GET 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books.json?auth_token=YOUR_API_KEY'
  
Example - Delete a DocumentType called books in the bookstore Engine
curl -X DELETE 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books?auth_token=YOUR_API_KEY'
  

Documents

Documents represent all of the pieces of content in an Engine. They are children of a DocumentType and conform to its field specification (note: you do not need to specify the fields ahead of time, they will be inferred by the contents of a document). When you perform a search on a DocumentType, you will receive Document results. external_id is the only required field for a Document. It can be any value, such as a numeric ID.

GET /api/v1/engines/{engine_id}/document_types/{document_type_id}/documents.json
POST /api/v1/engines/{engine_id}/document_types/{document_type_id}/documents.json
GET /api/v1/engines/{engine_id}/document_types/{document_type_id}/documents/{external_id}.json
PUT /api/v1/engines/{engine_id}/document_types/{document_type_id}/documents/{document_id}/update_fields.json
POST /api/v1/engines/{engine_id}/document_types/{document_type_id}/documents/create_or_update.json
DELETE /api/v1/engines/{engine_id}/document_types/{document_type_id}/documents/{document_id}.json
DELETE /api/v1/engines/{engine_id}/document_types/page/documents/destroy_url?url={url}
Example - List every Document in the books DocumentType
    curl -X GET 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books/documents.json?auth_token=YOUR_API_KEY'
  
Example - Create a Document in the books DocumentType
curl -X POST 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books/documents.json' \
-H 'Content-Type: application/json' \
-d '{
      "auth_token": "YOUR_API_KEY",
      "document": {
        "external_id": "2",
        "fields": [
            {"name": "title", "value": "my title 1", "type": "string"},
            {"name": "page_type", "value": "user", "type": "enum"}
      ]}
    }'
  
Example - Get a specific Document with external_id=1 from the books DocumentType
curl -X GET 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books/documents/1?auth_token=YOUR_API_KEY'
  

Updating Fields

Updates fields on an existing document. This method will only update existing fields on the document. It will not create new fields on an existing document. In most cases you probably want to use bulk_create_or_update_verbose, which will create new fields on existing documents.

Existing fields will not change if not listed in the request.

Example - Update fields for Document with external_id 1.
curl -X PUT 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books/documents/1/update_fields' \
-H 'Content-Type: application/json' \
-d '{
      "auth_token": "YOUR_API_KEY",
      "fields": {"title": "my title 2", "page_type": "user2"}
    }'
  
Example - Create or update a Document in the books DocumentType
curl -X POST 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books/documents/create_or_update.json' \
-H 'Content-Type: application/json' \
-d '{
      "auth_token": "YOUR_API_KEY",
      "document": {
        "external_id": "2",
        "fields": [
            {"name": "title", "value": "my new title", "type": "string"},
            {"name": "page_type", "value": "user", "type": "enum"}
        ]}
      }'
  
Example - Delete a Document with external_id=1 from the books DocumentType
curl -X DELETE 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books/documents/1?auth_token=YOUR_API_KEY'
  
Example - Delete a Document with url=http://example.com/section/page.html
curl -X DELETE 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/page/documents/destroy_url?url=http://example.com/section/page.html&auth_token=YOUR_API_KEY'
  

Bulk Operations

Bulk operations will allow you to perform rapid indexing updates and avoid the latency overhead of making repeated requests.

POST /api/v1/engines/{engine_id}/document_types/{document_type_id}/documents/bulk_create
PUT /api/v1/engines/{engine_id}/document_types/{document_type_id}/documents/bulk_update
POST /api/v1/engines/{engine_id}/document_types/{document_type_id}/documents/bulk_create_or_update_verbose
POST /api/v1/engines/{engine_id}/document_types/{document_type_id}/documents/bulk_create_or_update
POST /api/v1/engines/{engine_id}/document_types/{document_type_id}/documents/bulk_destroy

bulk_create

To create Documents in bulk, use bulk_create. bulk_create will return an error if a Document with a given external_id already exists. In most cases, it is easier to use bulk_create_or_update_verbose.

Example - Create Documents in bulk
curl -X POST 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books/documents/bulk_create' \
  -H 'Content-Type: application/json' \
  -d '{
        "auth_token": "YOUR_API_KEY",
        "documents":
          [
            {"external_id": "3", "fields": [{"name": "title", "value": "my title 1", "type": "string"},
                                            {"name": "page_type", "value": "user", "type": "enum"}]},
            {"external_id": "4", "fields": [{"name": "title", "value": "my title 1", "type": "string"},
                                            {"name": "page_type", "value": "user", "type": "enum"}]}
          ]
      }'
  

bulk_update

To update Documents in bulk, use bulk_update. bulk_update will return an error if a Document with a given external_id does not exist. In most cases, it is easier to use bulk_create_or_update_verbose.

Example - Update Documents in bulk
curl -X PUT 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books/documents/bulk_update' \
  -H 'Content-Type: application/json' \
  -d '{
        "auth_token": "YOUR_API_KEY",
        "documents":
          [
            {"external_id": "1", "fields": {"title": "my title 5", "page_type": "user2"}},
            {"external_id": "2", "fields": {"title": "my title 4", "page_type": "user2"}}
          ]
      }'
  

bulk_create_or_update_verbose

To create or update Documents in bulk, use the bulk_create_or_update_verbose endpoint. This endpoint returns an array of responses, one for each Document. If the create or update was successful, the response will be true.

Example - Create or update Documents in bulk (all succeeded)
curl -X POST 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books/documents/bulk_create_or_update_verbose' \
  -H 'Content-Type: application/json' \
  -d '{
        "auth_token": "YOUR_API_KEY",
        "documents":
          [
            {"external_id": "3", "fields": [{"name": "title", "value": "my title 1", "type": "string"},
                                            {"name": "page_type", "value": "user", "type": "enum"}]},
            {"external_id": "4", "fields": [{"name": "title", "value": "my title 1", "type": "string"},
                                            {"name": "page_type", "value": "user", "type": "enum"}]}
          ]
      }'
  

Example Response

[true, true]

If a create or update fails, the response for that Document will be a string containing an error message.

Example - Error messages from create or update in bulk (all failed)
curl -X POST 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books/documents/bulk_create_or_update_verbose' \
  -H 'Content-Type: application/json' \
  -d '{
        "auth_token": "YOUR_API_KEY",
        "documents":
          [
            {"fields": [{"name": "title", "value": "my title 1", "type": "string"},
                        {"name": "page_type", "value": "user", "type": "enum"}]},
            {"external_id": "4", "fields": [{"name": "title", "value": "my title 1", "type": "bogus"},
                                            {"name": "page_type", "value": "user", "type": "enum"}]}
          ]
      }'
  

Example Response

["Missing external_id", "Invalid field type: Invalid type for \"title\": \"bogus\""]

There is also a non-verbose version which only returns false when a create or update fails. This version is not recommended but it is still available for backwards compatibility.

bulk_destroy

To remove Documents in bulk, use bulk_destroy. The output is an array of true/false values. If an individual destroy succeeds, its entry in the result array will be true. If it fails, its entry in the result array will be false.

Example - Delete Documents in bulk
curl -X POST 'https://api.swiftype.com/api/v1/engines/bookstore/document_types/books/documents/bulk_destroy' \
  -H 'Content-Type: application/json' \
  -d '{
        "auth_token": "YOUR_API_KEY",
        "documents": ["1","2"]
      }'
  

Example Response

[true, true]

Crawler-based engines

If you do not wish to create Documents by hand, you can create a domain in an Engine and our crawlers will automatically crawl the domain and create a Document for each page it finds.

GET /api/v1/engines/{engine_id}/domains.json
POST /api/v1/engines/{engine_id}/domains.json
GET /api/v1/engines/{engine_id}/domains/{domain_id}.json
DELETE /api/v1/engines/{engine_id}/domains/{domain_id}.json
PUT /api/v1/engines/{engine_id}/domains/{domain_id}/recrawl.json
PUT /api/v1/engines/{engine_id}/domains/{domain_id}/crawl_url.json

Creating a domain

Example - Create a Domain for example.com in the bookstore Engine. The crawler begins indexing immediately
curl -X POST 'https://api.swiftype.com/api/v1/engines/bookstore/domains.json' \
  -H 'Content-Type: application/json' \
  -d '{
        "auth_token": "YOUR_API_KEY",
        "domain": {"url": "http://example.com"}
      }'
  

Get a domain

Example - Get every Domain in an Engine
curl -X GET 'https://api.swiftype.com/api/v1/engines/bookstore/domains.json?auth_token=YOUR_API_KEY'
  

Delete a domain

Example - Get a specific Domain with ID 4fcec5182f527673a0000006 in the bookstore Engine
curl -X GET 'https://api.swiftype.com/api/v1/engines/bookstore/domains/4fcec5182f527673a0000006.json?auth_token=YOUR_API_KEY'
  

Recrawl a domain

You may trigger a recrawl of a domain via the API. The frequency of the requests are dictated by your specific plan limitations.

Example - Recrawl a Domain with ID 4fcec5182f527673a0000006 in the bookstore Engine
curl -X PUT -H 'Content-Length: 0' 'https://api.swiftype.com/api/v1/engines/bookstore/domains/4fcec5182f527673a0000006/recrawl.json?auth_token=YOUR_API_KEY'
  

Crawl a single URL

You may trigger the crawl of a single URL by using the domain's crawl_url endpoint with the URL as a parameter. If the URL exists in the domain, it will be updated. If the URL does not exist in the domain, it will be added.

Example - Crawl a URL in the Domain with ID 4fcec5182f527673a0000006 in the bookstore Engine.
curl -X PUT 'https://api.swiftype.com/api/v1/engines/bookstore/domains/4fcec5182f527673a0000006/crawl_url.json' \
  -H 'Content-Type: application/json' \
  -d '{
        "auth_token": "YOUR_API_KEY",
        "url": "http://example.com/new-page"
      }'
  

Note: Crawling the URL happens asynchronously, so there is limited error checking. There are several scenarios that will prevent the URL from being added as a document:

  • The URL does not respond successfully (for example, if it responds HTTP 404 or HTTP 503).
  • The URL does not belong to the domain (for example, you try to crawl http://anotherdomain.net/contact on the example.com domain).
  • The URL is a duplicate of an existing document.
  • The URL is excluded by robots.txt, a robots meta tag, or a path whitelist or blacklist.