search mobile facets autocomplete spellcheck crawler rankings weights synonyms analytics engage api customize documentation install setup technology content domains user history info home business cart chart contact email activate analyticsalt analytics autocomplete cart contact content crawling custom documentation domains email engage faceted history info install mobile person querybuilder search setup spellcheck synonyms weights engage_search_term engage_related_content engage_next_results engage_personalized_results engage_recent_results success add arrow-down arrow-left arrow-right arrow-up caret-down caret-left caret-right caret-up check close content conversions-small conversions details edit grid help small-info error live magento minus move photo pin plus preview refresh search settings small-home stat subtract text trash unpin wordpress x alert case_deflection advanced-permissions keyword-detection predictive-ai sso

Indexing Documents

App Search is here to help you build and curate an index, so that you may craft fluid, delightful and relevant search experiences. To create an index, one must ingest: upload - create, retain, store - objects from a datastore. Those objects are then organized into documents, according to a schema which pairs each Field with a Type of value.

It is fun and easy to get started! Soon, your visitors will be finding the right things faster than ever before.

Pssst: You may also find the API Overview to be a helpful companion as you begin.

What is a Document?

Applications are full of objects.

A document is a representation of an object. An object becomes a document when it is indexing by your Engine to adhere to a schema. Your objects will still exist within your database or backend API, but you will have a set of matching documents within your Engine. These documents can be searched upon using clever and customizable algorithms.

For example, if you have an online storefront that sells fruit, then a piece of fruit would be an object:

{
  "fruit": "apple",
  "quantity": 32,
  "location": "37.7894758,-122.3940638",
  "expiry": "2018-10-13"
}

As a popular store, you would have more than one type of fruit. You would have many. If you were selling cars, you would have many cars, or tea, or RubyGems, or blog posts - whatever it may be. These objects would live inside of your database, be it MongoDB, PostgreSQL, SQLite, MySQL, or one of the many others.

The goal of App Search, and nifty open-source utilities like Elasticsearch, is to help you transform your objects into indexed documents so that you may search upon them in an intelligent and expedient way.

For that, one needs a Schema.

View the Documents API Reference.

What is a Schema?

During the indexing process, a schema is created based on the key/value pairings of the object. By default, you do not need to define a schema. Upon your initial import, a default text based schema is created for you.

In our example fruit case, the default schema would be:

{
  "fruit": "text",
  "quantity": "text",
  "location": "text",
  "expiry": "text"
}

While this is useful for getting your data indexed as quickly as possible, text only fields - while flexible - do not provide validation. If the date were to be incorrect within the expiry field or if the quantity values were formatted in an odd way, it might still be permitted as text. In addition, there are number, date, and location specific searching features that can only be applied once the correct field type is set.

An essential benefit of defining a schema is that each Field will be associated with a standardized Type. If you knew in advance which Field Types you were going to have, then you could apply them using either the dashboard or the Schema API endpoint before indexing your full set of data:

Update Schema via Dashboard - Reviewing and modifying the Engine's schema.
Reviewing and modifying the engine schema

If you are not sure of what your schema will be or just want to get all of your data into your Engine first, then you can use either of these methods at anytime to adjust your schema. But do note that once a field has been created, you are enable to delete it.

Our custom-tailored schema would now be:

{
  "fruit": "text",
  "quantity": "number",
  "location": "geolocation",
  "expiry": "date"
}

Ahhh ~ the power of indexing becomes clear.

Each Field Type has different parameters for enabling search capabilities and input sanitization:

textA full-text, deeply analyzed string value. This is the default type for all new fields. Supports full-text search, highlighting, filtering, faceting, value boosting and sorting.
numberA finite double-precision floating point value: 3.14 or 42. Supports filtering, faceting, functional boosting, value boosting and numeric sorting.
dateAn RFC3339 formatted date string value: "2018-04-27T16:20:00Z". The time parameters may be omitted: YYYY-MM-DD. Supports filtering, faceting, functional boosting, value boosting and time based sorting.
geolocationA lat/long formatted string value: "37.7894758, -122.3940638". Supports filtering, geo distance faceting and functional boosting. Note the spacing within the above example is the prefered spacing. The separating space after the comma may be omitted: "37.7894758,-122.3940638".

To summarize: An object is indexed into your App Search Engine, becoming a document. A document is organized by Field Type around a schema.

Now, to add your many documents.

View the Schema API Reference.

How can I add Documents to an Engine?

Within App Search, there are three different ways to index objects into your Engine:

  1. Paste a raw JSON array during on-boarding or after Engine creation.
  2. Import a .json file during on-boarding or after Engine creation.
  3. Send a POST request to the Documents API endpoint.

Once an Engine has been created, option #3 will be the standard method of indexing documents:

POST /api/as/v1/engines/{ENGINE_NAME}/documents
Example - A robust example of a complex array of vehicle object.
curl -X POST 'https://host-2376rb.api.swiftype.com/api/as/v1/engines/rent-a-car/documents' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer private-xxxxxxxxxxxxxxxxxxxx' \
-d '[
  {
    "id": "441376183",
    "make": "Toyota",
    "model": "4RUNNER",
    "year": "2017",
    "color": "blue",
    "drivetrain": "AWD",
    "description": "A gorgeous three wheel drive that handles rapid stops, highway speeding, and cruising through red stop signs with ease.",
    "category": "Full-Size SUV",
    "seats": 7,
    "status": "available",
    "daily_rate": 79,
    "ac": true,
    "mileage_policy": "unlimited",
    "current_location": "37.6213, -122.3790",
    "tags": [
      "child proof doors",
      "three wheel drive",
      "ludicrous speed",
      "premium gas",
      "red leather",
      "red seats"
    ]
  },
  {
    "id": "781226450",
    "make": "Ford",
    "model": "Fiesta",
    "year": "2018",
    "color": "red",
    "drivetrain": "FWD",
    "description": "For when you want to party. Fiesta means \"festival\" or \"party\" in Spanish. Who wants a car that is a party pooper?",
    "category": "Compact",
    "seats": 5,
    "status": "available",
    "daily_rate": 27,
    "ac": true,
    "mileage_policy": "unlimited",
    "current_location": "37.6213, -122.3790",
    "tags": [
      "children approved",
      "fuel efficient"
    ]
  },
  {
    "id": "66867a",
    "make": "Toyota",
    "model": "Camry",
    "year": "2017",
    "tags": [
      "child safe",
      "power steering"
    ]
  }
]'

Example - A robust example of a complex array of vehicle object.
import com.swiftype.appsearch.Client;
import com.swiftype.appsearch.ClientException;
import java.util.*;

Client client = new Client("host-2376rb", "private-xxxxxxxxxxxxxxxxxxxx"); String engineName = "rent-a-car";

Map<String, Object> doc1 = new HashMap<>(); doc1.put("id", "441376183"); doc1.put("make", "Toyota"); doc1.put("model", "4RUNNER"); doc1.put("year", "2017"); doc1.put("color", "blue"); doc1.put("drivetrain", "AWD"); doc1.put("description", "A gorgeous three wheel drive that handles rapid stops, highway speeding, and cruising through red stop signs with ease."); doc1.put("category", "Full-Size SUV"); doc1.put("seats", 7); doc1.put("status", "available"); doc1.put("daily_rate", 79); doc1.put("ac", true); doc1.put("mileage_policy", "unlimited"); doc1.put("current_location", "37.6213, -122.3790"); doc1.put("tags", Arrays.asList( "child proof doors", "three wheel drive", "ludicrous speed", "premium gas", "red leather", "red seats" ));

Map<String, Object> doc2 = new HashMap<>(); doc2.put("id", "781226450"); doc2.put("make", "Ford"); doc2.put("model", "Fiesta"); doc2.put("year", "2018"); doc2.put("color", "red"); doc2.put("drivetrain", "FWD"); doc2.put("description", "For when you want to party. Fiesta means &quot;festival&quot; or &quot;party&quot; in Spanish. Who wants a car that is a party pooper?"); doc2.put("category", "Compact"); doc2.put("seats", 5); doc2.put("status", "available"); doc2.put("daily_rate", 27); doc2.put("ac", true); doc2.put("mileage_policy", "unlimited"); doc2.put("current_location", "37.6213, -122.3790"); doc2.put("tags", Arrays.asList( "children approved", "fuel efficient" ));

Map<String, Object> doc3 = new HashMap<>(); doc3.put("id", "66867a"); doc3.put("make", "Toyota"); doc3.put("model", "Camry"); doc3.put("year", "2017"); doc3.put("tags", Arrays.asList( "child safe", "power steering" ));

List<Map<String, Object>> documents = Arrays.asList(doc1, doc2, doc3);

try { List<Map<String, Object>> indexDocumentResults = client.indexDocuments(engineName, documents); } catch (ClientException e) { System.out.println(e); }

Example - A robust example of a complex array of vehicle object.
const SwiftypeAppSearchClient = require('swiftype-app-search-node')

const client = new SwiftypeAppSearchClient('host-2376rb', 'private-xxxxxxxxxxxxxxxxxxxx') const engineName = 'rent-a-car' documents = [ { id: '441376183', make: 'Toyota', model: '4RUNNER', year: '2017', color: 'blue', drivetrain: 'AWD', description: 'A gorgeous three wheel drive that handles rapid stops, highway speeding, and cruising through red stop signs with ease.', category: 'Full-Size SUV', seats: 7, status: 'available', daily_rate: 79, ac: true, mileage_policy: 'unlimited', current_location: '37.6213, -122.3790', tags: [ 'child proof doors', 'three wheel drive', 'ludicrous speed', 'premium gas', 'red leather', 'red seats' ] }, { id: '781226450', make: 'Ford', model: 'Fiesta', year: '2018', color: 'red', drivetrain: 'FWD', description: 'For when you want to party. Fiesta means "festival" or "party" in Spanish. Who wants a car that is a party pooper?', category: 'Compact', seats: 5, status: 'available', daily_rate: 27, ac: true, mileage_policy: 'unlimited', current_location: '37.6213, -122.3790', tags: [ 'children approved', 'fuel efficient' ] }, { id: '66867a', make: 'Toyota', model: 'Camry', year: '2017', tags: [ 'child safe', 'power steering' ] } ]

client.indexDocuments(engineName, documents) .then(response => console.log(response)) .catch(error => console.log(error))

Example - A robust example of a complex array of vehicle object.
require 'swiftype-app-search'

host_identifier = 'host-2376rb' api_key = 'private-xxxxxxxxxxxxxxxxxxxx' client = SwiftypeAppSearch::Client.new(host_identifier: host_identifier, api_key: api_key)

engine_name = 'rent-a-car'

documents = [ { id: '441376183', make: 'Toyota', model: '4RUNNER', year: '2017', color: 'blue', drivetrain: 'AWD', description: 'A gorgeous three wheel drive that handles rapid stops, highway speeding, and cruising through red stop signs with ease.', category: 'Full-Size SUV', seats: 7, status: 'available', daily_rate: 79, ac: true, mileage_policy: 'unlimited', current_location: '37.6213, -122.3790', tags: [ 'child proof doors', 'three wheel drive', 'ludicrous speed', 'premium gas', 'red leather', 'red seats' ] }, { id: '781226450', make: 'Ford', model: 'Fiesta', year: '2018', color: 'red', drivetrain: 'FWD', description: 'For when you want to party. Fiesta means "festival" or "party" in Spanish. Who wants a car that is a party pooper?', category: 'Compact', seats: 5, status: 'available', daily_rate: 27, ac: true, mileage_policy: 'unlimited', current_location: '37.6213, -122.3790', tags: [ 'children approved', 'fuel efficient' ] }, { id: '66867a', make: 'Toyota', model: 'Camry', year: '2017', tags: [ 'child safe', 'power steering' ] } ]

begin puts client.index_documents(engine_name, documents) rescue SwiftypeAppSearch::ClientException => e puts e end

Example - A robust example of a complex array of vehicle object.
from swiftype_app_search import Client
from swiftype_app_search.exceptions import SwiftypeAppSearchError

host_identifier = 'host-2376rb' api_key = 'private-xxxxxxxxxxxxxxxxxxxx' client = Client(host_identifier, api_key) engine_name = 'rent-a-car' documents = [ { 'id': '441376183', 'make': 'Toyota', 'model': '4RUNNER', 'year': '2017', 'color': 'blue', 'drivetrain': 'AWD', 'description': 'A gorgeous three wheel drive that handles rapid stops, highway speeding, and cruising through red stop signs with ease.', 'category': 'Full-Size SUV', 'seats': 7, 'status': 'available', 'daily_rate': 79, 'ac': True, 'mileage_policy': 'unlimited', 'current_location': '37.6213, -122.3790', 'tags': [ 'child proof doors', 'three wheel drive', 'ludicrous speed', 'premium gas', 'red leather', 'red seats' ] }, { 'id': '781226450', 'make': 'Ford', 'model': 'Fiesta', 'year': '2018', 'color': 'red', 'drivetrain': 'FWD', 'description': 'For when you want to party. Fiesta means &quot;festival&quot; or &quot;party&quot; in Spanish. Who wants a car that is a party pooper?', 'category': 'Compact', 'seats': 5, 'status': 'available', 'daily_rate': 27, 'ac': True, 'mileage_policy': 'unlimited', 'current_location': '37.6213, -122.3790', 'tags': [ 'children approved', 'fuel efficient' ] }, { 'id': '66867a', 'make': 'Toyota', 'model': 'Camry', 'year': '2017', 'tags': [ 'child safe', 'power steering' ] } ]

try: print(client.index_documents(engine_name, documents)) except SwiftypeAppSearchError as e: print(e)

Example - A robust example of a complex array of vehicle object.
No Javascript example available, showing cURL
curl -X POST 'https://host-2376rb.api.swiftype.com/api/as/v1/engines/rent-a-car/documents' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer private-xxxxxxxxxxxxxxxxxxxx' \
-d '[
  {
    "id": "441376183",
    "make": "Toyota",
    "model": "4RUNNER",
    "year": "2017",
    "color": "blue",
    "drivetrain": "AWD",
    "description": "A gorgeous three wheel drive that handles rapid stops, highway speeding, and cruising through red stop signs with ease.",
    "category": "Full-Size SUV",
    "seats": 7,
    "status": "available",
    "daily_rate": 79,
    "ac": true,
    "mileage_policy": "unlimited",
    "current_location": "37.6213, -122.3790",
    "tags": [
      "child proof doors",
      "three wheel drive",
      "ludicrous speed",
      "premium gas",
      "red leather",
      "red seats"
    ]
  },
  {
    "id": "781226450",
    "make": "Ford",
    "model": "Fiesta",
    "year": "2018",
    "color": "red",
    "drivetrain": "FWD",
    "description": "For when you want to party. Fiesta means &quot;festival&quot; or &quot;party&quot; in Spanish. Who wants a car that is a party pooper?",
    "category": "Compact",
    "seats": 5,
    "status": "available",
    "daily_rate": 27,
    "ac": true,
    "mileage_policy": "unlimited",
    "current_location": "37.6213, -122.3790",
    "tags": [
      "children approved",
      "fuel efficient"
    ]
  },
  {
    "id": "66867a",
    "make": "Toyota",
    "model": "Camry",
    "year": "2017",
    "tags": [
      "child safe",
      "power steering"
    ]
  }
]'

You can send in 100 documents per request. Learn more about the Documents API Reference endpoint.

How many documents can you index into your Engine?

That depends upon your plan.

  • Standard Plan: 50,000 documents
  • Pro Plan: 100,000 documents

You may purchase space for more documents for $20 per month, per 25,000 documents.

What's Next?

You should have a solid grip on document indexing basics: adding data to your Engine and then building a schema. If you have not been acquainted with the Documents endpoint, then that may be a great next step. Otherwise, the Search is where you should venture to further increase your conceptual understanding of the core searching concepts. Alternatively, if you want to learn more about Schemas and Field Types, that would be a helpful read.


Stuck? Looking for help? Contact Support!