search mobile facets autocomplete spellcheck crawler rankings weights synonyms analytics engage api customize documentation install setup technology content domains user history info home business cart chart contact email activate analyticsalt analytics autocomplete cart contact content crawling custom documentation domains email engage faceted history info install mobile person querybuilder search setup spellcheck synonyms weights engage_search_term engage_related_content engage_next_results engage_personalized_results engage_recent_results success add arrow-down arrow-left arrow-right arrow-up caret-down caret-left caret-right caret-up check close content conversions-small conversions details edit grid help small-info error live magento minus move photo pin plus preview refresh search settings small-home stat subtract text trash unpin wordpress x alert case_deflection
Swiftype Documentation / tutorials: Building a Django Application with Swiftype Integration

Building a Django application with Swiftype integration

This tutorial will walk through how to use the swiftype-py library to index content in a Django application, as well as how to incorporate autocomplete and full-text search using the jQuery plugins.

The Django Blog

We'll be starting with a very simple blogging application that is built using the Django framework. If you're new to Django, you can find download instructions here and an excellent step-by-step guide for setting up your first Django application here. In our example project we can create, edit, and delete posts. By the end of this tutorial, we'll be using Swiftype to index content and serve search results for our blog's content.

You can find an example Django application, with working Swiftype integration, here.

Step 1: Setup

We'll use the django-admin.py utility to generate our project's initial structure. From your terminal, run the following (feel free to change myblog to whatever you'd like to name your blog):

django-admin.py startproject myblog

You should now have a new folder called myblog that contains all of the necessary files to start your Django application. Change into the new myblog directory and run:

python manage.py runserver

This will boot the Django web server for your new project. In a web browser, while the server is running, visit http://127.0.0.1:8000/ and you'll see a "Welcome to Django" page.

Now that the basic project is working, let's make a few modifications to support the desired blogging functionality. First, modify your project's structure to resemble the following (where myblog is the name of our project):

myblog/
    manage.py
    blog/
        __init__.py
        admin.py
        models.py
        tests.py
        views.py
    myblog/
        __init__.py
        settings.py
        urls.py
        wsgi.py
    static/
    templates/
        blog/
            index.html
            post.html
        base.html

We'll use a model, Post, to represent blog posts. This is defined in blog/models.py

from django.db import models

class Post(models.Model):
    title = models.CharField(max_length=2555)
    slug = models.SlugField(unique=True, max_length=255)
    content = models.TextField()
    created = models.DateTimeField(auto_now_add=True)

We'll use the admin interface for creating, editing, and deleting Posts. This is defined in blog/admin.py:

from django.contrib import admin
from blog.models import Post

class PostAdmin(admin.ModelAdmin):
  list_display = ['title']
  list_filter = ['created']
  search_fields = ['title', 'content']
  save_on_top = True
  prepopulated_fields = {"slug": ("title",)}

admin.site.register(Post, PostAdmin)

And we'll define the view functions to support listing posts (with pagination) and displaying individual posts. This is defined in blog/views.py

from django.core.paginator import Paginator, EmptyPage, PageNotAnInteger
from django.shortcuts import render, get_object_or_404
from blog.models import Post

def index(request):
    all_posts = Post.objects.all()
    paginator = Paginator(all_posts, 5)
    page = request.GET.get('page')

    try:
        posts = paginator.page(page)
    except PageNotAnInteger:
        posts = paginator.page(1)
    except EmptyPage:
        posts = paginator.page(paginator.num_pages)

    return render(request, 'blog/index.html', {'posts': posts})

def post(request, slug):
    post = get_object_or_404(Post, slug=slug)
    return render(request, 'blog/post.html', {'post': post })

Next, we need to add our blog/ app to the list of INSTALLED_APPS, and set the template and static directories. This is set in myblog/settings.py:

# ... other settings ...

INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'blog'
)

TEMPLATE_DIRS = (
    os.path.join(BASE_DIR, 'templates')
)

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.6/howto/static-files/

STATIC_URL = '/static/'

STATICFILES_DIRS = (
    os.path.join(BASE_DIR, "static"),
)

# ... other settings ...

Next, we'll define the URL patterns in myblog/urls.py as such:

from django.conf.urls import patterns, include, url
from django.contrib import admin

admin.autodiscover()

urlpatterns = patterns('',
    url(r'^admin/', include(admin.site.urls)),
    url(r'^$', 'blog.views.index'),
    url(r'^(?P<slug>[\w\-]+)/$', 'blog.views.post'),
)

And finally, we'll create a few simple templates for the blog. A base template in templates/base.html:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>
      {% block title %}{% endblock %}
    </title>
  </head>
  <body>
    <div class="content">
      {% block content %}
      {% endblock %}
    </div>
  </body>
</html>

A template for displaying all posts in templates/blog/index.html:

{% extends 'base.html' %}

{% block title %} Blog Archive {% endblock %}

{% block content %}

  <h1> My Blog Archive </h1>
  {% for post in posts %}
  <div class="post">
    <h2>
      <a href="/{{post.slug}}">
        {{post.title}}
      </a>
    </h2>
    <p>
      Posted on
      <time datetime="{{post.created|date:'c' }}">
      {{post.created|date}}
      </time>
    </p>
  </div>
  {% endfor %}

  <div class="pagination">
    <span class="post-links">
      {% if posts.has_previous %}
        <a href="?page={{ posts.previous_page_number }}">previous</a>
      {% endif %}

      <span class="current">
        Page {{ posts.number }} of {{ posts.paginator.num_pages }}.
      </span>

      {% if posts.has_next %}
        <a href="?page={{ posts.next_page_number }}">next</a>
      {% endif %}
    </span>
  </div>

{% endblock %}

And a template for displaying individual blog posts in templates/blog/post.html:

{% extends 'base.html' %}

{% block title %}{{post.title}}{% endblock %}

{% block content %}
<article>
  <header>
    <h1> {{post.title}} </h1>
    <p>
      Posted on
      <time datetime="">
      {{post.created|date}}
      </time>
    </p>
  </header>
  <p class="description">
    {{post.description}}
  </p>
  {{post.content|safe}}
</article>
{% endblock %}

Since our blog will be using a number of database tables, we'll need to create those as well. From your terminal, run the following (you'll need to stop the development server if that's still running):

python manage.py syncdb

If you haven't already, you'll be prompted to create a superuser account to access the Django Admin portion of the application. Follow the prompts to complete this.

With the database tables created, restart your development server and visit http://127.0.0.1:8000/ to see your blog's new home page. To create your first post, visit http://127.0.0.1:8000/admin/ and log in as your superuser account to navigate the admin interface.

Step 2: Creating a search engine on Swiftype

First, if you haven't already, sign up for a Swiftype account.

Next, log into your Swiftype account and create an API-based search engine called "My Blog" with a Document Type named "posts". After you create the engine, you'll see your API key, Engine Slug, and Engine Key.

We'll store these values as environment variables, rather than hard-coding the keys in the application itself. You can set these values as such (replacing your_api_key and your_engine_key with their respective values from your dashboard) from your terminal:

export SWIFTYPE_API_KEY=your_api_key
export SWIFTYPE_ENGINE_KEY=your_engine_key

Now, we'll edit myblog/settings.py to read these values when starting the application:

# ... other settings ...

SWIFTYPE_API_KEY = os.environ['SWIFTYPE_API_KEY']
SWIFTYPE_ENGINE_KEY = os.environ['SWIFTYPE_ENGINE_KEY']

# ... other settings ...

Step 3: Indexing blog posts on Swiftype

This is a good opportunity to get familiar with the Swiftype API. If you haven't already, read about the different resources we'll be working with in the Swiftype API here.

We've already created an Engine called "My Blog" with a DocumentType of "posts". Each of our blog post will act as Document when interacting with the Swiftype API.

Swiftype Documents have an External ID that ties a record in Swiftype to a record in our database. This must be unique within the DocumentType. For this value, we'll use the primary key of the Post model: id.

Next, we'll need to choose which field types to associate with each Document we send to Swiftype. Field types determine how and where a Document will be returned in search results. Swiftype provides many different field types such as string, text, enum, and date (learn more here).

For this example, a good choice is to map the Post's title attribute to a string field so it can be used for autocompletion; the content attribute maps to a text field because it will be used for full-text search, but not autocompletion; the created attribute maps to a date field and can be used for range queries. There is no url attribute on the Post model, but we can construct it using the slug value we've defined on the Post model. The url should be stored as an enum because it must be stored verbatim. (For more details on designing your search schema, see the Search Engine Schema Design tutorial.)

Now, with our search schema designed, we'll want to install the swiftype-py library, which allows us to easily talk to the Swiftype API. You can install this library with pip from your terminal:

pip install swiftype

We'll use Django's signal dispatcher to keep Swiftype informed of new posts and any changes to existsing posts (including when posts are deleted). Django provides built-in post_save and post_delete signals, which are triggered whenever an object is saved to, or deleted from, the database. These signals act as a good mechanism for notifying the Swiftype API of changes to Post content. We'll implement seperate handlers for the post_save and post_delete signals.

Edit blog/models.py as such:

from django.db import models
from django.db.models import signals
from django.conf import settings
from swiftype import swiftype

class Post(models.Model):
    title = models.CharField(max_length=2555)
    slug = models.SlugField(unique=True, max_length=255)
    content = models.TextField()
    created = models.DateTimeField(auto_now_add=True)

# Signal handlers

def publish_document(sender, instance, **kwargs):
    client = swiftype.Client(api_key=settings.SWIFTYPE_API_KEY)

    client.create_or_update_document('my-blog', 'posts', {
        'external_id': instance.id,
        'fields': [
            {'name': 'title', 'value': instance.title, 'type': 'string'},
            {'name': 'content', 'value': instance.content, 'type': 'text'},
            {'name': 'url', 'value': "/%s" % instance.slug, 'type': 'enum'},
        ]
    })

def destroy_document(sender, instance, **kwargs):
    client = swiftype.Client(api_key=settings.SWIFTYPE_API_KEY)
    client.destroy_document('my-blog', 'posts', instance.id)

signals.post_save.connect(publish_document, sender=Post)
signals.post_delete.connect(destroy_document, sender=Post)

At this point, creating, saving and deleting posts should be reflected in our Swiftype dashboard. We're now ready to add search functionality to our blog!

Note that signal dispatch and handling occurs synchronously within the request-response cycle. You may want to use something like Celery to process the Swiftype API calls asynchronously, in the background.

Step 4: Autocomplete

We'll be using the Swiftype jQuery Autocompletion plugin to power autocomplete functionality on our blog. Download those files if you haven't already.

Add the jquery.swiftype.autocomplete.js file to the static/ directory, as well as the autocomplete.css file if you want to use the default styling. We'll need to include to these files, as well as the jQuery plugin, wherever we want to access the plugin. Since we may want to access the plugin on every page, we'll include them in our base.html file as such:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>
      {% block title %}{% endblock %}
    </title>
    <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
    {% load staticfiles %}
    <link href="{% static '/static/autocomplete.css' %}" rel="stylesheet" type="text/css">
    <script src="{% static '/static/jquery.swiftype.autocomplete.js' %}"></script>
  </head>
  <body>
    <div class="content">
      {% block content %}
      {% endblock %}
    </div>
  </body>
</html>

Next, we'll add a search box to the index.html file. The Swiftype JavaScript plugin targets this by a CSS selector. A DOM ID like st-search-input is namespaced to avoid CSS conflicts. Using this ID, we'll configure this Swiftype JavaScript plugin after the DOM is ready (place this at the bottom of your index.html file):

<script>
  $(function() {
    $('#st-search-input').swiftype({
      engineKey: {{ swiftype_engine_key }}
    });
  });
</script>

Note that the above won't work because we aren't yet passing the swiftype_engine_key to our view. So, let's include this value in our view function for the index page:

blog/views.py:

# ...

from django.conf import settings

def index(request):
    all_posts = Post.objects.all()
    paginator = Paginator(all_posts, 5)
    page = request.GET.get('page')

    try:
        posts = paginator.page(page)
    except PageNotAnInteger:
        posts = paginator.page(1)
    except EmptyPage:
        posts = paginator.page(paginator.num_pages)

    return render(request, 'blog/index.html', {'posts': posts, 'swiftype_engine_key': settings.SWIFTYPE_ENGINE_KEY})

# ...

Again, we're grabbing the SWIFTYPE_ENGINE_KEY from the environment variable to avoid hardcoding the engine key in our code.

We should now have a working search box with autocompletion! Swiftype will perform autocompletion against all the string fields, which in this example is only title. For more information on restricting the fields that the plugin matches against, take a look at the documentation for the jQuery plugin.

Step 5: Full-text search

We'll use the Swiftype jQuery Search plugin to perform full-text searches through blog-content and display results in their own container on the page.

First, download the assets from the repository to your project's static/ folder and link to the jquery.swiftype.search.js, jquery.ba-hashchange.min.js (and optional search.css file for styling) from the base.html file:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>
      {% block title %}{% endblock %}
    </title>
    <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
    {% load staticfiles %}
    <link href="{% static '/static/autocomplete.css' %}" rel="stylesheet" type="text/css">
    <link href="{% static '/static/search.css' %}" rel="stylesheet" type="text/css">
    <script src="{% static '/static/jquery.swiftype.autocomplete.js' %}"></script>
    <script src="{% static '/static/jquery.ba-hashchange.min.js' %}"></script>
    <script src="{% static '/static/jquery.swiftype.search.js' %}"></script>
  </head>
  <body>
    <div class="content">
      {% block content %}
      {% endblock %}
    </div>
  </body>
</html>

Next, add a container element to the blog/index.html page where search results will be displayed. You may want this directly beneath the search form:

<div id="st-results-container" class="st-result-listing"></div>

And finally, configure the jQuery search plugin to display the results in the container we just added (you can place this in the same script tag we used for the autocomplete plugin in the index.html file):

<script>
  $(function() {
    $('#st-search-input').swiftype({
      engineKey: {{ swiftype_engine_key }}
    });

    $('#st-search-input').swiftypeSearch({
      resultContainingElement: '#st-results-container',
      engineKey: {{ swiftype_engine_key }}
    });
  });
</script>

With this added, we can now perform searches across all of our blog content. Type a query into the search box and press enter, the results should display in the st-result-listing div.

The plugin is very customizable with configuration options for what fields to fetch and how to render the results. For details, check out the README file and the Customizing Search Results with the Swiftype jQuery Plugin tutorial.