InDepth Djangosphinx Tutorial

Again, I still suck at documentation, and my “tutorials” aren’t in-depth enough. So hopefully this covers all of the questions regarding using the django-sphinx module.

The first thing you’re going to need to do is install the Sphinx search software. You will be able to get this throughhttp://www.sphinxsearch.com/, or probably even port or aptitude.

Configure Sphinx

Once you have successfully installed Sphinx you need to configure it. Follow the directions in on their website for the basic configuration, but most importantly, you need to configure a search index which can relate to one of your models.

Here is an example of an index from Curse’s File model, which let’s you search via name, description, and tags on a file. Please note, that “base” is a base source definition we created which has a few defaults which we use, but this is unrelated to your source definition.

source files_file_en : base
{
sql_query			= \
SELECT files_file.id, files_file.name, files_data.description, files_file.tags as tag \
FROM files_file JOIN files_data \
ON files_file.id = files_data.file_id \
AND files_data.lang = 'en' \
AND files_file.visible = 1 \
GROUP BY files_file.id
sql_query_info		= SELECT * FROM files_file WHERE id=$id
}

Now that you have your source defined, you need to build an index which uses this source. I do recommend placing all of your sphinx information somewhere else, maybe /var/sphinx/data.

index files_file_en
{
source			= files_file_en
path			= /var/data/files_file_en
docinfo			= extern
morphology			= none
stopwords			=
min_word_len		= 2
charset_type		= sbcs
min_prefix_len		= 0
min_infix_len		= 0
}

Configure Django

Now that you’ve configured your search index you need to setup the configuration for Django. The first step to doing this is to install the django-sphinx wrapper. First things first, download the zip archive, or checkout the source fromhttp://code.google.com/p/django-sphinx/.

Once you have your files on the local computer or server, you can simple do sudo python setup.py install to install the library.

After installation you need to edit a few settings in settings.py, which, again, being that I suck at documentation, isn’t posted on the website.

The two settings you need to add are these:

SPHINX_SERVER = 'localhost'
SPHINX_PORT = 3312

Setup Your Model

Now you are fully able to utilize Sphinx within Django. The next step is to actually attach your search index to a model. To do this, you will need to import djangosphinx and then attach the manager to a model. See the example below:

1 2 3 4 5 6 7 8 9

from django.db import models import djangosphinx class File(models.model): name = models.CharField() tags = models.CharField() # We actually store tags for efficiency in tag,tag,tag format here objects = models.Manager() search = djangosphinx.SphinxSearch(index="files_file_en")

The index argument is optional, and there are several other parameters you can pass, but you’ll have to look in the code (or pydoc if I did it right, but probably not).

Once we’ve defined the search manager on our model, we can access it via Model.manager_name and pass it many things like we could with a normal object manager in Django. The typical usage is Model.search.query('my fulltext query')which would then query sphinx, grab a list of IDs, and then do a Model.objects.filter(pk__in=[list of ids]) and return this result set.

Search Methods

There are a few additional methods which you can use on your search queryset besides the default query method.order_by, filter, count, and exclude to name a few. These don’t *quite* work the same as Django’s as they’re used directly within the search wrapper. So here’s a brief rundown of these:

• query
  This is your basic full-text search query. It works exactly the same as passing your query to the full-text engine. It’s search type will be based on the search mode, which, by default, is SPH_MATCH_EXTENDED.
• filter/exclude
  The filter and excludes method holds the same idea as the normal queryset methods, except that it is used directly in Sphinx. What this means, is that you can only filter on attribute fields that are present in your search index.
• order_by
  The order_by method also passes its parameters to Sphinx, with one exception. There are four reserved keywords: @id,@weight, @rank, and @relevance. These are detailed in the Sphinx documentation.
• select_related
  This method is directly passed onto the Django queryset and holds no value to Sphinx.
• index_on
  Allows you to specify which index(es) you are querying for. To query for multiple indexes you need to include a “content_type” name in your fields.