index Module

Rom - the Redis object mapper for Python

Copyright 2013-2016 Josiah Carlson

Released under the LGPL license version 2.1 and version 3 (you can choose which you’d like to be bound under).

class rom.index.GeneralIndex(namespace)

Bases: object

This class implements general indexing and search for the rom package.

Warning

You probably don’t want to be calling this directly. Instead, you should rely on the Query object returned from Model.query to handle all of your query pre-processing.

Generally speaking, numeric indices use ZSETs, and text indices use SETs built using an ‘inverted index’.

Say that we have words hello world in a column c on a model with primary key MyModel:1. The member 1 will be added to SETs with keys:

MyModel:c:hello
MyModel:c:world

Text searching performs a sequence of intersections of SETs for the words to be searched for.

Numeric range searching performs a sequence of intersections of ZSETs, removing items outside the requested range after each intersection.

Searches will pre-sort intersections from smallest to largest SET/ZSET prior to performing the search to improve performance.

Prefix, suffix, and pattern matching change this operation. Given a key generated of hello on a column c on a model with primary key MyModel:1, the member hello\01 with score 0 will be added to a ZSET with the key name MyModel:c:pre for the prefix/pattern index. On a suffix index, the member olleh\01 with score 0 will be added to a ZSET with the key name MyModel:c:suf.

Prefix and suffix matches are excuted in Lua with a variant of the autocomplete method described in Redis in Action. These methods ensure a runtime proportional to the number of matched entries.

Pattern matching also uses a Lua script to scan over data in the prefix index, exploiting prefixes in patterns if they exist.

count(conn, filters)

Returns the count of the items that match the provided filters.

For the meaning of what the filters argument means, see the .search() method docs.

search(conn, filters, order_by, offset=None, count=None, timeout=None)

Search for model ids that match the provided filters.

Arguments:

  • filters - A list of filters that apply to the search of one of the following two forms:

    1. 'column:string' - a plain string will match a word in a text search on the column

    Note

    Read the documentation about the Query object for what is actually passed during text search

    1. ('column', min, max) - a numeric column range search, between min and max (inclusive by default)

    Note

    Read the documentation about the Query object for information about open-ended ranges

    1. ['column:string1', 'column:string2'] - will match any of the provided words in a text search on the column

    2. Prefix('column', 'prefix') - will match prefixes of words in a text search on the column

    3. Suffix('column', 'suffix') - will match suffixes of words in a text search on the column

    4. Pattern('column', 'pattern') - will match patterns over words in a text search on the column

  • order_by - A string that names the numeric column by which to sort the results by. Prefixing with ‘-’ will return results in descending order

Note

While you can technically pass a non-numeric index as an order_by clause, the results will basically be to order the results by string comparison of the ids (10 will come before 2).

Note

If you omit the order_by argument, results will be ordered by the last filter. If the last filter was a text filter, see the previous note. If the last filter was numeric, then results will be ordered by that result.

  • offset - A numeric starting offset for results

  • count - The maximum number of results to return from the query

class rom.index.GeoIndex(name, callback)

Bases: tuple

callback

Alias for field number 1

name

Alias for field number 0

rom.index.Geofilter

alias of Geo

class rom.index.Pattern(attr, pattern)

Bases: tuple

attr

Alias for field number 0

pattern

Alias for field number 1

class rom.index.Prefix(attr, prefix)

Bases: tuple

attr

Alias for field number 0

prefix

Alias for field number 1

class rom.index.Suffix(attr, suffix)

Bases: tuple

attr

Alias for field number 0

suffix

Alias for field number 1

rom.index.estimate_work_lua(conn, index, prefix)

Estimates the total work necessary to calculate the prefix match over the given index with the provided prefix.

rom.index.redis_prefix_lua(conn, dest, index, prefix, is_first, pattern=None)

Performs the actual prefix, suffix, and pattern match operations.