RediSearch

Full-Text and Secondary Index engine over Redis

Data Lifecycle

Create a Schema

4 index types

Text
Numeric
Tag
Geospatial

Add documents

Hashes indexed automatically (RediSearch 2.0)

Search/Aggregate

Search
Aggregate
Delete documents as needed
Drop the whole index

Query Language

Goals

Intentionally not SQL
Familiar
Exposable to end-users

Simple

No knowledge of data structures needed

Powerful

With knowledge, zero in on data

Simple Word Matching

ford

Prefix Matching

ford | chev*

Negation

(ford | chev*) -explorer

Optional Term

(ford | chev*) -explorer ~truck

Fuzzy Matching

(ford | chev*) -explorer ~truck %raptur%

Numeric Filter

(ford | chev*) -explorer ~truck %raptur%
@year:[2001 2011]

Geo Filter

(ford | chev*) -explorer ~truck %raptur%
@year:[2001 2011]
@location:[-77.02 38.78 10 mi]

Tag Filter

(ford | chev*) -explorer ~truck %raptur%
@year:[2001 2011]
@location:[-77.02 38.78 10 mi]
@condition:{good | very good}

Full-Text Search

Stop Words

"a fox in the woods" → "fox woods"

Stemming

"going" → "going" "go" "gone"

Slop

"glass pitcher", slop 2 → "glass gallon beer pitcher"

With or without content

"To be or not to be" → Hamlet (without the whole play)

Matched text highlight

"To be or not to be" → <b>To be, or not to be</b> that is the question

Synonyms

"Bob" → Find documents with "Robert"

Spell Check

"a fxo in the woods" → Did you mean "a fox in the woods"

Phonetic Search

"John Smith" → "Jon Smyth"

Weights & Scores

Each field can have a weight which influences the rank in the returned result
Each document can have a score to influence rank

Scoring Functions

Default: TF-IDF
- Variant: DOCNORM
- Variant: BM25
DISMAX (Solr’s default)
DOCSCORE
HAMMING for binary payloads

Aggregations

Process/transform
Same query language as search
Can group, sort and apply transformations
Follows pipeline of composable actions

Aggregation Pipeline

Grouping

COUNT

COUNT_DISTINCT

COUNT_DISTINCTISH

SUM

MIN

MAX

AVG

STDDEV

QUANTILE

TOLIST

FIRST_VALUE

RANDOM_SAMPLE

Functions — Strings

substr(upper('hello'),0,3)

→ HEL

Functions — Numbers

sqrt(log(foo) * floor(@bar/baz)) + (3^@qaz % 6)

Functions — Time

timefmt(@time, "%b %d %Y %H:%M”)

→ Feb 24 2018 00:05

Aggregate Command

FT.AGGREGATE shipments "@box_area:[300 +inf]"
	APPLY "year(@shipment_timestamp / 1000)" AS shipment_year
	GROUPBY 1 @shipment_year REDUCE COUNT 0 AS shipment_count
	SORTBY 2 @shipment_count DESC
	LIMIT 0 3
	APPLY "format(\"%sk+ Shipments\",floor(@shipment_count / 1000))"
          AS shipment_count

Auto-complete

aka Suggestions
in the module but separate storage

Radix Tree-based

Optimized for real-time, as-you-type completions

Simple API

FT.SUGADD — add a suggestion
FT.SUGGET — get a suggestion
FT.SUGDEL — delete a suggestion

RediSearch

RediSearch

Data Lifecycle

Create a Schema

Add documents

Search/Aggregate

Query Language

Goals

Simple

Powerful

Simple Word Matching

Prefix Matching

Negation

Optional Term

Fuzzy Matching

Numeric Filter

Geo Filter

Tag Filter

Full-Text Search

Stop Words

Stemming

Slop

With or without content

Matched text highlight

Synonyms

Spell Check

Phonetic Search

Weights & Scores

Scoring Functions

Aggregations

Aggregation Pipeline

Grouping

Functions — Strings

Functions — Numbers

Functions — Time

Aggregate Command

Auto-complete

Radix Tree-based

Simple API

RediSearch vs Elasticsearch

Indexing Performance

Querying Performance