Hybrid Search - Upstash Documentation

Full-text search finds documents that contain the words of your query. Semantic search finds documents with a similar meaning, even when they use different words. Hybrid search runs both and merges the results into one list. Hybrid search has two parts:

HYBRID fields are indexed for both full-text and semantic search.
Retrievers run the searches. $ann runs a semantic search, and $rrf merges the results of several retrievers into one list.

Hybrid fields

A HYBRID field is a TEXT field that also stores a vector embedding. Full-text queries work on it the same way as on a TEXT field, and $ann searches its embeddings by meaning. See Hybrid Fields for the field options.

An index can have more than one HYBRID field.
If the text of a document cannot be embedded, the document is still searchable by its text.
HYBRID fields work with JSON, hash, string, and stream indexes.

How embeddings are created

You don’t create vectors yourself. You don’t need your own embedding provider, an API key, or a separate vector database. Redis Search creates the embeddings for you:

When you write a document, Search sends the text of its HYBRID fields to an OpenAI embedding model (one of text-embedding-3-small or text-embedding-3-large) in the background and stores the returned vectors in the index.
When you run an $ann query, Search creates an embedding of the query text with the same model and finds the documents with the closest vectors.

Your application only reads and writes text. None of the examples on this page call an embedding API.

The text of HYBRID fields and of $ann queries is sent to OpenAI to create embeddings. Use TEXT instead of HYBRID for fields whose content must not be sent to a third party.

Pricing

Hybrid search has no extra cost, and creating embeddings is included. Commands are billed the same way as all other Redis commands, whether the index has HYBRID fields or not. See Pricing.

Create an index

Declare the fields you want to search by meaning as HYBRID:

TypeScript
Python
Redis CLI

import { Redis, s } from "@upstash/redis";

const redis = Redis.fromEnv();

const index = await redis.search.createIndex({
  name: "products",
  dataType: "json",
  prefix: "product:",
  schema: s.object({
    name: s.string(),
    description: s.hybrid(),
    category: s.keyword(),
    price: s.number("F64"),
  }),
});

from upstash_redis import Redis

redis = Redis.from_env()

index = redis.search.create_index(
    name="products",
    data_type="json",
    prefixes="product:",
    schema={
        "name": "TEXT",
        "description": "HYBRID",
        "category": "KEYWORD",
        "price": {"type": "F64", "fast": True},
    },
)

SEARCH.CREATE products ON JSON PREFIX 1 product: SCHEMA name TEXT description HYBRID category KEYWORD price F64 FAST

Add documents

Write documents as usual. Search indexes them and creates their embeddings:

TypeScript
Python
Redis CLI

await redis.json.set("product:1", "$", { name: "Studio Headphones", description: "Over-ear headphones with active noise cancelling", category: "audio", price: 199 });
await redis.json.set("product:2", "$", { name: "Travel Earplugs", description: "Soft foam earplugs that keep airplane cabin sound out", category: "travel", price: 9 });
await redis.json.set("product:3", "$", { name: "Sport Earbuds", description: "Sweat-proof wireless earbuds for running", category: "sports", price: 79 });

// Wait until the documents are indexed
await index.waitIndexing();

redis.json.set("product:1", "$", {"name": "Studio Headphones", "description": "Over-ear headphones with active noise cancelling", "category": "audio", "price": 199})
redis.json.set("product:2", "$", {"name": "Travel Earplugs", "description": "Soft foam earplugs that keep airplane cabin sound out", "category": "travel", "price": 9})
redis.json.set("product:3", "$", {"name": "Sport Earbuds", "description": "Sweat-proof wireless earbuds for running", "category": "sports", "price": 79})

# Wait until the documents are indexed
index.wait_indexing()

JSON.SET product:1 $ '{"name": "Studio Headphones", "description": "Over-ear headphones with active noise cancelling", "category": "audio", "price": 199}'
JSON.SET product:2 $ '{"name": "Travel Earplugs", "description": "Soft foam earplugs that keep airplane cabin sound out", "category": "travel", "price": 9}'
JSON.SET product:3 $ '{"name": "Sport Earbuds", "description": "Sweat-proof wireless earbuds for running", "category": "sports", "price": 79}'

# Wait until the documents are indexed
SEARCH.WAITINDEXING products

Search by meaning

Use $ann to find the documents that are closest in meaning to a text. This query can find the earplugs, even though their description does not contain the word “noise”:

TypeScript
Python
Redis CLI

await index.query({
  filter: { $ann: { field: "description", query: "block noise on a flight" } },
});

// Search by meaning within one category
await index.query({
  filter: {
    $ann: { field: "description", query: "headphones for running", filter: { category: "sports" } },
  },
});

index.query(filter={"$ann": {"field": "description", "query": "block noise on a flight"}})

# Search by meaning within one category
index.query(
    filter={
        "$ann": {
            "field": "description",
            "query": "headphones for running",
            "filter": {"category": "sports"},
        }
    }
)

SEARCH.QUERY products '{"$ann": {"field": "description", "query": "block noise on a flight"}}'

# Search by meaning within one category
SEARCH.QUERY products '{"$ann": {"field": "description", "query": "headphones for running", "filter": {"category": "sports"}}}'

Combine full-text and semantic search

Use $rrf to run a full-text query and a semantic search, and merge their results into one list. Documents that rank high in both come first:

TypeScript
Python
Redis CLI

await index.query({
  filter: {
    $rrf: {
      retrievers: [
        { description: "noise cancelling" },
        { $ann: { field: "description", query: "noise cancelling" } },
      ],
    },
  },
});

index.query(
    filter={
        "$rrf": {
            "retrievers": [
                {"description": "noise cancelling"},
                {"$ann": {"field": "description", "query": "noise cancelling"}},
            ]
        }
    }
)

SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "noise cancelling"}, {"$ann": {"field": "description", "query": "noise cancelling"}}]}}'

To give one side more influence, set its weight. A full-text query takes a weight when you wrap it in $standard:

TypeScript
Python
Redis CLI

await index.query({
  filter: {
    $rrf: {
      retrievers: [
        { $standard: { query: { description: "noise cancelling" }, weight: 2 } },
        { $ann: { field: "description", query: "noise cancelling" } },
      ],
    },
  },
});

index.query(
    filter={
        "$rrf": {
            "retrievers": [
                {"$standard": {"query": {"description": "noise cancelling"}, "weight": 2}},
                {"$ann": {"field": "description", "query": "noise cancelling"}},
            ]
        }
    }
)

SEARCH.QUERY products '{"$rrf": {"retrievers": [{"$standard": {"query": {"description": "noise cancelling"}, "weight": 2}}, {"$ann": {"field": "description", "query": "noise cancelling"}}]}}'

Next steps

See Retrievers for all retriever parameters and rules.
See Schema Definition for field types and options.
See SEARCH.QUERY for all query options.

​Hybrid fields

​How embeddings are created

​Pricing

​Create an index

​Add documents

​Search by meaning

​Combine full-text and semantic search

​Next steps

Hybrid fields

How embeddings are created

Pricing

Create an index

Add documents

Search by meaning

Combine full-text and semantic search

Next steps