hybrid-vector-keyword-search
verifiedf1334551-0772-44f4-be74-f25492ca6bd8
Combine BM25 keyword and dense vector retrieval with reciprocal rank fusion (RRF) for search that catches both exact and semantic matches.
Metadata
Skill file
# Hybrid Keyword + Vector Search with RRF
Use when a single retrieval method keeps missing queries: dense vector search
handles paraphrase and conceptual queries well but can underweight exact rare
terms, product codes, and named entities; plain BM25 nails exact tokens but can't
match a differently-worded question. Neither wins every time. Hybrid search runs
both and fuses the ranked lists.
## Why RRF instead of averaging scores
BM25 and vector similarity produce scores on **incompatible scales**. Averaging
or min-max-normalizing raw scores are both fragile: normalization that looks fine
for one query distribution distorts another, and a single dominating source can
swamp the other entirely.
**Reciprocal Rank Fusion (RRF)** ignores raw scores and fuses on *ranks*:
```
score(d) = Σ over each retriever r of 1 / ( k + rank_r(d) )
```
where `k` is a small constant (commonly 60, sometimes 0 in toy examples) that
controls how much being ranked high in one list helps. Because it uses rank
position, not score magnitude, it stays stable across wildly different score
distributions and needs no normalization. A document ranked #1 by *one* retriever
gets a strong fused score even if the other retriever ranked it low — this is the
desired behavior.
## Pipeline
1. **Retrieve a surfeit** — pull `retrieval_k` (e.g. 50) from both BM25 and vector
search *concurrently*. Pull more than your final `top_k` so RRF has candidates.
2. **Fuse** the two ranked lists with RRF into one ranked list.
3. **Optionally rerank** the top (e.g. `top_k * 3`) candidates with a
cross-encoder (a dedicated reranker that scores query-document pairs, not just
independent embeddings). Cross-encoders are slower but much more accurate than
bi-encoder dot products; they fix the cases where fused recall is right but the
ordering is off. Over-retrieve → rerank top few is the standard pattern.
Pseudo-shape:
```python
dense = dense_retriever.search(query, top_k=retrieval_k)
sparse = bm25_retriever.search(query, top_k=retrieval_k)
fused = reciprocal_rank_fusion([dense, sparse], k=60)
if reranker:
return rerank(fused[: top_k * 3])[: top_k]
return fused[: top_k]
```
## When it's worth it vs not
- Worth it: mixed query streams (some exact phrase/product lookups, some
paraphrase questions), when users complain specific terms come back empty.
- Often the pragmatic destination is **BM25 + dense + RRF** before reaching for
more exotic sparse encoders like SPLADE (learned sparse vectors). Measure with
BM25+dense+RRF first; adopt SPLADE only if recall still lags and you can absorb
the operational complexity.
- Store/infra: many vector DBs (Weaviate, Qdrant, OpenSearch) ship hybrid + RRF
built in; OpenSearch 2.19+ added RRF to its Neural Search plugin. Using the
built-in beats hand-rolling when available.
## Pitfalls
- Fusing with only `top_k` from each retriever (too few candidates → misses).
- Naive score averaging/normalization that one retriever dominates.
- Speaking of RRF `k` as if it's a magic number — tune it on your data; the
effect of `k` on rank-vs-score is modest and standard values like 60 usually
work, but verify.
- Reranking only the final few results (nothing to fix); rerank a comfortably
larger candidate set.
- Forgetting metadata filters — apply filters in each retriever so fusion doesn't
surface out-of-scope docs.
## Verify
- Build a query set of exact-match, paraphrase, and mixed cases.
- Report precision@k / recall@k for: dense-only, BM25-only, hybrid-RRF, hybrid+rerank.
- Confirm hybrid ≥ the best single retriever on your real complaint queries before
shipping.