# Cross-Encoder Model Comparison

## Popular cross-encoder models

| Model | Size | Domain | Notes |
|-------|------|--------|-------|
| `cross-encoder/ms-marco-MiniLM-L-6-v2` | 80 MB | General IR | Fast, good baseline |
| `cross-encoder/ms-marco-MiniLM-L-12-v2` | 130 MB | General IR | Better accuracy, slower |
| `BAAI/bge-reranker-base` | 1.1 GB | General / multilingual | Strong on BGE ecosystem |
| `BAAI/bge-reranker-v2-m3` | 2.3 GB | Multilingual | State-of-the-art for 2024 |
| `Cohere Rerank v3` | Hosted | General | API, no infra needed |
| `Jina Reranker v2` | Hosted | General, multilingual | API with good code retrieval |

## Latency benchmarks (approximate)

| Model | Docs/sec (CPU) | Docs/sec (GPU) | Best for |
|-------|---------------|----------------|----------|
| MiniLM-L-6 | ~200 | ~2000 | High throughput, tight budget |
| bge-reranker-base | ~40 | ~800 | Quality-first RAG |
| Cohere Rerank API | ~50 (net) | Same | Zero infra |

## When to rerank

**Rerank when:**
- First-stage retrieval has high recall but poor precision
- You need the LLM to see the best 3-5 passages, not just "probably relevant" ones
- Your eval shows nDCG@k lift from reranking

**Skip reranking when:**
- First-stage retrieval already places correct docs at position 1-2
- Added latency (>100ms) breaks your SLO
- No labeled eval set to measure the benefit

## Budget formula

```
Rerank budget N = top_N_after_fusion (typically 50-100)
Final context K = top_K_after_rerank (typically 3-5)
Cost: O(N) cross-encoder calls per query
```