BGE-M3 Multilingual Embeddings for Hybrid Retrieval

BGE-M3 is an embedding model from the Beijing Academy of AI (BAAI) that unifies three retrieval modes — dense, sparse, and multi-vector — in a single 1024-dimensional model trained across 100+ languages. It removes the usual tradeoff between embedding quality, language coverage, and retrieval mode.

Why a unified model matters

Traditional RAG stacks run two or three models side by side: a dense embedder (e.g. OpenAI text-embedding-3), a sparse lexical model (BM25 or SPLADE), and a reranker (cross-encoder or ColBERT). BGE-M3 emits all three vector types from one forward pass. The operational savings are concrete: one model to serve, one weights cache, one tokenizer, one evaluation pipeline. For a self-hosted Ollama deployment, serving three specialised models on the same hardware is often impractical, so unification is the only path to hybrid retrieval.

The three heads

• Dense — 1024-dim float vector suited for cosine similarity. Produces the primary semantic recall signal. Stored in Qdrant under the dense named vector.
• Sparse — token-weighted lexical scores, functionally similar to SPLADE. Strong for exact-match, code, proper nouns. Stored in Qdrant as a sparse vector with modifier IDF.
• Multi-vector (ColBERT-style) — one 1024-dim vector per input token, scored at query time with MAX_SIM. Provides late-interaction reranking without a second network round-trip to a cross-encoder.

Cross-lingual behaviour

BGE-M3’s training corpus is ~2B pairs spanning ≥100 languages. In practice, an English query against a Japanese document set retrieves the correct document with no translation step, and vice versa. This is the lever that makes bilingual personal knowledge bases viable — authors can write in whichever language fits the content and readers query in whichever language fits the moment.

Hybrid fusion strategies

When all three heads are available, retrieval fuses them via one of:

• Reciprocal Rank Fusion (RRF) — rank-based, k=60 is the common default. Robust to score-scale differences across heads.
• Weighted Sum — requires min-max normalisation per head before weighted averaging. Sensitive to outlier scores; only preferred when head weights are tuned for the corpus.
• Dense-only fallback — for small corpora (<1000 documents) sparse adds little; the sparse head is worth skipping until the corpus grows past the embedding’s representation capacity.

Deployment notes for Ollama

Ollama serves BGE-M3 via the bge-m3 tag. The /api/embed endpoint returns only the dense head as of late 2025 — sparse and colbert require a direct HuggingFace pipeline or a FastAPI wrapper. For dense-only MVPs this is sufficient; sparse/colbert can be added later without re-embedding the corpus (additional named vectors can be upserted incrementally in Qdrant).

When BGE-M3 is the wrong choice

• English-only corpus — text-embedding-3-large outperforms on purely English benchmarks and has a hosted SLA.
• Sub-512-token passages — BGE-M3’s strength is its 8192-token context; a 384-token passage set wastes the context budget without quality gains.
• Sub-second latency requirements — the multi-vector head is slow at query time. Latency-sensitive systems should emit only the dense head.