Hybrid Search for RAG: Combining Vector Search and BM25 with Query Expansion
Why vector search alone misses answers in professional domains, and the retrieval pipeline that fixes it
Most RAG tutorials stop at one step: embed the question, find the nearest chunks in a vector database, and send them to the LLM. That works for general questions. It breaks in professional domains, where an answer can hinge on an exact drug name, a rubric, a section number or a product code.
HomeoAI, an AI assistant used by more than 1,400 homeopathic practitioners, hit exactly this problem. Its builder, Tejesh Vaish, explained on the Codersarts Podcast how he solved it: expand each question into several queries, run both vector search and BM25 keyword search, merge the results, and let the LLM re-rank them before answering.
This post explains how hybrid search works, why it matters, how to merge results, and how to implement it in Python. It is written for AI engineers, CTOs and technical founders building RAG systems on specialist knowledge. For the full HomeoAI architecture, read How HomeoAI Was Built.
Why vector search alone isn't enough
Vector search converts text into embeddings and finds chunks with similar meaning. It is excellent at matching paraphrases: "can't tolerate cold" and "cold intolerance" land close together.
It is weaker in three situations that are common in professional knowledge:
Exact terms and identifiers. Drug names, remedy names, legal citations, error codes and part numbers carry meaning in their exact spelling. Embeddings can place a similar-looking but different term nearby.
Rare vocabulary. Specialist words appear rarely in the data embedding models were trained on, so their embeddings may be less precise.
Near-identical entries. In homeopathy, one symptom can map to a hundred medicines. In law or tax, many provisions use the same language with one decisive difference. Semantic similarity alone can't tell which one matters.
Keyword search has the opposite profile. It is precise on exact terms but misses paraphrases. Hybrid search runs both and combines the strengths.
How BM25 keyword search works
BM25 is the standard ranking function behind most keyword search engines. For each query term, it scores a document higher when:
The term appears often in the document (term frequency), with diminishing returns, so the tenth mention adds less than the first.
The term is rare across the whole collection (inverse document frequency), so a specialist term counts for much more than a common word.
The document is not unusually long (length normalisation), so long chunks don't win just by containing more words.
Two parameters control this: k1, which sets how quickly repeated terms stop adding score (commonly between 1.2 and 2.0), and b, which sets how strongly length is normalised (commonly 0.75). The defaults work well for most corpora.
Because BM25 rewards rare, exact terms, it catches precisely the matches that vector search tends to blur.
The hybrid retrieval pipeline
HomeoAI's retrieval service is a Python Flask API that the model calls as a tool. When a doctor enters a list of symptoms, it runs these steps:
Query expansion. The LLM rewrites the symptom list into five or six alternative queries covering different phrasings and interpretations.
Parallel vector search. Each query is embedded with Google's Gemini embedding model and searched across several Chroma collections at once.
BM25 keyword search. The same queries run through BM25 to catch exact terms.
Merge and de-duplicate. The top five results from vector search and the top five from BM25 are combined, and duplicates are removed.
Re-rank and answer. The LLM re-ranks the merged passages and writes the answer, with the source passages shown to the doctor.
Each step fixes a specific failure:
Step | Failure it prevents |
Query expansion | The user's single phrasing doesn't match how the source describes it |
Vector search | Paraphrases and descriptive language are missed |
BM25 | Exact terms, names and identifiers are missed |
Merge and de-duplicate | One method's blind spots dominate the results |
Re-ranking | Relevant but less useful passages crowd out the best ones |
Query expansion: one question, several searches
A user's question is one way of describing a need. The source document may describe it another way. Query expansion closes that gap by asking the LLM to generate alternative versions of the query before searching.
For a list of symptoms, the variants might group the symptoms differently, use the domain's formal terms, or focus on the most distinctive symptom. Each variant is searched separately, and the results are pooled.
Practical tips:
Keep the original query in the set, alongside four to six variants.
Ask for domain vocabulary in the rewrite prompt, so variants use the terms the source documents use.
Run the searches in parallel to keep latency down.
Offer a fast path. HomeoAI added a fast endpoint for quick answers and a deeper endpoint for detailed analysis, because more queries and re-ranking add time.
Merging results: three options
Vector search and BM25 produce scores on different scales, so you can't simply add them. There are three common ways to merge:
1. Take the top results from each and re-rank
This is HomeoAI's approach: take the top five from each method, remove duplicates and let the LLM re-rank. It is simple, and it works well when the final re-ranking step is strong.
2. Reciprocal rank fusion (RRF)
RRF ignores raw scores and uses rank positions. Each result gets 1 / (k + rank) from every list it appears in, and the totals are summed. A value of k = 60 is a common default. Results that rank well in several lists, including across expanded queries, rise to the top. RRF is robust and needs no tuning, which makes it our usual starting point.
3. Weighted score fusion
Normalise each method's scores to the same range, then combine them with weights, for example 0.6 for vector and 0.4 for BM25. It gives fine control but needs tuning against an evaluation set, and the weights may need adjusting when data changes.
Re-ranking: the final quality gate
Retrieval casts a wide net; re-ranking picks the best catch. Two common approaches:
LLM re-ranking: the LLM reads the candidate passages and orders them by relevance to the question. HomeoAI uses its main model (GPT-4.1) for this, as part of writing the answer.
Cross-encoder re-rankers: smaller models trained to score how well a passage answers a query. They are faster and cheaper per call than a large LLM.
Re-rank a manageable candidate set, typically 10 to 30 passages, then pass only the top few to the answer step. This keeps answers focused and keeps token costs down.
Python implementation sketch
This example shows the core pattern with BM25 from the rank_bm25 package, a Chroma collection and reciprocal rank fusion. It is a starting point, not production code.
from rank_bm25 import BM25Okapi
# corpus: list of chunks, e.g. {"id": "kent-123", "text": "...", "source": "Kent Repertory"}
tokenized = [c["text"].lower().split() for c in corpus]
bm25 = BM25Okapi(tokenized)
def expand_query(question, llm, n=5):
prompt = (
"Rewrite this query in different ways using the domain's formal terms. "
f"Return {n} rewrites, one per line.\n\nQuery: {question}"
)
rewrites = [line.strip() for line in llm(prompt).splitlines() if line.strip()]
return [question] + rewrites[:n]
def vector_search(query, collection, k=5):
# The collection must be created with your embedding function (e.g. Gemini embeddings)
result = collection.query(query_texts=[query], n_results=k)
return result["ids"][0]
def bm25_search(query, k=5):
scores = bm25.get_scores(query.lower().split())
top = sorted(range(len(scores)), key=lambda i: scores[i], reverse=True)[:k]
return [corpus[i]["id"] for i in top]
def reciprocal_rank_fusion(ranked_lists, k=60):
scores = {}
for ranked in ranked_lists:
for position, doc_id in enumerate(ranked):
scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + position + 1)
return sorted(scores, key=scores.get, reverse=True)
def hybrid_retrieve(question, collection, llm, top_n=10):
ranked_lists = []
for query in expand_query(question, llm):
ranked_lists.append(vector_search(query, collection))
ranked_lists.append(bm25_search(query))
return reciprocal_rank_fusion(ranked_lists)[:top_n]
# Next: re-rank these candidates with the LLM or a cross-encoder,
# then send the top passages and their sources to the answer step.
Before production, add:
A better tokenizer for BM25 than a simple split: one that handles punctuation, hyphenated terms and stop words in your domain.
Parallel execution of the searches, for example with a thread pool or async calls.
Metadata filters, such as a tenant ID on every chunk, so retrieval never returns another customer's data.
Source metadata on every result, so answers can cite the exact book, section or row.
Caching for repeated queries and for embeddings of common queries.
Measure it: does hybrid search actually help?
Don't assume hybrid search is better for your data; measure it. HomeoAI's evaluation came from a golden dataset of real queries rated by 34 doctors, and every retrieval change was tested against it. Changes that lowered the score were reverted.
For retrieval specifically, track:
Recall@k: how often the correct passage appears in the top k results.
Mean reciprocal rank (MRR): how high the first correct passage ranks, on average.
Answer accuracy: expert ratings of the final answers, which is what users actually feel.
Compare vector-only, BM25-only and hybrid on the same query set. In domains with exact terminology, hybrid usually wins on recall. Keep the comparison as a regression test whenever you change chunking, embeddings or merging.
Hybrid search checklist
Area | Question to ask | Good answer |
Failure analysis | Where does vector-only search miss? | Known failing queries with exact terms or near-identical entries |
Keyword search | Is BM25 running alongside vector search? | Both run on every query, with a domain-aware tokenizer |
Query expansion | Is the user's single phrasing the only query? | Original plus 4–6 rewrites in the domain's vocabulary |
Merging | How are differently scaled results combined? | RRF or top-k from each, then re-ranking |
Re-ranking | Does a final step order candidates by usefulness? | LLM or cross-encoder re-ranking on 10–30 candidates |
Latency | Is the extra work slowing answers? | Parallel searches, caching and a fast path for simple queries |
Isolation | Can retrieval return another tenant's documents? | Tenant ID metadata filtered on every search |
Citations | Can users see where each answer came from? | Source metadata returned and shown with the answer |
Evaluation | Is hybrid proven better on your data? | Recall@k, MRR and expert answer ratings on a golden set |
Build a RAG system that finds the right answer with Codersarts
Retrieval quality is where most RAG systems succeed or fail. Codersarts designs and fixes retrieval pipelines for specialist knowledge:
Hybrid retrieval design: vector and keyword search, query expansion, fusion and re-ranking tuned to your documents.
RAG audits: if your assistant misses obvious answers or returns the wrong passages, we find the failure points and fix them.
Evaluation setup: golden datasets and retrieval metrics so every change is measured.
Domain AI builds: complete assistants with cited answers, data isolation and production deployment.
Talk to us: https://www.codersarts.com/contact
Explore our AI services: https://ai.codersarts.com
FAQ
What is hybrid search in RAG?
Hybrid search runs semantic vector search and keyword search, usually BM25, on the same query and merges the results. Vector search finds passages with similar meaning, and keyword search catches exact terms, names and codes. Together they retrieve more of the right passages than either alone.
Is BM25 still useful with modern embedding models?
Yes. Embeddings are strong on meaning but can blur exact and rare terms, which are common in professional domains. BM25 rewards exactly those terms, so it complements embeddings rather than competing with them.
What is reciprocal rank fusion?
Reciprocal rank fusion merges ranked result lists using positions instead of raw scores. Each result earns 1 / (k + rank) from every list it appears in, commonly with k = 60, and the totals decide the final order. It needs no score normalisation or tuning.
What is query expansion in RAG?
Query expansion uses an LLM to rewrite a user's question into several alternative queries before searching. Each version is searched, and the results are pooled, so passages described in different words from the user's question are still found.
Does hybrid search make RAG slower?
It adds work, but running searches in parallel keeps the extra latency small. Re-ranking adds the most time. Offering a fast path for simple questions and a deeper path for complex ones, as HomeoAI does, balances speed and quality.
Can Codersarts improve the retrieval quality of our RAG app?
Yes. Codersarts audits and rebuilds RAG retrieval with hybrid search, query expansion, fusion, re-ranking and evaluation tuned to your data. Contact us at https://www.codersarts.com/contact.
Related reading:
Want to be a guest on the Codersarts Podcast? Email jitendra@codersarts.com with your name, role and the topic you'd like to talk about.
Comments