How to teach keyword search to understand what you actually mean
Imagine you're building a search engine for an online grocery store.
A user searches for “parmesan.”
Your search engine looks through thousands of products and returns… nothing.
That sounds ridiculous. The catalog contains multiple products that are clearly related:
A human would immediately recognize that these are all reasonable matches for “parmesan.”
But a traditional keyword search engine doesn't think like a human. It only sees words. And none of those documents contain the exact word “parmesan.”
This is one of the oldest problems in information retrieval: the vocabulary mismatch problem. The user and the document are talking about the same thing, but they're using different words. And the search engine, for all its speed, cannot bridge that gap.
For decades, search systems relied on methods like BM25. These systems are incredibly fast, easy to explain, and surprisingly effective. You type a word, they find that word. Simple.
But they rank documents using word statistics, not meaning. They know how often a word appears. They don't know what a word means in context.
“Parmesan” and “grated hard cheese” share zero words in common. So to BM25, they might as not exist. The search returns nothing. The user leaves. The grocery store loses a sale.
This is the fundamental limitation of classical keyword search: it matches strings, not ideas.
Modern search systems solve this with a two-stage approach. First, BM25 grabs a broad set of candidate documents. Then, a heavy neural model called a reranker reads each candidate carefully and re-sorts them by relevance.
This works. But it's expensive. Every query means running a large model hundreds of times. At scale, the compute costs add up fast. And you still need to maintain two separate systems.
As search systems became more sophisticated, researchers started asking a simple question:
What if keyword search could understand context without giving up the efficiency of sparse retrieval?
What if the first step, the cheap retrieval step, was already smart enough?
Here's the intuition. Instead of assigning word weights based purely on how often a word appears in a document, what if we used a machine learning model to assign those weights? A model that actually understands what words mean in context.
The output would still look like a traditional bag of words, a sparse vector where most entries are zero and only a few words have weight. But those weights would be smarter. They'd carry meaning, not just frequency counts.
This question eventually led to a family of techniques known as sparse neural retrieval. And one model in particular became the poster child of the approach.
Its name is SPLADE.
SPLADE stands for Sparse Lexical AnD Expansion Model. It's a mouthful, but the idea is straightforward.
SPLADE takes BERT, a language model trained to understand context by guessing missing words, and repurposes it to produce smart, sparse term weights. It was built by researchers at NAVER Labs and works best with English text out of the box.
What makes SPLADE special is that it doesn't just assign weights to words already in the text. It expands the text with new words that fit the context, even if those words never appeared originally.
Q: “cheese” → [cheese, dairy, food, dish, …]
D: “Mac and cheese” → [mac, cheese, restaurant, brand, …]
This expansion is exactly what solves our parmesan problem. When the document “grated hard cheese” gets indexed, SPLADE expands it to include words like “parmesan,” “dairy,” “cheddar,” “grated.” Later, when someone searches for “parmesan,” that query also gets expanded. The two now share enough vocabulary to find each other.
Let's walk through what happens inside the model. It's less complicated than it sounds.
First, SPLADE breaks your text into small pieces called tokens using a fixed vocabulary of about 30,000 entries. The word “cheese” becomes token 8808. “Mac” becomes token 6097. This is standard stuff, similar to what any language model does.
The tokens pass through BERT's encoder, which builds a rich understanding of each word based on the words around it. At this stage, the model knows that in “Mac and cheese,” the word “Mac” refers to food, not a computer. This contextual awareness is the foundation for everything that follows.
This is where the magic happens. BERT was trained to predict masked words in a sentence. So at each position in the text, the model looks at the context and scores every word in its 30,000-word vocabulary for how well it fits.
At the position where “cheese” appears, it considers words like “dairy,” “food,” “restaurant,” “milk,” “grated.” At the position where “parmesan” would naturally fit (had it been written), the model recognizes the affinity.
Words that don't fit get a score of zero and are discarded. Only the fitting ones survive.
Not every candidate word is equally important. The model assigns a weight to each. “Parmesan” at the “cheese” position might score very high. “Cheddar” also fits, but slightly lower. Words that barely fit get low weights. Words that don't fit at all get zero and vanish from the representation.
The scores are aggregated across all positions. If “dairy” comes up as a good fit at multiple positions, all those scores combine. The result is a single sparse vector of roughly 30,000 entries, where only a few dozen have non-zero values. These are the words that represent the document, including words that were never in the original text.
Here's the clever part. Even though this vector is 30,000 dimensions wide, it's stored exactly like a traditional search index. For each word with a non-zero weight, the document ID gets added to that word's posting list. When a query comes in, the engine looks up the query's words, finds matching documents, and scores them using the weights. Same infrastructure as BM25. Smarter vocabulary.
Let's revisit the parmesan problem with SPLADE in the picture.
At index time, each product description went through SPLADE and got expanded:
At query time, the user types “parmesan.” SPLADE expands it: parmesan, cheese, dairy, grated, italian, food, topping, hard, aged.
Now when the search runs, all three products match. They share expanded terms like “parmesan,” “dairy,” “cheese,” “grated” with the query. The products that were invisible to BM25 are now visible and ranked correctly. The user finds what they need. The store makes the sale.
The original SPLADE was already a breakthrough. But a year later, the same team released SPLADE++ with three improvements that pushed quality significantly higher.
SPLADE++ starts from a stronger base model called CoCondenser. Think of it as BERT that did extra homework: additional training that made it especially good at understanding relationships between sentences. Starting from this stronger foundation, SPLADE++ immediately performed better.
SPLADE++ uses a technique called knowledge distillation. A large, slow, very accurate model scores millions of query-document pairs, capturing not just which documents are relevant but how relevant they are relative to each other. SPLADE++ then trains to mimic those scores. It's like learning to cook by watching a master chef, not just reading recipes. The student never surpasses the master, but it gets much closer than it would on its own.
The team tried two approaches. SelfDistil, where one SPLADE model teaches another. And EnsembleDistil, where multiple teachers combine their knowledge. EnsembleDistil performed better: two teachers know more than one.
Most models train on easy negatives, documents that are clearly irrelevant. But real-world search is mostly ambiguous. SPLADE++ introduced hard negative mining: it deliberately picks documents that look almost relevant but aren't, and trains the model to tell the difference. Like a pilot training on simulated emergencies instead of perfect weather.
No model is perfect. SPLADE++ has a few quirks worth knowing about.
SPLADE works with subword tokens, not whole words. “Parmesan” gets broken into “par” + “##mes” + “##an.” “Grated” becomes “gr” + “##ated.” This makes it harder to look at a match and immediately understand why it happened. You're reading pieces of words, not the words themselves.
Because SPLADE++ expands aggressively, it can add words that don't feel relevant. “Mac and cheese” might get expanded with “apple” or “brand”, words that relate statistically but not meaningfully. This makes results harder to explain and the vectors heavier.
The fix is fine-tuning. If you train SPLADE++ on your own domain (say, grocery products) it learns that “mac” expands to “pasta” and “dinner,” not “computer.” Fine-tuning aligns the expansion with your world.
SPLADE++ is the most famous sparse neural retriever, but it's not the only game in town. Qdrant has open-sourced two alternatives that take a different approach.
miniCOIL is my current recommendation for most new projects. It behaves like BM25 but with contextual ranking. For the query “apple slicer,” miniCOIL ranks “apple cutter” higher than “apple charger” because it understands “apple” as a fruit in context. It only matches exact words, so you keep full explainability but the ranking of those matches reflects meaning.
BM42 is a lighter alternative built on the BM25 formula itself, using attention weights from a transformer instead of raw term frequency.
When to use miniCOIL: when you need exact keyword matches (common in e-commerce, legal, medical) but want contextual ranking. When not to: when queries and relevant documents share no keywords at all. In that case, SPLADE's synonym expansion is exactly what you need.
| BM25 | miniCOIL | SPLADE | |
|---|---|---|---|
| How it matches | Exact words | Exact words | Words + synonyms |
| Works out of the box | Any language, any topic | English mostly | English mostly; needs fine-tuning for specialized domains |
| Can you explain results? | Yes, easily | Mostly yes | Harder without fine-tuning |
| Speed | Instant | Fast | Moderate |
| Vector size | Very compact | Compact | Larger (~8x BM25) |
| Best for | Simple, fast, any domain | BM25 upgrade with smarter ranking | When synonym matching matters |
| Example: “apple slicer” | “apple charger” and “apple cutter” ranked equally | “apple cutter” ranked higher (understands fruit) | Can match “fruit peeler” with no keyword overlap |
Let's come back to where we started. The user searching for “parmesan.” The store selling “grated hard cheese.” The search engine returning nothing.
This gap, between what users say and what documents contain, is the central challenge of information retrieval. Classical keyword search ignores it. Dense retrieval solves it but brings infrastructure baggage. Rerankers work but add cost and complexity.
Sparse neural retrieval offers a different path. It keeps the speed, transparency, and infrastructure of keyword search while adding semantic awareness through learned term expansion. The documents and queries get expanded with contextually fitting words at index time, so the matching happens on meaning, not just exact strings.
SPLADE++ is the most proven model in this space. It's not perfect: the token-level operations and occasional over-expansion can be frustrating. But for most real-world search needs, it strikes a balance that nothing else quite matches: semantically aware, fast, cheap, and transparent.
And that parmesan problem? With SPLADE++, the user types “parmesan,” the search engine finds “grated hard cheese,” and everyone walks away happy.
If you want to go deeper, start with the SPLADE++ paper. Pre-trained models are available on HuggingFace