Sparse Neural Retrieval with SPLADE++

How to teach keyword search to understand what you actually mean


Imagine you're building a search engine for an online grocery store.

A user searches for “parmesan.”

Your search engine looks through thousands of products and returns… nothing.

That sounds ridiculous. The catalog contains multiple products that are clearly related:

A human would immediately recognize that these are all reasonable matches for “parmesan.”

But a traditional keyword search engine doesn't think like a human. It only sees words. And none of those documents contain the exact word “parmesan.”

This is one of the oldest problems in information retrieval: the vocabulary mismatch problem. The user and the document are talking about the same thing, but they're using different words. And the search engine, for all its speed, cannot bridge that gap.


The old way: fast but limited

For decades, search systems relied on methods like BM25. These systems are incredibly fast, easy to explain, and surprisingly effective. You type a word, they find that word. Simple.

But they rank documents using word statistics, not meaning. They know how often a word appears. They don't know what a word means in context.

“Parmesan” and “grated hard cheese” share zero words in common. So to BM25, they might as not exist. The search returns nothing. The user leaves. The grocery store loses a sale.

This is the fundamental limitation of classical keyword search: it matches strings, not ideas.


The expensive workaround

Modern search systems solve this with a two-stage approach. First, BM25 grabs a broad set of candidate documents. Then, a heavy neural model called a reranker reads each candidate carefully and re-sorts them by relevance.

This works. But it's expensive. Every query means running a large model hundreds of times. At scale, the compute costs add up fast. And you still need to maintain two separate systems.

As search systems became more sophisticated, researchers started asking a simple question:

What if keyword search could understand context without giving up the efficiency of sparse retrieval?

What if the first step, the cheap retrieval step, was already smart enough?


The idea behind sparse neural retrieval

Here's the intuition. Instead of assigning word weights based purely on how often a word appears in a document, what if we used a machine learning model to assign those weights? A model that actually understands what words mean in context.

The output would still look like a traditional bag of words, a sparse vector where most entries are zero and only a few words have weight. But those weights would be smarter. They'd carry meaning, not just frequency counts.

This question eventually led to a family of techniques known as sparse neural retrieval. And one model in particular became the poster child of the approach.

Its name is SPLADE.


What is SPLADE?

SPLADE stands for Sparse Lexical AnD Expansion Model. It's a mouthful, but the idea is straightforward.

SPLADE takes BERT, a language model trained to understand context by guessing missing words, and repurposes it to produce smart, sparse term weights. It was built by researchers at NAVER Labs and works best with English text out of the box.

What makes SPLADE special is that it doesn't just assign weights to words already in the text. It expands the text with new words that fit the context, even if those words never appeared originally.

Query Expansion

Q: “cheese”  →  [cheese, dairy, food, dish, …]

Document Expansion

D: “Mac and cheese”  →  [mac, cheese, restaurant, brand, …]

This expansion is exactly what solves our parmesan problem. When the document “grated hard cheese” gets indexed, SPLADE expands it to include words like “parmesan,” “dairy,” “cheddar,” “grated.” Later, when someone searches for “parmesan,” that query also gets expanded. The two now share enough vocabulary to find each other.

“SPLADE doesn't just represent what a document says. It represents what a document means: by bringing in all the words that could have been used.”

How does SPLADE actually work?

Let's walk through what happens inside the model. It's less complicated than it sounds.

SPLADE pipeline Flowchart of six steps: tokenize text, encode with BERT, score vocabulary fit at each position, assign importance weights, combine into a sparse vector, and store in an inverted index. 1. Tokenize Break text into ~30k subword tokens 2. Encode with BERT Build contextual meaning per token 3. Score vocabulary fit At each position, score all 30k words 4. Assign importance weights Non-fitting words drop to zero 5. Combine into sparse vector ~30k dims, only a few dozen nonzero 6. Store in inverted index Same infra as BM25, smarter words Reading the text Expanding meaning Packing for search

Step 1: Turn text into tokens

First, SPLADE breaks your text into small pieces called tokens using a fixed vocabulary of about 30,000 entries. The word “cheese” becomes token 8808. “Mac” becomes token 6097. This is standard stuff, similar to what any language model does.

Step 2: Read the sentence in context

The tokens pass through BERT's encoder, which builds a rich understanding of each word based on the words around it. At this stage, the model knows that in “Mac and cheese,” the word “Mac” refers to food, not a computer. This contextual awareness is the foundation for everything that follows.

Step 3: At each position, ask what else could fit here

This is where the magic happens. BERT was trained to predict masked words in a sentence. So at each position in the text, the model looks at the context and scores every word in its 30,000-word vocabulary for how well it fits.

At the position where “cheese” appears, it considers words like “dairy,” “food,” “restaurant,” “milk,” “grated.” At the position where “parmesan” would naturally fit (had it been written), the model recognizes the affinity.

Words that don't fit get a score of zero and are discarded. Only the fitting ones survive.

“Grated hard cheese” grated shredded sliced ground hard aged firm mature cheese parmesan dairy cheddar
At each position, the model imagines other words that could fit. “Parmesan” naturally appears as a fit for the “cheese” position.

Step 4: Assign importance scores

Not every candidate word is equally important. The model assigns a weight to each. “Parmesan” at the “cheese” position might score very high. “Cheddar” also fits, but slightly lower. Words that barely fit get low weights. Words that don't fit at all get zero and vanish from the representation.

Step 5: Combine everything into one sparse vector

The scores are aggregated across all positions. If “dairy” comes up as a good fit at multiple positions, all those scores combine. The result is a single sparse vector of roughly 30,000 entries, where only a few dozen have non-zero values. These are the words that represent the document, including words that were never in the original text.

Step 6: Store it in a standard inverted index

Here's the clever part. Even though this vector is 30,000 dimensions wide, it's stored exactly like a traditional search index. For each word with a non-zero weight, the document ID gets added to that word's posting list. When a query comes in, the engine looks up the query's words, finds matching documents, and scores them using the weights. Same infrastructure as BM25. Smarter vocabulary.

“SPLADE upgrades the vocabulary without changing the engine. Every optimization that made BM25 fast still applies.”

Back to our grocery store

Let's revisit the parmesan problem with SPLADE in the picture.

At index time, each product description went through SPLADE and got expanded:

At query time, the user types “parmesan.” SPLADE expands it: parmesan, cheese, dairy, grated, italian, food, topping, hard, aged.

Query and document expansion bridging vocabulary mismatch A query for "parmesan" and a document containing "grated hard cheese" each get expanded by SPLADE into related terms. The expanded sets overlap on words like parmesan, cheese, and dairy, allowing a match that plain keyword search would miss. Query: "parmesan" Doc: "grated hard cheese" SPLADE expands SPLADE expands Expanded query terms parmesan cheese dairy grated italian food Expanded doc terms grated hard cheese parmesan dairy aged Shared terms found Document now matches the query

Now when the search runs, all three products match. They share expanded terms like “parmesan,” “dairy,” “cheese,” “grated” with the query. The products that were invisible to BM25 are now visible and ranked correctly. The user finds what they need. The store makes the sale.


The next version: SPLADE++

The original SPLADE was already a breakthrough. But a year later, the same team released SPLADE++ with three improvements that pushed quality significantly higher.

A better starting point

SPLADE++ starts from a stronger base model called CoCondenser. Think of it as BERT that did extra homework: additional training that made it especially good at understanding relationships between sentences. Starting from this stronger foundation, SPLADE++ immediately performed better.

Learning from a teacher

SPLADE++ uses a technique called knowledge distillation. A large, slow, very accurate model scores millions of query-document pairs, capturing not just which documents are relevant but how relevant they are relative to each other. SPLADE++ then trains to mimic those scores. It's like learning to cook by watching a master chef, not just reading recipes. The student never surpasses the master, but it gets much closer than it would on its own.

The team tried two approaches. SelfDistil, where one SPLADE model teaches another. And EnsembleDistil, where multiple teachers combine their knowledge. EnsembleDistil performed better: two teachers know more than one.

Harder training examples

Most models train on easy negatives, documents that are clearly irrelevant. But real-world search is mostly ambiguous. SPLADE++ introduced hard negative mining: it deliberately picks documents that look almost relevant but aren't, and trains the model to tell the difference. Like a pilot training on simulated emergencies instead of perfect weather.


Where SPLADE++ has trouble

No model is perfect. SPLADE++ has a few quirks worth knowing about.

Tokens aren't words

SPLADE works with subword tokens, not whole words. “Parmesan” gets broken into “par” + “##mes” + “##an.” “Grated” becomes “gr” + “##ated.” This makes it harder to look at a match and immediately understand why it happened. You're reading pieces of words, not the words themselves.

Expansion sometimes overreaches

Because SPLADE++ expands aggressively, it can add words that don't feel relevant. “Mac and cheese” might get expanded with “apple” or “brand”, words that relate statistically but not meaningfully. This makes results harder to explain and the vectors heavier.

The fix is fine-tuning. If you train SPLADE++ on your own domain (say, grocery products) it learns that “mac” expands to “pasta” and “dinner,” not “computer.” Fine-tuning aligns the expansion with your world.


Alternatives to SPLADE++

SPLADE++ is the most famous sparse neural retriever, but it's not the only game in town. Qdrant has open-sourced two alternatives that take a different approach.

miniCOIL is my current recommendation for most new projects. It behaves like BM25 but with contextual ranking. For the query “apple slicer,” miniCOIL ranks “apple cutter” higher than “apple charger” because it understands “apple” as a fruit in context. It only matches exact words, so you keep full explainability but the ranking of those matches reflects meaning.

BM42 is a lighter alternative built on the BM25 formula itself, using attention weights from a transformer instead of raw term frequency.

When to use miniCOIL: when you need exact keyword matches (common in e-commerce, legal, medical) but want contextual ranking. When not to: when queries and relevant documents share no keywords at all. In that case, SPLADE's synonym expansion is exactly what you need.


Which one fits your use case?

BM25 miniCOIL SPLADE
How it matches Exact words Exact words Words + synonyms
Works out of the box Any language, any topic English mostly English mostly; needs fine-tuning for specialized domains
Can you explain results? Yes, easily Mostly yes Harder without fine-tuning
Speed Instant Fast Moderate
Vector size Very compact Compact Larger (~8x BM25)
Best for Simple, fast, any domain BM25 upgrade with smarter ranking When synonym matching matters
Example: “apple slicer” “apple charger” and “apple cutter” ranked equally “apple cutter” ranked higher (understands fruit) Can match “fruit peeler” with no keyword overlap

Wrapping up

Let's come back to where we started. The user searching for “parmesan.” The store selling “grated hard cheese.” The search engine returning nothing.

This gap, between what users say and what documents contain, is the central challenge of information retrieval. Classical keyword search ignores it. Dense retrieval solves it but brings infrastructure baggage. Rerankers work but add cost and complexity.

Sparse neural retrieval offers a different path. It keeps the speed, transparency, and infrastructure of keyword search while adding semantic awareness through learned term expansion. The documents and queries get expanded with contextually fitting words at index time, so the matching happens on meaning, not just exact strings.

SPLADE++ is the most proven model in this space. It's not perfect: the token-level operations and occasional over-expansion can be frustrating. But for most real-world search needs, it strikes a balance that nothing else quite matches: semantically aware, fast, cheap, and transparent.

And that parmesan problem? With SPLADE++, the user types “parmesan,” the search engine finds “grated hard cheese,” and everyone walks away happy.

If you want to go deeper, start with the SPLADE++ paper. Pre-trained models are available on HuggingFace