The integration of LEMUR into txtai addresses a practical problem in late-interaction retrieval models: these models retain one vector for each token in a query or document, giving them finer detail than representing each text with a single vector, but making them more difficult to use with traditional dense-search indexes. txtai provides a path that converts this multi-vector representation into a fixed-dimensional vector that can be searched using the same type of index, while adding a tunable option called mean centering to better exploit directional differences in token vectors when they are highly similar.
The material, published on August 13, 2026, on the Hugging Face blog as a community article written by Morgan Carr, discusses the motivation for the change, specific experiment results, and how to train and load a LEMUR artifact in txtai. It does not present the result as a general ranking among retrieval techniques, since the comparisons were limited to one model, three datasets, one device, and exact search.
From Multiple Vectors to an Indexable Vector
In late-interaction search, every query token is compared with document tokens using MaxSim: the strongest match for each query token is retained, and these values are then summed. LEMUR, or Learned Multi-Vector Retrieval, learns a fixed-dimensional encoding that attempts to approximate the result of this interaction while preserving the ability to use a traditional vector index.
The LEMUR artifact in txtai consists of a feature encoder, output-normalization statistics, and a sample of token vectors. During inference, the query is converted into a sum of learned features, while the document is represented by a set of ordinary least-squares weights over the stored sample. The inner product between the two fixed vectors then approximates the original late-interaction score.
The resulting artifact is specific to the dataset used for training and must be trained before loading an embeddings index in txtai. The artifact is also saved as config.json and model.safetensors. The epochs=100 setting enables the quality-oriented MLP path, while epochs=0 selects deterministic random ELM features as a lower-cost alternative.
Training-Data Selection Was a Decisive Factor
An ablation study on the nfcorpus dataset showed that the vector distribution used to learn the feature map had a greater effect than some other training choices. When the MLP was trained on encoder-token vectors, it achieved an NDCG@10 of 0.15870, lower than the untrained ELM result of 0.19187. When query-encoder vectors were used, the score rose to 0.24868, then reached 0.25534 after selecting the number of epochs based on validation.
According to the analysis in the material, the choice of learning distribution explained 93% of the measured improvement between the data-vector experiment and the final result, while the choice of epochs contributed the remaining 7% improvement. Therefore, LemurTrainer defaults learncategory to query, while allowing data to be selected or a separate training set to be passed.
What Did the Comparative Test Show?
The comparison used the colbert-ir/colbertv2.0 model, an NVIDIA GeForce RTX 4080 SUPER, torch 2.13.0+cu130, and exact Faiss search through IDMap,Flat. The default MUVERA version with a width of 10,240 dimensions was compared with a MUVERA version reduced to 2,048 dimensions and with LEMUR using an MLP encoding with a width of 2,048 dimensions.
- On nfcorpus, LEMUR scored 0.25524, compared with 0.16299 for MUVERA at the same size and 0.23544 for MUVERA at its default size.
- On scifact, LEMUR scored 0.54910, compared with 0.36757 for reduced MUVERA and 0.50021 for the default version.
- On arguana, LEMUR scored 0.42556, compared with 0.26280 for reduced MUVERA and 0.34614 for the default version.
This corresponds to improvements over MUVERA at the same size of 56.6% on nfcorpus, 49.4% on scifact, and 61.9% on arguana. Compared with the default MUVERA vector, the improvements were 8.4%, 9.8%, and 22.9%, respectively. The LEMUR indexes used in the measurement also occupied one-fifth the space of the default MUVERA indexes because the vector width was reduced from 10,240 to 2,048 dimensions.
However, the scope of these figures matters: the fiqa and scidocs experiments were not completed because the required dataset was unavailable, and the measurements used one model, one device, and exact search. txtai uses exact search up to 5,000 rows, then switches to an IVF index. In the scifact experiment, the default IVF reduced LEMUR’s NDCG@10 by 43% compared with exact search, versus a 25% decrease for MUVERA. Therefore, it cannot be assumed that the exact-search result will remain the same when scaling up or using approximate search.
Why Was Mean Centering Added?
When moving from ColBERTv2 to lightonai/LateOn, the test found that the token vectors were highly directionally symmetric. In a sample of 5,000 pairs, the average cosine similarity between vectors was 0.9508, and the spread of MaxSim was 0.0559. After subtracting the dataset mean and renormalizing, similarity fell to 0.0033 and the MaxSim spread rose to 0.4772. This is not itself a retrieval metric, but it indicates that centering gave the fixed-dimensional encoder a larger directional space to capture.
Retrieval tests confirmed that the benefit depends on the model. In the LateOn matrix, batch-level centering was stronger than disabling it or using the collection mean in all four measured cells. For example, LEMUR’s score on nfcorpus rose from 0.00000 without centering to 0.33309 with batch centering, while on scifact it rose from 0.04985 to 0.69016. However, experiments on ColBERTv2 showed that centering can be neutral or harmful for MUVERA, so it was not presented as an unconditional global option.
The default rule in the merged changes enables batch centering when the loaded model contains more than one torch.nn.Linear layer. Models with zero or one layer retain their previous behavior, with the option to override the decision manually. Centering occurs after normalizing the token vectors and before LEMUR or MUVERA, followed by another normalization operation. The scope can be set to document, batch, or collection; the collection scope also accepts an embedded mean or a Safetensors file containing center.mean.
What Changes for the User in Practice?
Using LEMUR requires a separate training step before creating the index, and the vector settings during training must be consistent with the loading settings. The published example uses center: true with a LEMUR artifact and pins Faiss to IDMap,Flat when exact search is practical. The latest released version of txtai in the material is v9.12.0, while the merged changes target v9.13.0; to try the source before the release, the master version of the GitHub repository can be installed in an isolated environment.
The practical conclusion is not that LEMUR or centering outperform in every case, but that txtai now provides two separate tools for two different problems: LEMUR reduces a late-interaction representation to a fixed vector, while centering addresses unsuitable token-vector geometry in some models. Broader testing across models, datasets, and tuned approximate indexes is still needed before the results can be generalized.