The integration of LEMUR into txtai addresses a practical problem in late-interaction retrieval models: these models retain one vector for each token in a query or document, giving them more precise detail than representing each text with a single vector, but making them more difficult to use with traditional dense-search indexes. txtai provides a pathway that converts this multi-vector representation into a fixed-dimensional vector that can be searched using the same type of index, with the addition of a tunable option called mean centering to make better use of directional differences in token vectors when they are highly similar.
The material, published on August 13, 2026, on the Hugging Face blog as a community article written by Morgan Carr, reviews the motivation behind the change, specific experiment results, and how to train and load the LEMUR artifact in txtai. It does not present the result as a general ranking among retrieval techniques, since the comparisons were limited to one model, three datasets, one device, and exact search.
From Multiple Vectors to an Indexable Vector
In late-interaction search, every query token is compared with document tokens using MaxSim: the strongest match for each query token is retained and these values are then summed. LEMUR, or Learned Multi-Vector Retrieval, learns a fixed-dimensional encoding that attempts to approximate the result of this interaction while preserving the ability to use a traditional vector index.
The LEMUR artifact in txtai consists of a feature encoder, output-normalization statistics, and a sample of token vectors. During inference, the query is converted into a sum of learned features, while the document is represented by a set of ordinary least-squares weights over the stored sample. The inner product between the two fixed vectors becomes an approximation of the original late-interaction score.
The resulting artifact is specific to the dataset used for training and must be trained before loading an embeddings index in txtai. The artifact is saved as config.json and model.safetensors. The epochs=100 setting enables the quality-oriented MLP path, while epochs=0 selects deterministic random ELM features as a lower-cost alternative.
Training-Data Selection Was a Decisive Factor
An ablation experiment on the nfcorpus dataset showed that the vector distribution used to learn the feature map had a greater effect than some other training choices. When the MLP was trained on data-encoder token vectors, NDCG@10 reached 0.15870, lower than the 0.19187 result from the untrained ELM. When query-encoder vectors were used, the result rose to 0.24868, then reached 0.25534 after selecting the number of epochs based on validation.
According to the analysis in the material, choosing the learning distribution accounted for 93% of the measured improvement between the data-vector experiment and the final result, while epoch selection contributed the remaining 7% improvement. Therefore, LemurTrainer defaults learncategory to query, while allowing data to be selected or a separate training set to be passed.
What Did the Comparative Test Show?
The comparison used the colbert-ir/colbertv2.0 model, an NVIDIA GeForce RTX 4080 SUPER, torch 2.13.0+cu130, and exact Faiss search using IDMap,Flat. It compared the default MUVERA version with a width of 10,240 dimensions, a MUVERA version reduced to 2,048 dimensions, and LEMUR with a 2,048-dimensional MLP encoding.
- On nfcorpus, LEMUR scored 0.25524, compared with 0.16299 for MUVERA at the same size and 0.23544 for MUVERA at its default size.
- On scifact, LEMUR scored 0.54910, compared with 0.36757 for reduced MUVERA and 0.50021 for the default version.
- On arguana, LEMUR scored 0.42556, compared with 0.26280 for reduced MUVERA and 0.34614 for the default version.
This corresponds to improvements over same-size MUVERA of 56.6% on nfcorpus, 49.4% on scifact, and 61.9% on arguana. Compared with the default MUVERA vector, the improvements were 8.4%, 9.8%, and 22.9%, respectively. The LEMUR indexes used in the measurement also occupied one-fifth the space of the default MUVERA indexes because vector width was reduced from 10,240 to 2,048 dimensions.
However, the scope of these figures matters: the fiqa and scidocs experiments were not completed from the required dataset, and the measurements relied on one model, one device, and exact search. txtai uses exact search up to 5,000 rows, then switches to an IVF index. In the scifact experiment, the default IVF reduced LEMUR's NDCG@10 by 43% compared with exact search, versus a 25% decrease for MUVERA. Therefore, it cannot be assumed that the exact-search result will remain the same at larger scale or when approximate search is used.
Why Was Mean Centering Added?
When moving from ColBERTv2 to lightonai/LateOn, the test found that the token vectors were highly directionally similar. In a sample of 5,000 pairs, the average cosine similarity between vectors was 0.9508, and the spread of MaxSim was 0.0559. After subtracting the group mean and renormalizing, similarity fell to 0.0033 and the MaxSim spread rose to 0.4772. This is not a retrieval metric by itself, but it indicates that centering gave the fixed-dimensional encoder a larger directional space to capture.
Retrieval tests confirmed that the benefit depends on the model. In the LateOn matrix, batch-level centering was stronger than disabling it or using the collection mean in all four measured cells. For example, the LEMUR result on nfcorpus rose from 0.00000 without centering to 0.33309 with batch centering, while on scifact it rose from 0.04985 to 0.69016. However, experiments on ColBERTv2 showed that centering may be neutral or harmful for MUVERA, so it was not presented as an unconditional global option.
The default rule in the merged changes enables batch centering when the loaded model contains more than one linear layer from torch.nn.Linear. Models with zero or one layer retain their previous behavior, with the option to override the decision manually. Centering occurs after normalizing token vectors and before LEMUR or MUVERA, followed by another normalization step. The scope can be set to document, batch, or collection; collection scope also accepts an embedded mean or a Safetensors file containing center.mean.
What Changes for the User in Practice?
Using LEMUR requires a separate training step before creating the index, and vector settings during training must be consistent with the loading settings. The published example uses center: true with a LEMUR artifact and pins Faiss to IDMap,Flat when exact search is practical. The latest released version of txtai in the material is v9.12.0, while the merged changes target v9.13.0; to try the source before the release, the master version of the GitHub repository can be installed in an isolated environment.
The practical conclusion is not that LEMUR or centering outperform in every case, but that txtai now provides two separate tools for two different problems: LEMUR reduces a late-interaction representation to a fixed vector, while centering addresses token-vector geometry that is unsuitable for some models. Broader testing across models, datasets, and tuned approximate indexes is still needed before the results can be generalized.