Follow the latest coverage, related explainers and connected technology stories.
bartowski presents an extensive experiment to improve the calibration data used with imatrix when quantizing models through llama.cpp, concluding that the greatest gains appear at low bit rates, particularly in mixture-of-experts (MoE) models. The new dataset is smaller and more focused, but combines diverse texts with tool-use conversations formatted for tool calling.