JetBrains has revealed engineering details about running the first version of Junie Local locally on an M5 MacBook using the Qwen3.6-27B model, explaining that achieving practical performance was not simply a matter of choosing a compressed model. It required modifications to the coding agent itself, the inference engine, the model settings, and the way context is managed.
The company says that running Junie entirely on-device, without cloud inference, is a long-term project targeting a broad range of hardware configurations. The first version currently available focuses on M5 MacBook devices, which explains why the announced optimizations focus on the characteristics of these chips.
Keeping Task Context Within the Session
Like other coding agents, Junie operates through an execution loop in which the user sends a task to the model. The model then executes tool calls such as Bash commands and reading and writing files before returning the results of those operations to the model. With each new request, the context expands, and the KV-cache data processed by the model in the previous request can be reused.
In cloud models, the agent can request a file again when needed because the prefill stage—that is, processing the initial context before generating the response—is relatively fast. JetBrains found, however, that reading files is time-consuming in local models. It therefore modified the local inference logic so that each new request is added directly to the running context, rather than retaining only the parts considered relevant to the new task. This keeps the file read by the model within the context, allowing KV-cache memory to be reused instead of processing it again.
The company also changed the order of the data Junie sends when starting a new coding session and added logic to the inference engine to cache the prefix up to the user request. As a result, this prefix can be reused in subsequent tasks within the same project. The project context that comes after the user request was not cached in the same way because it is relatively small and consists mostly of top-level files that may change frequently.
Modifications Related to Model Behavior
Qwen3.6 did not handle progress updates in the way Junie expects from cloud models. It would usually ignore the XML-like block dedicated to status updates, but it would write a textual description of its actions alongside tool calls. JetBrains took advantage of this behavior and displayed the generated text as a user update. The company explains that this adaptation is model-specific, as other models may not print any text or may produce excessive text.
JetBrains also disabled optional requests from the model, including the logic that generates a short task description. The company believes that the limited compromise in the user experience is acceptable in exchange for reducing the number of requests. It also disabled multi-agent mode because sequential processing is more efficient on an M5 device according to its tests, while parallel requests would remain constrained by inference speed.
Why Was Qwen3.6-27B Chosen?
JetBrains decided to disable reasoning entirely in the local version. According to its internal tests of the cloud version of Qwen3.6-27B, enabling reasoning did not produce a significant increase in quality. Since reasoning tokens count among the tokens generated by the inference engine, disabling them reduced the required number of tokens by roughly two- to threefold. The company translated this into an approximately twofold speedup in task execution, with what it described as an insignificant impact on quality.
The company used the 4-bit quantized version because it was only slightly worse than the 8-bit version in benchmark tests, and because memory-bound token generation was roughly twice as fast as with the 8-bit version. However, prefill speed did not differ between the 4-bit, 8-bit, and 16-bit versions in the initial tests, prompting the team to examine the computations inside the engine.
The Invisible Bottleneck: The Prefill Stage
According to JetBrains' figures, prefill speed can reach approximately 3,700 tokens per second on an RTX 5090 under default settings, compared with approximately 650 tokens per second on an M5 before optimization. In tasks involving investigation within files, most of the time was spent processing context rather than generating the response itself.
The team found that a large portion of the matrix operations during prefill were performed at 16-bit precision, even when the weights were compressed to 4-bit, because the weights were converted to 16-bit before the operations were performed. Since the M5 has special instructions for 8-bit computation, JetBrains applied a patch to the MLX-VLM package to move some matrix operations in the self-attention layers to 8-bit, achieving an approximately 40% increase in prefill speed. The modification did not include the full-attention layers, whose weights remain at 16-bit precision even with quantization.
The company links its current focus on the M5 specifically to these computational instructions. M4 chips do not have them, and JetBrains says that 16-bit computations on the M4 are 20% to 30% slower during prefill than on the M5.
Accelerating Generation and Choosing the Model
JetBrains enabled two speculative decoding techniques together: multi-token prediction (MTP) using a separate draft model, and n-gram matching, which searches for repeated sequences in the context to predict the next tokens. In some cases, approximately three proposed tokens through MTP and up to eight additional tokens through n-gram matching can be accepted, giving generation a speedup of up to twofold.
The company explains that Qwen3.8-27B was not the best option on Mac devices because it needs reasoning to perform well. When reasoning is disabled, quality deteriorates sharply, and the model may become stuck in a loop in which it repeatedly makes the same tool call. Enabling it at a medium level increases the number of generated tokens by approximately fivefold, which practically results in a slowdown of nearly fourfold because prefill time does not change much. Therefore, Qwen3.6-27B remains, for now, the most suitable option for running Junie locally on Mac hardware, according to JetBrains' assessment.
certi.news' Take: What Actually Changes?
This experiment shows that measuring a local coding agent solely by the number of tokens generated per second can provide an incomplete picture. The user also waits for files to load, context to be reused, and tool calls to be executed—stages that can become bottlenecks before generation even begins. In practice, the Junie modifications reduce the need to reread files, while the MLX-VLM patch increases context-processing speed on the M5, and disabling reasoning reduces the number of required tokens.
However, the limitations are also clear: the first version focuses on the M5, disabling reasoning may not suit models that rely on it to maintain quality, and the benefits of progress updates are tied to Qwen3.6's behavior. JetBrains says it has prototypes supporting DGX Spark and the RTX 5090, and is also considering cards with 24 GB of capacity, but the material provides no timeline or specifications for making these versions available. Junie Local therefore represents an important technical step toward more usable local coding agents, not evidence that the experience has become equivalent to all cloud models or is available on every type of hardware.