A fast local-model demo is not enough to make it useful in a product. It must generate tokens quickly, correctly interpret when to call a tool, and leave reliable signals so someone can measure what happened. Three successive llama.cpp releases on August 2 make that dependency tangible.

The project added support for a form of speculative decoding for DeepSeekV4, adjusted Qwen3 tool-call parsing, and fixed an issue that could prevent profiling data from being written by the OpenCL backend. They are different changes, but together they make the same point: performance, protocol, and observability are not separate layers once AI leaves the lab.

Acceleration means proposing and checking

The DeepSeekV4 MTP + DSpark change, released in build b10228, adds support for the DSpark head and early work for multi-token prediction (MTP), a speculative decoding technique. Rather than have the main model compute every next token alone, an auxiliary component proposes several tokens; the main model verifies them and accepts only those that agree.

The contributor’s own warning matters: the recent checkpoints referenced in the change do not ship MTP, only the DSpark head. MTP therefore should not be assumed to be available for every DeepSeekV4 model. For a compatible DSpark checkpoint, the author reported measurements on a DGX Spark that reduced the aggregate time for their nine-request set from 102.15 to 55.95 seconds. That is evidence for one configuration, not a universal performance promise.

This changes the operational question for local AI. It is not enough to ask whether speculative decoding is enabled. Teams need to confirm which checkpoint carries the needed component, how many drafts are accepted, and whether the workload retains that advantage. A proposal rejected by the main model still consumed work.

A tool call cannot rely on perfect punctuation

In build b10227, the specialized Qwen3 parser began recognizing a case maintainers had observed: some models can emit <tool_call> immediately after their reasoning section, without the closing delimiter the server expected. The update also handles older models that sometimes omit the opening marker and permits arguments in different orders.

This does not give the model new tools. It changes the bridge between text generated by the model and the structured action the software performs. Previously, a formatting variation could become plain text, break a valid call, or require stricter constraints in the grammar sampler. The parser can now accept more expected forms without treating the end of reasoning as a lost instruction.

There is an engineering consequence. Tool compatibility is not just a JSON function definition. It depends on control tokens, delimiters, grammar, and tolerance for model variation. Updating weights without testing this complete path tests only half of an agent.

Measurement is part of the deliverable

Build b10229 fixes reference counting during OpenCL initialization. Without the increment, the counter could end negative and skip part of cleanup, including writing profiling data when that feature was enabled. The fix is small, but its effect is recognizable: a run can appear to work while losing the record needed to understand its cost.

Profiling does not change the answer a model generates. It makes it possible to see where time went, compare versions, and decide whether an optimization actually moved the bottleneck. If that evidence disappears at program exit, the team has a performance impression rather than a reproducible measurement.

Together, these changes form a simple test for any local-inference update:

  1. Does the fast path work with the actual weights and hardware?
  2. Do structured outputs reach the layer that performs actions?
  3. Do metrics survive to the end of execution?

Mature local AI is not merely AI that produces a response quickly. It can propose work without wasting it, turn output into action without guessing the protocol, and leave enough evidence for the next decision to improve on the last.