Skip to content

Releases: ggml-org/llama.cpp

b10686

b10686 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 29 Aug 16:46
3173a56

b10685

b10685 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 29 Aug 16:23
17252c7

b10684

b10684 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 29 Aug 15:57
cc83d7b

sycl: make --fit respect --fit-target better (#27629)

improve the --fit algorithm to take into account the actual peak
required VRAM for a given context size on a SYCL backend.

This includes both properly accounting for how much VRAM is required
when the allocated context is fully used (which makes the reported
context drop below what it did before, but stop it OOMing) as well
as preventing some overly-conservative calculations which meant too much
VRAM was being reserved.

Tested on a Arc b70 with unsloth's qwen3.8 (Q4_K_XL), able to get 262144 context,
fully usable, with q8_0 KV and MTP and 4k ubatch size using --fit-target 1

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10683

b10683 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 29 Aug 15:25
c9ca51c

vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs (#27526)

  • vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs

  • remove one more fastdiv

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10682

b10682 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 29 Aug 15:01
5ea1b12

b10681

b10681 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 29 Aug 14:34
77f132c

vulkan: Change mul_mat_id to pad K rather than N (#27925)

The N padding is needed for mul_mat, but not mul_mat_id. For mul_mat_id,
we indirect the row index through a shared memory lookup table which avoids
any OOB row coordinate. But that callback doesn't bounds check K, so we
actually need K padding instead.

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10679

b10679 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 28 Aug 19:23
50f068f

b10678

b10678 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 28 Aug 18:51
6fe7498

b10677

b10677 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 28 Aug 18:27
b387ddf

vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize (#27812)

  • vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize

is_src_of doesn't treat two views of one tensor as dependent, so the optimizer reorders nodes across aliased reads and writes.

Result: silently wrong tokens under greedy decoding, different output on every server start, and invalid speculative-decoding acceptance, with nothing logged.

Hits Qwen3.8's recurrent state (and any model with view-aliased state) on AMD and NVIDIA Vulkan. CUDA is clean.

Compare view_src bases on both sides.

Fixes #27805

  • vulkan: don't treat view/no-op nodes as aliasing dependencies

Nodes whose op is NONE, RESHAPE, TRANSPOSE, VIEW or PERMUTE execute nothing, so aliasing through them is not a real dependency. The previous base comparison matched them anyway, which only costs the optimizer reordering freedom.

Co-authored-by: Jeff Bolz jbolz@nvidia.com

  • vulkan: make the lambda parameter const and capture is_empty in is_src_of

Code will not compile without these changes.
is_src_of has an empty capture list, so is_empty was not visible inside it, and is_empty took a non-const pointer, while is_src_of receives const ones. Other call sites pass non-const pointers, which still convert as usual.


Co-authored-by: Jeff Bolz jbolz@nvidia.com

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10676

b10676 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 28 Aug 18:01
a43c398

ggml : fix conv_transpose_2d for multiple batches (#26132)

  • ggml : fix conv_transpose_2d for multiple batches

ggml_compute_forward_conv_transpose_2d_impl only computed the first
batch (ne[3] of the destination); every batch after the first was left
as zero. Both the src1 permutation and the main compute loop now iterate
over the batch dimension, and the work buffer size in ggml_graph_plan is
scaled by the src1 batch count so the extra permuted batches fit. A
multi-batch test case is added to test-backend-ops.

Fixes ggml-org/ggml#1448

  • metal : fix conv_transpose_2d for multiple batches

The kernel only computed batch 0 of the input (src1->ne[3]); every
output batch after the first was left as zero, so multi-batch
conv_transpose_2d results diverged from the CPU reference.

The grid now covers all batches (OW x OH x OC x N), the kernel decodes
the batch from the grid z coordinate and offsets both the input and
destination indices accordingly. nb3 is passed in the kernel args.

Assisted-by: pi:llama.cpp/Qwen3.8-27B


Co-authored-by: Georgi Gerganov ggerganov@gmail.com

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI: