cuda : use std::string instead of VLA for archName (#30463)

• cuda : use std::string instead of VLA for archName

Modernize the code (old C style) and fix the following warnings:

ggml/src/ggml-cuda/ggml-cuda.cu:180:19: warning: variable length arrays in C++ are a Clang extension [-Wvla-cxx-extension]
  180 |     char archName[archLen + 1];
      |                   ^~~~~~~~~~~
ggml/src/ggml-cuda/ggml-cuda.cu:180:19: note: read of non-const variable 'archLen' is not allowed in a constant expression
ggml/src/ggml-cuda/ggml-cuda.cu:179:9: note: declared here
  179 |     int archLen = strlen(devName);
      |         ^

Signed-off-by: Adrien Gallouët angt@huggingface.co

• Add a comment about archName

Signed-off-by: Adrien Gallouët adrien@gallouet.fr

Signed-off-by: Adrien Gallouët angt@huggingface.co
Signed-off-by: Adrien Gallouët adrien@gallouet.fr

common, grammar: add recursive depth limit, make it a requirement (#31153)

• common, grammar: add recursive depth limit, make it a requirement

• catch more recursive functions

• common : tidy up peg parser changes

• grammar : reject candidates in a loop, without a depth limit

The depth limit in llama_grammar_reject_candidates_for_stack counted
the code points of a vocab token, so any token of 256 chars or more
was always rejected under a grammar, such as the 512-space token of
GLM-4.7-Flash or the long space tokens of Laguna-S-2.1.

The rejection now walks the code points with its own stack of levels
instead of recursing: there is no limit and no stack risk, whatever
the length of a token from the GGUF, and the masks match master.

• grammar : track rejected candidates as indices into the input

At a given code point a candidate only differs from its input by the
position of its code point pointer, so the levels now carry indices
into the input candidates instead of copies. The first level needs no
special case and the rejects map back to the input directly.

Grammar sampling gets faster than master on GLM-4.7, Laguna, Qwen3.6
and gemma-4 with the json, list and spaces grammars, same masks.

• nit : shorten the comment of llama_grammar_reject_step

• common : add depth_guard for schema walks

• grammar : move GRAMMAR_RECURSIVE_LIMIT into the .cpp

• common : reduce recursion limit for peg parser

• improve

Co-authored-by: Alde Rojas hello@alde.dev
Co-authored-by: Pascal admin@serveurperso.com

tests : adjust default threads for test-llama-context (#31157)

qwen: merge tensors on convert (#31028)

• qwen: merge tensors on convert

• convert: use generator functions

• qwen4exp: remove conts and pre-scale

llama: fix exhausting CPU and MEM caused by dry sequence breaker (#31035)

llama-quant : fix 4D tensor quantization using ggml_nrows() and name-based expert detection (#30820)

• llama-quant : fix quantizing 4D tensors

nrows_total was ne[1]*ne[2], so for a 4D tensor only the first ne[2]
slices were quantized and the rest of the rows were dropped. Use
ggml_nrows(tensor) to count all rows, including ne[3].

Fixes #29916

• llama-quant : detect expert tensors by name and fix imatrix size for 4D tensors

The MXFP4_MOE path used ne[2] > 1 to detect MoE expert tensors. This is
wrong for non-expert tensors with 3 or more dims, e.g. MLA projections
(BailingMoE3) and 4D conv kernels. Use the tensor name instead: all
tensors used by GGML_OP_MUL_MAT_ID are named *_exps or *_chexps. This
also removes the BailingMoE3 special case.

The imatrix holds one vector of ne[0] values per 2D slice, so the
expected size is ne[0]*ne[2]*ne[3], not ne[0]*ne[2]. This matches how
the imatrix is indexed per row for 4D tensors in llama_tensor_quantize_impl.

Assisted-by: Claude Opus 5.5

grammar: limit recursion depth to prevent stack overflow (#29495) (#29755)

llama : respect no_alloc in all memory caches (#31087)

• llama : respect no_alloc in memory caches

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)

• llama : TODO: move no_alloc out of llama_hparams

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)

• llama : TODO: move no_alloc out of llama_hparams

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)

tools : reject truncated GGUF tensor data in merge (#31044)

Co-authored-by: Shreya Agrawal shreyaagrawal@Shreyas-MacBook-Air.local

CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows (#29948)

• cuda: fuse FFN gate/up and GLU in MMQ with interleaved gate/up rows

For prompt processing, run the gate and up matmuls of a dense FFN as one
mul_mat_q launch and apply the GLU in the write-back.

• The loaders take the gate weights as a second pointer and fill the weight
  tile with up and gate rows interleaved in chunks of 8, so each thread holds
  an up value and the matching gate value (rows r and r + 8 of an MMA tile).
• The MMA loop is unchanged: same tile, warp layout, vec_dot and one sum[]
  per thread. The fused write-back computes the GLU from registers and writes
  half the rows.
• The fused path uses the regular tiling (no stream-k) with the column tiles
  innermost so that neighbouring blocks share weight tiles.
• Enabled for Q4_K, Q5_K, Q6_K, Q8_0 and IQ4_XS on NVIDIA Turing and newer.

• cuda: assert the fused MMQ gate argument and trim redundant comments

• cuda: move the fused MMQ hint check to should_fuse and add TODOs

• cuda: pass prec_src1 to the fused MMQ write-back

• cuda: fuse only with efficient tiling and no fallback, merge the GLU write-back

• cuda: require has_fusion explicitly, drop the fusion-only src1 precision check

• cuda: launch the fused MMQ kernel through the stream-k kernel with one block per tile

• cuda: inline the MMQ tiling-efficiency check

• tests: one fused gate/up MMQ case per type and GLU op

• cuda: always launch the fused MMQ kernel with one block per tile

• tests: shorten the fused gate/up MMQ test comment

• cuda: select the fused MMQ loader and write-back in ggml_cuda_mmq_get_util_funcs

• cuda: run the fused MMQ kernel on the plain tiling path

• cuda: put the column tiles innermost in the MMQ plain tiling path