[velonica0]

Description

Two RVV kernels for riscv64, closing the remaining fp32 gaps between the RVV and SVE dispatch tables:

1. MlasLinearAttentionDispatchRvv (riscv64/linear_attention_kernel_rvv.cpp) — the fused recurrence for all four rules (linear / gated / delta / gated-delta), 1/2/4/8 readout heads, any d_v, d_k ≤ 256. Other shapes and head counts fall back to the portable kernel as on NEON/SVE.

   Where this differs from the fixed-width kernels: a column panel is a single LMUL register group with vl from vsetvl, so there is no per-lane macro grid and the trailing partial panel is an ordinary iteration rather than a predicated copy of the body. LMUL is chosen per head from a register budget (NOUT+2, NOUT+3 or 2·NOUT+3 groups, one kept spare) and narrowed to cover d_v. Two further reductions in S traffic, both bit-exact with the unfused form:

   • single-pass rules process two tokens per sweep of S at 1–2 heads (one state load/store serves both);
   • two-pass rules fuse pass 2 of token t with pass 1 of token t+1, so S is read once and written once per token; the per-column retrieval/readout accumulators cross the token boundary through two small stack buffers (d_v ≤ 256; the plain two-pass form remains above that).

   The decayed update is evaluated as (dec·S_old) + k·upd, multiplying the loaded row in place — the other association holds a product live alongside the row and spills at LMUL 4. Inner loops are verified spill-free.

2. MlasReduceMinimumMaximumF32KernelRvv (riscv64/softmax_kernel_rvv.cpp) — LMUL 4, tail-undisturbed accumulators, one reduction per extremum at the end, matching the existing RVV max kernel. MlasFindMinMaxElement only consulted the platform slot under MLAS_TARGET_AMD64 || MLAS_USE_SVE; the guard now includes RVV builds, and the riscv64 default block initializes the slot so ORT_MLAS_RISCV_FORCE_SCALAR=1 no longer dereferences a null kernel.

Motivation and Context

LinearAttention gained fused kernels for AVX-512 (#31674), NEON (#32178) and SVE (#32356); riscv64 was still on the scalar portable path. ReduceMinimumMaximumF32 feeds DynamicQuantizeLinear and was the last F32 compute kernel with an SVE implementation but no RVV one.

Test

Measured on a SpacemiT K3 (onnxruntime_mlas_benchmark, BM_LinearAttention, 8 threads), against the portable dispatch on the same build:

Shape (dk=dv=128 unless noted)	linear	gated	delta	gated-delta
T=1024, H=16, dk=32/dv=64	$7.4\times$	$7.8\times$	$6.6\times$	$7.7\times$
T=1024, H=16	$3.8\times$	$4.1\times$	$3.2\times$	$3.8\times$
T=4096, H=16	$3.8\times$	$5.9\times$	$3.5\times$	$4.2\times$
T=1024, Hq=32/Hkv=8 (GQA 4)	$3.6\times$	$4.8\times$	$4.4\times$	$5.0\times$
T=1024, Hq=64/Hkv=8 (GQA 8)	$3.1\times$	$3.1\times$	$3.6\times$	$4.1\times$
T=1024, Hq=16/Hkv=32 (inverse GQA)	$3.8\times$	$5.6\times$	$2.8\times$	$3.9\times$
B=8, T=512, H=16	$3.9\times$	$6.8\times$	$3.1\times$	$3.6\times$
T=4096, H=1 (single thread)	$5.4\times$	$7.8\times$	$4.5\times$	$5.9\times$

X100 cores, VLEN=256. The single-thread gated-delta case runs at ≈96% of the core's vector-issue bound. On the board's A100 cores (VLEN=1024) the same shapes are 5–14×.

MlasFindMinMaxElement: 1024 floats 1677 → 172 ns (9.7×), 64K floats 8.1×, 4M floats 2.4× (DRAM-bound).

LinearAttention and MinMaxElement unit tests pass at VLEN=256 and VLEN=1024 (which take different LMUL paths) and with ORT_MLAS_RISCV_FORCE_SCALAR=1.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Copilot review overview

🔵 Needs a closer look

The large architecture-specific vector kernel requires final review and validation on supported RVV hardware.

Review effort: Balanced
Findings: None

What changed in this PR

Adds riscv64 RVV acceleration for linear attention and FP32 min/max reduction.

Changes:

• Implements fused RVV linear-attention kernels and dispatch.
• Adds RVV minimum/maximum reduction.
• Registers kernels with runtime dispatch and CMake.

File	Description
riscv64/​softmax_kernel_rvv.cpp	Adds RVV min/max reduction.
riscv64/​linear_attention_kernel_rvv.cpp	Implements RVV linear attention.
quantize.cpp	Enables riscv64 RVV min/max dispatch.
platform.cpp	Registers scalar and RVV kernels.
mlasi.h	Declares new RVV entry points.
onnxruntime_mlas.cmake	Builds the RVV attention source.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🟡 Changes recommended

The new kernels lack coverage for important multi-vector and exact-buffer-boundary paths.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 · 1

Open (2)

• Add gated-delta RVV coverage for d_v=256 · New
• Test reduction loops across multiple RVV vectors · New

[Copilot]

Copilot review overview

🔵 Needs a closer look

The large architecture-specific RVV kernel remains hardware- and compiler-sensitive despite comprehensive static review and reported testing.

Review effort: Balanced
Findings: None

Resolved since last review (2)

• Add gated-delta RVV coverage for d_v=256
• Test reduction loops across multiple RVV vectors