[swetha097]

[MLAS] NCHWc HardSwish fusion for MobileNetV3 FP32 CPU inference

Implementation of the hardswish fusion kernel for mobilenetv3 models.

Implementation changes done:

• Fused the decomposed HardSwish pattern (Mul(x, HardSigmoid(x))) into the preceding NCHWc Conv as a native HardSwish activation, eliminating
  two standalone nodes per occurrence.
• Added TryFuseNchwcHardSwish in nchwc_transformer.cc to detect and apply the fusion when the HardSigmoid parameters match HardSwish values
  (α=1/6, β=1/2) and the Mul consumes the same conv output.

Performance:

Performance numbers on AMD Ryzen AI 9 365 (Strix Point)

NOTE: Performance taken on 31st July

STRIX 365 configration:
AMD Ryzen AI 9 365 (Strix Point) w/ Radeon 880M

Target Model: MobilenetV3-Large and MobilenetV3-Small (FP32, CPU)

Unit Tests
test_nchwc_hardswish.cpp

• Verifies TryFuseNchwcHardSwish correctly fuses the Mul(x, HardSigmoid(x)) diamond pattern into the preceding NCHWc Conv
• Confirms fusion is not applied when α/β do not match HardSwish values (standalone HardSigmoid falls through to standard activation path)
• Top-1 accuracy unchanged after fusion

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Hariharan Seshadri (hariharans29)]

Review: PR #31957 — Hardswish Fusion implement for Mobilenetv3 models (head 5be2cb8)

Author: @swetha097 with @jenetscaria-mcw (external, AMD). 9 commits, +217 / −0 across 5 files. CI: 0 / 1 checks OK (external gate — needs maintainer /azp run). No reviewers assigned. CLA not yet signed — bot has prompted. 1 participant.

Verdict: approve on correctness grounds once CLA and CI are cleared. Well-designed three-part change: (1) new MlasHardSwishActivation SIMD kernel in MLAS, (2) fused-activation string wiring, and (3) an NCHWc-transformer pass that reconstructs the ONNX-decomposed Mul(x, HardSigmoid(x)) diamond into a single NCHWc Conv with activation="HardSwish". The ONNX-spec math is correct, the pattern-matcher's guards are appropriately tight, and the fallthrough behavior (standalone HardSigmoid still gets fused as HardSigmoid, not misidentified as HardSwish) is directly tested.

Performance number on AMD Ryzen AI 9 365 (Strix Point) targets MobileNetV3-Large / MobileNetV3-Small FP32 CPU — perf table linked in PR description.

---

What each piece does

1. mlas.h + activate.cpp: the SIMD kernel

New MLAS_ACTIVATION_FUNCTION<MlasHardSwishActivation> mirrors the HardSigmoid template exactly. Vector body:

MLAS_FLOAT32X4 Gate = MlasMultiplyAddFloat32x4(Value, AlphaBroadcast, BetaBroadcast);  // x/6 + 0.5
Gate = MlasMinimumFloat32x4(MaximumBroadcast, Gate);                                    // min(1, ·)
Gate = MlasMaximumFloat32x4(MinimumBroadcast, Gate);                                    // max(0, ·)
return MlasMultiplyFloat32x4(Value, Gate);                                              // x * clamped

with constants Alpha = 1/6, Beta = 1/2, Minimum = 0, Maximum = 1. This is the exact ONNX HardSwish definition: y = x · max(0, min(1, α·x + β)) with α = 1/6, β = 1/2. Correctness spot-checks:

x	Gate = clamp(x/6 + 0.5, 0, 1)	y = x·Gate	Expected
−3	clamp(0, 0, 1) = 0	0	0 ✓
0	0.5	0	0 ✓
3	clamp(1, 0, 1) = 1	3	3 ✓
6	clamp(1.5, 0, 1) = 1	6	6 ✓

Scalar fallback correctly uses the same math with std::min/std::max. On SSE2 the scalar path routes through the SIMD kernel via _mm_set_ss/_mm_cvtss_f32, which is what HardSigmoid does — style-consistent. ✓

2. fused_activation.cc

Adds one else-if to the GetFusedActivationAttr string→enum lookup, placed in the parameter-less branch (correct — HardSwish has no runtime params, unlike HardSigmoid). One-liner. ✓

3. nchwc_transformer.cc: the pattern matcher

TryFuseNchwcHardSwish is called from the top of TransformActivation when the activation node is HardSigmoid consuming an NCHWc conv output. The pattern to match:

    conv (NCHWc) → x
                   ├→ HardSigmoid(x, α=1/6, β=1/2) → gate
                   └→ Mul(x, gate) → output

Guards (all correct, each necessary):

• hardsigmoid.OpType() == "HardSigmoid" and exactly one output edge.
• |α − 1/6| ≤ 1e-6 and |β − 0.5| ≤ 1e-6 (with defaults α = 0.2, β = 0.5 — so a default HardSigmoid never matches, which is the right disambiguation from ONNX default).
• Producing conv is com.microsoft.nchwc.Conv.
• nchwc_input.starting_original_uses_ == 2 — exactly two consumers of the conv output (the HardSigmoid + Mul, nothing else). ✓ This is the crucial guard against fusing away a conv output that has a third consumer we'd break.
• Conv doesn't already have an activation attribute (no double-fusion).
• The sibling Mul has exactly two inputs, is in the ONNX domain, and consumes both conv_output_arg and the HardSigmoid output. Handles ONNX's commutative Mul operand order (checks either position).
• Neither HardSigmoid nor Mul produces a graph output.

Rewrite:

nchwc_node.AddAttribute("activation", std::string("HardSwish"));
nchwc_input.remaining_original_uses_--;  // HardSigmoid use
nchwc_input.remaining_original_uses_--;  // Mul use
graph_utils::RemoveNodeOutputEdges(graph_, hardsigmoid);
removed_nodes_.push_front(hardsigmoid.Index());
FuseNchwcArgument(*mul_node, nchwc_input);
removed_nodes_.push_front(mul_node->Index());

Decrement-by-2 correctly matches the double-consumption. FuseNchwcArgument(mul_node, ...) routes the Mul's output NodeArg onto the NCHWc arg — same pattern the transformer already uses for Conv-Add-Activation fusion. ✓

Fallthrough is preserved and tested

If TryFuseNchwcHardSwish returns false, TransformActivation continues to the standard activation-fusion path — so a standalone HardSigmoid still becomes an NCHWc-Conv activation="HardSigmoid". This is verified by the second new test ConvNchwcHardSigmoidNotHardSwish. ✓ Excellent test-of-negative.

---

Correctness spot-checks

Consumer walking

const auto consumers = graph_.GetConsumerNodes(conv_output_arg->Name());
for (const Node* consumer : consumers) {
    if (consumer == &hardsigmoid) continue;
    Node* candidate = graph_.GetNode(consumer->Index());
    // ... check candidate is a matching Mul ...
    mul_node = candidate;
}

Since starting_original_uses_ == 2 was already asserted, there are exactly two consumers, one is the HardSigmoid, so this loop iterates exactly once for the candidate. If the candidate isn't a matching Mul, the function returns false. If it is, mul_node is set. ✓

Note: if a pathological input somehow had Mul(x, x) instead of Mul(x, HardSigmoid(x)), then consumes_hs would be false and the check returns false. ✓ Also caught: the starting_original_uses_ != 2 guard would trip first because Mul(x, x) counts as two uses on x from the same Mul, plus one from HardSigmoid = 3.

orig_input_arg capture ordering

if (nchwc_input != nullptr) {
    NodeArg* orig_input_arg = input_defs[0];  // ← captured BEFORE rewrite
    if (TryFuseNchwcHardSwish(node, orig_input_arg, *nchwc_input)) return;
    input_defs[0] = nchwc_input->nchwc_arg_;   // ← rewrite happens after
    ...
}

TryFuseNchwcHardSwish needs to consult graph_.GetConsumerNodes(conv_output_arg->Name()), which must key off the pre-rewrite name. Capturing before the rewrite is correct. ✓ (A follow-on commit 7381125 — "scope orig_input_arg to nchwc guard; clarify fallthrough" — explicitly scoped this to reinforce the ordering.)

Default HardSigmoid safety

ONNX HardSigmoid defaults are α = 0.2, β = 0.5. The code correctly reads alpha = 0.2f when no attribute is present. Then |0.2 - 1/6| = 0.033... > 1e-6f → bail. So a plain HardSigmoid without an explicit α=1/6 is never misidentified. ✓ This is precisely what ConvNchwcHardSigmoidNotHardSwish verifies.

Epsilon slack

1e-6f tolerance on |α − 1/6| and |β − 0.5| accepts float-exporter roundoff without being lax. 1.0f/6.0f in single precision is 0x3E2AAAAB = 0.16666667…, so exporters that emit exactly 0.16666666 (bit-diff 1 ulp) still match. Reasonable.

Test coverage

• ConvNchwcHardSwishFusion: builds the diamond with the correct HardSwish params, checks HardSigmoid == 0, Mul == 0, and nchwc.Conv == 2. ✓
• ConvNchwcHardSigmoidNotHardSwish: default-param HardSigmoid still fuses as HardSigmoid activation, verified HardSigmoid == 0 (fused-away, not misidentified). ✓

Both tests are strong. A small strengthening (non-blocking, see nits) would be to look up the resulting NCHWc Conv and assert activation="HardSwish" explicitly.

---

Nits (non-blocking)

1. CLA. Bot has prompted @swetha097. Must be signed before merge — hard gate.
2. Full CI. 0/1 — external contributor, @microsoft/onnxruntime maintainer needs to /azp run to unlock Azure Pipelines.
3. Native HardSwish opset-14 op is not covered. This PR handles the decomposed Mul(x, HardSigmoid(x)) pattern (which is what current MobileNetV3 exporters emit — reasonable scope). A future follow-up could extend TransformActivation's recognized-op set to include standalone HardSwish as an opType, so post-opset-14 exporters that don't decompose also benefit. Note in follow-up.
4. Test strengthening. ConvNchwcHardSwishFusion could add:

   for (const auto& node : session.GetGraph().Nodes()) {
     if (node.OpType() == "Conv" && node.Domain() == kMSNchwcDomain) {
       const auto* activation_attr = graph_utils::GetNodeAttribute(node, "activation");
       ASSERT_NE(activation_attr, nullptr);
       EXPECT_EQ(activation_attr->s(), "HardSwish");
     }
   }

   to positively assert the resulting attribute (rather than just op count).
5. 9 commits, mostly polish (style alignment, member declaration order, comment clarifications, blank-line spacing). Will squash cleanly — no history concern.
6. No reviewers assigned. Needs MLAS ownership (@yufenglee, @chenfucn) + graph-optimizer eyes (@edgchen1, @hariharans29).
7. Perf attribution. PR description shows a screenshot with numbers for MobileNetV3-L/S on Strix Point. Actual delta values aren't in the fetch output, but the PR text asserts "Top-1 accuracy unchanged after fusion" — good qualitative claim. Perf table review is left to the maintainer.

---

Merge state

• CI: 0 / 1 — needs /azp run from a maintainer.
• CLA: pending. Author must reply to the bot with @microsoft-github-policy-service agree (with company="AMD" if applicable, given the "AMD Ryzen AI" perf attribution).
• Approvals: none. Reviewers unassigned.
• Labels: none. MLAS, perf, or similar would help retention.

---

Bottom line

Well-scoped, correctly-bounded diamond-pattern fusion with matching SIMD kernel. Every guard in TryFuseNchwcHardSwish earns its keep, the ONNX HardSwish math is exact, and the fallthrough behavior for non-HardSwish HardSigmoid is directly tested. External-contributor plumbing (CLA, /azp run) is the only real gate on merge — code is ready.

Action items:

1. Author: reply to CLA bot.
2. Maintainer: /azp run to unlock full CI, then assign yourself + one MLAS + one optimizer reviewer.
3. Post-merge follow-up: extend TransformActivation to also accept standalone HardSwish opType (opset 14+ exporters), so this PR's benefit extends beyond decomposed patterns. Track as an issue.

Ready to merge once CLA, /azp run, and one MSFT-side approval are in.

[Copilot]

Pull request overview

Adds NCHWc Conv–HardSwish fusion for FP32 MobileNetV3 inference.

Changes:

• Adds MLAS HardSwish activation support.
• Detects and fuses decomposed HardSwish diamonds.
• Adds optimizer fusion tests.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
onnxruntime/core/mlas/inc/mlas.h	Adds the HardSwish activation kind.
onnxruntime/core/mlas/lib/activate.cpp	Implements and dispatches HardSwish.
onnxruntime/contrib_ops/cpu/fused_activation.cc	Maps fused HardSwish attributes to MLAS.
onnxruntime/core/optimizer/nchwc_transformer.cc	Adds NCHWc HardSwish pattern fusion.
onnxruntime/test/optimizer/nchwc_optimizer_test.cc	Adds positive and fallback optimizer tests.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[mirounga]

Verified effective GFLOPS increase 477->583 (mv3_large) and 257->347 (mv3_small) while the output is bitwise identical.

[swetha097]

swetha097 please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

• (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

• (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree company="Multicoreware"