🏆 Live Soccer TV: rust lang/rust/pull/

obi1kenobi · 2026-07-11T05:11:42Z

Inferred cross-crate inlining currently makes an otherwise-unannotated function available to downstream crates only when its optimized MIR is small and contains no ordinary calls, with intrinsic calls exempted.

This misses small functions whose remaining calls are statically dispatched trait calls to implementations explicitly marked #[inline], since the MIR call identifies the trait item but the relevant inline attribute belongs to the implementation selected for that call.

This PR addresses that edge case by resolving the selected implementation during cross-crate inlining cost analysis. When resolution produces a statically selected, explicitly inline item that is not externally exported, the call no longer disqualifies the enclosing function.

Motivation

The nom-json runtime benchmark uses nom 7.1.3, and its hot path includes an unannotated implementation equivalent to:

impl FindToken<char> for &str {
    fn find_token(&self, token: char) -> bool {
        self.chars().any(|character| character == token)
    }
}

The iterator operations here are marked #[inline], but prior to this PR that wasn't sufficient — find_token() was not made available for downstream cross-crate inlining.

The benchmarked function uses the JSON parser and repeatedly uses find_token() to look for ". Making find_token() available for downstream inlining allows constant propagation to replace the general-case search with a direct comparison, which is much more efficient.

The same optimization already happens in LTO-enabled builds. However, LTO is much more expensive than regular cargo build --release. This PR recovers that optimization for regular cargo build --release builds.

The same optimization can also be recovered by marking find_token() with the #[inline] attribute. While that's still probably a reasonable idea, I still think it's a good idea to offer better out-of-the-box performance for default (non-LTO) release builds even when users haven't realized that their small cross-crate trait function implementation is a good #[inline] candidate.

r? oli-obk

AI disclosure: The optimization opportunity here was discovered as part of a systematic probe for missed optimizations using a combination of both traditional and AI tools. The code here was initially prototyped and vetted by AI tools, followed by additional manual work. I secured approval in advance from the reviewer I pinged. I stand behind the quality of the code I'm submitting, and I vouch it's as good or better compared to if I had written every line by my own hand.

Kobzol · 2026-07-11T05:43:15Z

@bors try @rust-timer queue

[perf] Enable inlining trait items that only contain inlined code.

rust-bors · 2026-07-11T07:50:13Z

☀️ Try build successful (CI)
Build commit: 20bd965 (20bd9651565dfb3135d6e60029dfd8f975ff5329)
Base parent: 48e8ec6 (48e8ec6f05057f942a97187dde770a118e05f42c)

rust-timer · 2026-07-11T08:31:03Z

Finished benchmarking commit (20bd965): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.2%, 1.7%]	17
Regressions ❌ (secondary)	0.4%	[0.0%, 0.7%]	29
Improvements ✅ (primary)	-1.7%	[-6.3%, -0.3%]	5
Improvements ✅ (secondary)	-0.3%	[-4.3%, -0.0%]	31
All ❌✅ (primary)	-0.0%	[-6.3%, 1.7%]	22

Max RSS (memory usage)

Results (primary 0.9%, secondary 1.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	3.0%	[2.7%, 3.3%]	2
Regressions ❌ (secondary)	3.1%	[3.1%, 3.1%]	1
Improvements ✅ (primary)	-3.2%	[-3.2%, -3.2%]	1
Improvements ✅ (secondary)	-0.9%	[-0.9%, -0.9%]	1
All ❌✅ (primary)	0.9%	[-3.2%, 3.3%]	3

Cycles

Results (primary -7.0%, secondary 5.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	10.1%	[1.8%, 19.7%]	11
Improvements ✅ (primary)	-7.0%	[-7.0%, -7.0%]	1
Improvements ✅ (secondary)	-3.3%	[-5.4%, -2.0%]	6
All ❌✅ (primary)	-7.0%	[-7.0%, -7.0%]	1

Binary size

Results (primary -0.0%, secondary 0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.1%, 0.7%]	4
Regressions ❌ (secondary)	0.3%	[0.1%, 0.4%]	2
Improvements ✅ (primary)	-0.3%	[-1.5%, -0.0%]	9
Improvements ✅ (secondary)	-0.3%	[-0.3%, -0.3%]	1
All ❌✅ (primary)	-0.0%	[-1.5%, 0.7%]	13

Bootstrap: 490.729s -> 491.057s (0.07%)
Artifact size: 389.29 MiB -> 389.83 MiB (0.14%)

Kobzol · 2026-07-11T08:41:17Z

Looks like this does indeed help specifically nom-json a lot (https://perf.rust-lang.org/compare.html?start=48e8ec6f05057f942a97187dde770a118e05f42c&end=20bd9651565dfb3135d6e60029dfd8f975ff5329&stat=instructions%3Au&tab=runtime).

nnethercote · 2026-08-03T06:17:25Z

@bors try @rust-timer queue

[perf] Enable inlining trait items that only contain inlined code.

obi1kenobi · 2026-08-03T07:04:38Z

+        match codegen_fn_attrs.inline {
+            InlineAttr::Always | InlineAttr::Force { .. } => true,
+            InlineAttr::Hint => !matches!(tcx.sess.opts.optimize, OptLevel::No),
+            InlineAttr::None | InlineAttr::Never => false,
+        }


This is probably worth considering more closely. Should we keep the behavior the same in both optimized and unoptimized builds?

If yes, I'm happy to change this.

If no, then perhaps we should consider not bothering to resolve the trait fn call in unoptimized builds, and instead add an early-exit for that case near the top of this function.

Feedback welcome!

View changes since the review

hm. Ideally we'd keep it entirely in sync with what cross_crate_inlineable does. We've seen very nice compile-time improvements in debug mode due to inlining.

let's do a perf run with this logic entirely deduplicated with cross_crate_inlineable, (and done at the top both of this function and cross_crate_inlineable

rustbot · 2026-08-03T07:05:36Z

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

rust-bors · 2026-08-03T08:23:57Z

☀️ Try build successful (CI)
Build commit: b146532 (b146532349830911b317d35346adeaff0bc0478d)
Base parent: 65dd30f (65dd30fb9e882a7e8f0be10caca62936db2a98b8)

rust-timer · 2026-08-03T09:06:28Z

Finished benchmarking commit (b146532): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.7%	[0.3%, 1.6%]	5
Regressions ❌ (secondary)	0.4%	[0.3%, 0.4%]	2
Improvements ✅ (primary)	-1.3%	[-6.2%, -0.3%]	7
Improvements ✅ (secondary)	-0.6%	[-4.3%, -0.2%]	12
All ❌✅ (primary)	-0.5%	[-6.2%, 1.6%]	12

Max RSS (memory usage)

Results (primary 1.0%, secondary -0.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	4.0%	[2.2%, 6.3%]	3
Regressions ❌ (secondary)	2.5%	[2.5%, 2.5%]	1
Improvements ✅ (primary)	-3.4%	[-4.1%, -2.8%]	2
Improvements ✅ (secondary)	-0.6%	[-0.9%, -0.5%]	8
All ❌✅ (primary)	1.0%	[-4.1%, 6.3%]	5

Cycles

Results (primary -0.7%, secondary -0.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.7%	[0.5%, 1.1%]	4
Regressions ❌ (secondary)	1.9%	[0.5%, 3.7%]	5
Improvements ✅ (primary)	-2.6%	[-6.3%, -0.5%]	3
Improvements ✅ (secondary)	-1.9%	[-4.8%, -0.5%]	9
All ❌✅ (primary)	-0.7%	[-6.3%, 1.1%]	7

Binary size

Results (primary -0.1%, secondary -0.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.1%, 0.9%]	14
Regressions ❌ (secondary)	0.3%	[0.1%, 0.4%]	2
Improvements ✅ (primary)	-0.3%	[-2.3%, -0.0%]	29
Improvements ✅ (secondary)	-0.5%	[-0.6%, -0.0%]	52
All ❌✅ (primary)	-0.1%	[-2.3%, 0.9%]	43

Bootstrap: 491.898s -> 491.326s (-0.12%)
Artifact size: 391.08 MiB -> 390.33 MiB (-0.19%)

oli-obk · 2026-08-06T07:19:44Z

+                    // if we haven't already given up (or are about to give up)
+                    // on cross-crate inlining due to other disqualifying findings.
+                    if self.cost_allows_cross_crate_inlining()
+                        && !matches!(unwind, UnwindAction::Cleanup(_))


document why we don't do this for calls which can unwind

View changes since the review

oli-obk · 2026-08-06T07:22:05Z

+            return false;
+        };
+
+        let InstanceKind::Item(_) = instance.def else {


what else can it be if it's a trait method and got successfully resolved to an actual instance?

View changes since the review

oli-obk · 2026-08-06T07:24:50Z

+
+        let typing_env = self.typing_env();
+        let Some((instance, _)) =
+            inline::try_resolve_call_instance(tcx, typing_env, def_id, ty::Binder::dummy(args))


cc @addiesh this is unreachable under your work because the const_fn_def call changed in visit_terminator below will ICE. Unless... this late in the MIR pipeline, do we even have any late bound vars left? it's not like they matter anymore and already all got resolved properly in borrowck

View changes since the review

oli-obk · 2026-08-06T07:30:43Z

+        match codegen_fn_attrs.inline {
+            InlineAttr::Always | InlineAttr::Force { .. } => true,
+            InlineAttr::Hint => !matches!(tcx.sess.opts.optimize, OptLevel::No),
+            InlineAttr::None | InlineAttr::Never => false,
+        }


hm. Ideally we'd keep it entirely in sync with what cross_crate_inlineable does. We've seen very nice compile-time improvements in debug mode due to inlining.

let's do a perf run with this logic entirely deduplicated with cross_crate_inlineable, (and done at the top both of this function and cross_crate_inlineable

oli-obk · 2026-08-06T07:31:45Z

-    checker.calls == 0
-        && checker.resumes == 0
-        && checker.landing_pads == 0
-        && checker.statements <= threshold
+
+    checker.cost_allows_cross_crate_inlining()


Make this a separate commit before the other changes in this PR

View changes since the review

saethlin · 2026-08-06T18:32:25Z

@@ -0,0 +1,74 @@
+//@ compile-flags: -Copt-level=3 -Zinline-mir=no -Zcross-crate-inline-threshold=100


I don't think it is a good idea to pin down the threshold in a test. You might think that makes the test less flaky, but actually it means we change from testing how rustc in the wild optimizes code to whether this specific heuristic works as intended, in its own private universe.

View changes since the review

rustbot assigned oli-obk Jul 11, 2026

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jul 11, 2026

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 11, 2026

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Jul 11, 2026

Auto merge of #159119 - obi1kenobi:pg/cross-crate-inlining, r=<try>

20bd965

[perf] Enable inlining trait items that only contain inlined code.

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jul 11, 2026

[perf] Allow cross-crate inlining through explicitly inline trait calls

8ba0090

obi1kenobi force-pushed the pg/cross-crate-inlining branch from 75fddd9 to 8ba0090 Compare August 3, 2026 06:15

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Aug 3, 2026

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Aug 3, 2026

Auto merge of #159119 - obi1kenobi:pg/cross-crate-inlining, r=<try>

b146532

[perf] Enable inlining trait items that only contain inlined code.

obi1kenobi commented Aug 3, 2026

View reviewed changes

obi1kenobi marked this pull request as ready for review August 3, 2026 07:05

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 3, 2026

rustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Aug 3, 2026

obi1kenobi changed the title ~~[perf] Enable inlining trait items that only contain inlined code.~~ [perf] Allow cross-crate inlining through explicitly inline trait calls Aug 3, 2026

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Aug 3, 2026

oli-obk requested changes Aug 6, 2026

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 6, 2026

saethlin reviewed Aug 6, 2026

View reviewed changes

		@@ -0,0 +1,74 @@
		//@ compile-flags: -Copt-level=3 -Zinline-mir=no -Zcross-crate-inline-threshold=100

Uh oh!

Uh oh!

Conversation

obi1kenobi commented Jul 11, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Uh oh!

Kobzol commented Jul 11, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors Bot commented Jul 11, 2026

Uh oh!

This comment has been minimized.

rust-timer commented Jul 11, 2026

Overall result: ❌✅ regressions and improvements - please read:

Uh oh!

Kobzol commented Jul 11, 2026

Uh oh!

nnethercote commented Aug 3, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

obi1kenobi Aug 3, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rustbot commented Aug 3, 2026

Uh oh!

rust-bors Bot commented Aug 3, 2026

Uh oh!

This comment has been minimized.

rust-timer commented Aug 3, 2026

Overall result: ❌✅ regressions and improvements - please read:

Uh oh!

oli-obk Aug 6, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oli-obk Aug 6, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oli-obk Aug 6, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oli-obk Aug 6, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

saethlin Aug 6, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

obi1kenobi commented Jul 11, 2026 •

edited by rustbot

Loading

obi1kenobi Aug 3, 2026 •

edited by rustbot

Loading

oli-obk Aug 6, 2026 •

edited by rustbot

Loading

oli-obk Aug 6, 2026 •

edited by rustbot

Loading

oli-obk Aug 6, 2026 •

edited by rustbot

Loading

oli-obk Aug 6, 2026 •

edited by rustbot

Loading

saethlin Aug 6, 2026 •

edited by rustbot

Loading