Make object order deterministic - #47
Open
bmwiedemann wants to merge 5 commits into
Open
Conversation
Without this patch, the parallel path of compile_objects_inner would return OK with a reduced result list. It logged the error, but that can easily be missed in a large non-interactive compile log. Then users could get missing symbols later at link time. With this change, it fails early to make debugging problems easier.
Verifies that compile_objects_inner returns Err when a file fails to compile, in both the parallel and the serial configuration. The test fails without the previous commit.
The error from run() only says "nonzero exit status", which is not enough when many files are compiled, especially in parallel where the nasm output of several jobs is interleaved on stderr.
bmwiedemann
force-pushed
the
deterministic-object-order
branch
from
August 2, 2026 05:44
6a4e71f to
dcd7942
Compare
Author
|
I split out the robustness improvement into #48 and rebased onto it. Note: without the fix, the regression test failed on |
With the `parallel` feature, compile_objects_inner() shares one work list between the calling thread and the jobs the helper thread spawns, and then concatenates the calling thread's results with the helper thread's. Which file lands in which of the two vectors depends on jobserver token timing, so the returned Vec comes out in a different order on every run. The non-parallel path returns the input order, so the two paths disagree as well. Callers pass the result straight on to cc::Build::object() or to archive(), so the order becomes the member order of the resulting .a, and from there the layout of the linked binary. That makes the build unreproducible: in rav1e, which assembles 39 files this way, two builds of the same source placed the assembly routines at different addresses every time. Carry each file's index through the work list and give the results one slot per input file, so a job drops its object where the caller expects it whatever order the jobs finish in. Tested with a 40-file crate and a nasm stub that sleeps a random moment: before, six builds produced six different orders; after, all six agree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
bmwiedemann
force-pushed
the
deterministic-object-order
branch
from
August 2, 2026 06:11
dcd7942 to
5634065
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While working on reproducible builds for openSUSE, I found that our
rav1epackage varied in every build and the 1st commit in this PR helps to make the build result deterministic.The 2nd commit was found along the way by Claude to increase robustness and even though my knowledge of rust is limited, it both looks good to me.