Repository navigation
Fix AsyncGRPOTrainer crashing on environment-owned datasets - #6824
Merged
Merged
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Member
Author
|
I am merging this as a hotfix. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes
AsyncGRPOTrainerraisingAttributeError: 'AsyncGRPOTrainer' object has no attribute 'args'when it is built without atrain_dataset.Bug introduced by PR:
train_datasetoptionalΒ #6349The feature #6349 added has never worked.
Motivation
AsyncGRPOTrainer.__init__readsself.argswhile building the placeholder dataset, butself.argsis only set byTrainer.__init__, which runs a few lines later. Every environment-owned run, meaningtrain_dataset=Nonetogether with anenvironment_factory, fails there before it can reach either the intendedmax_stepsValueErroror the placeholder dataset construction.The existing regression test,
TestAsyncGRPOTrainer::test_environment_owned_data_requires_max_steps, does cover this, but it never ran:tests/experimentalis excluded from the regular suite throughnorecursedirs, and intests-experimental.ymlthe whole class is gated behindis_ampere_or_newer(), which was false on the T4 runner. Moving that job to an L40S in #6741 makes the test execute for the first time and surfaces the failure.Solution
Use the local
argsinstead ofself.argsin the pre-super().__init__()region, consistent with every other config read in that block.AsyncDistillationTraineris unaffected: it assignsself.args = argsbefore its ownsuper().__init__().Changes
max_steps,per_device_train_batch_sizeandgradient_accumulation_stepsfrom the localargswhen building the placeholder dataset inAsyncGRPOTrainer.__init__Note
Cursor Bugbot is generating a summary for commit cfa4913. Configure here.