Repository navigation
tests: unblock nightly sync integ tests on pytest 9.1 - #9189
Conversation
| LOG = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| # pytest 9.1 deprecation, inapplicable here: these fixtures assign to the class, not self. Remove before pytest 10. |
There was a problem hiding this comment.
[GENERAL] The justification in the comment is only half true, and that matters for what happens next.
The fixtures do assign results to the class (TestSyncCodeBase.template_path, TestSyncCodeBase.stack_name), but they also read instance state and call unittest methods on self:
@pytest.fixture(scope="class")
def execute_infra_sync(self):
if self.test_data_path is None: ...
TestSyncCodeBase.stack_name = self._method_to_stack_name(self.id()) # instance-derived
... dependency_layer=self.dependency_layer, image_repository=self.ecr_repo_name, kms_key_id=self.kms_key
@pytest.fixture(autouse=True, scope="class")
def sync_code_base(self, execute_infra_sync):
self.assertEqual(...) # unittest assertion on a per-test instanceThe same holds in the other two files: deploy_testing_stack reads self.test_template_folder and logs_base / sync_code_base call self.assertEqual. That is exactly the binding the deprecation targets β the self a class-scoped fixture receives is an instance created for one particular test in the class (here, the first one), so self.id() and the parameterized_class attributes come from an arbitrary member of the class rather than from the class itself. So the pattern is being used, not "inapplicable"; the suppression defers the migration rather than documenting a false positive.
Two consequences worth addressing before merge:
Remove before pytest 10reads as "delete this line," but deleting it without converting the fixtures will hard-fail the suite. Please state the required follow-up explicitly (e.g., move the shared setup intosetUpClass/arequest.cls-based fixture, or make the fixture a@classmethod/@staticmethodand replaceself.assertEqualwith plainassert), and ideally link a tracking issue.- Because the mark sits on the class, it silences this warning for every class-scoped fixture added to these classes later, including cases where the instance binding genuinely is a bug. Applying the filter narrowly (only where needed) or keeping it in one place with a pointer to the follow-up work limits that blast radius.
The unblock itself is reasonable; the ask is that the comment describe why the suppression is temporary rather than assert the deprecation does not apply.
There was a problem hiding this comment.
I agree with the bot. If the extra info we're adding is not correct, let's just change to something simpler so it's not confusing.
# Deprecated in pytest 9.1. Fix before pytest 10.
There was a problem hiding this comment.
Thanks for the suggestion, applied the fix
sync-code and sync-watch have failed on every scheduled run since pytest was bumped 9.0.3 -> 9.1.1 (#9095, 2026-07-31). Latest: 37 passed, 52 errors. The last pre-bump run passed with 89 passed, 1 rerun. Reproducible in ten lines, no AWS: class TestFixt: @pytest.fixture(scope="class") def fixt(self): yield def test_1(self, fixt): pass def test_2(self, fixt): pass pytest 9.0.3 -> 2 passed pytest 9.1.1 -W error -> 2 errors 1. pytest 9.1 deprecated class-scoped fixtures declared as instance methods. 2. `filterwarnings = error` in pytest.ini makes that a setup failure on the first test of every affected class. 3. pytest then raises its own internal AssertionError ("assert not self._finalizers", fixtures.py:1221) for the remaining tests in the class -- upstream pytest-dev/pytest#14775, still open. Root cause is fixtures.py:1147: finish() early-returns when cached_result is None without clearing _finalizers. 4. `--reruns 3` reports only the final attempt, so the real warning is discarded and only the internal AssertionError survives. That is why the CI logs contain zero occurrences of the actual cause. Counts line up: 52 errors x 3 reruns = 156 reruns (sync-watch: 12 x 3 = 36). Suppressing is safe here specifically: these fixtures assign to the class (`TestSyncCodeBase.stack_name = ...`), not to `self`, so the hazard the deprecation warns about -- attributes set on self being invisible to tests -- does not apply. Marked on the three classes that declare these fixtures rather than in pytest.ini, so the rest of the suite still fails on this warning. A class mark rather than a module-level pytestmark because TestSyncCodeBase is subclassed from other modules (test_sync_build_in_source.py, test_sync_adl.py) and a module mark would not reach them, which is why one base class took out both sync-code and sync-watch. Converting the fixtures is required before pytest 10 and is deliberately not done here: execute_infra_sync derives its stack name from self._method_to_stack_name(self.id()), and unittest's id() needs an instance, so it changes live CloudFormation stack names. Testing: verified against the repo's real pytest.ini that the same parameterized class-scoped shape goes 6 errors -> 6 passed with the mark, and that an unmarked one still errors, so suite-wide protection is intact. Confirmed pytest resolves the mark through the MRO for subclasses that carry their own marks. tests/integration/{sync,logs,traces} collect 266 tests. black and ruff clean.
166ee89 to
d373c81
Compare
Review feedback: keying the filter on the message prose means it stops applying if
upstream rewords the warning, and that failure is silent.
Confirmed the failure mode rather than assuming it. `filterwarnings` marks reach
warnings.filterwarnings, where the message becomes re.compile(msg, re.I).match(...), so a
stale phrase simply never matches: with a deliberately reworded pattern the probe goes
back to 4 errors / 8 reruns showing only the bare "assert not self._finalizers", exactly
the opaque failure this change exists to prevent.
Also checked the suggested compromise of combining message and category. It does not
help -- warnings.filterwarnings requires both fields to match, so the combined form is as
brittle as message-only:
stale message, message only -> error
stale message, message + category -> error
stale message, category only -> passes
So the filter is now category-only. `pytest.PytestRemovedIn10Warning` is exported by
pytest 9.1.1, and this matches existing convention: test_build_cmd.py:58 uses
"ignore::ResourceWarning" and pytest.ini has six ignore::Category entries.
The tradeoff is that it also suppresses the other eight PytestRemovedIn10Warning messages
in pytest 9.1.1 inside these three classes. That is acceptable: they cover config.inicfg,
pytest.console_main, FixtureDef.has_location, fixture baseid/nodeid, parsefactories
nodeid, monkeypatch.syspath_prepend and --pastebin -- plugin and config level APIs these
integration test classes do not touch.
Testing: unmarked probe still errors (6 errors, 18 rerun) so suite-wide protection is
intact; the category-marked equivalent passes. tests/integration/{sync,logs,traces}
collect 266 tests. black and ruff clean.
Which issue(s) does this change fix?
N/A
Why is this change necessary?
sync-code and sync-watch have failed on every scheduled integration run since pytest was bumped 9.0.3 β 9.1.1 (#9095, 2026-07-31).
The logs show only pytest's own internal assertion,
assert not self._finalizers(_pytest/fixtures.py:1221), with nothing pointing at the real cause. Reproducible in ten lines, no AWS needed:Four links:
PytestRemovedIn10Warning).TestSyncCodeBase.execute_infra_sync/sync_code_baseare exactly this shape.pytest.inisetsfilterwarnings = error, turning that warning into a setup failure on the first test of every affected class.AssertionErrorfor the remaining tests in the class β upstream pytest-dev/pytest#14775, still open. Root cause isfixtures.py:1147:finish()early-returns whencached_result is Nonewithout clearing_finalizers.--reruns 3hides the cause.pytest-rerunfailuresreports only the final attempt, so the real warning is discarded and only the internalAssertionErrorsurvives β which is why the CI logs contain zero occurrences of the actual error.Confirmed link 4 directly, same test with and without
--reruns:The counts line up exactly: 52 errors Γ 3 reruns = 156 reruns (sync-watch: 12 Γ 3 = 36).
How does it address the issue?
Marks the warning as ignored on the three classes that declare these fixtures β
TestSyncCodeBase,LogsIntegTestCases,TestTracesCommand. Six lines total.Safe here specifically: these fixtures assign to the class (
TestSyncCodeBase.stack_name = ...), not toself, so the hazard the deprecation warns about β attributes set onselfbeing invisible to tests β does not apply to them.A class mark rather than a
pytest.inientry, so the rest of the suite still fails on this warning. And a class mark rather than a module-levelpytestmark, becausepytestmarkis module-scoped whileTestSyncCodeBaseis subclassed from other modules (test_sync_build_in_source.py,test_sync_adl.py) β which is precisely why one base class took out both sync-code and sync-watch.What side effects does this change have?
Deferred, not avoided: converting these fixtures is required before pytest 10 removes the behaviour. Left out on purpose β it is not mechanical.
execute_infra_syncderives its stack name fromself._method_to_stack_name(self.id()), andunittest'sid()needs an instance, so any conversion changes live CloudFormation stack names. All three autouse fixtures also assert viaself.assertEqual, which under@classmethodwould bind the first argument asselfrather than fail loudly. That belongs in its own PR, verified against real AWS.Testing
pytest.ini, the same parameterized class-scoped shape goes 6 errors β 6 passed with the markTestLogsCommandWithRegularStack)tests/integration/{sync,logs,traces}collect 266 tests;blackandruffcleanMandatory Checklist
PRs will only be reviewed after checklist is complete