You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug DefaultAcknowledgementSetManagerTests.testExpirations() intermittently fails in CI with:
java.lang.AssertionError:
Expected: <0>
but: was <1>
at org.opensearch.dataprepper.core.acknowledgements.DefaultAcknowledgementSetManagerTests.testExpirations(
DefaultAcknowledgementSetManagerTests.java:113)
To Reproduce
Run ./gradlew :data-prepper-core:test --tests "org.opensearch.dataprepper.core.acknowledgements.DefaultAcknowledgementSetManagerTests.testExpirations" --rerun repeatedly. The failure is intermittent and more likely under CPU load.
Expected behavior
The test should pass consistently regardless of system load.
Environment (please complete the following information):
Additional context
The test sleeps for TEST_TIMEOUT * 2 (800ms) and then immediately asserts that the acknowledgement set monitor size is 0. This assumes the monitor's cleanup thread has already removed the expired set during the sleep window. Under CI load, the cleanup may not have completed in time, leaving the size at 1.
The subsequent await().atMost(TEST_TIMEOUT.multipliedBy(3)) only checks the result field, not the monitor size, so the stale monitor size is never retried.
A fix would be to move the getSize() == 0 assertion into an await().untilAsserted() block, similar to the pattern used in testBasic().
Hey — I've been seeing this exact pattern a lot. Timing-sensitive tests like testExpirations() that fail under CPU load are some of the most costly flaky tests because they fail unpredictably in any CI run, not just heavy ones.
Flaky tests like this usually cost teams $30-40 per rerun in CI time + developer context-switching. Over a month it adds up fast.
I built a tool that detects flaky tests and shows the dollar cost: kleore.com. Here's what a report looks like on a real repo: sample report.
60-second install as a GitHub App, free tier shows the cost. Might help here — at minimum it'll show you how much this specific test is costing in reruns.
Activity
[-][BUG] Flaky DefaultAcknowledgementSetManagerTests.testExpirations[/-][+][BUG] Flaky DefaultAcknowledgementSetManagerTests[/+]rsmihir389 commentedon Apr 6, 2026
Hey — I've been seeing this exact pattern a lot. Timing-sensitive tests like
testExpirations()that fail under CPU load are some of the most costly flaky tests because they fail unpredictably in any CI run, not just heavy ones.Flaky tests like this usually cost teams $30-40 per rerun in CI time + developer context-switching. Over a month it adds up fast.
I built a tool that detects flaky tests and shows the dollar cost: kleore.com. Here's what a report looks like on a real repo: sample report.
60-second install as a GitHub App, free tier shows the cost. Might help here — at minimum it'll show you how much this specific test is costing in reruns.