Skip to content

[BUG] Flaky DefaultAcknowledgementSetManagerTests #6719

Closed

Description

@lawofcycles

Describe the bug
DefaultAcknowledgementSetManagerTests.testExpirations() intermittently fails in CI with:

java.lang.AssertionError:
Expected: <0>
    but: was <1>
at org.opensearch.dataprepper.core.acknowledgements.DefaultAcknowledgementSetManagerTests.testExpirations(
DefaultAcknowledgementSetManagerTests.java:113)

To Reproduce
Run ./gradlew :data-prepper-core:test --tests "org.opensearch.dataprepper.core.acknowledgements.DefaultAcknowledgementSetManagerTests.testExpirations" --rerun repeatedly. The failure is intermittent and more likely under CPU load.

Expected behavior
The test should pass consistently regardless of system load.

Environment (please complete the following information):

Additional context
The test sleeps for TEST_TIMEOUT * 2 (800ms) and then immediately asserts that the acknowledgement set monitor size is 0. This assumes the monitor's cleanup thread has already removed the expired set during the sleep window. Under CI load, the cleanup may not have completed in time, leaving the size at 1.

The subsequent await().atMost(TEST_TIMEOUT.multipliedBy(3)) only checks the result field, not the monitor size, so the stale monitor size is never retried.

A fix would be to move the getSize() == 0 assertion into an await().untilAsserted() block, similar to the pattern used in testBasic().

Activity

  1. changed the title [-][BUG] Flaky DefaultAcknowledgementSetManagerTests.testExpirations[/-] [+][BUG] Flaky DefaultAcknowledgementSetManagerTests[/+] on Apr 3, 2026
  2. rsmihir389 commented on Apr 6, 2026

    @rsmihir389

    Hey — I've been seeing this exact pattern a lot. Timing-sensitive tests like testExpirations() that fail under CPU load are some of the most costly flaky tests because they fail unpredictably in any CI run, not just heavy ones.

    Flaky tests like this usually cost teams $30-40 per rerun in CI time + developer context-switching. Over a month it adds up fast.

    I built a tool that detects flaky tests and shows the dollar cost: kleore.com. Here's what a report looks like on a real repo: sample report.

    60-second install as a GitHub App, free tier shows the cost. Might help here — at minimum it'll show you how much this specific test is costing in reruns.

  3. added
    maintenanceIssues to help maintain the project, such as improving builds, testing, etc.
    and removed on Apr 7, 2026
  4. added this to the v2.16 milestone on Apr 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingmaintenanceIssues to help maintain the project, such as improving builds, testing, etc.

Type

No type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions