Skip to content

OpenSearch source should do a single scan to discover indexes #6169

Closed

Description

@san81

Is your feature request related to a problem? Please describe.
OpenSearch source make use of Source Coordinator functionality which stores the state in a DDB table with 7 day TTL. In case if the ingestion taking more than 7 days, source starting to ingest the indexes from the start since the original ddb entires were lost.

Describe the solution you'd like
OpenSearch source to implement single scan to discover the indexes

Describe alternatives you've considered (Optional)
Optionally, single scan can also be an optional configuration set by the user as needed

Activity

  1. dlvenable commented on Oct 21, 2025

    @dlvenable
    Member

    The proposal here is to scan one time. Then ingest and not scan again.

  2. added this to the v2.16 milestone on Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions