You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
OpenSearch source should do a single scan to discover indexes #6169
Is your feature request related to a problem? Please describe.
OpenSearch source make use of Source Coordinator functionality which stores the state in a DDB table with 7 day TTL. In case if the ingestion taking more than 7 days, source starting to ingest the indexes from the start since the original ddb entires were lost.
Describe the solution you'd like
OpenSearch source to implement single scan to discover the indexes
Describe alternatives you've considered (Optional)
Optionally, single scan can also be an optional configuration set by the user as needed
Activity
dlvenable commentedon Oct 21, 2025
The proposal here is to scan one time. Then ingest and not scan again.
Add SINGLE_SCAN discovery mode to OpenSearch source (#6169) (#6783)
Add SINGLE_SCAN discovery mode to OpenSearch source (opensearch-proje…
Add SINGLE_SCAN discovery mode to OpenSearch source (opensearch-proje…