Repository navigation
Feature Request: Prometheus Remote Write v1 Source for OpenSearch TSDB #6533
Description
Feature Request: Prometheus Remote Write v1 Source for Data Prepper
Is your feature request related to a problem?
Currently, there is no native way to ingest metrics from Prometheus servers directly into OpenSearch. Organizations using Prometheus for metrics collection must build and maintain custom exporters to convert Prometheus Remote Write format, which is time-consuming and error-prone.
What solution would you like?
Implement a Prometheus Remote Write v1.0 source plugin for Data Prepper that:
- Receives Prometheus Remote Write requests on a configurable HTTP endpoint (default:
http://data-prepper:9090/api/v1/write) - Parses the protocol (Snappy decompression + Protocol Buffer parsing)
- Converts to multiple output formats (TSDB, OTEL, OpenSearch, Prometheus)
- Supports OTLP collectors via OpenTelemetry Collector's
prometheusremotewriteexporter
Configuration:
Prometheus side (2-line change):
# prometheus.yml
remote_write:
- url: "http://data-prepper:9090/api/v1/write"Data Prepper pipeline:
prometheus-pipeline:
source:
prometheus:
port: 9090
path: "/api/v1/write"
# Source emits Metric events (Data Prepper internal model)
sink:
- opensearch:
hosts: ["https://opensearch:9200"]
index_type: tsdb # Sink converts Metric → TSDB format
index: metricsFormat Specifications
TSDB Format
OpenSearch TSDB expects documents with space-separated labels format:
Format:
{
"labels": "__name__ http_requests_total method POST handler /api/items status 200",
"timestamp": 1633072800000,
"value": 1.1
}
Specification:
- https://github.com/opensearch-project/time-series-db
- Labels are space-separated key-value pairs: key1 value1 key2 value2
- See https://github.com/opensearch-project/time-series-db#index-some-metrics
Sink Configuration:
sink:
- opensearch:
hosts: ["https://opensearch:9200"]
index_type: tsdb
index: metrics
OTEL Format
OpenTelemetry Metrics format following the OTEL specification:
Format:
{
"kind": "gauge",
"name": "http_requests",
"value": 100,
"attributes": {"method": "GET"},
"time": "2024-02-12T15:30:00.000Z"
}
Specification:
- https://opentelemetry.io/docs/specs/otel/metrics/data-model/
- Data Prepper's internal Metric interface aligns with OTEL model
Sink Configuration:
sink:
- opensearch:
hosts: ["https://opensearch:9200"]
index_type: otel_metrics # If supported
index: metrics
OpenSearch Format
Standard OpenSearch document format with conventional field names:
Format:
{
"@timestamp": "2024-02-12T15:30:00Z",
"metric_name": "http_requests",
"metric_value": 100,
"labels": {"method": "GET"}
}
Specification:
- Standard OpenSearch document structure
- Compatible with existing dashboards and queries
Sink Configuration:
sink:
- opensearch:
hosts: ["https://opensearch:9200"]
index: metrics # Regular index
Prometheus Format
Preserve original Prometheus Remote Write structure for compatibility:
Format:
{
"name": "http_requests",
"labels": {
"__name__": "http_requests",
"method": "GET"
},
"value": 100,
"timestamp": 1707523200000
}
Specification:
- https://prometheus.io/docs/specs/remote_write_spec/
- Used for debugging or forwarding to other Prometheus systems
Sink Configuration:
sink:
- prometheus:
url: "https://prometheus:9200" # Amazon Managed Prometheus
Examples
Basic Example
Input: Prometheus Remote Write
TimeSeries {
labels: [{name: "__name__", value: "http_requests"}, {name: "method", value: "GET"}]
samples: [{value: 100, timestamp: 1707523200000}]
}
Output (TSDB format):
{"labels": "__name__ http_requests method GET", "timestamp": 1707523200000, "value": 100}Output (OTEL format):
{"kind": "gauge", "name": "http_requests", "value": 100, "attributes": {"method": "GET"}}Output (OpenSearch format):
{"@timestamp": "2024-02-12T15:30:00Z", "metric_name": "http_requests", "metric_value": 100, "labels": {"method": "GET"}}Output (Prometheus format):
{"name": "http_requests", "labels": {"__name__": "http_requests", "method": "GET"}, "value": 100, "timestamp": 1707523200000} ## Multi-Sink Example
Send to both Prometheus and OpenSearch TSDB simultaneously:
```yaml
prometheus-pipeline:
source:
prometheus:
port: 9090
path: "/api/v1/write"
sink:
- prometheus:
url: "https://aps-workspaces.us-west-2.amazonaws.com/workspaces/ws-xxx"
- opensearch:
hosts: ["https://opensearch:9200"]
index_type: tsdb
index: metrics
Activity
dlvenable commentedon Feb 18, 2026
Thank you @srikanthpadakanti for opening this issue and contributing toward the project.
Data Prepper has an internal event model for data in the pipeline. We use this model to maintain compatibility between the plugins. In addition to our Event model, we have some predefined specialized types. One of these is the Metric type. This new
prometheussource should emit Data Prepper events that implement theMetricinterface.Right now, the
prometheussink can sendMetricevents to Prometheus. So by sendingMetricevents, from this source you could write those same events back to Prometheus without any additional processors.(The Prometheus sink only supports Amazon Managed Prometheus presently, but once it supports open-source this should be valid).
Now, what about writing to other sinks? I believe you want to write to the new OpenSearch-TSDB. I'm not familiar with it, but it looks like it accepts normal OpenSearch documents that come with a certain mapping. Data Prepper already supports similar scenarios in the
opensearchsink for trace data. The sink will send documents using mappings that work with the observability plugin. You can see this example for how we support it.As with the
prometheussink we should be able to write to OpenSearch using theopensearchsink, ideally without processors. Here is a possible approach to how this could work (again, I'm not familiar with TSDB, so perhaps more is needed):One advantage to avoid processors is that we can also send to both sinks in one pipeline.
In your description above you mention a few types to support - TSDB, OTEL, OpenSearch, Prometheus. Can you clarify what each of these formats is and include a link to the spec or documentation for each? That should help us better define the requirement for the source.
mishail commentedon Feb 18, 2026
I suggest we focus on the "source" side of the things in this issue. The new source shall be able to accept data in Prometheus Remote Write v1 format and convert them to
Metric.Transformation shall be handled by processors, and those can be added later when needed.
srikanthpadakanti commentedon Feb 19, 2026
Thank you @dlvenable
I understand now that the source should emit Data Prepper
Metricevents (internal model), and the sink handles format conversion viaindex_type. This is much cleaner than using a processor. I've updated the feature request to clarify the format specifications:Questions:
srikanthpadakanti commentedon Mar 10, 2026
Hello @dlvenable Please take a look into the PR #6627
Thanks.