OpenSearch Client Certificate Authentication

Is your feature request related to a problem? Please describe.

Data Prepper supports HTTP Basic authentication to OpenSearch clusters. However, it does not support client certificate authentication. As a client, Data Prepper can authenticate using a client certificate. This can replace HTTP Basic authentication or be combined with HTTP Basic authentication.

Describe the solution you'd like

Provide a mechanism for client client certificate authentication to OpenSearch clusters. Data Prepper pipeline authors would need to supply as input paths to both a certificate file and a private key file. Additionally, Data Prepper should allow this in conjunction with HTTP Basic authentication, for any cluster which requires both.

sink:
  - opensearch:
      authentication:
        mutual_tls:
          private_key: -----BEGIN PRIVATE KEY----- ...

Additional context

This was originally requested in a discussion as part of #310.

We'd also very much like this functionality.
I feel that data-prepper should support most if not all supported authentication methods supported by OpenSearch.

I've got mTLS working on all other components of my OTel stack. Now, if I want to use Data Prepper -> OpenSearch, I have to go back and disable mTLS for all existing components that communicate with OpenSearch.

@dlvenable

@dlvenable Please assign this to me. Thanks

@dlvenable

@dlvenable I am leaning toward keeping client_cert and client_key at the top level rather than nesting them under authentication.mutual_tls, mainly for consistency since trust_cert is already a top level field.

lmk your thoughts. Thanks

We currently have

sink:
  - opensearch:
      authentication:
        username: myuser
        password: secret
        api_token: abc123

I think we should try to keep the authentication aspects together, which is also aligns with other sinks using authentication.

sink:
  - opensearch:
      authentication:
        client_certificate: -----BEGIN CERTIFICATE----- ...
        client_key: -----BEGIN PRIVATE KEY----- ...

We are also trying to avoid the term cert in favor of the full word certificate going forward.

@dlvenable

@dlvenable Thanks David, that makes sense. I will move client_certificate and client_key under authentication and rename it to client_certificate instead of client_cert. I will make sure the implementation also supports both file paths and inline PEM content so users can configure it either way.

Navigation Menu

Repository navigation

OpenSearch Client Certificate Authentication #633

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

OpenSearch Client Certificate Authentication #633

Description

Activity

faxm0dem commented on Apr 15, 2025

sheariley commented on Jan 30, 2026

srikanthpadakanti commented on Apr 6, 2026

srikanthpadakanti commented on Apr 6, 2026

dlvenable commented on Apr 6, 2026

srikanthpadakanti commented on Apr 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions