Skip to content

Add ClusterIP Service for node-exporter and use service DNS for scraping - #304

Merged
sky333999 merged 2 commits into
aws-observability:release-6.1.0from
petruanica:fix-node-exporter-tls
Apr 30, 2026
Merged

sky333999 merged 2 commits into
aws-observability:release-6.1.0from
petruanica:fix-node-exporter-tls

Conversation

@petruanica

@petruanica petruanica commented Apr 29, 2026 •

Copy link
Copy Markdown
Collaborator

Description of changes:

PR #297 removed insecure_skip_verify: true from the node-exporter Prometheus scrape config, which broke node-exporter metric collection due to TLS handshake error.

Fix:

  • Add a ClusterIP Service with internalTrafficPolicy: Local for node-exporter to enable scraping via service DNS name
  • Update OTEL prometheus config to target service name instead of HOST_IP
  • Add node-exporter-service SANs to both cert-manager and auto-generated TLS certificates

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@petruanica petruanica changed the title Add insecure_skip_verify to node_exporter TLS config Add ClusterIP Service for node-exporter and use service DNS for scraping Apr 29, 2026
ports:
- name: metrics
containerPort: 9487
hostPort: 9487

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Dont think you need hostPort anymore.

@sky333999
sky333999 merged commit 391f2de into aws-observability:release-6.1.0 Apr 30, 2026
27 of 30 checks passed
mitali-salvi added a commit that referenced this pull request Jun 10, 2026
* Add OTEL Container Insights infrastructure (#282)

* Add v6.0.0 release notes

* Cluster-scraper deployment now uses cloudwatch-agent SA (cluster-scraper SA retained as dead resource)

* Bump chart version to 6.0.1 and add release notes

* Scope config-map permission for cloudwatch-agent to namespace only (#287)

* scope configmap permission to namesapce

* integ tests

* updates

* Retain configmap get permission at cluster scope (#291)

* Fix template whitespace control and hardcoded test names

- Use {{- if instead of {{ if in cloudwatch-agent-role.yaml and
  cloudwatch-agent-rolebinding.yaml to prevent leading blank lines
- Extract agentName constant in configmap_permission_scoping_test.go
  to replace hardcoded strings

* Keep configmaps get at cluster scope, scope create/update to namespace

- Add configmaps get-only rule to ClusterRole
- Narrow namespace-scoped Role to create and update only
- Update test assertions to match

* Run integration tests on PRs targeting release-* branches

* Set hostNetwork: true for default $.agent

* Feature/ksm enrichment and label preservation (#296)

* Fix agent.config bleed-through into cluster-scraper deployment (#298)

* Fix scrapeTimeout, add missing scrape_timeout, remove dead RBAC, cleanup docs and tests (#297)

* Add ClusterIP Service for node-exporter and use service DNS for scraping (#304)

* Refactor fluent-bit region-specific (isolated) config using tpl conditionals (#271)

* Pin GitHub Actions to commit SHAs (#306)

* minor fixes (#307)

* release v6.1.0

* Add LIS CSI metrics support to OTEL container insights config (#308)

Add prometheus receiver, scope transform, attribute promote transform,
and pipeline for local instance store CSI driver metrics collection,
mirroring the existing EBS CSI pattern. Scrapes pods with label
app=ec2-instance-store-plugin in kube-system namespace and promotes
instance_id and volume_id to resource attributes.

* feat(otel): Add k8sattributes processor node filter to daemonset (#311)

* fix(integration-test): include node-exporter-service in EKS resource counts (#312)

* Add OTEL-native Container Insights log pipelines (app, host) (#310)

* release v6.2.0

* Address PR review: use v0.18 CRD schema, add version bumps to release notes

---------

Co-authored-by: Louise Allen <147041218+louisall@users.noreply.github.com>
Co-authored-by: Kaushik Surya <kausyas@amazon.com>
Co-authored-by: Kaushik Surya <108111936+sky333999@users.noreply.github.com>
Co-authored-by: Petru Anica-Popa <66786772+petruanica@users.noreply.github.com>
Co-authored-by: Trey Richbourg <63218430+treyrichbourg@users.noreply.github.com>
Co-authored-by: POOJA REDDY NATHALA <poojardy@amazon.com>
Co-authored-by: Michael O'Neill <miconeil@amazon.com>
Co-authored-by: Mitali Salvi <mitsalvi@amazon.com>
vaishnavi-30-beep pushed a commit to vaishnavi-30-beep/helm-charts that referenced this pull request Jul 2, 2026
* Add OTEL Container Insights infrastructure (aws-observability#282)

* Add v6.0.0 release notes

* Cluster-scraper deployment now uses cloudwatch-agent SA (cluster-scraper SA retained as dead resource)

* Bump chart version to 6.0.1 and add release notes

* Scope config-map permission for cloudwatch-agent to namespace only (aws-observability#287)

* scope configmap permission to namesapce

* integ tests

* updates

* Retain configmap get permission at cluster scope (aws-observability#291)

* Fix template whitespace control and hardcoded test names

- Use {{- if instead of {{ if in cloudwatch-agent-role.yaml and
  cloudwatch-agent-rolebinding.yaml to prevent leading blank lines
- Extract agentName constant in configmap_permission_scoping_test.go
  to replace hardcoded strings

* Keep configmaps get at cluster scope, scope create/update to namespace

- Add configmaps get-only rule to ClusterRole
- Narrow namespace-scoped Role to create and update only
- Update test assertions to match

* Run integration tests on PRs targeting release-* branches

* Set hostNetwork: true for default $.agent

* Feature/ksm enrichment and label preservation (aws-observability#296)

* Fix agent.config bleed-through into cluster-scraper deployment (aws-observability#298)

* Fix scrapeTimeout, add missing scrape_timeout, remove dead RBAC, cleanup docs and tests (aws-observability#297)

* Add ClusterIP Service for node-exporter and use service DNS for scraping (aws-observability#304)

* Refactor fluent-bit region-specific (isolated) config using tpl conditionals (aws-observability#271)

* Pin GitHub Actions to commit SHAs (aws-observability#306)

* minor fixes (aws-observability#307)

* release v6.1.0

* Add LIS CSI metrics support to OTEL container insights config (aws-observability#308)

Add prometheus receiver, scope transform, attribute promote transform,
and pipeline for local instance store CSI driver metrics collection,
mirroring the existing EBS CSI pattern. Scrapes pods with label
app=ec2-instance-store-plugin in kube-system namespace and promotes
instance_id and volume_id to resource attributes.

* feat(otel): Add k8sattributes processor node filter to daemonset (aws-observability#311)

* fix(integration-test): include node-exporter-service in EKS resource counts (aws-observability#312)

* Add OTEL-native Container Insights log pipelines (app, host) (aws-observability#310)

* release v6.2.0

* Address PR review: use v0.18 CRD schema, add version bumps to release notes

---------

Co-authored-by: Louise Allen <147041218+louisall@users.noreply.github.com>
Co-authored-by: Kaushik Surya <kausyas@amazon.com>
Co-authored-by: Kaushik Surya <108111936+sky333999@users.noreply.github.com>
Co-authored-by: Petru Anica-Popa <66786772+petruanica@users.noreply.github.com>
Co-authored-by: Trey Richbourg <63218430+treyrichbourg@users.noreply.github.com>
Co-authored-by: POOJA REDDY NATHALA <poojardy@amazon.com>
Co-authored-by: Michael O'Neill <miconeil@amazon.com>
Co-authored-by: Mitali Salvi <mitsalvi@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants