Skip to content

Refactor: Use tpl conditionals for fluent-bit region-specific config - #271

Merged
movence merged 1 commit into
aws-observability:mainfrom
treyrichbourg:main
Apr 28, 2026
Merged

movence merged 1 commit into
aws-observability:mainfrom
treyrichbourg:main

Conversation

@treyrichbourg

@treyrichbourg treyrichbourg commented Feb 4, 2026 •

Copy link
Copy Markdown
Contributor

Issue #, if available:
N/A (follow-up to #257)

Description of changes:
This change aims to simplify isolated region configuration using single extraFiles with tpl functions provided by Helm.

Currently, the chart maintains separate configuration for adcRegionExtraFiles resulting in:

  • Duplication
  • Confusing user experience (User defined extraFiles will be overridden in isolated regions by the template which can be painful to debug)
  • Difficult maintenance when there are configuration changes
  • Risk of configuration drift between regions

This change uses Helm's tpl function with inline conditionals in values.yaml to dynamically render the correct default fluent-bit configuration for all regions from a single source. Currently values set to On or true I simply set to Off or false for isolated regions but conditional can easily be used to simply remove this inline during rendering if needed. I think this is also a good chance to evaluate the parity in these configurations and if they can simply be enabled for these regions.

Configuration differences were identified via diff between commercial and isolated region configs:

comet@pop-os:~/Projects/observability% diff commercial.txt adc.txt 
39,45d38
<             Name                aws
<             Match               application.*
<             az                  false
<             ec2_instance_id     false
<             Enable_Entity       true
< 
<           [FILTER]
59d51
<             Use_Pod_Association On
67a60
>             endpoint            logs.${AWS_REGION}.${ADC_REGION_ENDPOINT}
69d61
<             add_entity          true
113a106
>             endpoint            logs.${AWS_REGION}.${ADC_REGION_ENDPOINT}
171a165
>             endpoint            logs.${AWS_REGION}.${ADC_REGION_ENDPOINT}

This change has a lot of benefits including:

  • Single configuration source eliminates configuration drift between regions
  • Consistent user experience when using advanced configuration (extraFiles) where intent is preserved for all regions
  • Self-documenting configuration using inline conditionals

Testing

wtr@dev:/local/home/wtr/Projects/helm-charts% helm lint charts/amazon-cloudwatch-observability --set region='us-east-1' --set clusterName="test"
==> Linting charts/amazon-cloudwatch-observability
[INFO] Chart.yaml: icon is recommended

1 chart(s) linted, 0 chart(s) failed
wtr@dev:/local/home/wtr/Projects/helm-charts% helm lint charts/amazon-cloudwatch-observability --set region='us-isob-east-1' --set clusterName="test"
==> Linting charts/amazon-cloudwatch-observability
[INFO] Chart.yaml: icon is recommended

1 chart(s) linted, 0 chart(s) failed
comet@pop-os:~/Projects/helm-charts% helm template test charts/amazon-cloudwatch-observability -f charts/amazon-cloudwatch-observability/values.yaml --set region=us-east-1 --set clusterName='testCluster' > rendered.yaml
comet@pop-os:~/Projects/helm-charts% helm template test charts/amazon-cloudwatch-observability -f charts/amazon-cloudwatch-observability/values.yaml --set region=us-isob-east-1 --set clusterName='testCluster' > renderedIso.yaml
comet@pop-os:~/Projects/helm-charts% diff rendered.yaml renderedIso.yaml 
---
---
omit for brevity
---
---
<     [FILTER]
<       Name                aws
<       Match               application.*
<       az                  false
<       ec2_instance_id     false
<       Enable_Entity       true
205,206c199
<       Use_Pod_Association On
<     
---
>       
215,216c208,209
<       add_entity          true
<     
---
>       endpoint            logs.${AWS_REGION}.sc2s.sgov.gov
>       
261a255
>       endpoint            logs.${AWS_REGION}.sc2s.sgov.gov
320a315
>       endpoint            logs.${AWS_REGION}.sc2s.sgov.gov
764c759
<         checksum/config: 3dc2029a374dd405bfc5878e0f77ee3eb00a016001cd384ac7c90d6d7a1e4a8d
---
>         checksum/config: 24ed911cefb206caeabd8a17f238ec4f70087470e628477aa4d40d31e8efe572
776c771
<           value: us-east-1
---
>           value: us-isob-east-1
793a789,790
>         - name: ADC_REGION_ENDPOINT
>           value: "sc2s.sgov.gov"
916c913
<           value: us-east-1
---
>           value: us-isob-east-1

This chart was also successfully deployed in us-west-2 and us-isob-east-1.
Validated addon activated, configmap was accurate, and logs emitting as expected. Screenshots from us-west-2 below.

image cwa2_pdx

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@treyrichbourg treyrichbourg changed the title Refactor fluent-bit-configmap extraFiles template in isolated regions Refactor: Use marker-based templating for fluent-bit configs Feb 4, 2026
@treyrichbourg
treyrichbourg force-pushed the main branch 3 times, most recently from 7df3ad3 to 69e4eee Compare February 4, 2026 20:14
@treyrichbourg treyrichbourg changed the title Refactor: Use marker-based templating for fluent-bit configs Refactor: Use tpl conditionals for fluent-bit region-specific config Feb 4, 2026
@movence
movence merged commit 92d73bb into aws-observability:main Apr 28, 2026
15 of 18 checks passed
@movence movence mentioned this pull request Apr 28, 2026
movence pushed a commit that referenced this pull request May 5, 2026
@movence movence mentioned this pull request May 5, 2026
mitali-salvi added a commit that referenced this pull request Jun 10, 2026
* Add OTEL Container Insights infrastructure (#282)

* Add v6.0.0 release notes

* Cluster-scraper deployment now uses cloudwatch-agent SA (cluster-scraper SA retained as dead resource)

* Bump chart version to 6.0.1 and add release notes

* Scope config-map permission for cloudwatch-agent to namespace only (#287)

* scope configmap permission to namesapce

* integ tests

* updates

* Retain configmap get permission at cluster scope (#291)

* Fix template whitespace control and hardcoded test names

- Use {{- if instead of {{ if in cloudwatch-agent-role.yaml and
  cloudwatch-agent-rolebinding.yaml to prevent leading blank lines
- Extract agentName constant in configmap_permission_scoping_test.go
  to replace hardcoded strings

* Keep configmaps get at cluster scope, scope create/update to namespace

- Add configmaps get-only rule to ClusterRole
- Narrow namespace-scoped Role to create and update only
- Update test assertions to match

* Run integration tests on PRs targeting release-* branches

* Set hostNetwork: true for default $.agent

* Feature/ksm enrichment and label preservation (#296)

* Fix agent.config bleed-through into cluster-scraper deployment (#298)

* Fix scrapeTimeout, add missing scrape_timeout, remove dead RBAC, cleanup docs and tests (#297)

* Add ClusterIP Service for node-exporter and use service DNS for scraping (#304)

* Refactor fluent-bit region-specific (isolated) config using tpl conditionals (#271)

* Pin GitHub Actions to commit SHAs (#306)

* minor fixes (#307)

* release v6.1.0

* Add LIS CSI metrics support to OTEL container insights config (#308)

Add prometheus receiver, scope transform, attribute promote transform,
and pipeline for local instance store CSI driver metrics collection,
mirroring the existing EBS CSI pattern. Scrapes pods with label
app=ec2-instance-store-plugin in kube-system namespace and promotes
instance_id and volume_id to resource attributes.

* feat(otel): Add k8sattributes processor node filter to daemonset (#311)

* fix(integration-test): include node-exporter-service in EKS resource counts (#312)

* Add OTEL-native Container Insights log pipelines (app, host) (#310)

* release v6.2.0

* Address PR review: use v0.18 CRD schema, add version bumps to release notes

---------

Co-authored-by: Louise Allen <147041218+louisall@users.noreply.github.com>
Co-authored-by: Kaushik Surya <kausyas@amazon.com>
Co-authored-by: Kaushik Surya <108111936+sky333999@users.noreply.github.com>
Co-authored-by: Petru Anica-Popa <66786772+petruanica@users.noreply.github.com>
Co-authored-by: Trey Richbourg <63218430+treyrichbourg@users.noreply.github.com>
Co-authored-by: POOJA REDDY NATHALA <poojardy@amazon.com>
Co-authored-by: Michael O'Neill <miconeil@amazon.com>
Co-authored-by: Mitali Salvi <mitsalvi@amazon.com>
vaishnavi-30-beep pushed a commit to vaishnavi-30-beep/helm-charts that referenced this pull request Jul 2, 2026
* Add OTEL Container Insights infrastructure (aws-observability#282)

* Add v6.0.0 release notes

* Cluster-scraper deployment now uses cloudwatch-agent SA (cluster-scraper SA retained as dead resource)

* Bump chart version to 6.0.1 and add release notes

* Scope config-map permission for cloudwatch-agent to namespace only (aws-observability#287)

* scope configmap permission to namesapce

* integ tests

* updates

* Retain configmap get permission at cluster scope (aws-observability#291)

* Fix template whitespace control and hardcoded test names

- Use {{- if instead of {{ if in cloudwatch-agent-role.yaml and
  cloudwatch-agent-rolebinding.yaml to prevent leading blank lines
- Extract agentName constant in configmap_permission_scoping_test.go
  to replace hardcoded strings

* Keep configmaps get at cluster scope, scope create/update to namespace

- Add configmaps get-only rule to ClusterRole
- Narrow namespace-scoped Role to create and update only
- Update test assertions to match

* Run integration tests on PRs targeting release-* branches

* Set hostNetwork: true for default $.agent

* Feature/ksm enrichment and label preservation (aws-observability#296)

* Fix agent.config bleed-through into cluster-scraper deployment (aws-observability#298)

* Fix scrapeTimeout, add missing scrape_timeout, remove dead RBAC, cleanup docs and tests (aws-observability#297)

* Add ClusterIP Service for node-exporter and use service DNS for scraping (aws-observability#304)

* Refactor fluent-bit region-specific (isolated) config using tpl conditionals (aws-observability#271)

* Pin GitHub Actions to commit SHAs (aws-observability#306)

* minor fixes (aws-observability#307)

* release v6.1.0

* Add LIS CSI metrics support to OTEL container insights config (aws-observability#308)

Add prometheus receiver, scope transform, attribute promote transform,
and pipeline for local instance store CSI driver metrics collection,
mirroring the existing EBS CSI pattern. Scrapes pods with label
app=ec2-instance-store-plugin in kube-system namespace and promotes
instance_id and volume_id to resource attributes.

* feat(otel): Add k8sattributes processor node filter to daemonset (aws-observability#311)

* fix(integration-test): include node-exporter-service in EKS resource counts (aws-observability#312)

* Add OTEL-native Container Insights log pipelines (app, host) (aws-observability#310)

* release v6.2.0

* Address PR review: use v0.18 CRD schema, add version bumps to release notes

---------

Co-authored-by: Louise Allen <147041218+louisall@users.noreply.github.com>
Co-authored-by: Kaushik Surya <kausyas@amazon.com>
Co-authored-by: Kaushik Surya <108111936+sky333999@users.noreply.github.com>
Co-authored-by: Petru Anica-Popa <66786772+petruanica@users.noreply.github.com>
Co-authored-by: Trey Richbourg <63218430+treyrichbourg@users.noreply.github.com>
Co-authored-by: POOJA REDDY NATHALA <poojardy@amazon.com>
Co-authored-by: Michael O'Neill <miconeil@amazon.com>
Co-authored-by: Mitali Salvi <mitsalvi@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants