Skip to content

docs(kiro): make blast radius a first-class review question - #4954

Merged
kddejong merged 1 commit into
aws-cloudformation:mainfrom
kddejong:docs/review-blast-radius
Oct 28, 2026
Merged

kddejong merged 1 commit into
aws-cloudformation:mainfrom
kddejong:docs/review-blast-radius

Conversation

@kddejong

Copy link
Copy Markdown
Contributor

Documentation only — three .kiro agent-context edits. No rule code, no schema, no test changes.

These came out of reviewing #4937, which fixes an IAM Sid false positive by editing uniq_keys in src/cfnlint/jsonschema/_utils.py. That helper has one caller, the uniqueKeys keyword, which governs 13 schema sites including other/resources/tagging.json — so it reaches every resource type's Tags, and ECS Environment, DynamoDB KeySchema and CodePipeline Stages all lose their duplicate-empty-key check. The review found those regressions and still recommended approval, because nothing in the context said blast radius outranks constraint correctness. These edits say it.

1. .kiro/context/false-positives.md — two questions at the top of the review checklist

The checklist's six questions were all about whether a constraint is correct; none asked what the change reaches. Added "What does this change reach?" and "What does it stop reporting?" as questions 1-2, renumbering the rest to 3-8. Judging a constraint is premature if you do not know what it governs.

Heading retitled "Reviewing someone else's rule" → "change", since question 1 is specifically about changes that are not rules. No other file links to that anchor.

2. .kiro/context/verification.md — rank a stack probe above a service API probe

Item 3 of the evidence ladder read as a caveat on an acceptable choice ("Second-best: the service and the CloudFormation provider do not always agree"). Added a Prefer a stack paragraph after the list: many deployment probes are free because a synchronous rejection creates nothing, so a service API probe is a fallback for when a stack probe would be costly or impossible — not for when it is merely easier.

Plus a recorded example, matching the existing recorded notes in that file.

Evidence — us-east-1, 2026-10-28, AWS::IAM::Role with inline Policies via CreateStack:

Template Result
two statements, both Sid: "" CREATE_COMPLETE
two statements, same non-empty Sid ROLLBACK_COMPLETE — Statement IDs (SID) in a single policy must be unique. (Service: Iam, Status Code: 400)
Sid: "A\n" ROLLBACK_COMPLETE — Statement IDs (SID) must be alpha-numeric. Check that your input satisfies the regular expression [0-9A-Za-z]*
two distinct Sids CREATE_COMPLETE

The provider surfaces IAM's own errors verbatim, so the service and provider levels agree here. Duplicate empty Sids are legal, which is the #4937 quirk.

3. .kiro/steering/rule-development.md — how to scope keyword behaviour to one rule

Nothing documented this, and it has caused two defects in a week: #4924 broke intrinsic-function validation inside the Rules section by changing which schema keyword descended, and #4937 loosens uniqueKeys for 13 sites to fix one IAM quirk. Both had a scoped option available; neither author knew it existed.

Added a short section after the Rule Types list: register the override in self.validators and delegate to the stock keyword, rather than editing the keyword in src/cfnlint/jsonschema/.

Claim verified before writing it — all four cited rules do register a patternProperties override in self.validators pointing at a _pattern_properties method that delegates to the stock patternProperties imported from cfnlint.jsonschema._keywords:

  • E1700 src/cfnlint/rules/rules/Configuration.py:32
  • src/cfnlint/rules/outputs/Configuration.py:44
  • src/cfnlint/rules/parameters/Configuration.py:43
  • src/cfnlint/rules/resources/Configuration.py:55 (also sets maxProperties and propertyNames to None)

No correction to the sentence was needed.

Verification

pre-commit run --all-files
pre-commit run --files .kiro/context/false-positives.md \
  .kiro/context/verification.md .kiro/steering/rule-development.md   # all Passed
git diff --stat -- test/ src/                                        # empty

Markdown only, so ruff/isort/bandit/mypy have no files to check. The all-files run reports end-of-file-fixer failing on three .kiro/agents/*.json files — pre-existing local tooling rewrites, deliberately not staged and not part of this PR. Every hook passes on the three files changed here, and test/ and src/ are untouched rather than assumed untouched.

Net: +35 / -7 across three markdown files.

Add two questions at the top of the review checklist: what the change
reaches, and what it stops reporting. A change to shared validator
machinery is a change to the engine, not to a rule.

Rank a stack probe above a service API probe, with a recorded IAM Sid
example showing the provider surfaces the service's own errors.

Document registering a keyword override in self.validators as the way to
scope keyword behaviour to one rule, instead of editing the keyword.
@kddejong
kddejong marked this pull request as ready for review October 28, 2026 21:19
@kddejong
kddejong merged commit 0270355 into aws-cloudformation:main Oct 28, 2026
22 checks passed
@kddejong
kddejong deleted the docs/review-blast-radius branch October 28, 2026 23:04
@codecov

codecov Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.69%. Comparing base (f302ad0) to head (e3621ad).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4954   +/-   ##
=======================================
  Coverage   94.69%   94.69%           
=======================================
  Files         442      442           
  Lines       16092    16092           
  Branches     3121     3121           
=======================================
  Hits        15238    15238           
  Misses        470      470           
  Partials      384      384           
Flag Coverage Δ
unittests 94.68% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant