Skip to content

fix(E3601): support state-level JSONata and JSONata numeric fields - #4947

Merged
kddejong merged 1 commit into
aws-cloudformation:mainfrom
hoganbr-amzn:fix/4881-jsonata-state-level
Nov 3, 2026
Merged

kddejong merged 1 commit into
aws-cloudformation:mainfrom
hoganbr-amzn:fix/4881-jsonata-state-level

Conversation

@hoganbr-amzn

Copy link
Copy Markdown
Contributor

Issue #, if available: Fixes #4881

Description of changes:

E3601 picks the JSONPath or JSONata schema from the top-level QueryLanguage only, so states that set their own QueryLanguage are validated against the wrong schema. This change:

  • Validates a state with QueryLanguage: JSONata against the JSONata state schema when the state machine uses JSONPath. A state without QueryLanguage inherits from the top-level field, not from a parent Map or Parallel state, so nested states keep the top-level language unless they set their own (docs).
  • Allows QueryLanguage, InputPath and OutputPath on Succeed states. Succeed now shares the common definition with the other state types, so it also gains two errors that match Step Functions: a JSONPath Succeed with Output is rejected (it passed before), and a JSONata Succeed with InputPath or OutputPath is rejected.
  • Allows a JSONata expression string for Map ItemSelector.
  • Allows a JSONata expression string for Task TimeoutSeconds and HeartbeatSeconds, and for Map MaxConcurrency, ToleratedFailureCount and ToleratedFailurePercentage. Under JSONata the matching *Path fields are rejected, along with Wait SecondsPath and TimestampPath, because Step Functions rejects them there.
  • Builds the JSONata schema once in __init__ instead of on every validation.

The state-level JSONata routes are appended to the end of the state allOf, so the schema_path of existing JSONPath errors does not change. In the existing "Invalid query languages" test, two route indices move (Task 5 to 13, Choice 0 to 8), and a JSONata Choice state now also reports And as an unexpected property, which Step Functions rejects too.

Testing:

  • Added test_query_language with a case for each shape above, including the E3601: JSONata validation misses state-level QueryLanguage, QueryLanguage on Succeed, and a string Map ItemSelector #4881 template, plus the silent cases: a boundary value (TimeoutSeconds: 1), a Ref in a numeric field, and a Ref as a state's QueryLanguage, and an Fn::If as a state's QueryLanguage that expects one error from each branch, proving both are checked. The cases compare (message, path) pairs rather than full ValidationError objects so that the expected values do not depend on allOf route indices, and assert that every error comes from E3601.
  • Checked 51 state machine definitions against the Step Functions ValidateStateMachineDefinition API (us-west-2, 2026-10-27): the E3601: JSONata validation misses state-level QueryLanguage, QueryLanguage on Succeed, and a string Map ItemSelector #4881 template, each shape above as its own definition, the matching invalid shapes (*Path fields under JSONata, Output on a JSONPath Succeed, out-of-range values), and a nested JSONata Map whose child state uses JSONPath. cfn-lint reports no error on any definition the API accepts.
  • Deployed a template with a JSONata root state machine and a state-level JSONata state machine covering the shapes above (us-west-2, 2026-10-28, CREATE_COMPLETE). cfn-lint 1.58.0 reports 15 E3601 errors on it, and this branch reports none.
  • Every template under test/fixtures/templates/good/ still lints with no E3601 finding.

Commands run:

python -m pytest test/unit/rules test/unit/module/test_rules_collections.py
pre-commit run --files src/cfnlint/rules/resources/stepfunctions/StateMachineDefinition.py src/cfnlint/data/schemas/other/step_functions/statemachine.json test/unit/rules/resources/stepfunctions/test_state_machine_definition.py
cfn-lint -t "test/fixtures/templates/good/**/*.yaml"
aws stepfunctions validate-state-machine-definition --type STANDARD --definition "$DEFINITION"
aws cloudformation create-stack --stack-name cfnlint-4881-probe --template-body file://deploy-template.yaml --capabilities CAPABILITY_IAM
deploy-template.yaml
AWSTemplateFormatVersion: "2010-09-09"
Description: cfn-lint 4881 probe, JSONata fields cfn-lint rejects
Resources:
  Role:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal: {Service: states.amazonaws.com}
            Action: sts:AssumeRole
  StateLevelJsonata:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      RoleArn: !GetAtt Role.Arn
      Definition:
        StartAt: Task
        States:
          Task:
            Type: Task
            QueryLanguage: JSONata
            Resource: arn:aws:states:::lambda:invoke
            Arguments: "{% {'FunctionName': $states.input.fn, 'Payload': $states.input} %}"
            TimeoutSeconds: "{% $states.input.timeout %}"
            HeartbeatSeconds: "{% $states.input.heartbeat %}"
            Next: Wait
          Wait:
            Type: Wait
            QueryLanguage: JSONata
            Seconds: "{% $states.input.seconds %}"
            Next: Map
          Map:
            Type: Map
            QueryLanguage: JSONata
            Items: "{% $states.input.items %}"
            ItemSelector: "{% $states.context.Map.Item.Value %}"
            MaxConcurrency: "{% $states.input.concurrency %}"
            ItemProcessor:
              StartAt: Child
              States:
                Child:
                  Type: Pass
                  InputPath: $
                  End: true
            Next: Pick
          Pick:
            Type: Choice
            Choices:
              - Variable: $.jsonata
                IsPresent: true
                Next: JsonataSucceed
            Default: JsonPathSucceed
          JsonPathSucceed:
            Type: Succeed
            InputPath: $
            OutputPath: $
          JsonataSucceed:
            Type: Succeed
            QueryLanguage: JSONata
            Output: "{% $states.input %}"
  RootJsonata:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      RoleArn: !GetAtt Role.Arn
      Definition:
        QueryLanguage: JSONata
        StartAt: Task
        States:
          Task:
            Type: Task
            Resource: arn:aws:states:::lambda:invoke
            Arguments:
              FunctionName: "{% $states.input.fn %}"
            TimeoutSeconds: "{% $states.input.timeout %}"
            HeartbeatSeconds: "{% $states.input.heartbeat %}"
            Next: Map
          Map:
            Type: Map
            Items: "{% $states.input.items %}"
            ItemSelector: "{% $states.context.Map.Item.Value %}"
            MaxConcurrency: "{% $states.input.concurrency %}"
            ToleratedFailureCount: "{% $states.input.count %}"
            ToleratedFailurePercentage: "{% $states.input.percent %}"
            ItemProcessor:
              ProcessorConfig: {Mode: DISTRIBUTED, ExecutionType: STANDARD}
              StartAt: Child
              States:
                Child: {Type: Pass, End: true}
            End: true

Known gaps:

  • Step Functions accepts a string in the numeric fields only when the whole value is a {% ... %} expression, so cfn-lint still accepts a plain string such as "TimeoutSeconds": "10". A pattern check would also apply to numbers, so it is left for a follow-up.
  • Every state route now depends on QueryLanguage, so a state whose QueryLanguage is a value cfn-lint cannot resolve (a Ref to a parameter, including one with AllowedValues or a Default) is not validated. Before this change such a state was validated as JSONPath, which reported JSONata expression strings as errors. Fn::If is unaffected: each branch is checked. The test Unresolvable QueryLanguage on a state skips the state records this.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@hoganbr-amzn
hoganbr-amzn marked this pull request as ready for review October 28, 2026 17:52
@kddejong
kddejong force-pushed the fix/4881-jsonata-state-level branch from 362c7b7 to e04868d Compare November 3, 2026 22:45
@codecov

codecov Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.83333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 94.80%. Comparing base (dbb3da3) to head (e04868d).

Files with missing lines Patch % Lines
.../resources/stepfunctions/StateMachineDefinition.py 95.83% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4947   +/-   ##
=======================================
  Coverage   94.80%   94.80%           
=======================================
  Files         441      441           
  Lines       16128    16148   +20     
  Branches     3130     3133    +3     
=======================================
+ Hits        15290    15309   +19     
  Misses        463      463           
- Partials      375      376    +1     
Flag Coverage Δ
unittests 94.79% <95.83%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kddejong
kddejong merged commit f8018a2 into aws-cloudformation:main Nov 3, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

E3601: JSONata validation misses state-level QueryLanguage, QueryLanguage on Succeed, and a string Map ItemSelector

2 participants