Skip to content

test(conformance): add normative tests for HTTPRoute retries - #4817

Merged
k8s-ci-robot merged 1 commit into
kubernetes-sigs:mainfrom
snorwin:httproute-retries-test
May 19, 2026
Merged

k8s-ci-robot merged 1 commit into
kubernetes-sigs:mainfrom
snorwin:httproute-retries-test

Conversation

@snorwin

@snorwin snorwin commented May 1, 2026

Copy link
Copy Markdown
Member

What type of PR is this?

/kind test
/area conformance-test

What this PR does / why we need it:
This PR adds a new conformance test, HTTPRouteRetries, which validates that implementations correctly handle the retry configuration on HTTPRoute rules.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/test area/conformance-test Issues or PRs related to Conformance tests. labels May 1, 2026
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 1, 2026
@snorwin

snorwin commented May 1, 2026

Copy link
Copy Markdown
Member Author

/cc @mikemorris

@k8s-ci-robot
k8s-ci-robot requested a review from mikemorris May 1, 2026 13:28
@snorwin
snorwin force-pushed the httproute-retries-test branch 3 times, most recently from 118e621 to 19739c0 Compare May 2, 2026 19:24
@snorwin

snorwin commented May 2, 2026

Copy link
Copy Markdown
Member Author

/retest

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 4, 2026
@snorwin
snorwin force-pushed the httproute-retries-test branch from 19739c0 to fcf74d0 Compare May 4, 2026 12:53
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 4, 2026
@snorwin
snorwin force-pushed the httproute-retries-test branch from fcf74d0 to 5fd2269 Compare May 4, 2026 12:54
Comment thread conformance/echo-basic/echo-basic.go Resolved
Comment thread conformance/echo-basic/echo-basic.go Resolved
codes:
- 500
attempts: 3
backoff: 100ms

@rikatz rikatz May 13, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are not really testing the backoff, right? Should we be adding it here, or open a followup for implementations that decide to support it, even because the API specified different possible behaviors for the backoff?

@mikemorris

@snorwin snorwin May 15, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we are not really testing the backoff here, I decided to skip it for now and just set the interval low so the test is fast. According to the spec, the backoff would only apply from the first retry attempt, and implementations may use an exponential or alternative backoff strategy for subsequent attempts, so the behavior isn't guaranteed across implementations.

My experience is that asserting timing often introduces flakiness in tests. I'm happy to open a followup PR to test e.g. backoff of 1s and 2s.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

followup looks good

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I swear we had some mechanism in the HTTPRouteTimeout tests to mock timing - maybe @kflynn remembers?

@mikemorris mikemorris Jun 3, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open to making the behavior more explicit if we wanted to add something like a strategy: { exponential, linear } field with most implementations only supporting one (and probably exponential as default to avoid verbosity and since it's almost always preferable).

Comment thread conformance/tests/httproute-retries.go Resolved
Comment thread conformance/tests/httproute-retries.go Resolved
@snorwin
snorwin force-pushed the httproute-retries-test branch from 5fd2269 to cb1f2f0 Compare May 15, 2026 14:43
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 15, 2026
@snorwin
snorwin force-pushed the httproute-retries-test branch from cb1f2f0 to 2694ac3 Compare May 15, 2026 14:46
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 15, 2026
@snorwin
snorwin force-pushed the httproute-retries-test branch from 2694ac3 to b068554 Compare May 15, 2026 14:48
Signed-off-by: Norwin Schnyder <norwin.schnyder+github@gmail.com>
@snorwin
snorwin force-pushed the httproute-retries-test branch from b068554 to 580c531 Compare May 19, 2026 08:06
@snorwin
snorwin requested a review from rikatz May 19, 2026 08:07
@rikatz

rikatz commented May 19, 2026

Copy link
Copy Markdown
Member

/approve
/hold
testing on some more implementation, this already passes on Istio and Envoy GAteway

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 19, 2026
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rikatz, snorwin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 19, 2026
@rikatz

rikatz commented May 19, 2026

Copy link
Copy Markdown
Member

/lgtm

tested on Istio, Envoy Gateway and kgateway, works as expected. Also I don't have further comments to be addressed here.

Thanks @snorwin

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 19, 2026
@rikatz

rikatz commented May 19, 2026

Copy link
Copy Markdown
Member

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 19, 2026
@rikatz

rikatz commented May 19, 2026

Copy link
Copy Markdown
Member

/retest-required

@rikatz

rikatz commented May 19, 2026

Copy link
Copy Markdown
Member

/test pull-gateway-api-verify

@k8s-ci-robot
k8s-ci-robot merged commit 6668509 into kubernetes-sigs:main May 19, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/conformance-test Issues or PRs related to Conformance tests. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/test lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants