Repository navigation
gep: align GEP-1731 retry semantics with implementations - #5167
Conversation
|
/cc @mikemorris @rikatz |
|
/retest |
Signed-off-by: Norwin Schnyder <norwin.schnyder+github@gmail.com>
Signed-off-by: Norwin Schnyder <norwin.schnyder+github@gmail.com>
|
/lgtm @youngnick @robscott for approval, as I am biased on this one |
|
/cc @youngnick @robscott |
| // | ||
| // +optional | ||
| // <gateway:experimental> | ||
| // +listType=set |
There was a problem hiding this comment.
This isn't the actual types, but we need to ensure that <gateway:experimental> stays until the feature graduates to Standard...
There was a problem hiding this comment.
Yes, this only applies to the GEP. The other fields that are also considered experimental don't have the marker too.
Once we have agreed on the changes, I'm going to add them to the API as well.
| // `100ms` will cause a backend request to first be retried approximately | ||
| // 100 milliseconds after timing out or receiving a response code configured | ||
| // to be retriable. | ||
| // The maximum duration a Gateway should wait before a retry attempt is |
There was a problem hiding this comment.
This mandates that implementations must use exponential backoff, right? We didn't want to mandate that before, although it seems like it's almost always a better idea.
There was a problem hiding this comment.
The most common backoff strategies are:
- none
- fixed
- linear
- exponential
(Each of these can be combined with jitter and with a max cap)
For HTTP traffic, IMO the most common choice is capped exponential backoff with full jitter.
HAProxy and NGINX support only none. Traefik and Envoy support capped exponential backoff with jitter.
Before this change, backoff was specified as a lower bound on the retry interval a semantic that none of these data plane technologies can satisfy.
There was a problem hiding this comment.
For posterity, my original intent with the phrasing of minimum here was to capture constant, linear and exponential backoff strategies (without being prescriptive, and none could be implemented as simply not supporting backoff at all, as exponential is often preferable but not implemented universally or necessarily with the exact same algorithm in all dataplanes), such that with backoff: 100ms various strategies could plausibly look like:
exponential
min(((Base * 2^(attempt-1)) + Jitter), MaxDelay)
5xx -> wait (100 * 2^1 - 1 = 1) = 100)ms -> 5xx -> wait (100 * 2^2 - 1 = 3) = 300)ms -> 5xx -> wait (100 * 2^3 - 1 = 7) = 700)ms
100ms initial backoff, 1100ms total delays excluding request time
linear
min(((Base * attempt) + Jitter), MaxDelay)
5xx -> wait (100 * 1 = 100)ms -> 5xx -> wait (100 * 2 = 200)ms -> 5xx -> wait (100 * 3 = 300)ms
100ms initial backoff, 600ms total delays excluding request time
constant
Base + Jitter
5xx -> wait 100ms -> 5xx -> wait 100ms -> 5xx -> wait 100ms
100ms initial backoff, 300ms total delays excluding request time
That said, I think the revisions here do capture the same intent I had, with a bit more precision in specifics of jitter (which I think is typically added though, not subtracted, which is how I got to a minimum value rather than maximum) and max delay implementation variables. I'm good with leaving this impl-specific for v1.7, but I am still curious whether a minimum value (with explicit behavior exceptions for .Retry-After headers as described at https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#envoy-v3-api-field-config-route-v3-retrypolicy-rate-limited-retry-back-off) is actually able to be successfully tested in conformance
EDIT: After investigating more closely in #5347 (comment), I'm now confident backoff can't move to Extended with conformance tests, and should graduate to standard as implementation-specific.
Signed-off-by: Norwin Schnyder <norwin.schnyder+github@gmail.com>
|
/retest |
|
@kubernetes-sigs/gateway-api-maintainers if we want HTTP retries graduating in v1.7, this needs to be merged soon, PTAL. |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: rikatz, snorwin The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
65b6c78
into
kubernetes-sigs:main
What type of PR is this?
/kind gep
What this PR does / why we need it:
GEP-1731 currently specifies
backoffas "the minimum duration a Gateway should wait between retry attempts". None of the current implementations provide this guarantee. Airlock Microgateway, Istio, Envoy Gateway and kgateway all mapbackoffto Envoy'sretry_back_off.base_interval.In addition, this PR clarifies the retry behavior on connection errors. The previous wording ("disconnect, reset, timeout, TCP failure") included conditions where the request may already have been sent and processed.
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?:
AI Disclosure
AI (Claude;Fable 5) was used only for spell-checking and minor rephrasing suggestions.