Repository navigation
Regex Rewrites - #4695
Regex Rewrites#4695
Conversation
|
Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Stevenjin8 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
| `pattern` will be a regular expression. | ||
| **ALL** instances of `pattern` in the url path MUST be replaced with `substitution`. | ||
| The url path includes the leading slash. | ||
| If the resulting path is invalid (does not have a leading slash, etc), the proxy MUST return a 500 Internal Server Error. |
There was a problem hiding this comment.
No proxy does this right now. Envoy and traefik both just forward the request with an invalid path. Don't remeber what nginx does but it was super weird.
| ( Required for Experimental status and above) | ||
|
|
||
| * At least one Feature Name must be listed. | ||
| * The `Conformance Details` must be filled out, with conformance test scenarios listed. |
There was a problem hiding this comment.
if we don't have specific APIs, how will we be able to fill conformance tests?
There was a problem hiding this comment.
I think the idea is to have a high level plan of what will happen, then have an specific API with conformance tests.
There was a problem hiding this comment.
yeah might be a bit too early
|
Hey Steven! Thanks for this proposal. I think moving this GAP to provisional totally makes sense. I also agree that supporting syntax/dialect etc should be vendor specific similar to path matchers. In conformance tests we could use some common regex syntax - not to test regex support correctness, but rather to tests API consistency and substitution correctness. |
|
/cc @snorwin |
|
Some unresolved comments: |
|
The biggest problem I see is determining conflicts. What happens if a regex replacement ends up the same as a Path match? Does that that not count because you've "entered" the regex path match first? (To be honest, that's kind of what I would expect, that matches happen in an order like: Exact, Prefix except for What happens when people want to use the captured groups from the path match in other places? Headers? Query parameters?Will we ever allow that? (My feeling would be no, at a minimum that should be a very specific Match type, but we should be specific about what the match groups can and cannot be used for). It's important to remember that, if we have any conformance tests at all, then the we are effectively blessing some subset of regular expression syntax as part of Gateway API, even if it's behind an extended feature. And if we don't have conformance tests, then we should leave this as a custom matcher, and have it be implementation specific. We can't define a behavior and then not require tests for it for something as widely used as this. |
| | Implementation | Support | | ||
| |----------------|------------| | ||
| | Envoy | [config.route.v3.RouteAction.regex_rewrite](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#envoy-v3-api-field-config-route-v3-routeaction-regex-rewrite) | | ||
| | HAProxy | [http-request replace-path](https://cbonte.github.io/haproxy-dconv/2.5/configuration.html#4.2-http-request%20replace-path) | | ||
| | NGINX | [ngx_http_rewrite_module.html#rewrite](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite) | |
There was a problem hiding this comment.
This section also needs to include information about which regex engine is used by each implementation. Should probably include the kgateway/agentgateway proxy as well. We'll need that info to be able to determine if we can do conformance testing at all.
There was a problem hiding this comment.
I dont think agent gateway supports regex substitutions
| This is a provisional GEP, so no specific API details, but at a high level there will be two fields: `pattern` and `substitution`. | ||
| `pattern` will be a regular expression. | ||
| **ALL** instances of `pattern` in the url path MUST be replaced with `substitution`. | ||
| The url path includes the leading slash. | ||
| We do not define what happens if the path is invalid after the substitution (e.g. missing leading slash) |
There was a problem hiding this comment.
This is a pretty big departure from how the rest of path matching works - it only allows matching on full path components - the text between / and /, not on substrings inside path components. That's not a deal breaker, but it's probably worth calling out and then explaining.
There was a problem hiding this comment.
I can see the callout, but this would be completely seperate from path matching. That is you can have an prefix or even exact path and have a regex rewrite configured. Also, fairly sure that normal RegularExpression path matching does not give path components special treatment.
There was a problem hiding this comment.
So really emphasizing the fact that the pattern here, is not the pattern in the pathMatch and this GEP is completely orthogonal to any type of path matching (unlike NGINX, where you can reference capture groups in the match pattern).
| The url path includes the leading slash. | ||
| We do not define what happens if the path is invalid after the substitution (e.g. missing leading slash) | ||
|
|
||
| To be consistent with [HTTP path match](https://gateway-api.sigs.k8s.io/reference/spec/#pathmatchtype), we let implementation define what flavor of regex they will support, and what features they might support. |
There was a problem hiding this comment.
Again, I don't think this is viable without adding Extended features indicating which flavor of regex the implementation supports or something, or defining a smaller subset of regex functionality that MUST be supported, with clear warnings that if you go outside that, you lose portability guarantees.
It's critical that the same HTTPRoute MUST work on another implementation, given the same set of Extended features available across the two implementations. So, we either need to ensure that any two implementations can support the same regex, or we add Extended features to cover the various regex engine options.
There was a problem hiding this comment.
My bad, I mean to delete this, but later I talk about using POSIX ERE as a baseline regex language. I couldn't find any trustworthy sources talking about the explicitly compatibility between POSIX ERE and RE2/PCRE, but from some reading, POSIX ERE seems like a fairly good baseline, especially because we don't need to think about unicode, newlines, control characters, etc.
|
@youngnick This is not relevant to this PR, but I'll add that the priority of RegularExpression matches is undefined by the gateway api spec. For example, Istio and Envoy Gateway give them different priorities (istio/istio#58641), which is a bit annoying, especially when trying to define catch-all routes, but its probably too late to change now. |
eb8e8ed to
42673be
Compare
| We do not define what happens if the path is invalid after the substitution (e.g. missing leading slash) | ||
|
|
||
| If the implementation deems `pattern` or `substitution` to be invalid (contains illegal characters, unmatched parenthesis, unsupported features), | ||
| it MUST not accept the HTTPRoute with a `Reason` of `invalid` and a descriptive `Message`. |
There was a problem hiding this comment.
Should the entire route be rejected? Can the invalid rule alone be rejected and have the appropriate error message (similar to an invalid backendref on one rule) ?
There was a problem hiding this comment.
Yeah, I agree that this behavior should be defined, and more, I think that rejecting the rule is probably right.
There was a problem hiding this comment.
@youngnick +1, we can just call the route "invalid", and let the implementation decide what to do subject to contrains
https://github.com/kubernetes-sigs/gateway-api/blob/main/apis/v1/shared_types.go#L435-L470
|
/retest |
|
@youngnick the GWAPI recording has not been uploaded, so most of this is from word-of-mouth. It seems like we want some very strong validation on the regex expression at the API-level. I'm happy to restrict the character set to something like |
|
@Stevenjin8, yeah, we want to make sure we limit this to a limited subset of regex characters as far as possible. Sadly, I think we'll need We'll need negative conformance tests for all the cases you call out in that section as well, maybe? Like, not supporting classes etc, so no |
|
The Kubernetes project currently lacks enough contributors to adequately respond to all PRs. This bot triages PRs according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
11238fc to
0ef626c
Compare
|
@youngnick @rikatz @robscott now that #4764 merged, would love to get another review. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: robscott, Stevenjin8 The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/release-note-none |
|
/release-note-none |
|
/lifecycle active |
|
/lgtm This looks good for provisional, we can iterate better into the how during Implementable and Experimental (I do expect more deep discussion happening on it) thanks @Stevenjin8 |
What type of PR is this?
/kind feature
Takeover of #4361
What this PR does / why we need it:
Does this PR introduce a user-facing change?:
No