Skip to content

Provisional: Endpointselector GEP - #4731

Open
keithmattix wants to merge 6 commits into
kubernetes-sigs:mainfrom
keithmattix:endpointselector-gep
Open

keithmattix wants to merge 6 commits into
kubernetes-sigs:mainfrom
keithmattix:endpointselector-gep

Conversation

@keithmattix

@keithmattix keithmattix commented Apr 6, 2026 •

Copy link
Copy Markdown
Contributor

This PR introduces a new EndpointSelector resource that allows implementations to get a "back of endpoints" for any number of pods matching a workload selector. As explained in the text, this GEP is an expedient measure meant to give us time to iterate and get feedback while an analogous KEP percolates through the upstream Kubernetes process. This resource should not progress past experimental

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot

Copy link
Copy Markdown
Contributor

Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 6, 2026
@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 6, 2026
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: keithmattix
Once this PR has been reviewed and has the lgtm label, please assign aojea for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the kind/gep PRs related to Gateway Enhancement Proposal(GEP) label Apr 6, 2026
Comment thread geps/gep-xxx/index.md Resolved
Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>
@keithmattix
keithmattix marked this pull request as ready for review April 23, 2026 19:09
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 23, 2026
Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>
Comment thread geps/gep-4914/index.md

There are two main interfaces that will trigger the creation of an `EndpointSelector`:

1. When a user manually creates an `EndpointSelector` resource. In this case, the user opts-in to managing the lifecycle of the resource (e.g. nothing will set ownerReferences). Most of the time, users will choose this UX when wanting to decorate an existing `Service` with context or functionality described in a higher-level resource (e.g. `Backend` type `EndpointSelector` (TODO: confirm naming once `Backend` GEP is finalized)). These resources would reference the `EndpointSelector` via an object reference field in their spec. CRDs that utilize this pattern SHOULD add status to their APIs to report the validity of the reference.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we expect the endpointselector to be also supported as a backendRef? Let's say today we already support it via InferencePool, and as I can see below we say that an endpointSelector will be created by it, but given a scenario I want to create it manually, and say "my backends should be acquired from this endpointSelector" instead of relying on inferencePool, would we support it?

Do we envision also something like an union feature with backendTLSPolicy, where we apply a policy to the selected endpoints?

@keithmattix keithmattix Apr 23, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question; I know @LiorLieberman was thinking about EndpointSelector as a backendRef, but I honestly struggle to think of a scenario where you wouldn't want a Backend (or some higher level resource) to generate an EndpointSelector. Same goes for TLS and policy attachment. It may/probably exists and I just can't think of it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that it doesn't seem like there's any reason why EndpointSelector as a backendRef would not work, but also, there doesn't seem to be a compelling reason why you would use it when you had Backend, InferencePool, or similar wrapper.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe another way to think about this is: if EndpointSelector does end up in upstream, would we allow it as a backendRef? Again, it doesn't seem like the best option, but there's no reason why it wouldn't work.

@bowei

bowei commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

I think there is interest to make this not just an Gateway API resource but to bring this into Core? I think it would good to discuss this at the main SIG Network meeting.

@keithmattix

Copy link
Copy Markdown
Contributor Author

@bowei yes the plan is to have a two timelines approach to make this a KEP as well as a GEP

Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 28, 2026
@thockin

thockin commented Apr 28, 2026

Copy link
Copy Markdown

Is the point that this EndpointSelector would generate EndpointSlice(s) ?

@keithmattix

Copy link
Copy Markdown
Contributor Author

@thockin yup exactly. To start, we'd have gateway implementations do it, but the design is meant to be scoped enough to where endpointslice-controller can do it without adding a ton of complexity.

@youngnick youngnick left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think frontrunning the upstream KEP like this is actually a pretty great idea, especially in Experimental. Whether this should move to Standard later will be a more interesting conversation.

Comment thread geps/gep-4914/index.md Resolved
@thockin

thockin commented Apr 30, 2026

Copy link
Copy Markdown

I think frontrunning the upstream KEP like this is actually a pretty great idea, especially in Experimental. Whether this should move to Standard later will be a more interesting conversation.

It worked out well for ReferenceGrant.

/s

Signed-off-by: Keith Mattix II <keithmattix2@gmail.com>
@aojea

aojea commented May 9, 2026 •

Copy link
Copy Markdown
Contributor

Thanks for putting this together.

I think the current proposal address two different problems:

  1. Defining an API object that represents a selection of pod endpoints (we talked a lot about this in multiple places, I do not think this is controversial, Service selector matchExpressions kubernetes/kubernetes#62795, service resource support set-based selector kubernetes/kubernetes#48528 and desired for all of us, experimental in gateway sounds like a good way to iterate fast on this.

  2. Defining how that object is consumed and managed by higher-level resources. This is the part that is more confusing to me because is designing EndpointSelector to be tightly coupled to a parent resource. In Kubernetes, Service and EndpointSlice are loosely coupled. A Service selects endpoints, and controllers manage the slices, but the relationship is implicit and I think this model scales better.

Is there any reason to not follow the existing relational model? it seems easier to think about to me

I think frontrunning the upstream KEP like this is actually a pretty great idea, especially in Experimental. Whether this should move to Standard later will be a more interesting conversation.

strongly agree

@keithmattix

Copy link
Copy Markdown
Contributor Author

@aojea

  1. Thanks for linking those matchExpression issues! I think EndpointSelector is a great place for those; I'll probably want to tackle those in a follow up to ensure we get the current state of Service in first.
  2. The GEP actually tries to offer both an explicit and implicit model. Users can create EndpointSelectors manually and attach them to higher level objects or controllers of those higher level objects will automatically create an EndpointSelector when appropriate. Sorry if that wasn't clear! Let me know if I need to tweak any of the examples

@aojea

aojea commented May 12, 2026

Copy link
Copy Markdown
Contributor

2. The GEP actually tries to offer both an explicit and implicit model. Users can create EndpointSelectors manually and attach them to higher level objects or controllers of those higher level objects will automatically create an EndpointSelector when appropriate

fair, I just wanted to be sure we can support existing Services ... Thanks

@keithmattix

Copy link
Copy Markdown
Contributor Author

fair, I just wanted to be sure we can support existing Services

One detail that's not in here that will be in the KEP is that we want Service controller to create (the in-tree version of) an EndpointSelector for compatibility (just like what you did with IPAddress)

@aojea

aojea commented May 12, 2026

Copy link
Copy Markdown
Contributor

One detail that's not in here that will be in the KEP is that we want Service controller to create (the in-tree version of) an EndpointSelector for compatibility (just like what you did with IPAddress)

Yeah, IIUIC the plan is to get XEndpointSelector to work and socialize and get the things working, and then will have the discussion to move to Gateway or directly to core, optimistically we move to core , is that the plan?

@keithmattix

Copy link
Copy Markdown
Contributor Author

@aojea depends on when 1.37 KEP freeze is 😅 if it's still possible to get a KEP accepted for that, I can work on something so that we're not head of line blocking on Gateway

@aojea

aojea commented May 12, 2026

Copy link
Copy Markdown
Contributor

@aojea depends on when 1.37 KEP freeze is 😅 if it's still possible to get a KEP accepted for that, I can work on something so that we're not head of line blocking on Gateway

I do not understand, first you need to get XEndpointSelector implemented and working and then later we evoluate the next steps, the KEP goes after not first ... so I took you are aiming for 1.38

@keithmattix

Copy link
Copy Markdown
Contributor Author

The plan I talked about with @LiorLieberman @robscott and others about is trying to do it at the same time so we don't lose 4 months on the KEP process. We were also hoping the implementation could live in the endpointslice controller so that implementations can just import the functionality without manually dealing with endpoints.

@aojea

aojea commented May 12, 2026

Copy link
Copy Markdown
Contributor

We were also hoping the implementation could live in the endpointslice controller so that implementations can just import the functionality without manually dealing with endpoint

I do not understand exactly what are you proposing and the technical details, you need to be more specific

@keithmattix keithmattix changed the title Endpointselector GEP Provisional: Endpointselector GEP May 28, 2026
@keithmattix

Copy link
Copy Markdown
Contributor Author

Part of #4914

Signed-off-by: Keith Mattix II <keithmattix2@gmail.com>
@szuecs

szuecs commented May 29, 2026

Copy link
Copy Markdown

Because we are discussing here a kind of pre-alpha core object: Can we have a "kind" based filter in the selector?
In our case it's quite often that people create a label selector that matches their deployment and their cronjob, which obviously leads to connection refused errors due to calls to the cronjob.
Maybe this common problem could be solved by the XEndpointSelector object and in the future by EndpointSelector.

@keithmattix

Copy link
Copy Markdown
Contributor Author

@szuecs I don't think we can include a kind since endpointslice-controller is going to be the component that is creating EndpointSlices and it's strictly coupled to pods (for good reason since EndpointSlice expects certain information)

@szuecs

szuecs commented May 30, 2026

Copy link
Copy Markdown

@szuecs I don't think we can include a kind since endpointslice-controller is going to be the component that is creating EndpointSlices and it's strictly coupled to pods (for good reason since EndpointSlice expects certain information)

Domage! 😊
I think it's a missed opportunity to make things more robust for end users.
I understand that it will be nicer from code point of view and maybe even API, but we lack of good ways to make sure that overlapping selections are intended and not errors.
If there are ideas how to fix the case of cronjob cs deployment in such a selector I am all hears and would be happy to have a solution.
It happens every year at least twice for the last 12 years....

@k8s-triage-robot

Copy link
Copy Markdown

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@kubernetes-prow kubernetes-prow Bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 17, 2026
Comment thread geps/gep-4914/index.md

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there was some discussion at kubecon to turn this into memorandum GEP.

In the backend GEP, I've added a section on the stop gap service-backing & future KEP.

Is there any reason we need this GEP? Is the current backend GEP missing any context that turning this into a memorandum would still be needed?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's in Backend already, we can probably just kill this GEP

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can just mark this as Declined. Still worthwhile merging it as Declined, maybe? To preserve that we discussed it?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. kind/gep PRs related to Gateway Enhancement Proposal(GEP) lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.