Repository navigation
feat: add node public IP support to AKSNodeClass (AKS Machine API) - #1982
Merged
Merged
Conversation
This was referenced Oct 9, 2026
Alex Leites (tallaxes)
force-pushed
the
tallaxes/node-public-ip-api
branch
from
October 9, 2026 21:10
c0437d7 to
d7d8ec4
Compare
Alex Leites (tallaxes)
force-pushed
the
tallaxes/node-public-ip-api
branch
2 times, most recently
from
October 10, 2026 01:38
230c88b to
0f71243
Compare
Matthew Christopher (matthchr)
previously approved these changes
Oct 10, 2026
Add an optional spec.nodePublicIP block to the v1beta1 AKSNodeClass, and mirror it in v1alpha2, so a NodeClass can give each node its own instance-level public IP: - enabled (optional bool) - prefixIDs (optional, 1 item): a Microsoft.Network/publicIPPrefixes ID - ipTags (optional, 1-4 unique items): ipTagType and tag CEL requires enabled=true when prefixIDs or ipTags is set, rejects combining prefixIDs with ipTags, and rejects empty lists. The nodeclass hash ignores the block when it is omitted or disabled, so existing nodes do not drift. Prefix IDs are compared case-insensitively and IP tag order does not matter. No provisioning mode supports node public IPs yet: enabling the field sets ValidationSucceeded=False with reason NodePublicIPUnsupportedProvisionMode. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Alex Leites (tallaxes)
force-pushed
the
tallaxes/node-public-ip-api
branch
from
October 10, 2026 08:51
8141cc8 to
708a13b
Compare
β¦1983) Enable spec.nodePublicIP for the aksmachineapi and aksmachineapiheaderbatch provisioning modes. NodeClasses using it in VM-based modes stay rejected by status validation. - The Machine template sets network.enableNodePublicIP, plus nodePublicIPPrefixID or nodePublicIPTags, only when the feature is enabled, so disabled machines keep their existing template and batch key. - Reusing a leftover Machine checks its node public IP settings against the NodeClass. A missing value counts as disabled and prefix IDs are compared case-insensitively. On a mismatch the Machine is not reused and is deleted, so a node with the wrong settings is never adopted under the current NodeClass hash. - Public IP failures from AKS stay on the existing unhandled error path: they are not treated as insufficient capacity, keep AKS's code and message, and leave every offering available. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Alex Leites (tallaxes)
marked this pull request as ready for review
October 10, 2026 18:42
Contributor
There was a problem hiding this comment.
π‘ Changes recommended
RoutingPreference can repeatedly select invalid zonal offerings, and the API is incorrectly mirrored into unserved v1alpha2.
2 open findings
What changed in this PR
Adds configurable node public IPs for AKS Machine API provisioning, including validation, hashing, request construction, reuse checks, and error handling.
Changes:
- Adds
nodePublicIPAPI fields and CRD validation. - Configures and validates AKS Machine network profiles.
- Adds extensive unit, acceptance, drift, batching, and error-path coverage.
| File | Description |
|---|---|
pkg/βproviders/βinstance/βofferings/βerrordetailhandlers_test.go |
Tests provisioning errors. |
pkg/βproviders/βinstance/βofferings/βcommonerrorhandlers_test.go |
Adds offering assertion helper. |
pkg/βproviders/βinstance/βofferings/βaksmachinebegincreateerrorhandlers_test.go |
Tests synchronous errors. |
pkg/βproviders/βinstance/βaksmachineinstancehelpers.go |
Builds public-IP network settings. |
pkg/βproviders/βinstance/βaksmachineinstancehelpers_test.go |
Tests network configuration and reuse validation. |
pkg/βproviders/βinstance/βaksmachineinstance.go |
Validates Machines before reuse. |
pkg/βproviders/βazclient/βaksmachinesheaderbatch/βbatchkey_test.go |
Verifies batching separation. |
pkg/βoperator/βoptions/βoptions_extensions.go |
Defines supported provisioning modes. |
pkg/βfake/βaksmachinesapi.go |
Adds public-IP failure fakes. |
pkg/βcontrollers/βnodeclass/βstatus/βvalidation.go |
Rejects unsupported modes. |
pkg/βcontrollers/βnodeclass/βstatus/βvalidation_test.go |
Tests mode validation. |
pkg/βcloudprovider/βsuite_aksmachineapi_offerings_test.go |
Tests offering error behavior. |
pkg/βcloudprovider/βsuite_aksmachineapi_integration_test.go |
Tests Machine reuse and rejection. |
pkg/βcloudprovider/βsuite_aksmachineapi_features_test.go |
Tests Machine request fields. |
pkg/βcloudprovider/βsuite_aksmachineapi_drift_test.go |
Tests public-IP drift. |
pkg/βapis/βv1beta1/βzz_generated.deepcopy.go |
Adds generated deep copies. |
pkg/βapis/βv1beta1/βcrd_validation_cel_test.go |
Tests API validation. |
pkg/βapis/βv1beta1/βaksnodeclass.go |
Defines and hashes the API. |
pkg/βapis/βv1beta1/βaksnodeclass_hash_test.go |
Tests hash semantics. |
pkg/βapis/βv1alpha2/βzz_generated.deepcopy.go |
Mirrors generated deep copies. |
pkg/βapis/βv1alpha2/βcrd_validation_cel_test.go |
Mirrors validation tests. |
pkg/βapis/βv1alpha2/βaksnodeclass.go |
Mirrors the API into v1alpha2. |
pkg/βapis/βv1alpha2/βaksnodeclass_hash_test.go |
Mirrors hash tests. |
pkg/βapis/βcrds/βkarpenter.azure.com_aksnodeclasses.yaml |
Updates generated CRD schemas. |
charts/βkarpenter-crd/βtemplates/βkarpenter.azure.com_aksnodeclasses.yaml |
Updates chart CRD template. |
Files not reviewed (2)
- pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
- pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file
π§ Review effort: Balanced
π‘ Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Matthew Christopher (matthchr)
previously approved these changes
Oct 10, 2026
An exhausted public IP prefix doesn't fail the Machine create request. AKS accepts it, and the Machine fails during provisioning with a detail that has no code and the message "No more IPs available in IpPrefix ...". Drop the PublicIpPrefixOutOfIpAddressesForVMScaleSet create-request case, replace the CreateOrUpdatePublicIPAddressError provisioning case with the observed shape, and use the observed GetPublicIPPrefixByResourceIDError message. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
β¦p-api # Conflicts: # pkg/apis/v1alpha2/zz_generated.deepcopy.go # pkg/apis/v1beta1/zz_generated.deepcopy.go
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Matthew Christopher (matthchr)
approved these changes
Oct 11, 2026
1 of 3 tasks
1 of 3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Description
fixes: #656
Adds an optional
spec.nodePublicIPblock to theAKSNodeClass, and supports it in the AKS Machine API provisioning modes (aksmachineapiandaksmachineapiheaderbatch). It lets a NodeClass give every node its own instance-level public IP address, optionally allocated from a public IP prefix or carrying IP tags such as routing preference. This is the Karpenter equivalent of an agent pool's node public IPs.AKS creates each node's public IP with the node and deletes it with the node, so Karpenter has no cleanup of its own to do. In the VM-based modes (
aksscriptless,bootstrappingclient), enabling the field setsValidationSucceeded=Falsewith reasonNodePublicIPUnsupportedProvisionMode, so the NodeClass isn't ready and no nodes are created from it.API (
v1beta1, mirrored inv1alpha2)enabled(optional bool).prefixIDs(optional list, 1 item). Each item must be aMicrosoft.Network/publicIPPrefixesARM ID.ipTags(optional list, 1 to 4 items). Each item has a requiredipTagTypeandtag, 1 to 256 characters each, and entries must be unique.enabled: truewhenprefixIDsoripTagsis set, and rejects setting bothprefixIDsandipTags. Empty lists are rejected rather than treated as omitted.{}and{enabled: false}all hash the same. Prefix IDs are compared case-insensitively, since ARM IDs are case-insensitive. Reordering IP tags doesn't change the hash; changing a tag, including its casing, does.AKS Machine API
network.enableNodePublicIP, plusnetwork.nodePublicIPPrefixIDornetwork.nodePublicIPTags, only when the feature is enabled. A disabled NodeClass produces the same request and batch key as before. In header-batch mode, the fields are in the shared template, so Machines from different NodeClass settings are never batched together.$expand). The E2E below confirms that GET and LIST return these fields.Example
or, with IP tags instead of a prefix:
This becomes the Machine's network profile:
ipTagsmap tonodePublicIPTagsthe same way.E2E results (
aksmachineapi, self-hosted; tests are in #1984)Machine readback. GET and LIST, without
$expand, returnenableNodePublicIP: trueand the configurednodePublicIPPrefixIDornodePublicIPTags. After the feature is disabled and the nodes drift, the replacement Machines reportenableNodePublicIP: false.Without a prefix. Each node's NIC gets a public IP, and the node reports it as
ExternalIP. Inbound traffic reaches the node on its public IP, and outbound traffic leaves from it. Disabling the feature drifts the nodes, and AKS releases the old IPs.From a prefix. Nodes in all three zones get IPs from a zone-redundant prefix, and AKS releases the IPs when the nodes are deleted.
With IP tags. Regional nodes (
karpenter.azure.com/placement-scope: regional) withRoutingPreference: Internetget a zone-redundant public IP carrying the tag.Error codes. Each failing identity was observed on its own:
LinkedAuthorizationFailed, naming Karpenter's identityLaunched=Unknown, reasonCreateInstanceFailed. The NodeClaim is kept and retried.LinkedAuthorizationFailed, naming the cluster identityAsyncProvisioningErrorevent on the NodeClaim, which is then deleted and replacedUnsupportedIPTagTypeLaunched=Unknown, reasonCreateInstanceFailedRoutingPreferenceon a zonal node (observed in a manual run)AsyncProvisioningErrorevent; the NodeClaim is deleted and replacedBoth identities need
Microsoft.Network/publicIPPrefixes/join/actionon the prefix. On NAP, Karpenter calls AKS as the cluster identity, so one grant covers both.How was this change tested?
v1beta1andv1alpha2): valid and invalid prefix IDs;prefixIDsoripTagswithoutenabled; more than one prefix;ipTagswithprefixIDs; empty, 256- and 257-character tag values; duplicate tags; more than 4 tags.v1beta1andv1alpha2): omitted,{}and{enabled: false}match the pre-feature hash; prefix ID casing and IP tag order don't change the hash; enabling, changing the prefix, and adding, changing or removing a tag do; hashing doesn't mutate the NodeClass.NodePublicIPUnsupportedProvisionMode, in the VM-based modes, and recovers when the feature is disabled; it's accepted in both AKS Machine API modes.make verifyandmake test.Does this change impact docs?
Release Note