Skip to content

feat: node image pinning - #1920

Merged
Pablo Trivino (PabloTriv) merged 119 commits into
mainfrom
pablotrivino/nodeImagePinningSplit
Sep 29, 2026
Merged

Pablo Trivino (PabloTriv) merged 119 commits into
mainfrom
pablotrivino/nodeImagePinningSplit

Conversation

@PabloTriv

@PabloTriv Pablo Trivino (PabloTriv) commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1220

Description

This PR adds node image pinning, as described in doc 0014.

How was this change tested?
Standalone and E2Es. Tested by pinning version, pinning invalid versions, seeding recentlyUsed to simulate rollback. E2E Run: https://github.com/Azure/karpenter-provider-azure/actions/runs/35620834867/job/106403188039 (This is using FIt so the run is marked as failed, but the actual tests pass).
*

Does this change impact docs?

  • Yes, PR includes docs updates
  • Yes, issue opened: #
  • No

Release Note


Copilot AI lite review requested due to automatic review settings September 17, 2026 16:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟑 Changes recommended

Build-breaking API signature changes and a status-patch rejection bug need fixing before merge.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR introduces a split/pinning model for NodeClass Kubernetes version and node image version selection, including new API fields, status tracking, controller logic, and validation to support explicit version pinning and rollbacks.

Changes:

  • Add spec.versions to AKSNodeClass and status.versions to track control-plane version, latest image version, and recently used versions.
  • Extend the NodeClass status reconcilers to validate requested Kubernetes versions, drive image resolution for requested versions, and support pinning/rollback behavior.
  • Wire a new AKS Managed Clusters client into the operator/providers and add integration/unit tests for pinning flows.
File summaries
File Description
test/suites/integration/nodeimagepinning_test.go New integration coverage for Kubernetes/node image pinning behavior
pkg/providers/kubernetesversion/kubernetesversion.go Add AKS supported-version check via ManagedClusters API; extend provider interface
pkg/providers/azclient/azclient.go Construct and expose ManagedClusters client; thread through AZClient factory
pkg/providers/azclient/azapi/api.go Add AKSManagedClustersAPI interface definition
pkg/operator/operator.go Wire managed clusters client + location into KubernetesVersionProvider construction
pkg/controllers/nodeclass/status/kubernetesversion.go Validate requested Kubernetes versions and record control-plane version in status
pkg/controllers/nodeclass/status/kubernetesversion_test.go Add unit tests for requested Kubernetes version validation paths
pkg/controllers/nodeclass/status/images.go Add image pinning/rollback logic and status fields for latest image version
pkg/controllers/nodeclass/status/images_test.go Add unit tests for requested versions, pinning, and rollback behavior
pkg/controllers/nodeclass/status/controller.go Snapshot β€œrecently used” versions on status changes
pkg/apis/v1beta1/aksnodeclass.go Add spec.versions type + CEL validations; add status-aware CEL rule for image pins
pkg/apis/v1beta1/aksnodeclass_status.go Add VersionsStatus + RecentlyUsedVersion types and embed in NodeClass status
pkg/apis/v1beta1/zz_generated.deepcopy.go Generated deepcopy updates for new API/status types
pkg/apis/crds/karpenter.azure.com_aksnodeclasses.yaml Generated CRD updates for new fields and validations
charts/karpenter-crd/templates/karpenter.azure.com_aksnodeclasses.yaml Helm CRD template updated to match generated CRD schema
Review details

Files not reviewed (1)

  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Suppressed comments (1)

pkg/providers/azclient/azclient.go:134

  • Severity: High | Category: Correctness

NewAZClientFromAPI now requires a managedClustersClient, but at least one call site still uses the old argument list (e.g. pkg/test/environment.go), which will break compilation. Ensure all NewAZClientFromAPI callers and relevant fakes are updated to pass an AKSManagedClustersAPI implementation.

func NewAZClientFromAPI(
	virtualMachinesClient azapi.VirtualMachinesAPI,
	azureResourceGraphClient azapi.AzureResourceGraphAPI,
	aksMachinesClient azapi.AKSMachinesAPI,
	aksMachinesBatchClient aksmachinesheaderbatch.AKSMachinesHeaderBatchAPI,
	agentPoolsClient azapi.AKSAgentPoolsAPI,
	managedClustersClient azapi.AKSManagedClustersAPI,
	virtualMachinesExtensionClient azapi.VirtualMachineExtensionsAPI,
	interfacesClient azapi.NetworkInterfacesAPI,
	subnetsClient azapi.SubnetsAPI,
	diskEncryptionSetsClient azapi.DiskEncryptionSetsAPI,
	loadBalancersClient loadbalancer.LoadBalancersAPI,
	networkSecurityGroupsClient networksecuritygroup.API,
	imageVersionsClient imagefamilytypes.CommunityGalleryImageVersionsAPI,
	nodeImageVersionsClient imagefamilytypes.NodeImageVersionsAPI,
	nodeBootstrappingClient imagefamilytypes.NodeBootstrappingAPI,
	skuClient skewer.ResourceClient,
	subscriptionsClient zone.SubscriptionsAPI,
	usageClient quota.UsageAPI,
	skuMixPlacementClient capacityrecommendation.SKUMixPlacementScoresAPI,
) *AZClient {
  • Files reviewed: 14/15 changed files
  • Comments generated: 5
  • Review effort level: Lite

πŸ’‘ Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread pkg/apis/v1beta1/aksnodeclass.go Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

πŸ”΅ Needs a closer look

Unresolved correctness, API compatibility, documentation, and integration-test issues block approval.

Review effort: Lite
Findings: 3 High severity Β· 1 Low severity

Open (4)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file
Previously missed (2)

In code that hasn't changed since last review

Medium severity Design documents incorrect control plane Kubernetes version field

pkg/​apis/​v1beta1/​aksnodeclass_status.go:127

[Medium][API Design] The new JSON field is currentControlPlaneKubernetesVersion, but design 0014 and the other changed references document status.observedVersions.controlPlaneKubernetesVersion. Consumers following the design will query a field that is never populated; either align the API name or update the design and all examples before release.

Low severity Design uses incorrect observed Kubernetes version field name

designs/​0014-node-image-and-k8s-version-controls.md:63

Low β€” Documentation: the implementation defines this status field as status.observedVersions.currentControlPlaneKubernetesVersion (see pkg/apis/v1beta1/aksnodeclass_status.go:124-127), but the design document advertises controlPlaneKubernetesVersion throughout. Readers following this design will query a field that the API does not expose; update all references and the sample type to the implemented name, or rename the API consistently.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Resolve version normalization, remove unsupported v1alpha2 additions, and replace focused integration specs.

Review effort: Lite
Findings: 2 High severity

Open (2)
Resolved since last review (3)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

πŸ”΅ Needs a closer look

Resolve the deprecated v1alpha2 API additions and correct integration-test supported-version setup.

Review effort: Lite
Findings: 2 High severity

Open (2)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Final review comments identify unresolved compatibility and reconciliation issues.

Review effort: Lite
Findings: 3 High severity

Open (3)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Comment thread pkg/apis/v1beta1/aksnodeclass.go Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

πŸ”΅ Needs a closer look

Moderate API contract and pinning correctness issues remain unresolved.

Review effort: Lite
Findings: 3 High severity

Open (3)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file
Previously missed (2)

In code that hasn't changed since last review

Medium severity Mark invalid image pins as validation failures

pkg/​controllers/​nodeclass/​status/​images.go:388

Severity: Medium β€” Category: API Design. Invalid or incompatible image pin requests are reported only through ImagesReady=False, but the later validation reconciler resets ValidationSucceeded to true on every pass (pkg/controllers/nodeclass/status/validation.go:127-129). This makes an unresolvable nodeImageVersion look like a valid spec to API consumers, whereas the design assigns NodeImageVersionInvalid and RollbackTargetKubernetesVersionMismatch to ValidationSucceeded; mark spec-level pin validation failures on ValidationSucceeded (while reserving ImagesReady for resolution failures) and test the final aggregate conditions.

Medium severity Mark invalid Kubernetes versions as validation failures

pkg/​controllers/​nodeclass/​status/​kubernetesversion.go:137

Severity: Medium β€” Category: API Design. An invalid requested Kubernetes version only sets KubernetesVersionReady=False here, while the status controller later runs validation and unconditionally sets ValidationSucceeded=True (pkg/controllers/nodeclass/status/validation.go:127-129). Consequently a NodeClass with an invalid spec.versions.kubernetesVersion reports that validation succeeded, contrary to the design's condition contract and misleading consumers that use ValidationSucceeded to distinguish invalid spec from an incompatible effective version; set ValidationSucceeded=False for request-format failures (and cover the resulting condition).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Unresolved API compatibility, AKS version normalization, image-status refresh, and Azure permission issues block approval.

Review effort: Lite
Findings: 4 High severity

Open (4)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

πŸ”΅ Needs a closer look

Unresolved API compatibility and image/Kubernetes readiness issues block approval.

Review effort: Lite
Findings: 4 High severity

Open (4)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

πŸ”΅ Needs a closer look

Address the three unresolved API validation, deprecated-version, and reconciliation findings.

Review effort: Lite
Findings: 4 High severity

Open (4)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

πŸ”΅ Needs a closer look

Unresolved compatibility and validation-condition issues remain.

Review effort: Lite
Findings: 4 High severity

Open (4)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

πŸ”΅ Needs a closer look

Unresolved API compatibility, version-format validation, and image-pinning reconciliation issues remain.

Review effort: Lite
Findings: 4 High severity

Open (4)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Generated deepcopy and CRD artifacts reference the wrong field, and the feature should not expand the deprecated v1alpha2 API.

Review effort: Lite
Findings: 4 High severity

Open (4)
Resolved since last review (4)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Unresolved generated-code, CRD schema, compatibility, and image-pinning correctness issues block approval.

Review effort: Lite
Findings: 2 High severity

Open (2)
Resolved since last review (4)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

πŸ”΅ Needs a closer look

Unresolved validation, image compatibility, and deprecated API-surface findings must be addressed.

Review effort: Lite
Findings: None

Resolved since last review (2)
Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Comment thread pkg/apis/v1beta1/aksnodeclass_status.go Resolved
Comment thread pkg/apis/v1beta1/aksnodeclass_status.go Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

πŸ”΅ Needs a closer look

Address the deprecated v1alpha2 API additions and preserve staged Kubernetes version status updates.

Review effort: Lite
Findings: None

Files not reviewed (2)
  • pkg/apis/v1alpha2/zz_generated.deepcopy.go: Generated file
  • pkg/apis/v1beta1/zz_generated.deepcopy.go: Generated file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for setting node image version for nodeClass

5 participants