Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Static Egress Gateway in Azure Kubernetes Service (AKS) routes pod egress traffic through a dedicated gateway node pool with fixed source IP addresses. This article shows you how to enable the feature, configure a gateway, route pod traffic, and monitor or remove the configuration.
Limitations and considerations
- Static Egress Gateway isn't supported in clusters with Azure CNI Pod Subnet.
- Static Egress Gateway supports IPv4-only clusters and isn't supported in dual-stack clusters.
- Kubernetes network policies don't apply to traffic that annotated pods route through the gateway node pool. Other cluster traffic isn't affected.
- Use the gateway node pool only for egress traffic, not for general-purpose workloads.
- Windows node pools can't be used as gateway node pools.
- You can't annotate
hostNetworkpods to use the gateway node pool. - Pods can only use a gateway node pool if they are in the same namespace as the
StaticGatewayConfigurationresource.
Prerequisites
- Azure CLI version 2.75.0 or later. Run
az versionto find your installed version. If you need to install or upgrade, see Install the Azure CLI.
Create or update an AKS cluster with Static Egress Gateway
Before you can create and manage gateway node pools, you must enable the Static Egress Gateway feature for your AKS cluster. You can do this when creating a new cluster or by updating an existing cluster using az aks update.
az aks create --name <cluster-name> --resource-group <resource-group> --enable-static-egress-gateway
To enable Static Egress Gateway on an existing cluster, run the following command:
az aks update --name <cluster-name> --resource-group <resource-group> --enable-static-egress-gateway
Create a gateway node pool
After enabling the feature, create a dedicated gateway node pool. This node pool handles the egress traffic through the specified public IP prefix. The --gateway-prefix-size is the size of the public IP prefix to be applied to the gateway node pool nodes. The allowed range is 28-31.
| Prefix size | Available IP addresses | Maximum gateway nodes |
|---|---|---|
/28 |
16 | 16 |
/29 |
8 | 8 |
/30 |
4 | 4 |
/31 |
2 | 2 |
az aks nodepool add --cluster-name <cluster-name> \
--name <nodepool-name> \
--resource-group <resource-group> \
--mode gateway \
--node-count <number-of-nodes> \
--gateway-prefix-size <prefix-size>
Note
- The number of nodes must fit within the capacity allowed by the selected prefix size. For example, a /30 prefix supports up to 4 nodes, and at least 2 nodes are required for high availability. You can manually scale the node pool within the prefix capacity, but cluster autoscaler isn't supported.
- You can define the SKU of the VM to use in your gateway node pool with the
--vm-sizeparameter. You should understand your specific needs and plan accordingly to ensure the right performance and cost balance.
Scale the gateway node pool
Manually scale the gateway node pool within the capacity of its public IP prefix. Gateway node pools don't support cluster autoscaler.
az aks nodepool scale --cluster-name <cluster-name> \
--name <nodepool-name> \
--resource-group <resource-group> \
--node-count <desired-node-count>
Create a StaticGatewayConfiguration resource
Define the gateway configuration by creating a StaticGatewayConfiguration custom resource. This configuration specifies which node pool and public IP prefix to use.
If you supply an existing public IP prefix, configure its access and namespace authorization before applying the manifest.
| Field | Required | Description |
|---|---|---|
gatewayNodepoolName |
Yes | Name of the gateway node pool that handles egress traffic. |
excludeCidrs |
No | Destination CIDRs that bypass the gateway and use the pod's primary network interface. |
publicIpPrefixId |
No | Resource ID of an existing public IP prefix. If omitted, the controller creates a prefix. |
apiVersion: egressgateway.kubernetes.azure.com/v1alpha1
kind: StaticGatewayConfiguration
metadata:
name: <gateway-config-name>
namespace: <namespace>
spec:
gatewayNodepoolName: <nodepool-name>
excludeCidrs: # Optional
- 10.0.0.0/8
- 172.16.0.0/12
- 169.254.169.254/32
publicIpPrefixId: /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Network/publicIPPrefixes/<prefix-name> # Optional
Save the manifest as static-gateway-configuration.yaml, and then apply it:
kubectl apply -f static-gateway-configuration.yaml
Wait for the controller to populate .status.egressIpPrefix before you create or restart annotated workloads. Run the following command until it returns the assigned prefix:
kubectl get staticgatewayconfiguration <gateway-config-name> \
--namespace <namespace> \
--output jsonpath='{.status.egressIpPrefix}'
Use your own public IP prefix
To use an existing public IP prefix, set spec.publicIpPrefixId to its Azure resource ID.
Grant Azure access to the cluster identity
Grant the AKS cluster identity the Network Contributor role at the existing public IP prefix's resource scope.
Authorize the namespace on the prefix
Ask the prefix administrator to set the Azure resource tag kube-egress-gateway-allowed-namespaces to the Kubernetes namespaces approved to use the prefix. The tag accepts comma- or semicolon-separated namespace names. For example, team-a,team-b authorizes configurations in those two namespaces. The value * authorizes all namespaces; use explicit namespace names when access should be limited.
Set the tag:
PREFIX_ID=$(az network public-ip prefix show \
--resource-group <prefix-resource-group> \
--name <prefix-name> \
--query id \
--output tsv)
az tag update \
--resource-id "$PREFIX_ID" \
--operation Merge \
--tags "kube-egress-gateway-allowed-namespaces=team-a,team-b"
Replace team-a,team-b with the complete list of approved namespaces, including any existing entries that should remain authorized. The merge operation preserves other resource tags, but replaces the value of this tag.
Important
Permission to modify this tag grants control over namespace authorization. Limit tag changes to trusted administrators. The tag doesn't grant Azure access to the cluster identity or Kubernetes access to create StaticGatewayConfiguration resources; configure those permissions separately.
This tag applies only when you supply an existing public IP prefix through spec.publicIpPrefixId. It isn't required for automatically created public IP prefixes or private IP configurations with spec.provisionPublicIps: false.
Configure Static Egress Gateway with private IP addresses
Important
Static private IP support requires clusters running Kubernetes version 1.34 or later.
If you need to keep egress traffic on private addresses, create a gateway node pool that uses the VirtualMachines VM set type. Node public IPs are disabled by default. The StaticGatewayConfiguration resource in the following steps separately disables public IP provisioning for egress.
az aks nodepool add --cluster-name <cluster-name> \
--name <nodepool-name> \
--resource-group <resource-group> \
--mode gateway \
--node-count <number-of-nodes> \
--vm-set-type VirtualMachines \
--gateway-prefix-size <prefix-size>
Tip
Specify --vnet-subnet-id to allocate the gateway nodes' private IP addresses from a custom subnet. A separate subnet can improve network isolation and IP address planning.
Important
Private IP mode doesn't create an outbound route. Before you annotate workloads, configure a route from the gateway node pool subnet to the intended destination by using a user-defined route with Azure Firewall or another network virtual appliance, or by using ExpressRoute. For more information, see Customize cluster egress with outbound types in AKS. Specifying --vnet-subnet-id alone doesn't provide outbound connectivity.
Set provisionPublicIps: false to disable public IP provisioning and keep the private IPs allocated to the gateway nodes for the lifetime of the StaticGatewayConfiguration. The gatewayNodepoolName value must match the gateway node pool that you created with --vm-set-type VirtualMachines.
apiVersion: egressgateway.kubernetes.azure.com/v1alpha1
kind: StaticGatewayConfiguration
metadata:
name: <gateway-config-name>
namespace: <namespace>
spec:
gatewayNodepoolName: <nodepool-name>
provisionPublicIps: false
Save the manifest as static-gateway-configuration.yaml, and then apply it:
kubectl apply -f static-gateway-configuration.yaml
Before you create or restart annotated workloads, run the following command until .status.egressIpPrefix contains the assigned private IP addresses:
kubectl get staticgatewayconfiguration <gateway-config-name> \
--namespace <namespace> \
--output jsonpath='{.status.egressIpPrefix}'
When you run kubectl describe StaticGatewayConfiguration <gateway-config-name> -n <namespace>, the egressIpPrefix field shows a comma-separated list of those static private IPs. You continue to use the same APIs and manifests for the rest of the workflow, including the StaticGatewayConfiguration resource and the pod annotations.
Annotate pods to use the gateway configuration
To route traffic from specific pods through the gateway node pool, add the kubernetes.azure.com/static-gateway-configuration annotation to the pod template. Set the annotation value to the metadata.name of a StaticGatewayConfiguration resource in the same namespace.
apiVersion: apps/v1
kind: Deployment
metadata:
name: <deployment-name>
namespace: <namespace>
spec:
template:
metadata:
annotations:
kubernetes.azure.com/static-gateway-configuration: <gateway-config-name>
Note
The CNI plugin on each node configures new pods to route traffic through the selected gateway node pool.
The CNI plugin configures the gateway when a pod is created. If you didn't replace existing pods when you added the annotation, restart the workload. For example, restart a deployment by using the following command:
kubectl rollout restart deployment <deployment-name> --namespace <namespace>
Monitor Static Egress Gateway configurations
To monitor a configuration, retrieve the complete StaticGatewayConfiguration resource:
kubectl get staticgatewayconfiguration <gateway-config-name> \
--namespace <namespace> \
--output yaml
In the status section, verify the following fields:
egressIpPrefixcontains the assigned public IP prefix or private IP addresses.gatewayServerProfile.ipandgatewayServerProfile.portidentify the gateway endpoint.gatewayServerProfile.publicKeycontains the public key for the WireGuard configuration, which defines the encrypted tunnel between pods and the gateway node pool.
Check BYO prefix authorization warnings
A Kubernetes Warning event with reason BYOPrefixAuthorizationRequired on a StaticGatewayConfiguration indicates that its BYO prefix doesn't authorize the configuration's namespace.
View events for a configuration:
kubectl describe staticgatewayconfiguration <gateway-config-name> -n <namespace>
List authorization warnings in a namespace:
kubectl get events -n <namespace> \
--field-selector involvedObject.kind=StaticGatewayConfiguration,reason=BYOPrefixAuthorizationRequired
Delete a gateway node pool
Before you delete a gateway node pool, remove the gateway annotation from pod templates and replace the existing pods, or delete the workloads that use it. Then, delete every StaticGatewayConfiguration that references the node pool. The command waits for the controller's cleanup finalizers to complete before it returns:
kubectl delete staticgatewayconfiguration <gateway-config-name> \
--namespace <namespace> \
--wait=true \
--timeout=5m
After you delete all configurations that reference the node pool, delete the gateway node pool:
az aks nodepool delete --cluster-name <cluster-name> \
--name <nodepool-name> \
--resource-group <resource-group>
Disable the Static Egress Gateway feature
If you no longer need Static Egress Gateway, disable the feature and uninstall the operator:
Delete every
StaticGatewayConfigurationand wait for its cleanup to finish.Delete each gateway node pool.
az aks nodepool delete --cluster-name <cluster-name> \ --name <nodepool-name> \ --resource-group <resource-group>Disable Static Egress Gateway.
az aks update --name <cluster-name> --resource-group <resource-group> --disable-static-egress-gateway