Kubernetes
v1.37.0Kubernetes CoreACTION 6CHECK 35PLAN 5OTHER 403
A feature-rich major API release with new capabilities, scheduling and node improvements, maturity promotions, and correctness and performance fixes. Upgrade planning includes API and feature-gate removals, deprecated interface cleanup, and SELinux and kubelet configuration changes that may require manifest, client, or cluster adjustments.
Action needed (6)
securityMalformed object deletion admission handling
An edge case that could allow malformed object deletion to bypass admission and graceful deletion of well-formed objects is removed.
breakingThe
WatchListCompressiondefaultThis behavior is enabled by default and can be disabled with the
WatchListCompressionfeature gate.breakingThe
EtcdRangeStreamfeature gateThe
EtcdRangeStreamfeature gate is enabled by default and promoted to Beta.breakingThe
AllowUnsafeMalformedObjectDeletionfeature gateThe
AllowUnsafeMalformedObjectDeletionfeature gate is promoted to Beta and enabled by default. List errors for objects that cannot be read from storage include the first underlying cause in the error message.breakingThe
PodLevelResourceManagersdefaultThe
PodLevelResourceManagersfeature gate is disabled by default because of issues found before release.breaking
kube-schedulerlist-type attribute evaluationkube-schedulerevaluates list-type attributes, the.function, and CEL macros even when theincludes ListTypeAttributesfeature gate is disabled. This prevents errors during rolling upgrades or when the feature gate is toggled.
Check if affected (35)
security
kubeconfigmigration permissionsApplies if you configure a
kubeconfigfile for migration.client-gonow preserves the source file's permissions when migrating akubeconfigfile, such as from~/.tokube/. kubeconfig ~/.. Previously, the destination file was created with broader permissions.kube/config breakingThe
SELinuxMountdefaultApplies if your cluster has SELinux enabled.
SELinuxMountis enabled by default inv1.. Existing workloads in clusters with SELinux enabled may break.37 breakingThe Workload-Aware Scheduling API types
Applies if you use
scheduling.ork8s. io/v1alpha2 scheduling..k8s. io/v1beta1 The core Workload-Aware Scheduling API types
WorkloadandPodGroupare promoted toscheduling..k8s. io/v1beta1 breakingThe
DisruptionModefield typeApplies if you use
DisruptionMode.The
DisruptionModeenum field is converted to a struct to support future extensibility.breakingThe
eventRecordQPSzero valueApplies if you configure
eventRecordQPS.eventRecordQPShandling inkubeletconfiguration now treats0as unlimited, with no rate limit.breakingThe deprecated
cAdvisorflagsApplies if you configure any of
--application-metrics-count-limit,--boot-id-file,--container-hints,--containerd,--containerd-namespace,--enable-load-reader,--event-storage-age-limit,--event-storage-event-limit,--global-housekeeping-interval,--log-cadvisor-usage,--machine-id-file,--storage-driver-user,--storage-driver-password,--storage-driver-host,--storage-driver-db,--storage-driver-table,--storage-driver-secure, or--storage-driver-buffer-duration.Deprecated
cAdvisorflags are no longer accepted.kubeletfails to start if any of these flags are set:--application-metrics-count-limit,--boot-id-file,--container-hints,--containerd,--containerd-namespace,--enable-load-reader,--event-storage-age-limit,--event-storage-event-limit,--global-housekeeping-interval,--log-cadvisor-usage,--machine-id-file,--storage-driver-user,--storage-driver-password,--storage-driver-host,--storage-driver-db,--storage-driver-table,--storage-driver-secure, and--storage-driver-buffer-duration.breakingThe
DeclarativeValidationTakeoverfeature gateApplies if you configure
DeclarativeValidationTakeover.The
DeclarativeValidationTakeoverfeature gate can no longer be set.breakingThe
PodCertificateRequestv1fieldsApplies if you use
PodCertificateRequest v1beta1.The deprecated
PKIXPublicKeyandProofOfPossessionfields inPodCertificateRequestv1beta1are removed from thev1API.breakingThe
PKIXPublicKeyandProofOfPossessionfieldsApplies if you use
PKIXPublicKeyorProofOfPossession.PKIXPublicKeyandProofOfPossessionare removed from thev1API.breakingThe
HostnameOverridefeature gateApplies if you configure
HostnameOverride.The
HostnameOverridefeature gate is locked to enabled.breakingThe
DRAPlugin.methodWatchHealthStatus Applies if you use
DRAPlugin..WatchHealthStatus DRAPlugin.is now a mandatory method.WatchHealthStatus breakingThe Workload-Aware Scheduling feature gates
Applies if you configure
GangSchedulingorWorkloadAwarePreemption.The
GangSchedulingandWorkloadAwarePreemptionfeature gates are removed.GenericWorkloadis the replacement feature gate.breakingThe
AnyVolumeDataSourcefeature gateApplies if you configure
AnyVolumeDataSource.The generally available
AnyVolumeDataSourcefeature gate is removed.breakingAdmission webhook round-trip load balancing
Applies if you configure
--enable-aggregator-routing=true.With
--enable-aggregator-routing=true,kube-apiserverevenly load-balances requests across admission webhook endpoints instead of routing concurrent requests to a single cached backend endpoint. TheWebhookRoundTripLoadBalancingfeature gate is Beta and enabled by default, with an opt-out option.breaking
HorizontalPodAutoscalerscaling to and from zeroApplies if you use the
HorizontalPodAutoscaler(HPA).Scaling the
HorizontalPodAutoscalerto and from zero is enabled by default.breakingThe
MaxUnavailableStatefulSetdefaultApplies if you use
MaxUnavailableStatefulSet.The
MaxUnavailableStatefulSetfeature is enabled by default.breakingAdmission webhook virtual resource exclusions
Applies if you use admission webhooks.
Admission webhooks now skip auth and authz virtual resources such as
tokenreviewsandsubjectaccessreviews, which are already excluded by ValidatingAdmissionPolicy and MutatingAdmissionPolicy. The BetaExcludeAdmissionWebhookVirtualResourcesfeature gate is enabled by default and provides an opt-out.breakingThe
NodeLocalCRISocketfeature gateApplies if you run
kubeadm.kubeadmremoves theNodeLocalCRISocketfeature gate, which graduated to GA and was locked to enabled by default in a previous release.breakingThe
HPAOptimizedSelectorStorefeature gateApplies if you run the HorizontalPodAutoscaler controller.
The Beta
HPAOptimizedSelectorStorefeature gate is enabled by default. It reduces lock contention in the HorizontalPodAutoscaler controller's selector overlap detection and improves reconciliation throughput at high HorizontalPodAutoscaler counts and concurrency.breakingThe
kubectl get--label-columnsconstraintApplies if you configure
--label-columns.kubectl getnow returns an error when--label-columnsis used with custom-columns output.breakingThe
image.validationreference Applies if you configure
image..reference Image volume validation now rejects empty
image.fields in Pod templates, including Deployment, StatefulSet, DaemonSet, and Job templates.reference breakingThe
kubeletlogs endpoint methodsApplies if you run
kubelet.kubeletenforces explicit HTTP method restrictions for logs-related endpoints. Read-onlykubeletserver endpoints reject non-GET methods with 405, whileNodeLogQueryaccepts only GET and POST.breakingThe
NFTablesNetlinkdefaultApplies if you run
kube-proxynftables mode.Netlink support is enabled by default in
kube-proxynftables mode.breakingThe PodGroup validation Alpha admission plugin
Applies if you use the Alpha admission plugin.
The Alpha admission plugin that validated PodGroup references to existing Workload resources and matched declared PodGroupTemplate specifications is removed.
breakingThe
kube-proxyNode state responseApplies if you run
kube-proxy.kube-proxyexits when the watched Node's IPs change or when the Node object is deleted, so it can restart with updated node networking state.breakingHugepage-aware
memory.evictionavailable Applies if your nodes use hugepages.
The
kubeleteviction manager excludes hugepage-reserved RAM fromAvailableByteswhen calculatingmemory.on nodes with hugepages. Theavailable HugepageAwareEvictionfeature gate is enabled by default and can be disabled to restore the previous behavior.breakingThe
DRAPrioritizedListfeature gateApplies if you use DRA.
The
DRAPrioritizedListfeature gate is locked to enabled by default. The Prioritized List feature reached GA inv1.and can no longer be disabled.36 breakingThe
RelaxedDNSSearchValidationfeature gateApplies if you configure
RelaxedDNSSearchValidation.The
RelaxedDNSSearchValidationfeature gate is removed.breakingThe locked GA feature gates
Applies if you configure
RetryGenerateName,BtreeWatchCache,OrderedNamespaceDeletion,StreamingCollectionEncodingToJSON,StreamingCollectionEncodingToProtobuf,APIServerTracing,ResilientWatchCacheInitialization, orConsistentListFromCache.The locked GA feature gates
RetryGenerateName,BtreeWatchCache,OrderedNamespaceDeletion,StreamingCollectionEncodingToJSON,StreamingCollectionEncodingToProtobuf,APIServerTracing,ResilientWatchCacheInitialization, andConsistentListFromCacheare removed.breakingThe
--concurrent-service-syncsflagApplies if you configure
--concurrent-service-syncs.The
--concurrent-service-syncskube-controller-managerflag is removed. It has been a no-op sincev1..31 breakingThe
PreventStaticPodAPIReferencesfeature gateApplies if you configure
PreventStaticPodAPIReferences.The
PreventStaticPodAPIReferencesfeature gate is removed. Static Pods can no longer reference API resources, and this behavior can no longer be disabled.breakingThe
SidecarContainersfeature gateApplies if you configure
SidecarContainers.The
SidecarContainersfeature gate, unconditionally enabled sincev1., is removed.33 breakingThe deprecated API server cache list metrics
Applies if you use
apiserver_cache_list_total,apiserver_cache_list_fetched_objects_total, orapiserver_cache_list_returned_objects_total.The deprecated Alpha metrics
apiserver_cache_list_total,apiserver_cache_list_fetched_objects_total, andapiserver_cache_list_returned_objects_totalare no longer exposed by default. Consumers should migrate to the unifiedapiserver_storage_list_*metrics with thestorage="watchcache"label.breakingThe
DefaultWatchCacheSizefieldApplies if you configure
DefaultWatchCacheSize.The no-op
DefaultWatchCacheSizefield ofk8s.is removed.io/apiserver/pkg/server/options. EtcdOptions breakingThe
kubeadmv1beta3API andPublicKeysECDSAfeature gateApplies if you use the
v1beta3API or configurePublicKeysECDSA.kubeadmremoves the deprecatedv1beta3API and thePublicKeysECDSAfeature gate.v1beta4usesClusterConfiguration.for ECDSA key support, and a placeholder experimentalEncryptionAlgorithm v1API is added but cannot yet be used.
Plan ahead (5)
deprecatedThe
kube-proxyipvs modedeprecated since v1.35Applies if you run
kube-proxy's ipvs mode.The
kube-proxyipvs mode is deprecated sincev1..35 deprecatedThe
v1alpha1DRAResourceHealth APIremoval planned in v1.40Applies if you use the
v1alpha1 DRAResourceHealthAPI.The
v1alpha1DRAResourceHealth API is deprecated and planned for removal inv1..40 deprecatedThe
kubectl run--filenameand-fflagsApplies if you configure
--filenameor-f.The ignored
--filenameand-fflags onkubectl runare deprecated.deprecatedThe old
client-goAPIsApplicability is not stated in the release notes.
The old
client-goAPIs are deprecated, with no plan to remove them.deprecatedThe
client-goleader election resourcelock typesApplies if you use
MultiLock,UnknownLeader, orConcatRawRecord.MultiLock,UnknownLeader, andConcatRawRecordare deprecated in theclient-goleader election resourcelock package.
All 403 other recorded changesvalue changes 171 · fixes 123 · additions 75 · constraints 17 · defaults 10 · renames 5 · removals 2
value changes (171)
- Updated
google.togolang. org/grpc v1.82. 1 - Updated the default etcd version to
v3..7. 0 - Updated the etcd client library to
v3..6. 10 - conditions to optionally include the
observedGeneration - encode collections item by item instead of all at once.
- Improved CEL error messages in Dynamic Resource Allocation
- Promoted DRA Workload resource claims to Beta.
- Promoted
kubeletvolume metrics - Promoted the DRA Device Taints and Tolerations feature to GA
- Promoted the DRA extended resource feature to GA in
v1..37 - Promoted the DRA metadata API to Beta.
- Promoted the
HPAConfigurableTolerancefeature gate to GA. - Promoted the
KubeletInUserNamespacefeature gate to Beta. - Promoted the
MemoryQoSfeature gate to Beta. - Promoted the
NodeDeclaredFeaturesfeature gate to GA. - Promoted the
VolumeLimitScalingfeature gate - Promoted the
metrics.API fromk8s. io v1beta1tov1without changes. - Updated CDI spec version selection to be dynamic, preventing the generation of incompatible CDI specifications.
- Empty Pod resources (
{},{requests:{}}, or{limits:{}}) no longer affect QoS calculation. - Updated the Alpha
DRANodeAllocatableResourcesfeature - Added
cpu_idsandmemoryfields at the pod level to thePodResourcesv1 API to report total allocated pod resources, while only returning container-level allocations for container-isolated containers. - Added structured
CauseTypevalues to PodDisruptionBudget-related evictionForbiddenerrors in the eviction API, allowing clients to programmatically distinguish PDB invalid-state errors from other forbidden errors without string-matching on the message. - Added the
cache_to_watcherstage to the Alphaapiserver_watch_events_dispatch_duration_secondsmetric to measure the latency incurred when pushing events to a watcher's result channel. - Added the
storage_to_cachestage to the Alphaapiserver_watch_events_dispatch_duration_secondsmetric to track the latency from backend decode to watch cache ingestion. - Changed PodGroup preemption to run after a failed PodGroup scheduling attempt for PodGroups with scheduling constraints.
- Demoted the
SchedulerPreQueueingHintsfeature gate from Beta to Alpha, disabled by default, because of issues found shortly before release. - Updated pod group preemption errors to be prefixed with
pod group preemption:message. - Graduated
WatchCacheInitializationPostStartHookto GA. - Graduated the
NativeHistogramsfeature gate to Beta. - Graduated the
RelaxedServiceNameValidationfeature gate to GA. - Graduated the
scheduler_plugin_execution_duration_secondsandscheduler_scheduling_algorithm_duration_secondsmetrics from Alpha to Beta. - Improved node health checks by verifying lease staleness with a live
GETbefore marking nodes unhealthy, avoiding false positives from stale cache. - Improved scheduling performance for required Pod affinity and anti-affinity with
topologyKey: kubernetes., behind theio/hostname InterPodAffinityHostnameFastPathfeature gate. - Optimized CEL admission policy evaluation by adopting a lazy zero-allocation reflection-based utility for object traversal, significantly reducing CPU usage and garbage collection overhead during request processing.
- Optimized
kube-schedulerperformance for Pods with PersistentVolumeClaim mounts by processing only delta counts between scheduling cycles. - Promoted
serviceaccount_legacy_tokens_total,serviceaccount_stale_tokens_totalandserviceaccount_valid_tokens_totalto Beta. - Promoted support for
kubectl get -o kyamlto Stable. - Promoted the
DRAResourceClaimDeviceStatusfeature gate to GA. - Promoted the
InPlacePodVerticalScalingInitContainersfeature gate to GA. - Promoted the
PLEGOnDemandRelistfeature gate to GA. - Promoted the
PodAndContainerStatsFromCRIfeature gate to Beta, disabled by default. - Promoted the
PodReadyToStartContainerscondition to GA. - Promoted the
kube-apiserverwebhook metricsapiserver_webhooks_x509_missing_san_totalandapiserver_webhooks_x509_insecure_sha1_totalto Beta and updated their documentation. - Promoted the
kubeletPodsAPI gRPC service to Beta. - Set the
nominatedNodeNamefield on pods from a PodGroup after a successful PodGroup preemption, consistent with single-pod preemption. - Updated the
apiserver_storage_list_*metrics to includestorageandindexlabels to distinguish the storage backend and lookup path used to serve LIST requests. - Updated the scheduler to avoid redundant preemption attempts during PodGroup scheduling when terminating victim pods are already present on the nominated nodes.
- Updated CoreDNS to
….×3 - Updated PodGroup scheduling to requeue remaining unscheduled Pods directly to the active queue (rather than backoff queue) after successful PodGroup scheduling, preserving their original timestamps so they retain scheduling precedence unless a higher priority entity is added.
- Updated PodGroup status to include the
pod group preemption found a placement for podgroup, preempting <victim_count> victimsmessage when workload-aware preemption finds a placement. - Updated cri-tools to v1.36.0.
- Updated default preemption to include the message
preemption: found a potential placement for pod on node <node_name>, preempting <victim_count> victimsin theFailedSchedulingevent andPodScheduledcondition when it finds a potential Node for a Pod. - Updated the Go version used to build Kubernetes to
….×5 - Updated the
WorkloadAwarePreemptionfeature to perform a single scheduling attempt with all potential victims removed. This significantly improves performance but can result in a less optimal choice of preemption victims. - Updated volume mount host path type mismatch errors to log the actual path type alongside the expected one.
- Updated workload-aware preemption to preempt victims so that as many as possible of the preemptor pods can be scheduled.
- Deferred syncing an HPA object in the HPA controller when the controller has not yet observed HPA status writes from the last time the object was synced.
kubeadm: Changed the preflightPort-xxchecks forkube-apiserver,kube-scheduler,kube-controller-manager, and etcd to bind to the address configured in the kubeadm config for the respective component (via thelocalAPIEndpoint.field or theaddress --bind-addressextraArgs override), instead of callingnet.without an address (which binds to all available unicast and anycast IP addresses for the port).Listen() kubelet: Deferred the deprecation removal timeline for the configuration flags (and the related fallback behavior) fromv1.to37 v1.to align with containerd38 v1.support.7 - Added the group name to the
kubectlerror message when a resource type is not found under the specified group, for examplethe server doesn't have a resource type "pdb" in group "hpa". - Avoided costly comparisons during SELinux metric emission.
- Changed
client-goRetryWatcherto log 410 Gone (resource expired) errors at debug verbosity (V(4)) instead of ERROR level during watch establishment. - Changed
kube-proxyto skip full-sync operations when operating in large-cluster mode (more than 1000 endpoints). - Changed the HPA controller to reconcile newly created and spec-changed HPAs immediately instead of waiting for the full resync period (default 15s).
- - Improved error reporting when invoking
kubectl exec. - - Improved the logic in
kubeadmaround warnings when a user sets a non-defaultbindAddressinKubeProxyConfiguration. - - Reverted the
cri-apiKeyValuevalue field to its pre-v1.JSON encoding behavior for compatibility with earlier releases.34 - - Surfaced the error reason when invalid service CIDRs are configured.
- - Updated
kubectl runerror messages for invalid--restartand--image-pull-policyvalues to list the accepted values. - - Updated the PodGroup
status.field to reflect the failure reason when scheduling is rejected due to mismatchedconditions .across Pods in a group.spec. schedulerName - - Updated the
PodReadyToStartContainerscondition to include a diagnostic message whenstatusisFalse, explaining why the pod sandbox is not ready (for example,pod sandbox has no IP addressorno pod sandbox exists). This improves debuggability for Pods stuck inContainerCreatingwithout requiring access to node logs. - - Updated the
kubeletto no longer emitV(4)"Label not found" logs for missing optional container annotations. - - Updated the version of the
nftbinary in thekube-proxyimage to nftablesv1.to fix issues resyncing0. 6. 1 kube-proxyin nftables mode on systems containing rules created by recent versions of nftables. - -
kubelet: Changed theDefaultPodSysctlsfeature to treat an unsetspec.as true when evaluatinghostUsers user.sysctls.* - - Changed MutatingAdmissionPolicy and MutatingAdmissionPolicyBinding storage in etcd to use the
admissionregistration.API version.k8s. io/v1 - - Changed ResourceClaim config status to leave the
requestsfield empty when the configuration applies to all requests. - - Changed the
kube-apiserverservice/proxysubresource to use EndpointSlices instead of Endpoints when proxying to a Service. This change only affects clusters that manually create Endpoints for a Service and have EndpointSlice mirroring disabled. - - Changed the scheduler's opportunistic batching to rescore the previously chosen node when it is still feasible, allowing it to compete with cached candidates for the next hint rather than always being skipped.
- - Improved
kubeletTopology Manager error messages when theprefer-closest-numa-nodespolicy option is enabled on Windows nodes that do not expose NUMA distance information, clarifying that the option is not supported on those nodes. - - Improved memory usage of
kube-proxyby dropping the.field, whichmetadata. managedFields kube-proxydoes not require. - - Promoted
apiserver_watch_events_totalandapiserver_watch_events_sizesto Beta. - - Stopped using maps for single-endpoint Services in
kube-proxynftables mode, increasing the speed of programming nftables. - - Switched StorageVersionMigration to use merge patch instead of SSA.
- - The
kube-apiserver--enable-logs-handlerflag, deprecated inv1., is no longer marked deprecated. It remains off by default.15 - - Updated the etcd client library to
v3..6. 11 - -
kubeadm: Updated the supported etcd version tov3.for supported control plane versions6. 11 v1.,34 v1., and35 v1..36 - - cyphar.com/go-pathrs: [v0.2.2 → v0.2.5](https://github.com/cyphar/libpathrs.git/compare/go-pathrs/v0.2.2...go-pathrs/v0.2.5)
- - github.com/Azure/go-ansiterm: [306776e → faa5f7b](https://github.com/Azure/go-ansiterm/compare/306776e...faa5f7b)
- - github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp: [v1.30.0 → v1.32.0](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/compare/v1.30.0...v1.32.0)
- - github.com/Microsoft/hnslib: [v0.1.2 → v0.1.3](https://github.com/Microsoft/hnslib/compare/v0.1.2...v0.1.3)
- - github.com/antlr4-go/antlr/v4: [v4.13.0 → v4.13.1](https://github.com/antlr4-go/antlr/compare/v4.13.0...v4.13.1)
- - github.com/cncf/xds/go: [ee656c7 → dba9d58](https://github.com/cncf/xds/compare/ee656c7534f5d7dc23d44dd611689568f72017a6...dba9d589def2cd10099a3a64887d859188c2f57a)
- - github.com/container-storage-interface/spec: [v1.9.0 → cd9e7ad](https://github.com/container-storage-interface/spec/compare/v1.9.0...cd9e7ad1ae0915cabcad179f2b8a660c0cb6eb9f)
- - github.com/containerd/containerd/api: [v1.10.0 → v1.11.1](https://github.com/containerd/containerd/compare/api/v1.10.0...api/v1.11.1)
- - github.com/containerd/ttrpc: [v1.2.7 → v1.2.9](https://github.com/containerd/ttrpc/compare/v1.2.7...v1.2.9)
- - github.com/containerd/typeurl/v2: [v2.2.3 → v2.3.0](https://github.com/containerd/typeurl/compare/v2.2.3...v2.3.0)
- - github.com/coredns/corefile-migration: [v1.0.31 → v1.0.34](https://github.com/coredns/corefile-migration/compare/v1.0.31...v1.0.34)
- - github.com/cyphar/filepath-securejoin: [v0.6.1 → v0.7.0](https://github.com/cyphar/filepath-securejoin/compare/v0.6.1...v0.7.0)
- - github.com/envoyproxy/go-control-plane/envoy: [v1.36.0 → v1.37.0](https://github.com/envoyproxy/go-control-plane/compare/envoy/v1.36.0...envoy/v1.37.0)
- - github.com/envoyproxy/protoc-gen-validate: [v1.3.0 → v1.3.3](https://github.com/envoyproxy/protoc-gen-validate/compare/v1.3.0...v1.3.3)
- - github.com/fxamacker/cbor/v2: [v2.9.0 → v2.9.1](https://github.com/fxamacker/cbor/compare/v2.9.0...v2.9.1)
- - github.com/go-jose/go-jose/v4: [v4.1.3 → v4.1.4](https://github.com/go-jose/go-jose/compare/v4.1.3...v4.1.4)
- - github.com/go-openapi/jsonpointer: [v0.21.0 → v1.0.0](https://github.com/go-openapi/jsonpointer/compare/v0.21.0...v1.0.0)
- - github.com/go-openapi/jsonreference: [v0.20.2 → v1.0.0](https://github.com/go-openapi/jsonreference/compare/v0.20.2...v1.0.0)
- - github.com/go-openapi/swag: [v0.23.0 → v0.27.1](https://github.com/go-openapi/swag/compare/v0.23.0...v0.27.1)
- - github.com/golang-jwt/jwt/v5: [v5.3.0 → v5.3.1](https://github.com/golang-jwt/jwt/compare/v5.3.0...v5.3.1)
- - github.com/google/cel-go: [v0.26.0 → v0.29.2](https://github.com/google/cel-go/compare/v0.26.0...v0.29.2)
- - github.com/google/pprof: [294ebfa → 545e8a4](https://github.com/google/pprof/compare/294ebfa9ad836ed3d00d43d54ea599339e403110...545e8a4df9364095d66e521b8f515f7af961e653)
- - github.com/grpc-ecosystem/grpc-gateway/v2: [v2.27.7 → v2.29.0](https://github.com/grpc-ecosystem/grpc-gateway/compare/v2.27.7...v2.29.0)
- - github.com/klauspost/compress: [v1.18.0 → v1.19.0](https://github.com/klauspost/compress/compare/v1.18.0...v1.19.0)
- - github.com/moby/term: [v0.5.0 → v0.5.2](https://github.com/moby/term/compare/main...v0.5.2)
- - github.com/onsi/ginkgo/v2: [v2.28.1 → v2.32.0](https://github.com/onsi/ginkgo/compare/v2.28.1...v2.32.0)
- - github.com/onsi/gomega: [v1.39.1 → v1.40.0](https://github.com/onsi/gomega/compare/v1.39.1...v1.40.0)
- - github.com/opencontainers/cgroups: [v0.0.6 → v0.0.7](https://github.com/opencontainers/cgroups/compare/v0.0.6...v0.0.7)
- - github.com/opencontainers/selinux: [v1.13.1 → v1.15.1](https://github.com/opencontainers/selinux/compare/v1.13.1...v1.15.1)
- - github.com/prometheus/client_golang: [v1.23.2 → v1.24.0](https://github.com/prometheus/client_golang/compare/v1.23.2...v1.24.0)
- - github.com/prometheus/common: [v0.67.5 → v0.70.0](https://github.com/prometheus/common/compare/v0.67.5...v0.70.0)
- - github.com/prometheus/procfs: [v0.19.2 → v0.21.1](https://github.com/prometheus/procfs/compare/v0.19.2...v0.21.1)
- - github.com/sirupsen/logrus: [v1.9.3 → v1.9.4](https://github.com/sirupsen/logrus/compare/v1.9.3...v1.9.4)
- - github.com/spf13/pflag: [v1.0.9 → v1.0.10](https://github.com/spf13/pflag/compare/v1.0.9...v1.0.10)
- - github.com/stretchr/objx: [v0.5.2 → v0.5.3](https://github.com/stretchr/objx/compare/v0.5.2...v0.5.3)
- - go.etcd.io/bbolt: [v1.4.3 → v1.5.0](https://github.com/etcd-io/bbolt/compare/v1.4.3...v1.5.0)
- - go.etcd.io/etcd/api/v3: [v3.6.8 → v3.7.0](https://github.com/etcd-io/etcd/compare/api/v3.6.8...api/v3.7.0)
- - go.etcd.io/etcd/client/pkg/v3: [v3.6.8 → v3.7.0](https://github.com/etcd-io/etcd/compare/client/pkg/v3.6.8...client/pkg/v3.7.0)
- - go.etcd.io/etcd/client/v3: [v3.6.8 → v3.7.0](https://github.com/etcd-io/etcd/compare/client/v3.6.8...client/v3.7.0)
- - go.etcd.io/etcd/pkg/v3: [v3.6.8 → v3.7.0](https://github.com/etcd-io/etcd/compare/pkg/v3.6.8...pkg/v3.7.0)
- - go.etcd.io/etcd/server/v3: [v3.6.8 → v3.7.0](https://github.com/etcd-io/etcd/compare/server/v3.6.8...server/v3.7.0)
- - go.etcd.io/raft/v3: [v3.6.0 → v3.7.0](https://github.com/etcd-io/raft/compare/v3.6.0...v3.7.0)
- - go.opentelemetry.io/contrib/detectors/gcp: [v1.39.0 → v1.43.0](https://github.com/open-telemetry/opentelemetry-go-contrib/compare/detectors/gcp/v1.39.0...detectors/gcp/v1.43.0)
- - go.opentelemetry.io/contrib/instrumentation/github.com/emicklei/go-restful/otelrestful: [v0.65.0 → v0.69.0](https://github.com/open-telemetry/opentelemetry-go-contrib/compare/instrumentation/github.com/emicklei/go-restful/otelrestful/v0.65.0...instrumentation/github.com/emicklei/go-restful/otelrestful/v0.69.0)
- - go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc: [v0.65.0 → v0.68.0](https://github.com/open-telemetry/opentelemetry-go-contrib/compare/instrumentation/google.golang.org/grpc/otelgrpc/v0.65.0...instrumentation/google.golang.org/grpc/otelgrpc/v0.68.0)
- - go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp: [v0.65.0 → v0.69.0](https://github.com/open-telemetry/opentelemetry-go-contrib/compare/instrumentation/net/http/otelhttp/v0.65.0...instrumentation/net/http/otelhttp/v0.69.0)
- - go.opentelemetry.io/contrib/propagators/b3: [v1.40.0 → v1.44.0](https://github.com/open-telemetry/opentelemetry-go-contrib/compare/propagators/b3/v1.40.0...propagators/b3/v1.44.0)
- - go.opentelemetry.io/otel: [v1.41.0 → v1.44.0](https://github.com/open-telemetry/opentelemetry-go/compare/v1.41.0...v1.44.0)
- - go.opentelemetry.io/otel/exporters/otlp/otlptrace: [v1.40.0 → v1.44.0](https://github.com/open-telemetry/opentelemetry-go/compare/exporters/otlp/otlptrace/v1.40.0...exporters/otlp/otlptrace/v1.44.0)
- - go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc: [v1.40.0 → v1.44.0](https://github.com/open-telemetry/opentelemetry-go/compare/exporters/otlp/otlptrace/otlptracegrpc/v1.40.0...exporters/otlp/otlptrace/otlptracegrpc/v1.44.0)
- - go.opentelemetry.io/otel/exporters/stdout/stdouttrace: [v1.40.0 → v1.44.0](https://github.com/open-telemetry/opentelemetry-go/compare/exporters/stdout/stdouttrace/v1.40.0...exporters/stdout/stdouttrace/v1.44.0)
- - go.opentelemetry.io/otel/metric: [v1.41.0 → v1.44.0](https://github.com/open-telemetry/opentelemetry-go/compare/metric/v1.41.0...metric/v1.44.0)
- - go.opentelemetry.io/otel/sdk: [v1.40.0 → v1.44.0](https://github.com/open-telemetry/opentelemetry-go/compare/sdk/v1.40.0...sdk/v1.44.0)
- - go.opentelemetry.io/otel/sdk/metric: [v1.40.0 → v1.44.0](https://github.com/open-telemetry/opentelemetry-go/compare/sdk/metric/v1.40.0...sdk/metric/v1.44.0)
- - go.opentelemetry.io/otel/trace: [v1.41.0 → v1.44.0](https://github.com/open-telemetry/opentelemetry-go/compare/trace/v1.41.0...trace/v1.44.0)
- - go.opentelemetry.io/proto/otlp: [v1.9.0 → v1.10.0](https://github.com/open-telemetry/opentelemetry-proto-go/compare/otlp/v1.9.0...otlp/v1.10.0)
- - go.yaml.in/yaml/v2: [v2.4.3 → v2.4.4](https://github.com/yaml/go-yaml/compare/v2.4.3...v2.4.4)
- - golang.org/x/crypto: [v0.47.0 → v0.54.0](https://go.googlesource.com/crypto/+/506e022208b864bc3c9c4a416fe56be75d10ad24^1..cdce021fa6c7d9c7eb2743bfbe551f0a98fd5d62/)
- - golang.org/x/exp: [944ab1f → 746e56f](https://go.googlesource.com/exp/+/944ab1f22d936eefb8f6260ecd2053101d8d7b2a^1..746e56fc9e2fafde18176275ce0b96b06ac53955/)
- - golang.org/x/mod: [v0.32.0 → v0.37.0](https://go.googlesource.com/mod/+/4c04067938546e62fc0572259a68a6912726bcdd^1..deb1dfcdb7c7fd98fb5afddc3e95dd36d5880874/)
- - golang.org/x/net: [v0.49.0 → v0.57.0](https://go.googlesource.com/net/+/d977772e17ccaa1903b2af736f6405ab3a9f05cc^1..b8f09f6f062ceb4531b7af4bd17a5c8fe9c4b2b5/)
- - golang.org/x/oauth2: [v0.34.0 → v0.36.0](https://go.googlesource.com/oauth2/+/acc38155b7f6f36aefcb58faff6f36d314dd915c^1..4d954e69a88d9e1ccb8439f8d5b6cbef230c4ef9/)
- - golang.org/x/sync: [v0.19.0 → v0.22.0](https://go.googlesource.com/sync/+/2a180e22fddcc336475e72aa950be958c1b68d33^1..1eb64d4bc0cde6da1bb8ebc7f178bb577508e5d0/)
- - golang.org/x/sys: [v0.40.0 → v0.47.0](https://go.googlesource.com/sys/+/2f442297556c884f9b52fc6ef7280083f4d65023^1..9e7e939dcafac07e8ab4cffa6e5fc74908413f00/)
- - golang.org/x/telemetry: [bd525da → 59b4966](https://go.googlesource.com/telemetry/+/bd525da824e2505db9e8ac44025316bf6f43a6f6^1..59b4966ccb57499277814ee2272936a2c01cfbcd/)
- - golang.org/x/term: [v0.39.0 → v0.45.0](https://go.googlesource.com/term/+/a7e5b0437ffa3159709172efbe396bc546550e23^1..9f69229da31ca6a34b522f59dbe07cad5ea21587/)
- - golang.org/x/text: [v0.33.0 → v0.40.0](https://go.googlesource.com/text/+/536231a9abc69feaab8d726b5ec75ee8d3620829^1..724af9c35838492dcaacc1ac51a8a0187c994c54/)
- - golang.org/x/time: [v0.14.0 → v0.15.0](https://go.googlesource.com/time/+/2b4e43900c03fd6b77109b7b2b6d77583f48bc1c^1..812b343c8714c317b0dad633efa6d103e554c006/)
- - golang.org/x/tools: [v0.41.0 → v0.47.0](https://go.googlesource.com/tools/+/2ad2b30edf98d0e3b67a7b3e8f6d1d6e41c963c3^1..fbf9f2e2c8124fbe1877f5ed2857111038d9fe12/)
- - gonum.org/v1/gonum: [v0.16.0 → v0.17.0](https://github.com/gonum/gonum/compare/v0.16.0...v0.17.0)
- - google.golang.org/genproto/googleapis/api: [8636f87 → 3dc84a4](https://github.com/googleapis/go-genproto/compare/8636f8732409467ddc8453f81f4429397739bb17...3dc84a4a5aaa87331e10f51e22e90d961f986894)
- - google.golang.org/genproto/googleapis/rpc: [8636f87 → 3dc84a4](https://github.com/googleapis/go-genproto/compare/8636f8732409467ddc8453f81f4429397739bb17...3dc84a4a5aaa87331e10f51e22e90d961f986894)
- - google.golang.org/grpc: [v1.79.3 → v1.82.1](https://github.com/grpc/grpc-go/compare/v1.79.3...v1.82.1)
- - k8s.io/gengo/v2: [ec3ebc5 → 25e2208](https://github.com/kubernetes/gengo/compare/ec3ebc5fd46b84f44dfb135e9684c6567791dd8e...25e2208e0dc371a827289e7faced19a2dbcd480b)
- - k8s.io/kube-openapi: [43fb72c → d427ff9](https://github.com/kubernetes/kube-openapi/compare/43fb72c5454a03ed83388cf20c070499ee359af8...d427ff9ee9ad05f5da435abbb7c5929cb713ac56)
- - k8s.io/utils: [b8788ab → be93311](https://github.com/kubernetes/utils/compare/b8788abfbbc27cab6c8732274b5c2ae213868854...be93311217bd4e42d0aa42a5987d08e8f5581ec0)
- - sigs.k8s.io/apiserver-network-proxy/konnectivity-client: [v0.34.0 → v0.36.0](https://github.com/kubernetes-sigs/apiserver-network-proxy/compare/konnectivity-client/v0.34.0...konnectivity-client/v0.36.0)
- - sigs.k8s.io/knftables: [v0.0.21 → v0.0.22](https://github.com/kubernetes-sigs/knftables/compare/v0.0.21...v0.0.22)
- - sigs.k8s.io/structured-merge-diff/v6: [v6.3.2 → v6.4.2](https://github.com/kubernetes-sigs/structured-merge-diff/compare/v6.3.2...v6.4.2)
fixes (123)
- Fixed CEL cost estimation for
metadata.andname metadata.generateName - Fixed DRA
CapacityRequestPolicyRangeto support fractional quantities in milli-scale. - Fixed Pod status validation for reported Linux container user UIDs to accept values above 2147483647 and up to the unsigned 32-bit UID limit.
- Fixed a
v1.+ regression handling containers with environment values set from Secret API objects containing binary non-utf8 data.34 - Fixed a bug in DRA consumable capacity
- Fixed the overestimation of a Pod's resource footprint during resize operations for multi-container Pods.
- Preserved data in the DRA-related Pod status fields
resourceClaimStatuses,extendedResourceClaimStatus, andnodeAllocatableResourceClaimStatuseswhen handling Pod status updates that omit those fields. This prevents updates from older clients from unsetting these DRA fields, which could leave Pods permanently stuck in Terminating. - Fixed a nil pointer dereference in
client-goevent key generation by adding nil checks ingetEventKey,getSpamKey, andEventAggregatorByReasonFunc, preventing panics when processing nil events. - Fixed a bug in
kube-schedulerwhen theDRADeviceTaintRulesfeature gate is enabled that could cause scheduler panics when DeviceTaintRules exist and ResourceSlices change, or cause new DeviceTaintRules changes to be ignored. - Fixed a bug where nomination of a gated pod wasn't preventing lower-priority pods from scheduling on the nominated space.
- Changed
kube-apiserverto validate the--advertise-addressIP when using--endpoint-reconciler-typemaster-countorlease, ensuring the specified IP address can be persisted to an Endpoints API object successfully. - Changed
kube-proxyto truncate nftables comments to the kernel's 128-byte limit before programming service maps, avoiding sync failures for long Service names. - DRA: Fixed a bug where a rare missed informer update of a ResourceClaim could cause Pods to remain pending until the unschedulable queue was flushed.
- Fixed 409 Conflict errors between the PVC protection controller and the PV binder during initial PVC binding.
- Fixed CEL behavior for
setandmaplists. - Fixed DRA scheduling bugs where the structured allocator incorrectly counted a device's shared counters while evaluating candidates.
- Fixed Pod-level MemoryQoS memory protection (
memory.andmin memory.) being silently dropped during in-place Pod resizelow - Fixed VolumeAttachment validation to report the correct maximum message size (1024 bytes) in error messages.
- Fixed Windows CPU affinity so that, when the CPU Manager
staticpolicy and the Memory Manager are both active under theWindowsCPUAndMemoryAffinityfeature gate, containers are pinned to the CPU Manager's allocated set - Fixed
DecodeMetadataFromStreamto skip only entries with unknown API versions and return errors for decode failures or malformed metadata in supported versions, preventing silent data loss. - Fixed
kube-apiserverhanging indefinitely onSIGTERMwhen it could not create its identity Lease - Fixed
kube-proxyto remove stale conntrack entries when a UDP Service no longer has any serving endpoints - Fixed
kubectl cluster-info dump --output-directorycreating world-readable dump files. - Fixed
kubectl get storageclassto show only the effective default StorageClass as "(default)" when multiple StorageClasses have the default annotation. - Fixed
kubeletapplying device health updates to the wrong Pod status when device plugins for different resources exposed devices with identical IDs. - Fixed
kubeletfailure starting on ZFS due to missingcadvisorplugin. - Fixed a DRA consumable-capacity scheduling bug where a device that consumes shared counters could have them counted twice
- Fixed a DRA issue where drivers might not recreate ResourceSlices that were deleted externally
- Fixed a DRA partitionable devices issue where counters published by a DRA driver outside the valid
int64range could be mutated in the informer cache - Fixed a DRA scheduling bug where the structured allocator keyed shared-counter caches by pool name only
- Fixed a Dynamic Resource Allocation (DRA) scheduler bug that could assign mutually exclusive device partitions to multiple Pods.
- Fixed a
kube-proxyIPVS-mode performance bug wheresyncProxyRulescould take tens of seconds in clusters with many Services - Fixed a
kube-proxyissue on Windows where transient HNS downtime during restart or recovery could cause incorrect LoadBalancer state reconciliation - Fixed a
kubeletbug where init containers could be skipped when a Pod sandbox was recreated - Fixed a
kubeletissue where Pods withsubPathmounts could become stuck in an error loop after FUSE or GlusterFS network filesystem disruptions. - Fixed a
kubeletmemory leak regression inv1.caused by leaked contexts on every Pod sync.36 - Fixed a
kubeletpanic in image pull credential verification whenmaxParallelImagePullsis configured above 31. - Fixed a
v1.regression that could cause a panic in the endpoint controller when processing services with empty IPFamilies field33 - Fixed a
v1.regression where exec readiness probes stopped executing35 - Fixed a bug in CEL where
quantity.mutated the receiver.Add - Fixed a bug in ImageLocality scoring where image volumes could receive a higher score than equivalent regular container images.
- Fixed a bug in
kube-apiserverwhere a request matching multiple ValidatingAdmissionPolicy bindings with audit actions only recorded the first validation failure in the audit annotation - Fixed a bug in the DRA
kubeletplugin helper where drivers with names longer than ~30 characters could not enable rolling updates - Fixed a bug that caused Pods in a PodGroup sharing a ResourceClaim to get stuck scheduling.
- Fixed a bug that could cause the admission controller to panic when evaluating CEL expressions against typed map lists with three keys.
- Fixed a bug when the
GenericWorkloadfeature gate is enabled that could prevent Pods in the same PodGroup sharing the same ResourceClaim from successfully scheduling. - Fixed a bug where Burstable Pod
memory.soft protection was ineffectivelow - Fixed a bug where Pod
.could flap between partial lists of claims when multiple claims were used in the Pod.status. resourceClaimStatuses - Fixed a bug where Pods in a PodGroup sharing a ResourceClaim could be scheduled to Nodes where the ResourceClaim is not available.
- Fixed a bug where Pods that share multi-node claims and also have per-node claims can get stuck in Pending.
- Fixed a bug where ResourceClaims using
allocationMode: Allwith consumable capacity could be partially allocated - Fixed a bug where ValidatingAdmissionPolicy and MutatingAdmissionPolicy evaluation could observe subtle differences
- Fixed a bug where
kubectl drain --disable-eviction --dry-run=serverhangs indefinitely. - Fixed a bug where a StatefulSet with the
OnDeleteupdate strategy never updatedStatus.CurrentRevision - Fixed a bug where disabling the
MemoryQoSfeature gate did not clear per-containermemory.cgroup valueshigh - Fixed a bug where enabling the
DRAListTypeAttributesfeature gate could prevent device allocation even when a valid combination existed. - Fixed a bug where kubelet would generate an event once per second for every image volume in a pod.
- Fixed a bug where non-admitted Pods could briefly count against the allocated budget
- Fixed a bug where pods with multiple subPath volume mounts on Windows would get stuck in Terminating state
- Fixed a bug where successfully scheduled Pods could be stuck with the
PodScheduled=Falsecondition. - Fixed a bug where the
kubeletnode shutdown manager could leak D-Bus connections on repeated failures - Fixed a bug where the kubelet did not enforce per-container ephemeral-storage limits on restartable init containers (sidecar containers)
- Fixed a case where Pods in a PodGroup that were successfully evaluated during a failed PodGroup scheduling cycle had
nominatedNodeNameset from that evaluation instead of from PodGroup preemption. - Fixed a concurrent map read/write data race in
handleSchedulingFailureduring scheduling failure handling. - Fixed a kube-scheduler panic when a DRA ResourceClaim using
allocationMode: Allselects a device that consumes shared counters. - Fixed a metrics leak in the scheduler
PriorityQueue - Fixed a panic caused by integer division by zero and incorrect ResourceSlice admission validation
- Fixed a panic in ResourceSlice validation that could occur when the
DRAConsumableCapacityfeature gate was enabled - Fixed a panic in
kube-controller-managerthat could occur when a StorageVersionMigration targeted a resource missing from the RESTMapper - Fixed a race condition in preemption, where a preemptor pod could get stuck in unschedulable state.
- Fixed a race in
kubeletwherePrepareResourcescould attach a Pod to a ResourceClaim that was concurrently being unprepared - Fixed a regression in Kubernetes
v1.where, with a Parallel Pod management policy, unavailable Pods from an older revision were incorrectly counted toward the35 maxUnavailablebudget. - Fixed a regression in Server-Side Apply where patching a container type (list or map) could return
422 requirederrors for apply requests that previously succeeded. - Fixed a regression in
v1.where modifications to scheduling directives (36 nodeSelector,tolerations,nodeAffinity) on suspended Jobs were rejected - Fixed a regression in retrying deferred resizes caused by changes to Pod resource footprint calculation.
- Fixed a regression where the Job controller could report
status.as 0 while replacement Pod creation was deferred due to pod-failure backoffactive - Fixed a regression where the
kubeletdid not clear stale cgroup v2memory.andmin memory.values when thelow MemoryQoSfeature gate was disabled after being previously enabled. - Fixed a scheduler bug in DRA consumable capacity where a ResourceSlice with a device capacity requirement stored as a high-precision decimal
- Fixed a scheduler bug where clearing
NominatedNodeNamecould leave Pods tracked under an empty node key in the scheduler's nominator. - Fixed a scheduler cache bug where assumed Pods were not removed correctly from
PodGroupStateafter receiving a deletion timestamp. - Fixed admission handling so that updates to namespaced objects that still exist after their namespace was deleted are allowed.
- Fixed an issue in the CronJob controller where it failed to adopt existing Jobs by erroneously using the empty namespace from the
jobTemplate. - Fixed an issue that could cause duplicate configuration entries to be reported in ResourceClaim status.
- Fixed an issue where PodGroup preemption that detected an ongoing preemption would clear
nominatedNodeNameon the PodGroup's Pods. - Fixed an issue where the StatefulSet controller's skip metrics were not properly registered.
- Fixed an issue where the
kubeletwould delete the CSI mount directory when a periodicNodePublishVolumecall (triggered by settingCSIDriver.to true) returned an errorspec. requiresRepublish - Fixed audit logging of malformed patch request bodies.
- Fixed capacity accounting in the DRA consumable-capacity allocator.
- Fixed duplicate logs when trying to attach to a pod fails.
- Fixed duplicated mount arguments in log string output from
MakeMountArgsSensitiveWithMountFlags. - Fixed handling of a certificate authority path outside the
.directory on Windowskube/config - Fixed inconsistent
ephemeral-storageformatting between capacity and allocatable values in Node status - Fixed incorrect error message formatting in the HPA controller when object metric retrieval fails.
- - Fixed inter-pod affinity, anti-affinity, and volume restriction evaluation in
kube-schedulerduring PodGroup scheduling cycles. The scheduler snapshot'sAssumePodandForgetPodmethods correctly maintain affinity node lists and PVC usage tracking. - - Fixed nil pointer dereference in Windows memory eviction threshold notifier when
GetPerformanceInfo()fails. - - Fixed queue hint for inter-pod anti-affinity in case there are multiple terms, which might have caused delays in scheduling.
- - Fixed regression in
kubectlresource printing on bigger data sets (100+ rows). - - Fixed stale remote HNS endpoint cleanup on Windows when a pod IP is reused across nodes in L2Bridge networks, preventing DNS timeouts caused by traffic being routed to the wrong node.
- - Fixed the DRA
kubeletplugin helper repeating the listen error instead of reporting why removing a stale Unix domain socket failed when it could not start its listener. - - Fixed the ResourceClaim controller mutating the shared informer cache when creating a ResourceClaim from a ResourceClaimTemplate that has annotations.
- - Fixed the
kube-apiserverto create metadata fields for create-via-update and created-via-apply requests like they are for create requests.UIDandresourceVersionpreconditions are still honored. - - Fixed the error message from
PodGroupPostFilterto contain the correct extension point name. - - Fixed the inconsistency between opportunistic batching and PodGroups that made the batching hints always infeasible during PodGroup scheduling cycle.
- - Fixed the wrong cause of the UnexpectedJob event/warning by checking the owner reference of the job correctly in the cron job controller.
- - Improved
kubeadm joinreliability by using theKubernetesAPICalltimeout (default 1 minute) when fetching thekubeadm-configConfigMap from the cluster, instead of the short 350ms retry previously used for optional component configs. A newshortConfigMapGetparameter was added toFetchInitConfigurationFromClusterso that callers likekubeadm resetcan still use the short retry. - - Improved scheduler handling of large PodGroups by reducing the likelihood of scheduling stalls when member Pods transiently fail to bind to Nodes, such as when many Pods share the same ResourceClaim.
- - Improved the resilience of
kubeadmetcd learner promotion by correctly handling cases where promotion succeeds but a transient client-side error is returned, preventing unnecessary etcd-join failures. - - Fixed
kubeletto recover from corrupted subpath mount points (for example, stale NFS file handle) during container restart instead of leaving the pod stuck inCreateContainerConfigError. - - Updated the
kubeletto emitFailedToRetrieveImagePullSecretevents only when an image pull has failed. - - Updated the
pods/bindingsubresource endpoint to validate the specified node name consistently. - -
kubeadm: Changedkubeadm jointo return a clear error message when the TLS bootstrap kubeconfig has a current-context that does not appear in the contexts list, instead of panicking with a nil pointer dereference. - -
kubeadm: Changed cluster-info discovery over HTTPS to check the HTTP response status code, so a non-200 response produces a clear error instead of a confusing kubeconfig parse failure. - -
kubeadm: Fixed MemberPromote to skip the etcd promote API call when the member is already a voting member, avoiding unnecessary retries and timeout. - -
kubeadm: Fixed a panic inkubeadmPKI key loading when the private key type and public key type mismatch. - -
kubeadm: Fixed kubeadm init phasecerts --dry-runto correctly copy existing CA files. - -
kubelet/DRA: Fixed a bug where retrying a partially failedPrepareResourcescaused duplicate CDI device IDs to be passed to the CRI runtime, which could cause container start to fail. - -
kubelet: Fixed a goroutine leak on shutdown by making the eviction manager's monitoring goroutine exit promptly when thekubeletcontext is cancelled. - -
kubelet: Fixed incorrect Pod-level CPU requests reported in status from the cgroup v2 readback. - - DRA: Fixed a potential crash in the scheduler, recovered after restart, when the ResourceSlice tracker encountered an OnDelete event for a DeviceTaintRule whose deleted object is unknown.
- - Fixed a bug in
kubeletDRA where deleting a Pod could unprepare resources still in use by another Pod. - - Fixed a race condition where server-side apply requests for custom resources could observe an updated CustomResourceDefinition before the apply path was fully synchronized, causing inconsistent dry-run behavior.
- - Fixed a theoretical issue where nodes might have been denied access to synthesized ResourceClaims for pods using extended resources (for example,
nvidia.), causing containers to get stuck incom/gpu ContainerCreating. Not observed in practice. - - Fixed server-side apply to correctly drop status changes when tracking field ownership for PodGroup, PodCompositeGroup, and PodCertificateRequest.
additions (75)
- Updated the
kubeletto log its effective configuration at startup. - Added Alpha support for DRA device compatibility groups
- Added Alpha support for binding service account tokens to webhook configurations with attestations
- Added Alpha support for defining the file owner of atomically written volume files
- Added CompositePodGroup support to the building block APIs and the
workloadbuilderlibrary. - Added Workload-aware scheduling (WAS) support to the Job controller
- Added
CheckpointPodandRestorePodRPCs to the CRIv1RuntimeService API - Added a
PreemptionPolicyfield to PodGroup - Added a
protocolfield tohttpGetprobes - Added a defense-in-depth check to the
NodeRestrictionadmission plugin for PodCertificateRequests. - These accounting fixes change the numbers reported in
v1..36 - Added an opt-in userspace TCP proxy to the nftables
kube-proxybackend - Added dry-run support to unsafe corrupt object deletion
- Added scheduler support for preempting lower-priority Pods
- Added support for dynamically resizing memory-backed volumes
- setting Unix permission bits (0000-01777) through the
modefield - specifying bind mount options (
noexec,nodev,nosuid) per container volume mount. - Added the API changes required for reporting volume health.
- Added the CompositePodGroup API to
scheduling..k8s. io/v1alpha3 - Added the Recreate update strategy for StatefulSet
- Added the
--concurrent-disruption-syncsflag - Added the
.Pod field, along with the EvictionRequest and Eviction resources.spec. evictionResponders - Added the
DefaultPodSysctlskubeletconfiguration field - Added the
GracefulNodeShutdownInProgress,DrainInProgress,Drained,MaintenancePlanned, andMaintenanceInProgressNode lifecycle conditions. - Added the
PodGroupPostFilterextension point - Added the
PreemptionPolicyfield to PodGroupTemplate - improving scheduling throughput.
- Added the core machinery for Conditional Authorization
- Added Alpha support for
DRAOptionalNodeOperations - PersistentVolumeClaims report the
Unusedcondition - adds the
preventPodSchedulingIfMissingfield to CSIDriver - Added TLS support for gRPC container probes
- Added GROUP, SCOPE, VERSIONS, and CREATED AT columns to
kubectl get crdoutput - Added Prometheus metrics for Windows kube-proxy (winkernel) load balancer operation failures
- Added ServiceName, PodManagementPolicy, and PersistentVolumeClaimRetentionPolicy to
kubectl describe statefulsetoutput. - Added a
--max-depthflag tokubectl explain --recursiveto limit the depth of nested fields displayed in the output. - Added a warning when
kube-proxyis started without an explicitly specified proxy mode (such asiptables,ipvs, ornftables), because the default mode on Linux will switch fromiptablestonftablesin a future release. - Added an Alpha feature gate,
ConsistentListFromCacheSkipTimeoutFallback. When enabled,kube-apiserverreturns HTTP 429 for consistent LIST requests that cannot be served from the watch cache within the timeout window, instead of falling back to storage. - Added an
erroroutcome to theroute_sync_totalmetric for failed route reconciles, alongside the existingchangedandnoopoutcomes. - Added metric
apiserver_watch_cache_initialization_duration_secondsrecording the duration of the most recent watch cache initialization, labeled by group and resource. - Added metrics for informer activity in
kube-apiserver. - Added progress reporting to StorageVersionMigration conditions, allowing users to see how many objects a migration has processed.
- Added scheduler metrics for the topology-aware scheduling (TAS) placement phases, available when the
TopologyAwareWorkloadSchedulingfeature gate is enabled:scheduler_generated_placements_total,scheduler_placement_evaluations_total, andscheduler_placement_evaluation_duration_seconds. - Added the Alpha
apiserver_watch_events_dispatch_duration_secondsmetric, recording the duration from when a watch event is decoded from etcd until it is written to the watcher's outgoing result channel. - Added the Alpha
kubeletmetrickubelet_pod_deferred_resize_duration_secondshistogram and thepriority_bucketlabel on thekubelet_pod_pending_resizesgauge. - Added the Alpha
kubeletmetricpod_level_resources_admission_totalto track adoption of Pod-Level Resources (KEP-2837) upon Pod admission, categorized by resource configuration mode and QoS class. - Added the
--proxy-urlflag tokubectlto override the proxy URL configured in the kubeconfig. - Added the
CompositePodGroupfeature gate to enable Composite Pod Group functionality. - Added the
EtcdRangeStreambeta feature gate. The watch cache initializes by streaming objects from etcd in a singleRangeStreamRPC instead of paginatedRangerequests. - Added the
KubeProxyIPVSfeature gate in preparation for deactivating and then removing theipvsmode ofkube-proxy. - Added the
PodGroupfield to thePodGroupInfoobject inkube-schedulerto enable plugins to obtain a consistent state throughout the scheduling cycle. - Added the
allocatedPodskubeletendpoint, which surfaces thekubelet's allocated Pod spec for debugging in-place Pod resizing and other Pod update issues. Requires theKubeletAllocatedPodsEndpointfeature gate. - Added the
client-goinformer metricsinformer_store_resource_version,informer_queued_items, andinformer_processing_latency_secondstokube-scheduler, labelledname="kube-scheduler". - Added the
owner_api_groupandowner_api_kindlabels to thedynamic_resource_allocation_resourceclaim_creates_totalmetric to distinguish ResourceClaims created for Pods from those created for PodGroups under theDRAWorkloadResourceClaimsfeature gate. - Added the
queued_entitiesandqueue_incoming_entities_totalscheduler metrics. - Added the
trigger(periodicornode_change) andoutcome(changedornoop) labels to the Alpha route controller metricroute_controller_route_sync_total. - Added the scheduler extension point
PlacementFeasibleto allow early termination of the PodGroup scheduling cycle. - Added the standard device attribute
resource.and sysfs-based helper functions for DRA drivers.kubernetes. io/numaNode - Applied
--field-selectorto pod metrics when invokingkubectl top pod. - Enhanced Pod-by-Pod preemption to support PodGroups as preemption victims.
- Made it possible for authorizers to return conditional decisions in addition to unconditional (Allow/Deny/NoOpinion).
- Retried binding API calls in
kube-schedulerwhen a transient error occurs. - Set the
KUBECTL_PATHenvironment variable to the path of thekubectlbinary when it executes a plugin. - Added three different subtypes of the cluster event resource "Pod": "AssignedPod", "UnscheduledPod", "TargetPod".
kubeadm: Added thekubeproxydaemonsetpatch target to allow patching the kube-proxy DaemonSet duringkubeadm initandkubeadm upgrade, consistent with the existingcorednsdeploymentpatch target.- Added
apiserver_storage_list_duration_seconds, a metric measuring end-to-end apiserver list latency (etcd read plus object decode), labelled by whether etcd RangeStream was used, so streamed and non-streamed lists can be compared directly. - added Pod-level
memory.enforcement when thehigh PodLevelResourcesfeature gate is enabled. - - Generated
metadata.andgeneration status.fields in HorizontalPodAutoscaler resources.observedGeneration - -
client-go: Added support for waiting for in-progress event handler runs to complete before closing the event handler. - -
kubeadm: Changedkubeadm initso that, when the defaultadmin.andconf super-admin.paths are used, the files are loaded but in-memory kubeconfigs are constructed pointing toconf InitConfiguration.instead oflocalAPIEndpoint ClusterConfiguration.. This resolved issues with delayed load balancers that are provisioned only after the firstcontrolPlaneEndpoint kube-apiserverinstance starts. - -
kubeadm: Used a dedicated ClusterRolesystem:kubelet-api-adminfor thekube-apiserverkubeletclient. - -
kubelet: PopulatedinvolvedObject.on node events on a best-effort basis once the node is registered, so node events can be correlated by UID, for example inuid kubectl describe node. The UID is resolved once and not refreshed afterward. If a node is deleted and recreated with a new UID while thekubeletkeeps running, its events continue to use the original UID until thekubeletrestarts. - -
kubelet: Set cgroup v2memory.for BestEffort containers whenhigh MemoryQoSis enabled (per KEP-2570). - - Added the
HasValidationFuncmethod toruntime.to report whether a declarative validation function is registered for a type.Scheme - - Logged a warning in the
kubeletif a static Pod defines an invalidpriorityorpriorityClassName.
constraints (17)
- Added support for derived attributes in DRA
- selecting ResourceSlices by pool name with the field selector
spec.pool. name - must explicitly select which versions to support in their metadata output.
- Relaxed container security context validation so that updates to Pods may set
allowPrivilegeEscalationtogether withCAP_SYSADMIN. - allow modifying
minCountafter creation. - Added
metrics.support tok8s. io/v1 kubectl top. - Added
net.andipv4. tcp_slow_start_after_idle net.to the allowed safe sysctls list.ipv4. tcp_notsent_lowat - Added support for CBOR encoding in discovery endpoints and structured error responses when the
CBORServingAndStoragefeature gate is enabled. - Added the 90s, 120s, 180s, and 300s buckets to the
watch_list_duration_secondsmetric. - Added validation to PodGroup scheduling that, when the
PodGroupPreemptionPolicyfeature gate is enabled, ensures that the preemption policies of Pods being evaluated for scheduling match the priority of the PodGroup. - Added validation to PodGroup scheduling which ensures priorities of the evaluated pods match the priority of the PodGroup.
- Reduced the scope of
EventedPLEGto only accelerate detection of unexpected container terminations. - Updated PodGroup scheduling to skip PostFilter plugins for Pods in a PodGroup cycle. Instead,
PodGroupPostFilterruns only when the entire PodGroup is unschedulable. kubeadm: The preflight checkContainerRuntimeVersionvalidates if the installed container runtime supports theRuntimeConfiggRPC method. For older kubelet versions thanv1., it will return a preflight warning.38 - -
kubeadm: Changed the etcd cluster status check to use a quorum approach instead of considering the health of all members, so the check no longer fails when there are sufficient healthy voting members. - - Changed
client-goto requestv2for aggregated discovery instead of falling back tov2beta1. - - Updated DRA so that the ResourceClaim controller creates ResourceClaims from ResourceClaimTemplates referenced by a Pod that is a member of a PodGroup only when the
DRAWorkloadResourceClaimsfeature gate is enabled. This prevents creating a ResourceClaim for an individual Pod when it is intended to be created for the PodGroup.
defaults (10)
- Changed
kubeadmto explicitly setKubeProxyConfiguration.to iptables whenmode KubeProxyConfigurationis not provided or when themodefield is empty. - enabled them by default.
memory.is not set unless explicitly configured.high - enabled it by default.
- The
storagemigration.API group is enabled by default.k8s. io/v1 - Added the
WatchListCompressionfeature gate (Beta, enabled by default) to compressWatchListresponses with gzip for clients that sendAccept-Encoding: gzip. RegularWatchrequests are unaffected. - Graduated the
ConcurrentWatchObjectDecodefeature gate to Beta, enabled by default. - Graduated the
ManifestBasedAdmissionControlConfigfeature gate to Beta and enabled it by default. - Promoted the
PodLevelResourceManagersfeature gate to Beta, enabled by default. - -
kubeadm: Skipped LocalAPIEndpoint defaulting onkubeadm joinfor worker nodes.
renames (5)
- Moved the
NodeSyncPeriodfield fromKubeCloudSharedConfigurationtoCloudControllerManagerConfiguration..NodeLifecycleController. NodeMonitorPeriod - Renamed signal enum keys in
cri-api - Renamed the PodGroup condition
PodGroupScheduledtoPodGroupInitiallyScheduled - replace
PodGroupTemplateRefwithWorkloadRef - -
kube-controller-managerandkube-schedulerboth exposedynamic_resource_allocation_resourceclaim_creates_totalas a metric for the number of ResourceClaims created, replacing the differently named metrics in each component. Thekube-controller-managermetricresource_claimswas moved to the samedynamic_resource_allocationsubsystem.
removals (2)
cAdvisorapplication/custom metrics are no longer collected- The
/metrics/cadvisorseriescontainer_cpu_load_average_10s,container_cpu_load_d_average_10s, andcontainer_tasks_stateare no longer exported.
A weekly email arrives when a release needs action. Like the security patches and breaking changes in this release.