リリース
CNCF graduated・incubatingプロジェクトのリリースノートのAI分析。
v1.17.3 fixes a chart upgrade blocker, corrects multi-cluster search visibility, prevents over-scheduling, and stabilizes cluster readiness detection during credential rotation.
breakingUnblock chart upgrades: ConfigMap size issue resolved
The karmada-operator-chart had a ConfigMap size overflow when embedding the Karmada CRD, blocking chart upgrades. This is fixed in v1.17.3. If you are stuck on an older chart version due to this error, upgrade immediately to restore your ability to update Karmada deployments.
breakingScheduler now respects cluster resource limits
The scheduler previously allowed multiple workload replicas to land on a single cluster even when it had insufficient resources. v1.17.3 fixes over-scheduling. Audit your current deployments after upgrade to verify actual placement against your expectations, and adjust replica counts if needed.
enhancementSearch now reflects recovered cluster resources correctly
karmada-search now correctly watches recovered clusters and reflects their resources without delay. If you rely on search functionality across cluster recovery scenarios (failover or restoration), upgrade to v1.17.3 to ensure visibility into newly recovered member clusters.
enhancementCluster readiness now waits for failure threshold
A temporary credential rotation failure (missing SecretRef) now waits for the ClusterFailureThreshold before marking a cluster NotReady, instead of failing instantly. This prevents hair-trigger failovers during routine rotation. No action required if you already have a stable credential rotation process; this reduces false positives for teams that do.
主な変更 (4)
- ConfigMap size limit exceeded when embedding Karmada CRD into operator chart—chart upgrades now work
- karmada-search watch connection now immediately reflects resources from recovered clusters
- karmada-scheduler no longer over-schedules workloads on resource-constrained clusters
- ClusterClientSetFunc transient failures no longer instantly mark clusters NotReady; now respects ClusterFailureThreshold
Karmada v1.16.6 fixes three reliability issues: watch-based resource discovery on cluster rejoin, scheduler over-committing resources under concurrent scheduling, and hair-trigger cluster failover on temporary credential errors.
breakingFix scheduler resource exhaustion vulnerability
The scheduler previously allowed workloads to land on clusters with insufficient resources if multiple template resources were being placed simultaneously. Upgrade to v1.16.6 to enforce proper resource availability checks. Review your cluster resource limits and current deployments to confirm they respect intended capacity constraints post-upgrade.
breakingCluster readiness now respects failure threshold
Transient credential issues during secret rotation no longer immediately mark clusters as unready and trigger failover. Karmada now respects the ClusterFailureThreshold before degrading cluster status. If you tune ClusterFailureThreshold or rely on rapid failover behavior, test cluster credential rotation in a staging environment to confirm failover timing matches your expectations.
enhancementWatch discovery now reflects recovered cluster resources
Karmada-search now properly mirrors resources from clusters that rejoin after network issues or restarts. If you run multi-cluster deployments with watch-based resource discovery, verify after upgrading that recovered clusters' resources reappear in search results without manual intervention. Monitor search logs for any delays in resource reconciliation on rejoined clusters.
主な変更 (3)
- karmada-search: watch connections now correctly reflect resources from recovered clusters
- karmada-scheduler: fixed premature resource scheduling when cluster capacity is insufficient
- karmada-controller-manager: cluster status no longer flips to unready on transient credential failures; respects ClusterFailureThreshold
v1.18.0 adds overflow cluster affinities for hybrid cloud burst scheduling and a scheduling overcommit protection mechanism. Mandatory step: upgrade to v1.17.3+ before applying this release.
securityAlpine base image updated to 3.23.4
The base Alpine image moved from 3.23.3 to 3.23.4. No action needed beyond the normal upgrade, but if you pin image digests in your deployment, update them accordingly.
breakingUpgrade to v1.17.3+ before moving to v1.18.x
Before upgrading to v1.18.x, you must first be on v1.17.3+. Skipping this step will break the operator upgrade. Check your current version with `karmadactl version` and upgrade to v1.17.3+ first if you are behind.
breakingDeprecated gRPC fields in scheduler-estimator require plugin updates
Several deprecated gRPC fields in karmada-scheduler-estimator are now replaced: `resourceRequest` → `resourceRequestBytes`, `nodeAffinity` → `nodeAffinityBytes`, `tolerations` → `tolerationsBytes`. If you have custom estimator plugins or tooling that reads these fields directly, update them before upgrading. The old fields still exist in v1.18 but are deprecated and will be removed in a future release.
breakingRemoved flags and metric labels will break existing configs
Two flags removed from karmada-controller-manager: `--cluster-lease-duration` and `--cluster-lease-renew-interval-fraction`. Also, `Etcd.Local.InitImage` is gone from Karmada Init Configuration. If your deployment scripts or Helm values reference these, remove them before upgrading or the components will fail to start. Also check your Prometheus dashboards: the `cluster` and `cluster_name` metric labels are gone, replaced by `member_cluster`.
enhancementCritical scheduler and eviction bug fixes
Two significant scheduler bugs are fixed in this release. First, bindings with insufficient cluster replicas were retrying via exponential backoff (1–10s) instead of the correct 5-minute timer queue — workloads may have appeared stuck. Second, a race condition could silently drop graceful eviction tasks when multiple controllers modified the same ResourceBinding concurrently, meaning workloads might not have been evacuated from failing clusters. If you have observed unexplained scheduling delays or failed evictions, upgrade to v1.18.0 and re-examine affected workloads.
enhancementEnable SchedulingOvercommitProtection in high-throughput clusters
SchedulingOvercommitProtection (disabled by default via feature gate) closes the window where back-to-back scheduling decisions could over-commit a cluster's capacity before Pods are actually bound to nodes. Enable it in high-throughput environments where you see workloads going Pending due to resource exhaustion shortly after scheduling. Set `--feature-gates=SchedulingOvercommitProtection=true` on karmada-scheduler and karmada-scheduler-estimator once you've validated in staging.
enhancementOverflow Cluster Affinities for hybrid cloud burst scheduling
The new `overflowAffinities` field in PropagationPolicy/ClusterPropagationPolicy lets you define fallback cluster groups. The scheduler fills the primary group first, then spills to supplementary groups in order. On scale-down, replicas are reclaimed from supplementary groups first. Useful for IDC-primary / public-cloud-overflow patterns. This is a new API field — existing policies are unaffected unless you add it.
主な変更 (6)
- Mandatory upgrade path: must be on v1.17.3+ before upgrading to v1.18.x (karmada-operator-chart requirement)
- New OverflowClusterAffinities API in PropagationPolicy enables progressive spill-over from primary to supplementary cluster groups with automatic reverse contraction on scale-down
- SchedulingOvercommitProtection feature gate (default: off) prevents resource over-commitment in rapid back-to-back scheduling by caching assumed workloads in the scheduler
- Deprecated gRPC fields in karmada-scheduler-estimator (resourceRequest, nodeAffinity, tolerations) replaced by *Bytes equivalents for Kubernetes 1.35+ compatibility
- Removed: --cluster-lease-duration, --cluster-lease-renew-interval-fraction flags; cluster/cluster_name Prometheus labels replaced by member_cluster; Etcd.Local.InitImage config field
- Critical bug fixes: scheduler mis-routing bindings to backoffQ, silent eviction task drops under concurrent controller writes, ClusterTaintPolicy dropping concurrent health taints
Karmada v1.17.2 is a patch release fixing scheduler misrouting bugs, operator reconciliation failures, and a Job replica assignment error — plus an Alpine base image bump for security.
securityRebuild or pull updated images to get the Alpine 3.23.4 base
The base image was bumped from Alpine 3.23.3 to 3.23.4. If you're running air-gapped or mirrored image setups, make sure to pull and mirror the new v1.17.2 images. Check Alpine's changelog for the specific CVEs addressed if your compliance process requires it.
breakingScheduler backoffQ misrouting may have masked real scheduling failures
Bindings with insufficient cluster replicas were incorrectly queued in backoffQ instead of unschedulableBindings. This means your scheduler may have been retrying workloads on an exponential backoff schedule rather than surfacing them as unschedulable. After upgrading, expect some previously silent failures to become visible in unschedulableBindings — review any workloads that seemed stuck or slow to schedule.
enhancementUpgrade if you use karmada-operator with custom tolerations/affinity or multi-cluster Jobs
Two high-impact fixes landed here: operator-managed deployments were silently ignoring tolerations and affinity configs, which could cause pods to land on unintended nodes. And Job completion counts were being assigned incorrectly across clusters, which could corrupt Job semantics in distributed workloads. Both are straightforward regressions worth patching immediately if either feature is in use.
主な変更 (6)
- karmada-operator now correctly applies tolerations and affinity settings to karmada-aggregated-apiserver and karmada-search deployments
- Fixed non-idempotent secret creation that caused operator init reconciliation failures on repeated runs
- Scheduler no longer misroutes bindings with insufficient replicas to backoffQ — they now correctly land in unschedulableBindings
- Schedule success events with ClusterAffinities now include proper cluster information
- Job completions are now distributed correctly across clusters instead of being assigned to wrong replicas
- Base Alpine image bumped from 3.23.3 to 3.23.4 to address security concerns
Karmada v1.16.5 is a focused patch release fixing scheduler queue misrouting, Job replica assignment errors, and operator reconciliation failures, plus an Alpine base image bump.
securityAlpine 3.23.4 base image — rebuild and redeploy your Karmada components
The Alpine bump from 3.23.3 to 3.23.4 addresses upstream security concerns. If you're running custom-built Karmada images or have image scanning in your pipeline, update your base images accordingly. The official images in this release already include the updated base.
breakingScheduler queue misrouting causes stalled workloads — patch immediately
The backoffQ vs unschedulableBindings misrouting bug means workloads with insufficient cluster replicas may have been stuck in the wrong queue, causing unexpected scheduling delays or failures. If you've seen bindings that appear perpetually pending despite enough cluster capacity becoming available, this fix directly addresses that. Upgrade to v1.16.5 and check for any bindings that are stuck — they should reschedule automatically after the upgrade.
breakingJob completions were assigned to wrong clusters — review existing Job distributions
The Job completion replica assignment bug could have led to incorrect work distribution across member clusters. Jobs that ran on this version may have had skewed completion counts per cluster. After upgrading, inspect any multi-cluster Jobs that were scheduled on v1.16.4 to verify their completion semantics were not affected in production.
主な変更 (5)
- karmada-operator: Secret creation made idempotent, fixing init reconciliation failures on retry
- karmada-scheduler: Bindings with insufficient cluster replicas now correctly land in unschedulableBindings queue instead of backoffQ
- karmada-scheduler: Schedule success events now include cluster info when using ClusterAffinities
- Job completions now assigned correctly per cluster instead of being misrouted across replicas
- Base image updated from alpine:3.23.3 to alpine:3.23.4 for security fixes
Karmada v1.15.8 is a targeted bug-fix patch addressing scheduler queue misrouting, missing cluster info in events, and an operator reconciliation failure. Alpine base image bumped for security.
securityRebuild and redeploy to pick up alpine:3.23.4 patches
The alpine base image bump from 3.23.3 to 3.23.4 addresses unspecified security concerns. If you're running custom Karmada images built from the upstream base, rebuild against the new base. If you're using official images, pulling v1.15.8 is sufficient.
breakingScheduler queue misrouting can cause workloads to be stuck — patch immediately
The backoffQ bug is particularly dangerous in production: when a binding hits insufficient replicas, it gets retried with exponential backoff instead of being placed in unschedulableBindings where it belongs. This means your workloads silently wait longer than expected before rescheduling. If you're running ClusterAffinities with replica constraints, upgrade to v1.15.8 — don't wait on this one.
enhancementOperator idempotency fix prevents init reconciliation crashes on restarts
The non-idempotent secret creation in karmada-operator meant that if the operator restarted mid-initialization, reconciliation would fail. This is now fixed with an idempotent approach. If you've been seeing operator init failures after pod restarts or rolling updates, this patch resolves the root cause.
主な変更 (4)
- karmada-operator: Fixed non-idempotent secret creation that caused init reconciliation failures on retry
- karmada-scheduler: Schedule success events now include cluster information when using ClusterAffinities
- karmada-scheduler: Bindings with insufficient cluster replicas now correctly land in unschedulableBindings queue instead of backoffQ
- Base image updated from alpine:3.23.3 to alpine:3.23.4 to address security concerns
Karmada v1.17.1 is a targeted patch fixing four bugs: a silent graceful eviction failure, a cert rotation loop, a Helm chart render issue, and an OpenAPI schema error.
breakingSilent eviction failures: patch immediately if you rely on graceful eviction
The race condition in graceful eviction is dangerous precisely because it fails silently. Workloads on tainted or failing member clusters may never get evacuated. If you run high-availability workloads across multiple clusters, upgrade to v1.17.1 now and verify any pending eviction tasks completed after the upgrade.
breakingCertificate rotation was broken for karmada-agent — check your agent certs
The SignerName mismatch meant CSRs submitted for cert rotation were never approved, leaving agents unable to renew certificates. Agents that have been running since v1.17.0 may have stale or near-expiry certs. After upgrading, check agent certificate expiry dates and manually trigger rotation if needed.
enhancementHelm chart users: re-run upgrades after patching to fix TLS config
If you upgraded karmada-chart on v1.17.0 and saw '{{ ca_crt }}' appear literally in your config, the CA cert was never injected. After upgrading to v1.17.1, re-apply your Helm upgrade to ensure the CA certificate is correctly rendered and TLS is not silently misconfigured.
主な変更 (4)
- Race condition fixed in graceful eviction: tasks could be silently dropped when multiple controllers modified the same ResourceBinding/ClusterResourceBinding simultaneously
- Certificate rotation CSRs now auto-approve correctly — a SignerName mismatch between cert_rotation_controller and agent_csr_approving was preventing rotation entirely
- Helm chart upgrade fix: unrendered '{{ ca_crt }}' placeholder no longer breaks TLS config during karmada-chart upgrades
- OpenAPI schema now uses fully qualified model names instead of Go type names, resolving unknown model errors
Karmada v1.16.4 patches three bugs: a broken certificate rotation loop in karmada-agent, a Helm chart rendering failure on upgrades, and a race condition that silently dropped graceful eviction tasks.
securityCertificate rotation was silently broken — rotate certificates after upgrading
The SignerName mismatch meant that agent certificate rotation CSRs were never approved, so agent certificates were quietly expiring without renewal. After upgrading to v1.16.4, check the remaining validity of your karmada-agent certificates across all member clusters and manually trigger rotation if any are close to expiry. Don't assume rotation was working before this fix.
breakingEviction race condition could leave workloads stranded on failing clusters
If you run multiple controller replicas or have high-churn ResourceBindings, the race condition fixed in this release means your workloads may have silently failed to evacuate from tainted or unhealthy clusters. Upgrade to v1.16.4 immediately and verify that any in-flight eviction tasks complete. Check cluster taints and ResourceBinding status after upgrading to confirm nothing was left behind.
breakingHelm chart upgrades with unrendered ca_crt will break TLS — re-run your upgrades
The '{{ ca_crt }}' template variable was not being rendered during Helm upgrades, which would produce invalid TLS configuration silently. If you upgraded via Helm between v1.16.3 and this release, inspect your deployed chart values and secrets to confirm ca_crt is properly populated. Re-apply or re-upgrade using v1.16.4 if you suspect the broken rendering affected your environment.
主な変更 (3)
- karmada-agent: Certificate rotation CSRs were never auto-approved due to SignerName mismatch between cert_rotation_controller and agent_csr_approving — fixed
- karmada-chart: Helm upgrades left '{{ ca_crt }}' unrendered, breaking TLS config — fixed
- karmada-controller-manager: Race condition causing graceful eviction tasks to be silently dropped when multiple controllers modified the same ResourceBinding or ClusterResourceBinding concurrently — fixed
Karmada v1.15.7 patches three bugs: a cert rotation deadlock in the agent, a Helm chart rendering failure on upgrade, and a race condition that silently dropped graceful eviction tasks.
securityAudit agent certificates for expiry before upgrading
The cert rotation bug could have left agents with expired certificates, creating an outage risk or a gap where mTLS wasn't enforced. Before upgrading, check the NotAfter field on agent certs. Post-upgrade, confirm the cert_rotation_controller and agent_csr_approving are now operating with matching SignerNames and that pending CSRs get approved.
breakingCheck if cert rotation has been silently failing in your agents
If your karmada-agent has been running for a while without certificate renewal, the SignerName mismatch bug means CSRs were never approved and certs may be expired or near expiry. After upgrading to v1.15.7, verify agent certificate validity and manually trigger rotation if needed. This is especially urgent in long-running clusters where cert TTLs are short.
breakingSilent eviction failures may have left workloads on bad clusters — audit now
The race condition in graceful eviction means any cluster that was tainted or marked unhealthy while multiple controllers were active may not have had its workloads evacuated. Before upgrading, check ResourceBinding and ClusterResourceBinding objects for stale taints or pending eviction conditions that were never resolved. Post-upgrade, re-trigger eviction for any affected bindings.
主な変更 (3)
- karmada-agent: Certificate rotation CSRs now auto-approve correctly — a SignerName mismatch between cert_rotation_controller and agent_csr_approving was blocking all cert renewals.
- karmada-chart: Helm upgrades no longer leave {{ ca_crt }} as a literal unrendered string, which would break TLS config silently.
- karmada-controller-manager: Race condition fixed where concurrent modifications to the same ResourceBinding or ClusterResourceBinding caused graceful eviction tasks to be silently dropped, leaving workloads stranded on tainted or failing clusters.
Karmada v1.16.3 is a bug-fix release addressing critical issues including controller race conditions, scheduler panics, and graceful eviction problems that could impact multi-cluster operations stability.
breakingUpgrade immediately to prevent scheduler crashes
The scheduler panic fix for divide-by-zero errors is critical for production stability. Schedule maintenance window to upgrade karmada-scheduler component, especially if you use spread constraints across clusters with dynamic availability.
enhancementReview CronFederatedHPA configurations after upgrade
The fix for CronFederatedHPA scale-up failures when replicas field is missing improves reliability. After upgrading, verify your CronFederatedHPA resources are scaling correctly from zero replicas, and consider adding explicit replicas fields to avoid edge cases.
enhancementMonitor graceful eviction behaviors post-upgrade
The graceful eviction timing fix prevents GracePeriodSeconds leakage between tasks. After upgrade, monitor your workload evictions to ensure they complete within expected timeframes and adjust grace periods if needed for improved resource cleanup.
主な変更 (5)
- Fixed job status aggregator race condition causing error loops in controller manager
- Resolved CronFederatedHPA scaling failures when replicas field is missing
- Corrected graceful eviction task timing issues preventing proper resource cleanup
- Fixed scheduler divide-by-zero panic when no valid clusters available for spread constraints
- Improved backoff queue sorting to prevent priority inversion in scheduler operations
Karmada v1.15.6 is a critical bug-fix release addressing race conditions, scheduler panics, and CronFederatedHPA failures that could impact production stability.
securityBase image security update included
Alpine base image updated to 3.23.3 which may include security patches. Review your container scanning results after upgrade and update any custom images based on Karmada components.
breakingCritical scheduler stability fixes require immediate update
The scheduler panic fix and backoff queue corrections address critical stability issues that could cause service disruptions. Plan immediate upgrade for production environments running multi-cluster workloads with spread constraints or high scheduling volumes.
enhancementCronFederatedHPA reliability improvements
If using CronFederatedHPA for autoscaling, this release fixes scale-up failures when replicas field is missing. Update to ensure consistent autoscaling behavior across your federated deployments.
主な変更 (5)
- Fixed job status aggregator race condition preventing error loops
- Resolved CronFederatedHPA scale-up failures when replicas field is missing
- Corrected graceful eviction task timing issues with GracePeriodSeconds
- Fixed scheduler panic from divide-by-zero errors in spread constraints
- Improved backoff queue sorting to prevent priority inversion issues
Karmada v1.14.10 is a patch release focused on stability improvements, fixing critical race conditions in job status handling, scheduler panics, and graceful eviction timing issues.
enhancementUpgrade to improve operational stability
This patch fixes several critical race conditions and panics that could affect workload scheduling and job management. Plan an upgrade during maintenance window to prevent potential scheduler crashes and job status inconsistencies in production clusters.
enhancementReview CronFederatedHPA configurations
The fix for CronFederatedHPA scale-up from zero when replicas field is missing means previously broken auto-scaling configurations will now work correctly. Audit your CronFederatedHPA resources to ensure they behave as expected after upgrade.
主な変更 (6)
- Fixed race condition in job status aggregator that could cause error loops
- Resolved CronFederatedHPA scale-up failures when replicas field is missing
- Fixed GracePeriodSeconds value leakage between eviction tasks
- Corrected scheduler panic from divide-by-zero in spread constraint calculations
- Fixed backoff queue sorting causing priority inversion in scheduler
- Resolved resource quota plugin namespace issue in scheduler estimator
Karmada v1.17.0 is a community-driven release with contributions from 30+ developers, focusing on stability improvements and feature enhancements for multi-cluster management.
enhancementPlan upgrade testing in staging environment
With 30+ contributors and significant community activity, this release likely contains substantial improvements. Test the upgrade path in your staging clusters first, paying attention to controller behavior and resource propagation patterns before rolling out to production.
enhancementReview cluster scheduling policies
Enhanced scheduling capabilities may offer new optimization opportunities for your workload distribution. Review your current PropagationPolicies and OverridePolicies to leverage potential new features for better resource utilization across clusters.
主な変更 (5)
- Enhanced multi-cluster resource propagation and scheduling capabilities
- Improved cluster federation management and workload distribution
- Strengthened API compatibility and controller reliability
- Extended observability and monitoring features for federated clusters
- Performance optimizations for large-scale cluster deployments