Karmada
v1.18.3Orchestration & ManagementACTION 4CHECK 1PLAN 3OTHER 108
기능 확장과 동작 정확성 개선, 의존성 유지보수가 함께 이뤄진 릴리스입니다. 보안 관련 의존성 업데이트와 사용 중단 및 제거된 설정 경로가 포함되어 있어 업그레이드 호환성과 설정 변경을 함께 검토해야 합니다.
조치 필요 (4)
securitymedium
github.의존성 업데이트com/vektra/mockery 보안 문제에 대응하기 위해
github.를com/vektra/mockery v3.로 업데이트했습니다. 관련 보안 권고 식별자는5. 5 GO-2025-3900입니다.security
alpine기본 이미지 업데이트보안 문제에 대응하기 위해 기본 이미지
alpine을alpine:3.에서23. 4 alpine:3.로 변경했습니다.24. 1 security
alpine기본 이미지 버전 변경보안 문제에 대응하기 위해 기본 이미지
alpine을alpine:3.에서23. 3 alpine:3.로 변경했습니다.23. 4 breaking멤버 클러스터 클라이언트 기본 제한 시간 추가
멤버 클러스터가 응답하지 않을 때 컨트롤러가 멈추지 않도록 멤버 클러스터 클라이언트에 기본 제한 시간
32s를 추가했습니다.
영향 확인 (1)
breaking
karmada-operator외부 etcd 필드 제거CAData,CertData,KeyData를 사용하는 경우에 적용됩니다.karmada-operator에서 외부 etcd 설정에 사용하던CAData,CertData,KeyData필드를 제거했습니다.
미리 준비 (3)
breaking외부 etcd 필드 제거
CAData,CertData,KeyData를 사용하는 경우에 적용됩니다.외부 etcd 설정에서 사용하던
CAData,CertData,KeyData필드를 제거했습니다.deprecated
--etcd-init-image플래그 사용 중단제거 시점 미정--etcd-init-image를 사용하는 경우에 적용됩니다.init명령의--etcd-init-image플래그는 더 이상 사용되지 않아 사용 중단으로 지정됐으며, 향후 릴리스에서 제거됩니다.breaking멤버 클러스터 메트릭 레이블 이름 변경제거 시점 미정
메트릭에
cluster또는cluster_name을 사용하는 경우에 적용됩니다.멤버 클러스터를 나타내는 Prometheus 메트릭 레이블
cluster와cluster_name을 사용 중단하고, 새 레이블 이름member_cluster를 사용합니다.cluster와cluster_name은1.릴리스에서 제거됩니다.18
그 외 기록된 변경 108건 전체fixes 42 · additions 37 · value changes 19 · constraints 8 · defaults 2
fixes (42)
- -
karmada-controller-manager: Fixed an issue where the taint-manager eviction queue would enqueue bindings with indefinite taint tolerations. ([#7775](https://github.com/karmada-io/karmada/pull/7775), @karmada-bot) - -
karmada-controller-manager: Fixed an issue whereCluster.could remain stale after an associatedstatus. remedyActions Remedyresource was removed. ([#7790](https://github.com/karmada-io/karmada/pull/7790), @karmada-bot) - -
helm chart: Fixed TLS certificate SAN mismatch when deploying to a custom namespace by adding systemNamespace SANs to certs.auto.hosts. ([#7686](https://github.com/karmada-io/karmada/pull/7686), @karmada-bot) - #### Bug Fixes -
karmada-search: Fixed the issue that watch connect cannot reflect resources from recovered clusters immediately. ([#7522](https://github.com/karmada-io/karmada/pull/7522), @Ady0333) - -
karmada-scheduler: Fixed the issue when cluster resources are insufficient, multiple template resources can still be scheduled. ([#7580](https://github.com/karmada-io/karmada/pull/7580), @jabellard) - -
karmada-controller-manager: Fixed the issue that a transientClusterClientSetFuncfailure (e.g. missingSecretRefduring credential rotation) would immediately set the clusterReady=Falsewithout respectingClusterFailureThreshold, potentially triggering unnecessary workload failover. ([#7578](https://github.com/karmada-io/karmada/pull/7578), @driegel1) - -
karmada-operator: Fixed init reconciliation failure by replacing non-idempotent secret creation with an idempotent approach. ([#7405](https://github.com/karmada-io/karmada/pull/7405), @anr) - -
karmada-scheduler: Fixed an issue where the schedule success event was missing cluster information when scheduling withClusterAffinities. ([#7419](https://github.com/karmada-io/karmada/pull/7419), @cotishq) - -
karmada-scheduler: Fixed incorrect error type propagation that caused bindings with insufficient cluster replicas to be misrouted tobackoffQinstead ofunschedulableBindings. ([#7354](https://github.com/karmada-io/karmada/pull/7354), @SujoyDutta) - - Fixed the issue that
Jobcompletions were assigned to the wrong replicas for each cluster. ([#7401](https://github.com/karmada-io/karmada/pull/7401), @Ady0333) - -
karmada-agent: Fixed the issue where certificate rotation CSRs were never auto-approved due to a SignerName mismatch betweencert_rotation_controllerandagent_csr_approving. ([#7310](https://github.com/karmada-io/karmada/pull/7310), @Denyme24) - -
karmada-chart: Fixed unrendered{{ ca_crt }}during upgrades. ([#7330](https://github.com/karmada-io/karmada/pull/7330), @AbhinavPInamdar) - -
karmada-controller-manager: Fixed a race condition where graceful eviction tasks could be silently dropped when multiple controllers concurrently modify the same ResourceBinding or ClusterResourceBinding, preventing workloads from being evacuated from tainted or failing clusters. ([#7307](https://github.com/karmada-io/karmada/pull/7307), @Ady0333) - -
karmada-controller-manager: Fixed the issue where the job status aggregator could enter an error loop due to a race condition when setting the initialstartTime. ([#7158](https://github.com/karmada-io/karmada/pull/7158), @rohan-019) - -
karmada-controller-manager: Fixed CronFederatedHPA scale-up from zero failure when the replicas field is missing. ([#7212](https://github.com/karmada-io/karmada/pull/7212), @zhengjr9) - -
karmada-controller-manager: Fixed an issue where a per-taskGracePeriodSecondsvalue could leak to subsequent graceful eviction tasks, causing premature or delayed evictions. ([#7187](https://github.com/karmada-io/karmada/pull/7187), @Ady0333) - -
karmada-controller-manager: Fixed an issue where dependency updates could overwrite other controller annotations during retry conflicts. ([#7216](https://github.com/karmada-io/karmada/pull/7216), @Ady0333) - -
karmada-scheduler: Fixed a scheduler panic caused by a divide-by-zero error when calculating spread constraints with no valid clusters. ([#7234](https://github.com/karmada-io/karmada/pull/7234), @XiShanYongYe-Chang) - -
karmada-scheduler: Fixed the bug in the backoff queue where the sorting function was incorrect, potentially causing high-priority items with long backoffs to block lower-priority items. ([#7231](https://github.com/karmada-io/karmada/pull/7231), @zhzhuang-zju) - -
karmada-scheduler-estimator: Fixed the issue where the resource quota plugin failed to list resource quotas due to a missing namespace in the gRPC request. ([#7238](https://github.com/karmada-io/karmada/pull/7238), @zhzhuang-zju) - #### Bug Fixes -
karmada-controller-manager: Fixed an issue where policy deletion could be blocked if a resource selector targeted a non-existent resource. ([#7083](https://github.com/karmada-io/karmada/pull/7083), @FAUST-BENCHOU) - -
karmada-scheduler: Fixed bug preventing multi-component workloads from being rescheduled during cluster failover events. ([#7130](https://github.com/karmada-io/karmada/pull/7130), @mszacillo) - -
karmada-webhook: Fixed an issue where thecondition.was not set toreason QuotaExceededwhen FederatedResourceQuota is exceeded. ([#7098](https://github.com/karmada-io/karmada/pull/7098), @kajal-jotwani) - -
karmadactl: Fixed the messy auto-completion suggestions for commands like 'get' and 'apply'. ([#7025](https://github.com/karmada-io/karmada/pull/7025), @zhzhuang-zju) - -
karmada-controller-manager: Fixed the issue where PP/CPP cannot be deleted because the resources API selected by the PP/CPP do not exist on the control plane. ([#7028](https://github.com/karmada-io/karmada/pull/7028), @XiShanYongYe-Chang) - -
karmada-controller-manager: Fixed the issue thatHelmReleasedid not defineobservedGenerationvariable in thestatusAggregationoperation. ([#7059](https://github.com/karmada-io/karmada/pull/7059), @FAUST-BENCHOU) - Fixed the issue that
rbSpec.is not updated when the template is updated.Components - Fixed the Job status cannot be aggregated issue due to the missing
JobSuccessCriteriaMetcondition when using kube-apiserver v1.32+ as Karmada API server. - Fixed the issue that attached resource changes were not synchronized to the cluster in the dependencies distributor.
- Fixed a panic when querying node metrics by name caused by using the wrong GroupVersionResource (PodsGVR instead of NodesGVR) when creating a lister.
- Fixed the issue that CRDs cannot be updated during upgrades of the Karmada instance.
- Fixed the issue where increasing the total number of replicas can cause some clusters to receive fewer replicas under the StaticWeight strategy by introducing the Webster algorithm.
- Fixed the issue that resourcebinding validating webhook may panic when ReplicaRequirements of a Component in rbSpec.Components is nil.
- Fixed the issue that the
registercommand still uses the cluster-info endpoint when registering a pull-mode cluster, even if the user provides the API server endpoint. - Fixed the issue that when an object API field name contains dots or colons, it would cause the resource interpreter to fail.
karmadactl: Fixed the issue that theregistercommand still uses the cluster-info endpoint when registering a pull-mode cluster, even if the user provides the API server endpoint.karmada-metrics-adapter: Fixed a panic when querying node metrics by name caused by using the wrong GroupVersionResource (PodsGVR instead of NodesGVR) when creating a lister.karmada-scheduler: Fixed the issue where increasing the total number of replicas can cause some clusters to receive fewer replicas under the StaticWeight strategy by introducing the Webster algorithm.karmada-controller-manager: Fixed the issue thatrbSpec.is not updated when the template is updated.Components ResourceInterpreter: Fixed the issue that when an object API field name contains dots or colons, it would cause the resource interpreter to fail.karmada-webhook: Fixed the issue that resourcebinding validating webhook may panic when ReplicaRequirements of a Component in rbSpec.Components is nil.karmada-operator: Fixed the issue that CRDs can not be updated during upgrades of the Karmada instance.
additions (37)
- In Karmada v1.16.0, we introduce **multi-component scheduling**, a new capability that enables the **complete and unified placement of multi-component workloads**—those composed of multiple interrelated components (e.g., jobManager and taskManagers of FlinkDeployment)—**into a single member cluster with sufficient resources**.
- - Estimate how many full sets of components can fit into a member cluster based on **ResourceQuota** limits.
- This release introduces an eviction queue with rate limiting capabilities for the Karmada taint manager. The eviction queue enhances the failover mechanism by controlling the rate of resource evictions using a configurable fixed rate parameter. The implementation also provides metrics for monitoring the eviction process, improving overall system observability.
- - **Configurable Fixed Rate Limiting**: Configure the eviction rate per second through the
--eviction-ratecommand-line flag. - - **Comprehensive Metrics Support**: Provides metrics for queue depth, resource kind, processing latency, and success/failure rates for monitoring and troubleshooting. By introducing the rate limiting mechanism, administrators can better control resource scheduling efficiency during cluster failover, balancing service stability with scheduling flexibility and efficiency.
- Introduced
Componentsfield toResourceInterpreterContextin theResourceInterpreterResponseto support interpreting components for webhook interpreter. - Introduced a
PodDisruptionBudgetfield to theCommonSettingsofKarmadaAPI for supporting PodDisruptionBudgets (PDBs) for Karmada control plane components. - The
initcommand now supports customizing Karmada component command line flags. - The
initcommand now supports customizing Karmada component command line flags via the configuration file. - Introduced built-in interpreter for Volcano
Job. - Introduced a built-in interpreter for Kubeflow Notebooks.
- Introduced built-in interpreter for SparkApplication.
- Introduced built-in interpreter for PyTorchJob.
- Introduced built-in resource interpreter for Kubernetes ReplicaSet workloads.
- Introduced built-in interpreter for MPIJob.
- Introduced
--resource-eviction-rateflag to specify the eviction rate during cluster failover. - Implemented
MaxAvailableComponentSetsinterface for general estimator based on resource summary. - Enabled the capability for multiple component estimation in the scheduler. The feature is gated behind MultiplePodTemplatesScheduling.
- Implemented
getMaximumSetsBasedOnResourceModelsin general estimator. - Introduced MaxAvailableComponentSetsRequest & MaxAvailableComponentSetsResponse for component scheduling.
- Added plugins in estimator for component scheduling.
- Added
ResourceQuotaplugin for multi-component scheduling. - Implemented maxAvailableComponentSets for the accurate estimator.
- Implemented the noderesource plugin for multi-component scheduling estimation.
- Enabled federated resource quota calculation for multi-component scheduling.
- Enabled
GetComponentsinterpreter operation through Webhook Interpreter. - Added maxAvailableComponentSets to estimator interface.
- Added PodDisruptionBudget (PDB) support to enable high availability guarantees for all control plane components during planned disruptions.
- Added a new Warning event
DependencyPolicyConflictto surface when dependency policies have conflicts. - Added new metrics for the failover eviction queue to enhance observability.
karmadactl: Theinitcommand now supports customizing Karmada component command line flags.karmada-scheduler-estimator: Introduce MaxAvailableComponentSetsRequest & MaxAvailableComponentSetsResponse for component scheduling.karmada-scheduler: ImplementedMaxAvailableComponentSetsinterface for general estimator based on resource summary.ResourceInterpreter: EnableGetComponentsinterpreter operation through Webhook Interpreter.ResourceInterpreter: Adding maxAvailableComponentSets to estimator interface.Helm chart: Added helm index for 1.15 release.- Multi-component scheduling is **disabled by default**.
value changes (19)
- - The base image
alpinehas now been promoted fromalpine:3.to23. 2 alpine:3.. ([#7162](https://github.com/karmada-io/karmada/pull/7162), @dependabot)23. 3 - - The base image
alpinehas been promoted fromalpine:3.to22. 2 alpine:3.. ([#7004](https://github.com/karmada-io/karmada/pull/7004), @dependabot)23. 0 - - The base image
alpinehas been promoted fromalpine:3.to23. 0 alpine:3.. ([#7037](https://github.com/karmada-io/karmada/pull/7037), @dependabot)23. 2 - - **Monotonic replica assignment**: Increasing the total replica count will never cause any cluster to lose replicas, ensuring consistent and intuitive behavior.
- - **Fair handling of remainder replicas**: When distributing replicas among clusters with equal weights, priority is given to the cluster with fewer current replicas. This "smaller-first" approach promotes balanced deployment and better satisfies high availability (HA) requirements. This update enhances the stability, fairness, and predictability of workload distribution across clusters, making replica scheduling more robust in multi-cluster environments.
- For controllers not built on controller-runtime, such as the detector controller, we extend this capability in release-1.16 by enabling the priority-queue functionality for all controllers that use async workers. **Now, we can achieve downtime reduction for all controllers after a restart or leader transition.**
- Computed effective field values for attached ResourceBindings when referenced by multiple ResourceBindings.
- Migrated dynamic weight assignment to use the Webster algorithm.
- Refactored the replica estimation logic by moving the node resource-based calculation into a dedicated, default plugin.
- Karmada is now built with Golang ….×2
- Kubernetes dependencies have been updated to v1.34.1.
- Updated
sigs.fromk8s. io/controller-runtime v0.to21. 0 v0..22. 4 - The base image
alpinehas been promoted from 3.22.1 to 3.22.2. - After enabling the
ControllerPriorityQueuefeature gate, the asyncWorker uses a priority queue based implementation, which affects the processing order of items in the resource detector and causes the monitoring metricworkqueue_depthto be split into multiple series. - The
ControllerPriorityQueuefeature gate now applies to all controllers, including async workers. Enable it with--feature-gates=ControllerPriorityQueue=true. - The base image
alpinenow has been promoted from 3.22.1 to 3.22.2. - Introduced
Componentsfield toResourceInterpreterContextin theResourceInterpreterResponseto support interpreting components for webhook interpreter. - we introduce the
Webster method
constraints (8)
- - Predict schedulability using **actual node resource availability** across clusters. Karmada v1.16.0 ships with built-in **resource interpreters** for the following multi-component workload types:
- -
FlinkDeployment(flink.)apache. org/v1beta1/FlinkDeployment - -
SparkApplication(sparkoperator.)k8s. io/v1beta2/SparkApplication - -
Volcano Job(batch.)volcano. sh/v1alpha1/Job - -
MPIJob(kubeflow.)org/v2beta1/MPIJob - -
RayCluster(ray.)io/v1/RayCluster - -
RayJob(ray.)io/v1/RayJob - -
TFJob(kubeflow.)org/v1/TFJob
defaults (2)
- The
initcommand's defaultkube-apiserverandkube-controller-managerimages have been updated from v1.31.3 to v1.34.1. And the defaultetcdimage has been updated from 3.5.16-0 to 3.6.0-0. - The default
kube-apiserverandkube-controller-managerimages have been updated from v1.31.3 to v1.34.1. The defaultetcdImage has been updated from 3.5.16-0 to 3.6.0-0.
조치가 필요한 릴리스가 나왔을 때 주간 메일로 알려드립니다. 이번 릴리스의 보안 패치와 브레이킹 체인지 같은 것들입니다.