Releases

Prometheus v3.13.0 is an LTS release leading with a HIGH-severity credential-forwarding fix (CVE-2025-4673, CVE-2023-45289) and a MEDIUM XSS fix (CVE-2026-44990), alongside four breaking changes to pagination tokens, path resolution, PromQL duration function names, and license file packaging, plus a broad set of new features and performance improvements.

• securityHIGH: Credentials no longer forwarded on cross-host redirects

  Credentials (Authorization header, basic auth, bearer token, OAuth2, and configured headers) are no longer forwarded when following a redirect to a different host. This affects scraping, remote read/write, alerting, and service discovery where credentials are configured and cross-host redirects occur. Tracked as CVE-2025-4673 and CVE-2023-45289; driven by upgrading prometheus/common from v0.68.x to v0.69.0. Review any targets or endpoints that rely on redirects, as credentials will now be dropped silently on host change.

• securityMEDIUM: XSS fix in UI dependency (CVE-2026-44990)

  sanitize-html was bumped to fix a cross-site scripting vulnerability (CVE-2026-44990) in the Prometheus UI. No configuration change needed; upgrade to v3.13.0.

• breakingPagination token algorithm changed from SHA-1 to SHA-256

  Rule group pagination tokens now use SHA-256 instead of SHA-1. Any client or tooling that stores or compares pagination tokens across versions will see different token values after upgrade.

• breaking--http.config.file relative path resolution changed

  Relative file paths inside the file passed to --http.config.file are now resolved relative to that config file's own directory, not its parent directory. If you use relative paths in that config, verify they still resolve correctly after upgrading.

• breakingDuration-expression functions min()/max() renamed to min_of()/max_of()

  If you use the experimental-duration-expr feature flag, the PromQL duration functions min() and max() have been renamed to min_of() and max_of(). Update any queries or rules that use the old names.

• breakingnpm_licenses.tar.bz2 removed; licenses now at /assets/third-party-licenses.txt

  npm_licenses.tar.bz2 is no longer shipped in release tarballs or container images. Third-party npm license text is now embedded in the binary and served at /assets/third-party-licenses.txt. Any automation that extracted that archive needs updating.

Fluentd v1.19.3 is a security and hardening patch that tightens input validation in out_http, buffer/in_http, and output tag handling, and locks down default access for in_monitor_agent and in_debug_agent. The rest of the release is routine bug fixes and small dependency/CI cleanup.

• securityStricter input validation across out_http, buffer/in_http, and output tag paths

  Applies unconditionally in v1.19.3. out_http now strictly validates hostnames for dynamic endpoints, buffer and in_http enforce size limits on decompressed payloads, and output enforces strict path boundary checks on tag values. Together these close off request-smuggling and decompression-bomb style abuse paths; review any configs that build hosts, tags, or payload sizes from untrusted input.

• breakingin_monitor_agent and in_debug_agent defaults now expose less by default

  Applies unconditionally in v1.19.3. in_monitor_agent changes default visibility of config, retry, and debug info, and in_debug_agent now accepts connections only from the local machine by default. If you relied on remote access to either agent or exposed their full output externally, update firewall rules or explicit config options to restore the access you need.

v0.155.0 is a breaking-change release for operators: seven stabilized feature gates are removed, memory_limiter metrics get a new prefix, and two config/service APIs are deprecated in favor of snapshot-based replacements. The rest is tooling work on mdatagen and the newly relocated schemagen CLI.

• breakingSeven stabilized feature gates removed

  Applies to any collector build referencing confighttp.framedSnappy, configoptional.AddEnabledField, confmap.newExpandedValueSanitizer, exporter.PersistRequestContext, otelcol.printInitialConfig, telemetry.UseLocalHostAsDefaultMetricsAddress, or pdata.enableRefCounting via --feature-gates flags or config. These gates are gone in v0.155.0; startup will fail if they're still referenced. Drop them from your CLI args and config before upgrading.

• breakingmemory_limiter metrics renamed with processor prefix

  Applies to anyone scraping or alerting on otelcol_processor_* metrics from the memory_limiter processor. They're renamed to otelcol_processor_memory_limiter_* to disambiguate from other processors. Update dashboards, alert rules, and metric queries to the new names.

• breakingmdatagen reaggregation_enabled setting removed

  Applies to custom components using mdatagen metadata.yaml with reaggregation_enabled. The setting is removed; per-metric reaggregation config is now always generated. Old files with the field are still accepted but the value is ignored, so check generated output matches expectations.

• breakingConfig watcher APIs deprecated for snapshot-based equivalents

  Applies to extension or core code using service.Settings.CollectorConf or extensioncapabilities.ConfigWatcher. Both are deprecated in favor of service.Settings.ConfigSnapshot and extensioncapabilities.ConfigSnapshotWatcher. No forced migration yet, but plan to move to the snapshot-based APIs.

Chaos Mesh v2.8.3 is a security-focused patch: critical/high CVEs addressed via Go 1.25.11 and containerd 1.7.32 upgrades, plus a NetworkChaos recovery bug fix.

• securityUpgrade to v2.8.3 to patch critical/high CVEs

  The Go toolchain and containerd in the container images had critical/high CVEs. If you're running Chaos Mesh on release-2.8, upgrade to v2.8.3 now. No config changes needed — this is a drop-in image update.

• breakingVerify NetworkChaos experiments involving crash-looping pods

  NetworkChaos recovery previously failed when the target container was in CrashLoopBackOff. v2.8.3 fixes this by falling back to the pause container's PID. If your test environments include intentionally crash-looping workloads, re-run those experiments to confirm recovery behaves correctly after the upgrade.

OTel Collector v0.154.0 fixes a startup crash in exporterhelper and changes --skip-get-modules behavior. Two minor enhancements add TLS and CORS config options.

• securityinclude_insecure_cipher_suites is opt-in — keep it that way

  The new configtls option lets you re-enable weak cipher suites, but they stay disabled by default. Do not set include_insecure_cipher_suites: true in production unless you're forced to by a legacy peer — and if you are, treat it as a temporary workaround and track it.

• breakingCheck custom build pipelines using --skip-get-modules

  If your OCB-based build scripts pass --skip-get-modules expecting go.mod to be regenerated as a side effect, that no longer happens. Audit any CI pipelines that relied on this implicit behavior and add an explicit go mod tidy step if needed.

• enhancementUpgrade if you use sending_queue with sizer enabled

  The nil-pointer panic fix in exporterhelper is critical for anyone who sets sending_queue.sizer alongside batch.enabled: false. If your collector was crashing on startup in this config, v0.154.0 resolves it — upgrade directly.

v2.19.0 ships the new /api/v3/trace-summaries endpoint and migrates the UI search to use it, plus TLS for ClickHouse and a configurable gRPC max message size.

• breakingUI search migration to /api/v3/trace-summaries requires compatible backend

  The UI now exclusively uses /api/v3/trace-summaries for search results. If you run a custom or older storage backend via the gRPC storage plugin, it must implement the SummaryReader interface or the fallback full-trace aggregation path kicks in. Verify your storage plugin supports this before upgrading. If you pin the UI separately from the backend, do not upgrade the UI to this version without also upgrading the backend to v2.19.0.

• breakingRename query.num_traces to query.search_depth in any API v3 clients

  Any scripts, dashboards, or integrations that pass query.num_traces to the v3 API will still work via the deprecated alias, but you should migrate to query.search_depth now. The deprecated alias will eventually be removed. Also check for snake_case HTTP query params — camelCase is now the canonical form, with snake_case kept as deprecated aliases.

• enhancementSet max_recv_msg_size_mib if you see gRPC message size errors in remote storage

  Teams using the gRPC remote storage plugin with large traces often hit the default gRPC message size cap. The new max_recv_msg_size_mib config option on the gRPC storage client lets you raise this limit explicitly. If you've been seeing 'received message larger than max' errors from your storage backend, configure this value in your Jaeger deployment config to fix it without workarounds.

• enhancementEnable TLS for ClickHouse storage in production

  The ClickHouse storage plugin now supports TLS configuration. If you're running ClickHouse as a Jaeger backend in any environment where the connection crosses a network boundary, configure TLS now rather than relying on network-level controls alone.

v1.120.3 patches two Go CVEs and ships a wide range of bug fixes across AWS, Azure, Oracle, and DigitalOcean providers, plus OVH support and cosign image signing.

• securityPatch vulnerable Go dependencies — upgrade immediately

  A Go dependency update patches GHSA-xmrv-pmrh-hhx2 and CVE-2026-34986. If you're running any v1.120.x release prior to v1.120.3, upgrade now — these are dependency-level vulnerabilities, not just code changes.

• breakingMCP server is now opt-in — check your config before upgrading

  The MCP server now defaults to disabled (MCP_SERVER_ENABLED=false). If you were relying on it being on by default, set the env var explicitly after upgrading. Check your deployment manifests before rolling out.

• enhancementEnable cosign image verification in your admission pipeline

  Container images are now signed with cosign keyless signing and include SLSA provenance attestations. If your admission policy requires image signature verification, you can now enforce it against OpenCost images. Update your policy tooling (e.g., Kyverno, Cosign verify) to validate these attestations in CI or at deploy time.

Prometheus 3.12.0 ships two security patches (Remote Write DoS, STACKIT SD secret leak), fixes a WAL race in Agent mode, and cuts TSDB range query CPU with a quadratic-to-constant head chunk lookup fix.

• securityPatch two security fixes now — especially if using STACKIT SD

  Two CVEs patched in this release. First: Remote Write now rejects snappy-compressed payloads where the declared decoded size exceeds 32 MB — this closes a DoS vector against your remote-write receiver endpoint. Second: STACKIT SD was leaking secrets in plaintext via the `/-/config` endpoint (GHSA-39j6-789q-qxvh). If you use STACKIT SD, rotate any credentials that may have been exposed before upgrading.

• breakingAgent mode WAL race and remote_write panic bug fixed — upgrade Agent deployments

  A race condition in the agent appender could produce duplicate in-memory series and duplicate WAL records when concurrent appends target the same label set. If you run Prometheus in Agent mode under high write concurrency, this bug could silently corrupt your WAL. Upgrade to 3.12.0 to fix it. Also, `remote_write` queue_config fields are now validated at load time — misconfigurations that previously caused silent runtime panics will now fail at startup, which is the right behavior but means you should test configs before rolling out.

• enhancementTSDB range query CPU cut and auto-reload-config is now stable

  TSDB head chunk lookup in range queries drops from quadratic to constant time, and mmap operations now skip series that don't need work. At production scale with large head chunks, this can meaningfully cut CPU. No config changes needed — just upgrade. Separately, `auto-reload-config` is now stable (no longer experimental), so you can drop any caveats around it in runbooks.

v0.153.0 stabilizes seven feature gates (breaking), fixes a critical memory corruption bug in gRPC Snappy compression, and ships several mdatagen enhancements for component authors.

• securityUpgrade immediately if you use gRPC with Snappy compression

  The Snappy fix in configgrpc addresses memory corruption that can cause fatal errors — this is a process-crash-level issue, not just a performance concern. If your exporters or receivers use gRPC with Snappy compression enabled, treat this as a priority upgrade. Check your exporter configs for compression: snappy settings.

• breakingAudit your feature gate overrides before upgrading

  All seven stabilized gates are now permanent behavior — you can no longer toggle them. If your collector config or startup flags reference configoptional.AddEnabledField, confmap.newExpandedValueSanitizer, exporter.PersistRequestContext, otelcol.printInitialConfig, pdata.useCustomProtoEncoding, telemetry.UseLocalHostAsDefaultMetricsAddress, or pdata.enableRefCounting, remove those references or the collector will fail to start. The metrics address change (UseLocalHostAsDefaultMetricsAddress) is the one most likely to surprise teams — your metrics endpoint is now localhost-bound by default.

• breakingmdatagen reaggregation config is now on by default

  If you maintain custom collector components using mdatagen, the reaggregation config fields are now generated by default. To preserve the old behavior (metrics config with only the enabled field), explicitly set reaggregation_enabled: false in your metadata.yaml. Run mdatagen on your components after upgrading and review the generated output before committing.

v0.152.1 is a bug-fix-heavy release with one useful new metric. The most operationally impactful changes are snappy decompression security fixes and a Prometheus metric naming regression fix.

• securityconfighttp snappy fixes limit memory exposure from compressed payloads

  Three snappy decompression fixes land in `pkg/confighttp`: body is now closed after reading, panics in decompression libraries return HTTP 400 instead of crashing with 500, and `max_request_body_size` is enforced before the decoded buffer is allocated. The size-check fix in particular prevents a potential memory spike from a maliciously crafted compressed payload. If you accept compressed OTLP over HTTP from untrusted sources, upgrade.

• breakingPrometheus metric name format may change if you customized telemetry host

  If you explicitly set the `host` field in the telemetry metrics section of your collector config, check whether your Prometheus metric names changed after upgrading to recent versions. The bug caused `WithoutScopeInfo`, `WithoutUnits`, and `WithoutTypeSuffix` to default to false instead of true in that code path, which means your metrics may have had unexpected suffixes or scope labels. This release restores the correct defaults — metric names may shift again on upgrade, so update dashboards and alerts accordingly.

• enhancementAdd in-flight exporter metric to your dashboards

  The new `otelcol_exporter_in_flight_requests` UpDownCounter metric is available in `pkg/exporterhelper`. Add it to your dashboards to see when exporters are queuing up requests or saturating worker pools — it's a direct signal of export backpressure that was previously hard to observe without custom instrumentation.

Litmus 3.29.0 patches a gRPC CVE, fixes duplicate chaos triggers under concurrent reconciles, and adds Prometheus metrics support for experiment observability.

• securityPatch CVE-2026-33186 by upgrading to 3.29.0 now

  The gRPC library was upgraded to v1.79.3 to fix CVE-2026-33186. If you're running any Litmus version prior to 3.29.0, your control plane is exposed. Upgrade immediately — this isn't a 'schedule it next sprint' situation.

• breakingVerify event-tracker behavior after the duplicate-trigger fix

  The fix for duplicate chaos experiment triggers under concurrent reconciles changes how the event-tracker handles race conditions. If you rely on the event-tracker for automated chaos injection, test your pipelines post-upgrade to confirm expected trigger counts. Duplicate runs may have been masking gaps in your experiment coverage.

• enhancementWire up Prometheus metrics to your existing dashboards

  Prometheus metrics are now natively exposed by ChaosCenter. The release includes a getting-started guide and unit tests, so integration is straightforward. Add Litmus as a scrape target and start tracking experiment pass/fail rates, run durations, and infra connectivity — this fills a long-standing observability gap for chaos workflows.

Jaeger v2.18.0 brings two breaking changes (OTEL metrics overhaul and removed min-step API), useful header-forwarding for ES/OpenSearch, and a massive UI refactor migrating state management from Redux to Zustand.

• breakingAudit your metrics dashboards before upgrading

  The OTEL collector package upgrade changed metric names and shapes. Before upgrading, compare your current Jaeger metrics against the new ones by running both versions in parallel or checking the PR diff. Any Grafana dashboards, Prometheus alerts, or monitoring rules built on Jaeger's internal metrics need to be reviewed and updated. The min_step removal is a smaller blast radius — only affects custom tooling that calls the metricstore API directly — but still requires a code or config change before upgrading.

• enhancementUse header forwarding if you authenticate ES/OpenSearch with custom headers

  If your Elasticsearch or OpenSearch cluster requires auth headers beyond basic username/password (e.g., custom JWT, IAM proxy headers), configure the new header-forwarding feature. This also helps in multi-tenant setups where storage routing depends on request headers. Configure via the storage backend settings in jaeger-query or the gRPC storage plugin config.

• enhancementEvaluate ClickHouse as a storage backend if you want SPM without Prometheus

  ClickHouse SPM support is experimental but now covers the full metrics trifecta: call rates, error rates, and latencies. TTL support is also added. If you're running ClickHouse already or want a single-store solution for both traces and metrics, this is the release to start testing against. Don't use it in production yet, but set up a staging environment to track maturity.

v0.152.0 fixes a silent Prometheus metric name regression when telemetry host is customized, changes AsString HTML-escaping behavior for map/slice values, and adds an in-flight exporter requests metric.

• breakingCheck Prometheus metric names if you customized telemetry host

  The Prometheus exporter boolean fields (WithoutScopeInfo, WithoutUnits, WithoutTypeSuffix) now default correctly when you explicitly set the telemetry host. Before this fix, those fields defaulted to false instead of true, so your metric names would silently change format. If you run a Collector with a custom telemetry host configured, verify your Prometheus metric names after upgrading — dashboards and alerts keyed on metric name format may need updating.

• breakingAsString output change for map/slice values — check downstream consumers

  pcommon.Value.AsString no longer HTML-escapes <, >, and & in map and slice values. If your pipeline or downstream consumers were relying on escaped output (e.g., &lt; or &gt;) from attribute maps or slices, the raw characters will now appear instead. Audit any log parsers, attribute processors, or exporters that inspect string representations of map/slice values.

• enhancementAdd otelcol_exporter_in_flight_requests to exporter dashboards

  A new UpDownCounter metric otelcol_exporter_in_flight_requests tracks concurrent export requests per exporter. Add it to your Collector dashboards to detect exporter worker pool saturation before it causes backpressure or dropped data. No config change needed — it emits automatically.

v0.151.0 has one API-level breaking change in the OTLP receiver config struct and changes the default behavior of the builder's go.mod generation. Dual-stack gRPC connection fixes and a pprofile data corruption fix are the most operationally relevant improvements.

• breakingUpdate OTLP receiver config field access in custom collectors

  The OTLP receiver's Config.Protocols is now a named field — any code accessing cfg.GRPC or cfg.HTTP directly will fail to compile. Update those references to cfg.Protocols.GRPC and cfg.Protocols.HTTP before upgrading collector builds that embed or extend the OTLP receiver.

• breakingCheck generated collector go.mod if you commit builder output

  cmd/builder now writes relative paths in Go module replace statements by default. For teams that build the collector, commit the generated source, and run it on multiple machines, this is an improvement. If you have tooling that depends on absolute paths (e.g., scripts parsing go.mod), set dist::use_absolute_replace_paths: true to preserve the old behavior while you migrate.

• enhancementFix dual-stack DNS connection issues with passthrough gRPC resolver

  Teams running OTLP gRPC exporters in dual-stack DNS environments have reported connection issues since the grpc.NewClient migration. You can now work around this by setting the endpoint to passthrough:///host:port in your exporter config. If you've seen intermittent connection failures in IPv4/IPv6 mixed environments, try this first.

v1.120.1 is a patch release focused on carbon cost fixes, AWS Spot pricing performance, memory optimization, and a security-relevant MCP server default change.

• breakingCheck if MCP server was relied upon — it's now off by default

  The MCP server flipped from opt-out to opt-in. If any tooling or integrations depended on the MCP server being active without explicit configuration, they will silently stop working after this upgrade. Before deploying, audit whether MCP_SERVER_ENABLED needs to be explicitly set to true in your environment config.

• enhancementAWS Spot users: Spot Price History caching reduces API pressure

  If you run workloads on AWS Spot instances, OpenCost was previously hitting the Spot Price History API more aggressively. The new caching layer cuts down on AWS API calls, which matters both for rate limiting and for cost attribution latency. No action required, but worth validating your Spot cost data accuracy after upgrading.

• enhancementCarbon cost data was wrong — upgrade if you track emissions

  The carbon cost fixes address incorrect provider detection and broken fallback logic, meaning prior carbon cost figures may have been inaccurate. If you surface carbon/emissions data to teams or dashboards, re-baseline your metrics post-upgrade rather than comparing historical data across this version boundary.

Cortex v1.21.0 graduates several experimental features to stable, fixes multiple PRW2 data corruption and panic bugs, and introduces a new Parquet mode for Store Gateway alongside an Overrides API.

• breakingAudit and update renamed flags before upgrading

  Several flags were renamed: -experimental.ruler.enable-api → -ruler.enable-api, -experimental.alertmanager.enable-api → -alertmanager.enable-api, -distributor.otlp.enable-type-and-unit-labels is now a no-op (consolidated into -distributor.enable-type-and-unit-labels), and parquet cache flags/configs dropped 'queryable' from their names. The users-scanner cleanup-interval became update-interval. Old flags are deprecated but not removed yet — audit your config files and Helm values now, before they disappear in a future release.

• breakingPRW2 data corruption and panic fixes require validation of existing pipelines

  Three separate bugs in the Remote Write V2 handler were fixed: shallow copying of Samples/Histograms caused data corruption, dirty sync.Pool reuse caused index-out-of-range panics, and Symbols backing array wasn't cleared on pool return causing memory leaks. If you're using PRW2, treat this upgrade as mandatory. Validate your ingestion pipeline after upgrading and check that downstream metrics are intact.

• enhancementBucket index is now on by default — verify storage permissions

  The bucket index is enabled by default in this release. If your object storage IAM policies restrict list operations or your Cortex deployment had bucket index disabled intentionally, verify that the bucket index can be written and read correctly post-upgrade. Cortex will generate and maintain the index automatically, but storage permission errors will surface as query failures rather than obvious configuration errors.

Litmus 3.28.0 is a focused maintenance release fixing a probe config leak, a subscriber crash on workflow events, and a Python vulnerability in the frontend base image.

• securityUpgrade frontend to resolve Python CVE in base image

  The frontend container was running on a ubi8 base image with a Python vulnerability. The fix upgrades it to ubi9. If you run Litmus in environments with strict image scanning policies (e.g., Trivy in CI, ACS, or Prisma), this upgrade will clear those alerts. Pull the 3.28.0 frontend image and redeploy — no config changes needed.

• breakingProbe config leak fix may change behavior for shared-type probe setups

  If you run multiple probes of the same type in a single experiment, previous releases silently shared stale config between them. The fix now isolates config per probe. Audit any experiments where multiple probes of the same type are defined — if you were unknowingly relying on config bleed-over, results may differ after upgrading.

• enhancementSubscriber crash fix improves reliability for GitOps and event-driven workflows

  Teams using Litmus in GitOps mode or triggering experiments via Argo Workflow ADD events with ChaosEngine nodes were hitting silent subscriber crashes. After upgrading to 3.28.0, those workflows should resume without manual subscriber restarts. No action required beyond the upgrade, but verify subscriber pod stability post-deploy if you've been seeing unexplained restarts.

OpenCost v1.120.0 adds OVH cloud provider support, AWS CUR 2.0 compatibility, and fixes several resource leaks and bugs across AWS, DigitalOcean, and S3 integrations.

• breakingCheck PV cost accuracy after upgrading — unit parsing was broken

  PV capacity was previously mishandled for Ki/Mi/Gi/Ti units, which means PV cost allocations may have been wrong in earlier versions. After upgrading, verify your PV cost data looks correct, especially if you were seeing anomalous persistent volume costs.

• enhancementMigrate to AWS CUR 2.0 if you haven't already

  AWS is deprecating CUR 1.0 — this release adds CUR 2.0 support. If you're still on CUR 1.0 exports, now is the time to update your AWS billing export configuration and point OpenCost at the new format. Don't wait for AWS to force the migration.

• enhancementSuppress noisy Spot feed warnings with the new toggle

  Teams not using AWS Spot instances were getting misleading log warnings about spot data feeds. You can now explicitly disable the Spot Data Feed via config. Set the toggle if you're not using Spot — it cleans up your logs and removes the misleading provider log noise fixed in this release.

Prometheus v3.11.2 patches a stored XSS vulnerability (CVE-2026-40179) in the web UI, plus two Consul SD fixes. Upgrade immediately if your UI is exposed.

• securityPatch CVE-2026-40179 immediately if your Prometheus UI is reachable

  A stored XSS can be triggered by crafted metric names or label values — meaning any scrape target you ingest data from could potentially inject malicious scripts into your UI. If your Prometheus endpoint is accessible to users beyond the ops team (internal dashboards, federated setups), treat this as urgent. Upgrade to v3.11.2 now. If you can't upgrade immediately, restrict UI access via network policy or auth proxy until you can.

• breakingVerify Consul SD behavior after the Health API filter fix

  The filter parameter was previously applied incorrectly to the Consul Health API. After upgrading, your Consul service discovery results may change if you were relying on that filter — intentionally or not. Review your Consul SD configs and validate that the services being scraped post-upgrade match your expectations before rolling out to production.

• enhancementUse `health_filter` in Consul SD to reduce noise from unhealthy services

  The new `health_filter` field lets you filter Consul Health API responses directly at the SD layer. If you've been working around this with relabeling rules to drop unhealthy instances, you can now clean that up. Set `health_filter` to `healthy` in your Consul SD config to only scrape passing services and reduce unnecessary target churn.

Prometheus v3.11.0 ships significant new discovery capabilities, TSDB experimental features, and a critical retention bug fix that could cause data to be kept 1,000,000x longer than configured.

• breakingFix TSDB retention unit bug before your disk fills up

  A unit mismatch in the config file parser caused `storage.tsdb.retention.time` to be interpreted as 1,000,000x the intended value. If you set retention via the config file (not CLI flags), your TSDB may be holding far more data than expected. Upgrade to 3.11.0 immediately and verify disk usage post-upgrade. Also note: CLI flag values are now the fallback when retention is removed from config, so review your configuration carefully.

• breakingMigrate Hetzner hcloud SD label references before July 2026

  `__meta_hetzner_datacenter` and `__meta_hetzner_hcloud_datacenter_location*` labels are deprecated for hcloud roles and will stop working after July 1, 2026. Audit all relabeling configs and recording rules that reference these labels. Replace with `__meta_hetzner_hcloud_location` and `__meta_hetzner_hcloud_location_network_zone`. The robot role keeps the old label for backward compatibility.

• enhancementCap TSDB disk usage with retention.percentage

  The new `storage.tsdb.retention.percentage` setting lets you define a maximum percentage of available disk that TSDB can use. This is a cleaner safety net than estimating byte-based retention limits, especially in environments with variable data volumes. Consider pairing it with the existing time-based retention rather than replacing it — both constraints apply.

v2.17.0 ships a security fix for XSS in the UI, two major UI features (side panel span details, trace log aggregation), and several backend stability fixes including a panic guard and clock skew correction.

• securityUpdate UI immediately to fix XSS vulnerability

  The UI was rendering user-supplied trace data using innerHTML, which opens the door to XSS if trace attribute values contain malicious HTML. This is fixed by switching to textContent. If you expose Jaeger UI to users who can influence span data (e.g., shared environments, multi-tenant setups), treat this as a priority upgrade. Pull the v2.17.0 image and redeploy.

• breakingClock skew fix changes span end timestamps — verify your trace data expectations

  The clock skew adjuster previously only corrected span start times, leaving end timestamps unadjusted. Now both are corrected. For teams with strict SLO calculations or latency assertions based on adjusted trace data, this behavioral change could shift computed durations. Validate your dashboards and alerting rules against real traces after upgrading.

• enhancementSide panel span view reduces trace analysis friction

  Span details previously expanded inline, collapsing the trace timeline context. The new side panel mode keeps the timeline visible while inspecting a span. Useful immediately for deep trace debugging — no configuration needed, just click a span in the UI. Worth showing to your team as a workflow improvement.

Chaos Mesh v2.8.2 is a security-focused patch that resolves CVEs across Go and UI dependencies, drops install.sh, and fixes cert-manager webhook conflicts.

• securityUpgrade now — this release exists specifically to fix CVEs

  Both Go and UI packages were upgraded in bulk to address vulnerabilities tracked in issue #4830. No specific CVE IDs are listed in the release notes, but the scope (all direct dependencies) suggests broad exposure. If you're running v2.8.1 or earlier, treat this as an urgent upgrade, especially for clusters where Chaos Mesh's dashboard is internet-accessible or exposed internally to untrusted users.

• breakingStop using install.sh — it's been deleted, not just deprecated

  install.sh is physically removed in this release. Any CI/CD pipelines, runbooks, or onboarding docs that reference it will break immediately. Migrate to the Helm chart or the official quick-start guide before upgrading. Check your automation scripts and internal documentation now.

• enhancementFix cert-manager server-side apply conflicts before they hit you in prod

  If you use cert-manager to manage Chaos Mesh's webhook certificates, the caBundle placeholder in webhook templates was causing server-side apply (SSA) field ownership conflicts — a subtle issue that silently breaks reconciliation. After upgrading to v2.8.2, re-apply your Helm release or manifests to let SSA recalculate field ownership cleanly. If you've been seeing unexpected webhook certificate errors or Helm diff noise, this is likely the cause.

Litmus 3.27.0 ships Job targeting for chaos experiments and fixes a cluster of stability bugs: nil pointer crashes, GitOps deadlocks, race conditions, and probe detection failures.

• breakingGitOps two-way sync was silently broken — verify your sync state after upgrading

  The GitOps sync handler goroutine was disabled, meaning changes in Git were not propagating back to Litmus in two-way mode. After upgrading to 3.27.0, manually trigger a sync and confirm experiment state matches your Git source of truth. Any drift that accumulated should be reconciled.

• enhancementTarget Kubernetes Jobs in chaos experiments

  If your team runs batch workloads or CI pipelines as Kubernetes Jobs, you can now inject chaos directly into them. Update your experiment manifests to reference Job resources and validate resilience of batch processing paths — something previously impossible without workarounds.

• enhancementMultiple nil pointer and race condition fixes reduce silent failures

  At least three separate nil pointer dereferences are fixed in this release (subscriber, QueryServerVersion, YAML parser), plus experiment run race conditions. If you've seen intermittent crashes or experiments stuck in inconsistent states, upgrading is the straightforward fix — no config changes needed.

Jaeger v2.16.0 introduces breaking changes requiring Go 1.25.7 and removes legacy remote sampling format, while improving ClickHouse performance and adding Elasticsearch health-check timeout support.

• breakingUpgrade Go to version 1.25.7 before deploying

  The Go version requirement is now enforced across the codebase. Teams running older Go versions will face build failures. Update your build environments, CI/CD pipelines, and container images to Go 1.25.7 before attempting to upgrade or build from source.

• breakingUpdate remote sampling client integrations

  The legacy remote sampling endpoint response format has been removed. If your applications use custom remote sampling clients that depend on the old format, they will break. Review your sampling configuration and update any custom clients to use the current format before upgrading.

• enhancementEnable Elasticsearch health-check timeout for improved startup reliability

  New startup health-check timeout configuration prevents indefinite blocking during Elasticsearch connectivity issues. Add health-check timeout settings to your Elasticsearch storage configuration to improve service startup reliability in environments with intermittent connectivity.

Fluentd v1.19.2 delivers critical bug fixes for socket leaks and connection timeouts, with Ruby 4.0 compatibility improvements. Priority fix for production stability issues affecting out_forward and HTTP server plugins.

• securityUpgrade immediately to fix socket leak vulnerability

  HTTP server plugin had socket leaks in POST requests that could lead to resource exhaustion. This is a critical fix for production environments using HTTP input plugins. Update immediately and monitor socket usage after deployment.

• breakingUpdate out_forward configurations for timeout handling

  The out_forward plugin now properly handles connection timeouts to prevent infinite loops. Review your forward output configurations and monitor for any connection behavior changes, especially in high-throughput environments where connection stability is critical.

• enhancementPrepare for Ruby 4.0 migration

  This release adds Ruby 4.0 compatibility with updated gem dependencies including ostruct and net-http. Plan your Ruby upgrade path and test this version in staging environments if you're planning to move to Ruby 4.0.