RATATOSKRATATOSK
Sign in

Releases

AI-analyzed release notes for CNCF graduated and incubating projects.

May 2026Clear ×

OpenCost

ObservabilityMay 29, 2026

v1.120.3 patches two Go CVEs and ships a wide range of bug fixes across AWS, Azure, Oracle, and DigitalOcean providers, plus OVH support and cosign image signing.

  • securityPatch vulnerable Go dependencies — upgrade immediately

    A Go dependency update patches GHSA-xmrv-pmrh-hhx2 and CVE-2026-34986. If you're running any v1.120.x release prior to v1.120.3, upgrade now — these are dependency-level vulnerabilities, not just code changes.

  • breakingMCP server is now opt-in — check your config before upgrading

    The MCP server now defaults to disabled (MCP_SERVER_ENABLED=false). If you were relying on it being on by default, set the env var explicitly after upgrading. Check your deployment manifests before rolling out.

  • enhancementEnable cosign image verification in your admission pipeline

    Container images are now signed with cosign keyless signing and include SLSA provenance attestations. If your admission policy requires image signature verification, you can now enforce it against OpenCost images. Update your policy tooling (e.g., Kyverno, Cosign verify) to validate these attestations in CI or at deploy time.

Key changes (7)
  • Security: vulnerable Go dependencies patched (GHSA-xmrv-pmrh-hhx2, CVE-2026-34986)
  • MCP server now opt-in via MCP_SERVER_ENABLED=false default
  • OVH cloud provider added; DigitalOcean and Oracle/Karpenter pricing fixes
  • AWS Spot Price History API now cached; toggle added to disable spot data feed entirely
  • Memory leak fixed in scrape target parsing; CPU usage counter overflow protection added
  • Container images now signed with cosign and include SLSA provenance attestations
  • CUR 2.0 support added for AWS cost data ingestion
Source

Prometheus

ObservabilityMay 28, 2026

Prometheus 3.12.0 ships two security patches (Remote Write DoS, STACKIT SD secret leak), fixes a WAL race in Agent mode, and cuts TSDB range query CPU with a quadratic-to-constant head chunk lookup fix.

  • securityPatch two security fixes now — especially if using STACKIT SD

    Two CVEs patched in this release. First: Remote Write now rejects snappy-compressed payloads where the declared decoded size exceeds 32 MB — this closes a DoS vector against your remote-write receiver endpoint. Second: STACKIT SD was leaking secrets in plaintext via the `/-/config` endpoint (GHSA-39j6-789q-qxvh). If you use STACKIT SD, rotate any credentials that may have been exposed before upgrading.

  • breakingAgent mode WAL race and remote_write panic bug fixed — upgrade Agent deployments

    A race condition in the agent appender could produce duplicate in-memory series and duplicate WAL records when concurrent appends target the same label set. If you run Prometheus in Agent mode under high write concurrency, this bug could silently corrupt your WAL. Upgrade to 3.12.0 to fix it. Also, `remote_write` queue_config fields are now validated at load time — misconfigurations that previously caused silent runtime panics will now fail at startup, which is the right behavior but means you should test configs before rolling out.

  • enhancementTSDB range query CPU cut and auto-reload-config is now stable

    TSDB head chunk lookup in range queries drops from quadratic to constant time, and mmap operations now skip series that don't need work. At production scale with large head chunks, this can meaningfully cut CPU. No config changes needed — just upgrade. Separately, `auto-reload-config` is now stable (no longer experimental), so you can drop any caveats around it in runbooks.

Key changes (18)
  • Security: Remote Write rejects snappy payloads with decoded size over 32 MB (DoS fix); STACKIT SD secret leak via /-/config endpoint patched (GHSA-39j6-789q-qxvh)
  • TSDB performance: head chunk range query lookup is now O(1) instead of O(n²); mmap skips clean series, reducing CPU at scale
  • PromQL: new experimental functions start(), end(), range(), step(); rate()/irate()/increase()/resets() updated to use start timestamps behind the use-start-timestamps feature flag
  • Service Discovery: DigitalOcean Managed Databases and Outscale VM added; AWS EC2 SD gains IPv6 support; AWS SD gets optional external_id for ECS/MSK/RDS/ElastiCache
  • Bug fixes: agent WAL race condition patched; scrape panics on malformed histograms fixed; TSDB native histogram query panic fixed; remote_write queue_config now validated at startup
  • UI: time series deletion and tombstone cleanup now available from the Status menu
  • auto-reload-config promoted to stable
  • OTLP gzip body size now capped to prevent decompression abuse
  • SD target updates propagate faster via dynamic backoff instead of static 5s interval
  • Consul SD health_filter fix for Catalog-only fields like ServiceTags
  • prometheus_sd_refresh and prometheus_sd_discovered_targets metrics cleaned up when scrape jobs are removed
  • PromQL warns when sort/sort_by_label used in range queries (no-op in that context)
  • sort/sort_by_label warning in range queries, NaN/infinite duration expressions now rejected
  • Scrape: st-synthesis feature flag added to synthesize start timestamps for cumulative metrics when using Remote Write 2.0
  • promtool query instant gains --header flag
  • aix/ppc64 compilation target added
  • /api/v1/status/self_metrics endpoint added
  • Tracing: OTLP HTTP insecure startup failure fixed
Source

OpenTelemetry

ObservabilityMay 25, 2026

v0.153.0 stabilizes seven feature gates (breaking), fixes a critical memory corruption bug in gRPC Snappy compression, and ships several mdatagen enhancements for component authors.

  • securityUpgrade immediately if you use gRPC with Snappy compression

    The Snappy fix in configgrpc addresses memory corruption that can cause fatal errors — this is a process-crash-level issue, not just a performance concern. If your exporters or receivers use gRPC with Snappy compression enabled, treat this as a priority upgrade. Check your exporter configs for compression: snappy settings.

  • breakingAudit your feature gate overrides before upgrading

    All seven stabilized gates are now permanent behavior — you can no longer toggle them. If your collector config or startup flags reference configoptional.AddEnabledField, confmap.newExpandedValueSanitizer, exporter.PersistRequestContext, otelcol.printInitialConfig, pdata.useCustomProtoEncoding, telemetry.UseLocalHostAsDefaultMetricsAddress, or pdata.enableRefCounting, remove those references or the collector will fail to start. The metrics address change (UseLocalHostAsDefaultMetricsAddress) is the one most likely to surprise teams — your metrics endpoint is now localhost-bound by default.

  • breakingmdatagen reaggregation config is now on by default

    If you maintain custom collector components using mdatagen, the reaggregation config fields are now generated by default. To preserve the old behavior (metrics config with only the enabled field), explicitly set reaggregation_enabled: false in your metadata.yaml. Run mdatagen on your components after upgrading and review the generated output before committing.

Key changes (5)
  • Seven feature gates stabilized and removed — including telemetry.UseLocalHostAsDefaultMetricsAddress, otelcol.printInitialConfig, and pdata.enableRefCounting — meaning any code still gating on these will break
  • Critical bug fix: memory corruption and fatal error in configgrpc's Snappy compression (CVE-adjacent, upgrade immediately if you use gRPC with Snappy)
  • pdata.useCustomProtoEncoding feature gate fully removed — no opt-out path remains, custom proto encoding is now always on
  • mdatagen now generates config documentation tables injected into README.md automatically, and enables reaggregation config generation by default
  • New Walker interface in xextension/storage enables storage migration and TTL-based garbage collection patterns
Source

OpenTelemetry

ObservabilityMay 19, 2026

v0.152.1 is a bug-fix-heavy release with one useful new metric. The most operationally impactful changes are snappy decompression security fixes and a Prometheus metric naming regression fix.

  • securityconfighttp snappy fixes limit memory exposure from compressed payloads

    Three snappy decompression fixes land in `pkg/confighttp`: body is now closed after reading, panics in decompression libraries return HTTP 400 instead of crashing with 500, and `max_request_body_size` is enforced before the decoded buffer is allocated. The size-check fix in particular prevents a potential memory spike from a maliciously crafted compressed payload. If you accept compressed OTLP over HTTP from untrusted sources, upgrade.

  • breakingPrometheus metric name format may change if you customized telemetry host

    If you explicitly set the `host` field in the telemetry metrics section of your collector config, check whether your Prometheus metric names changed after upgrading to recent versions. The bug caused `WithoutScopeInfo`, `WithoutUnits`, and `WithoutTypeSuffix` to default to false instead of true in that code path, which means your metrics may have had unexpected suffixes or scope labels. This release restores the correct defaults — metric names may shift again on upgrade, so update dashboards and alerts accordingly.

  • enhancementAdd in-flight exporter metric to your dashboards

    The new `otelcol_exporter_in_flight_requests` UpDownCounter metric is available in `pkg/exporterhelper`. Add it to your dashboards to see when exporters are queuing up requests or saturating worker pools — it's a direct signal of export backpressure that was previously hard to observe without custom instrumentation.

Key changes (7)
  • New `otelcol_exporter_in_flight_requests` metric tracks concurrent export requests per exporter, useful for detecting worker pool saturation
  • Three `pkg/confighttp` fixes for snappy decompression: panic recovery (now returns 400), body cleanup, and pre-allocation size enforcement
  • `pcommon.Value.AsString` no longer HTML-escapes `<`, `>`, `&` in map and slice values — output may change if you relied on escaped output
  • Noisy gRPC disconnect messages (`connection reset by peer`) no longer emit at WARN level during normal client disconnects
  • Prometheus config default mismatch fixed: explicitly setting telemetry host no longer silently changes metric name format
  • Return noop tracer provider when no trace processors are configured, avoiding unnecessary overhead
  • API: `xconfmap.Validator` deprecated; migrate to `confmap.Validator` and `confmap.Validate`
Source

OpenCost

ObservabilityMay 18, 2026

v1.120.2 is a substantial release with security fixes, memory leak patches, AWS CUR 2.0 support, OVH cloud provider, and supply chain security improvements via cosign image signing.

  • securityUpgrade immediately for CVE-2026-34986 fix

    This release patches GHSA-xmrv-pmrh-hhx2 and CVE-2026-34986 in Go dependencies. If you're running any prior v1.120.x or v1.119.x version, upgrade to v1.120.2 now. Check your vulnerability scanner output to confirm the affected packages are resolved post-upgrade.

  • enhancementVerify cosign image signatures in your admission pipeline

    OpenCost images are now signed with cosign keyless signing and include SLSA provenance attestations. If your cluster uses an admission controller (e.g., Kyverno, Connaisseur), add a policy to enforce signature verification on opencost images. This closes a real supply-chain gap for teams running OpenCost in regulated environments.

  • enhancementEnable AWS CUR 2.0 and tune spot data feed behavior

    If your AWS billing is already on CUR 2.0, you can now configure OpenCost to use it directly. Additionally, if you don't use the AWS spot data feed, set the new toggle to disable it — this suppresses noisy warnings and avoids unnecessary config polling. Teams with spot-heavy workloads should also benefit from the new spot price API caching layer reducing API call volume.

Key changes (5)
  • Security: Patched vulnerable Go dependencies (GHSA-xmrv-pmrh-hhx2, CVE-2026-34986) — upgrade promptly
  • AWS CUR 2.0 support added, plus a configurable toggle to disable the spot data feed and spot price API caching
  • Memory leak fixed in scrape target parsing; CPU counter overflow/reset protection added
  • Container images now signed with cosign keyless signing and SLSA provenance attestation
  • OVH cloud provider added; PV pricing can now be set via annotations; PV capacity parsing fixed for Ki/Mi/Gi/Ti units
Source

Litmus

ObservabilityMay 18, 2026

Litmus 3.29.0 patches a gRPC CVE, fixes duplicate chaos triggers under concurrent reconciles, and adds Prometheus metrics support for experiment observability.

  • securityPatch CVE-2026-33186 by upgrading to 3.29.0 now

    The gRPC library was upgraded to v1.79.3 to fix CVE-2026-33186. If you're running any Litmus version prior to 3.29.0, your control plane is exposed. Upgrade immediately — this isn't a 'schedule it next sprint' situation.

  • breakingVerify event-tracker behavior after the duplicate-trigger fix

    The fix for duplicate chaos experiment triggers under concurrent reconciles changes how the event-tracker handles race conditions. If you rely on the event-tracker for automated chaos injection, test your pipelines post-upgrade to confirm expected trigger counts. Duplicate runs may have been masking gaps in your experiment coverage.

  • enhancementWire up Prometheus metrics to your existing dashboards

    Prometheus metrics are now natively exposed by ChaosCenter. The release includes a getting-started guide and unit tests, so integration is straightforward. Add Litmus as a scrape target and start tracking experiment pass/fail rates, run durations, and infra connectivity — this fills a long-standing observability gap for chaos workflows.

Key changes (5)
  • Security: gRPC bumped to v1.79.3 to address CVE-2026-33186
  • Bug fix: concurrent reconciliation no longer triggers duplicate chaos experiments via the event-tracker
  • New feature: Prometheus metrics added to ChaosCenter for experiment observability, with unit tests and a getting-started guide
  • Bug fix: experiments can now be stopped even when the connected infra is disconnected
  • Bug fix: CronWorkflow run history no longer shows a blank page in the UI
Source

Jaeger

ObservabilityMay 13, 2026

Jaeger v2.18.0 brings two breaking changes (OTEL metrics overhaul and removed min-step API), useful header-forwarding for ES/OpenSearch, and a massive UI refactor migrating state management from Redux to Zustand.

  • breakingAudit your metrics dashboards before upgrading

    The OTEL collector package upgrade changed metric names and shapes. Before upgrading, compare your current Jaeger metrics against the new ones by running both versions in parallel or checking the PR diff. Any Grafana dashboards, Prometheus alerts, or monitoring rules built on Jaeger's internal metrics need to be reviewed and updated. The min_step removal is a smaller blast radius — only affects custom tooling that calls the metricstore API directly — but still requires a code or config change before upgrading.

  • enhancementUse header forwarding if you authenticate ES/OpenSearch with custom headers

    If your Elasticsearch or OpenSearch cluster requires auth headers beyond basic username/password (e.g., custom JWT, IAM proxy headers), configure the new header-forwarding feature. This also helps in multi-tenant setups where storage routing depends on request headers. Configure via the storage backend settings in jaeger-query or the gRPC storage plugin config.

  • enhancementEvaluate ClickHouse as a storage backend if you want SPM without Prometheus

    ClickHouse SPM support is experimental but now covers the full metrics trifecta: call rates, error rates, and latencies. TTL support is also added. If you're running ClickHouse already or want a single-store solution for both traces and metrics, this is the release to start testing against. Don't use it in production yet, but set up a staging environment to track maturity.

Key changes (6)
  • Breaking: OTEL collector package upgrades changed metric names/shapes — dashboards and alerts referencing old metric names will break
  • Breaking: min_step API removed from metricstore — any tooling calling this endpoint needs updating
  • New: UI auto-detects base path from browser URL, removing the need to manually configure UI base paths in reverse-proxy setups
  • New: Configurable header forwarding to ES/OpenSearch and gRPC storage backends — useful for auth tokens and custom routing headers
  • Experimental: ClickHouse SPM (Service Performance Monitoring) now has call rates, error rates, and latencies, plus TTL support — the storage backend is maturing fast
  • UI breaking: Legacy browser support dropped — IE and very old Chromium/Firefox users will have problems
Source

OpenTelemetry

ObservabilityMay 11, 2026

v0.152.0 fixes a silent Prometheus metric name regression when telemetry host is customized, changes AsString HTML-escaping behavior for map/slice values, and adds an in-flight exporter requests metric.

  • breakingCheck Prometheus metric names if you customized telemetry host

    The Prometheus exporter boolean fields (WithoutScopeInfo, WithoutUnits, WithoutTypeSuffix) now default correctly when you explicitly set the telemetry host. Before this fix, those fields defaulted to false instead of true, so your metric names would silently change format. If you run a Collector with a custom telemetry host configured, verify your Prometheus metric names after upgrading — dashboards and alerts keyed on metric name format may need updating.

  • breakingAsString output change for map/slice values — check downstream consumers

    pcommon.Value.AsString no longer HTML-escapes <, >, and & in map and slice values. If your pipeline or downstream consumers were relying on escaped output (e.g., &lt; or &gt;) from attribute maps or slices, the raw characters will now appear instead. Audit any log parsers, attribute processors, or exporters that inspect string representations of map/slice values.

  • enhancementAdd otelcol_exporter_in_flight_requests to exporter dashboards

    A new UpDownCounter metric otelcol_exporter_in_flight_requests tracks concurrent export requests per exporter. Add it to your Collector dashboards to detect exporter worker pool saturation before it causes backpressure or dropped data. No config change needed — it emits automatically.

Key changes (5)
  • New otelcol_exporter_in_flight_requests metric tracks concurrent in-flight export requests per exporter
  • pcommon.Value.AsString no longer HTML-escapes <, >, & in map/slice values — behavior now consistent with ValueTypeStr
  • Prometheus config defaults bug fixed: explicit telemetry host config no longer silently flips metric name format
  • pkg/confighttp: snappy decompression now enforces max_request_body_size before allocating buffer, and panics in decompression libs return HTTP 400 instead of 500
  • Noisy gRPC 'connection reset by peer' log lines no longer emit at WARN during normal client disconnects
Source
Browse by month