RATATOSKRATATOSK
ログイン

Prometheus

v3.16.0Observability
2026年11月5日

ACTION 6CHECK 3OTHER 76

オペレーターから見える既定値や入力検証、PromQLとヒストグラムの扱い、性能と正確性に関わる変更を含むリリースです。ストレージ、API、スクレイピング、リモートライトにも動作修正があります。

要対応 (6)

  • breakingRemote Write 2.0キューの非推奨メタデータメトリクスを削除

    Remote Write 2.0キューで、非推奨のprometheus_remote_storage_metadata_total、prometheus_remote_storage_metadata_failed_total、prometheus_remote_storage_metadata_retried_total、prometheus_remote_storage_metadata_bytes_totalメトリクスを公開しなくなりました。

  • breaking重複したラベル名を持つexemplarを拒否

    ラベル名が一意でないexemplarを受け付けなくなりました。これにより、保存されたexemplarが/api/v1/query_exemplarsで重複したJSONキーとして返されることを防ぎます。

  • breakingSmoothedおよびAnchored vector selectorを常時有効化

    Smoothed vector selectorとAnchored vector selectorが安定版になり、常に有効になりました。

  • breakingtimeoutとlookback_deltaのオーバーフロー値を拒否

    timeoutとlookback_deltaがint64の範囲を超える値の場合、負のdurationに変換せず拒否するようになりました。

  • breaking未来に偏りすぎたタイムスタンプのexemplarを破棄

    現在時刻より10分を超えて未来のタイムスタンプを持つexemplarを破棄するようになりました。これらを保存して同じ時系列の後続exemplarを妨げることはありません。

  • breakingバケットインデックスがオーバーフローするnative histogramを拒否

    span offsetがint32のバケットインデックスでオーバーフローするnative histogramを拒否するようになりました。破損したバケットを持つ状態で保存されることはありません。

影響確認 (3)

  • breaking実験的な検索エンドポイントのbatch_size上限を1000に変更

    実験的な検索エンドポイントを利用している場合に適用されます。

    実験的な検索エンドポイントで、batch_sizeの上限が1000になりました。

  • breakingTLS有効時の--web.external-url既定値をhttpsに変更

    --web.config.fileを使用している場合に適用されます。

    --web.config.fileでTLSを有効にすると、--web.external-urlの既定値がhttpsになります。

  • breakingNHCB変換でsum-only classic histogramを拒否

    NHCB変換を使用している場合に適用されます。

    NHCB変換で、sumだけを持つclassic histogramを拒否するようになりました。

その他の記録済み変更 76 件すべてfixes 52 · value changes 14 · additions 10

fixes (52)

  • Abort failed streaming responses without writing duplicate headers or appending plain-text errors to the protobuf stream
  • Fix loss of out-of-order samples when a WAL checkpoint drops a series record that the out-of-order write-behind log still needs
  • Fix queries that seek within a head chunk returning samples committed after the query started
  • Preserve mapped chunks when WAL replay merges multiple references for the same series, preventing missing out-of-order samples and duplicate in-order chunks after restart
  • Fix auto-reload logic to reload after reverting a failed configuration change
  • Do not match metric family metadata for doubled _total and _info suffixes in experimental search API.
  • Resolve metric family metadata for suffixed metric names in search results, in experimental search API.
  • Describe each ElastiCache cluster once instead of twice for clusters that are not replication group members.
  • Allow installing promtool with version-suffixed go install commands
  • Fix a panic in Azure service discovery when a VM tag has a null value.
  • fix corruption of float native histograms.
  • Fix ts_of_last_over_time for samples before the Unix epoch.
  • Fix ConvertNHCBToClassic inflating classic bucket counts for integer native histograms whose spans have gaps.
  • Preserve rule and TSDB block values in JSON logs.
  • Count only out-of-order exemplars in prometheus_api_otlp_out_of_order_exemplars_total, not every rejected exemplar.
  • Do not drop exemplars that have a trace_id or span_id filtered attribute in addition to a trace or span ID.
  • Do not emit le="+Inf" twice when a histogram's explicit bounds end at +Inf, and warn about the bound.
  • Stop dropping exemplars that arrive out of timestamp order within a data point.
  • Attach histogram exemplars to matching classic _bucket series and preserve them when scraping both native and classic histograms.
  • Avoid intermediate overflow in rad() and deg() conversions.
  • Consistently ignore and drop __type__ and __unit__ labels in without aggregations, absent(), info(), and binary operator fill matching.
  • Do not panic when printing the partial AST returned alongside a parse error.
  • Drop __type__ and __unit__ in binary operations so they are transparent to users, similar to how __name__ is handled.
  • Fix ==, !=, changes() and count_values on native histograms giving wrong results inside histogram_count(), histogram_sum() and histogram_avg().
  • Fix histogram_fraction returning wrong results for native histograms with custom buckets when a bucket boundary is exactly 0.
  • Fix histogram_quantile and histogram_quantiles interpolating within the wrong bucket for native histograms whose sum is NaN.
  • Fix lexing of subqueries whose range is a duration keyword expression, e.g. foo[range():1s].
  • Fix warning and error locations for duration expressions min_of and max_of.
  • Ignore NaN classic histogram bucket boundaries in histogram_quantile, histogram_quantiles and histogram_fraction and return correct value instead of NaN if there are normal buckets.
  • Preserve infinite values at exact quantile ranks.
  • Preserve original operand sides for experimental fill modifiers with group_right matching.
  • Preserve subquery step expressions when printing queries after preprocessing.
  • Range vector functions no longer raise a spurious "vector cannot contain metrics with the same labelset" error when series have different metric names but the same remaining labels, as long as their timestamps do not overlap within the evaluated range window
  • do not count subquery materialized samples in total samples statistics
  • Prevent future-dated exemplars from scaling shards down to the minimum.
  • Prevent future sent timestamps from advancing WAL truncation beyond the minimum retention window.
  • Restore firing alerts with for durations shorter than --rules.alert.for-grace-period after a restart.
  • fix failing scrape when float histogram is emitted over protobuf with count, zero count and buckets all zero, for example a result of recording rate(x[1m]) over constant histogram.
  • Return the lexically smallest label values when limit is set on /api/v1/search/label_values, instead of an arbitrary subset
  • Fix WAL watcher permanently skipping checkpoint series garbage collection when lagging behind head truncation.
  • Avoid panics when the requested time range has no participating storage.
  • Fix a race in selected-series and stale-series compaction where a sample committed after the block was written but before the series was evicted from the head could be silently lost.
  • Fix a rare case where a sample could be lost during selected-series or stale-series compaction if an unrelated write to the same series ran cleanup that erased the evidence needed to protect it.
  • Fix spurious "unknown series references" warnings during WAL replay after a restart, when the only blocks on disk were produced by selected-series or stale-series compaction.
  • Preserve series identities needed to recover exemplars after compaction and restart.
  • fix no passing metadata-wal-records to TSDB from main feature flag.
  • Preserve IPv6 addresses with dotted zone identifiers in stripDomain.
  • Don't automatically open an autocomplete dropdown after closing a function call body or aggregation operator with a closing parenthesis.
  • Fix HTTP/2 negotiation silently falling back to HTTP/1.1 under Go 1.27+ when TLS is enabled with http2: true. The server now correctly advertises h2 via ALPN and multiplexes concurrent requests over a single connection.
  • Fix double-counting of failed samples in the PrometheusRemoteStorageFailures alert denominator.
  • Honor --enable-feature when evaluating rules in test rules.
  • Ensure UTF-8 works with the experimental OpenMetrics 2.0

value changes (14)

  • Subquery samples are now included in the query TotalSamples and TotalSamplesPerStep statistics, and SamplesRead now counts reads from subqueries with an @ modifier in range queries.
  • Count Remote Write 2.x samples and histograms ingested without metadata in prometheus_api_remote_write_without_metadata_appended_samples_total even when metadata-wal-records is disabled.
  • Warn when participating remote-read storage does not support search.
  • Cache repeated OTLP histogram metric names.
  • Reuse classic histogram and summary expansion buffers across scrapes.
  • Decode only histogram sum/count (not buckets) for timestamp(), start_timestamp() and absent().
  • Only select the fixed window for subqueries with an @ modifier.
  • Deduplicate per-series metadata structs in memory with unique.Handle.
  • Avoid some copies of scrape response bodies.
  • Improve XOR2 compression and append performance by selecting value windows based on encoding cost.
  • Reduce memory used writing and reading checkpoint metadata records.
  • Convert classic histograms to NHCB directly in the OpenMetrics 2.0 parser without wrapping in NHCBParser.
  • Reduce OpenMetrics 2.0 parser allocations and CPU overhead by lazily constructing composite series and exemplar labels.
  • Increment prometheus_tsdb_wal_corruptions_total when WAL corruption is detected during checkpoint creation, so the metric reflects corruption found at compaction time rather than only after a restart-triggered repair.

additions (10)

  • Allow overriding PromQL Start Timestamp processing on a per-query basis using the X-Prometheus-Use-Start-Timestamps HTTP header.
  • Support metadata-wal-records in Prometheus Agent mode.
  • Add experimental integral() function for computing integral-over-time of range vectors behind the --enable-feature=promql-experimental-functions flag.
  • Add experimental query cost estimation, actual cost reporting, and configurable execution limits, enabled with --enable-feature=query-cost.
  • Use start timestamps to detect counter resets when float timeseries are queried using extended range selector modifiers (anchored, smoothed). This feature is hidden behind use-start-timestamps feature flag.
  • print test filename during promtool tests
  • Allow unquoted Unicode letters and dots behind a feature flag
  • Add native histogram prometheus_remote_read_handler_gate_wait_duration_seconds to measure concurrency-gate wait duration, including requests canceled while waiting.
  • Add experimental flag convert_nhcb_to_classic to convert NHCB to classic histograms for external systems that dont support NHCBs.
  • Add --allow-incompatible-block-duration so users can specify any max-block-duration instead of being restricted to the default Prometheus block durations.
Prometheusをスタックに追加

対応が必要なリリースが出たときに、週次メールでお知らせします。 今回の破壊的変更も、その一例です。

スタックに追加 →
公式リリースノート ↗Prometheusのリリース一覧 →