RATATOSK
← All releases

KServev0.16.0

Security 2Breaking changes 19Deprecations 1Important fixes 28Enhancements 43Dependency updates 8Other changes 49

Get KServe alerts

Email is sent weekly only when there are changes to check; critical security advisories are sent immediately.

Security2

RuntimeDefault seccomp profiles added to KServe controllers

AffectsDeployments using the KServe controller, LocalModel controller, or LocalModelNode agent manifests, including Helm chart controller deployments(Inferred)

Previously, the shown pod security contexts did not set seccompProfile, and the Helm chart's documented controller security context default was {"runAsNonRoot":true}. The affected contexts now specify seccompProfile.type: RuntimeDefault.

Show release-note lines
* Fix pss restricted warnings by @akagami-harsh in https://github.com/kserve/kserve/pull/4327
github.com/kserve/kserve/releases/tag/v0.16.0

Storage-initializer HF token exposure in pod manifests fixed

Affectsstorage-initializer workloads using an HF token stored in a Kubernetes Secret(Inferred)
Additional actionHF_TOKEN is injected using valueFrom with a secretKeyRef to the Kubernetes Secret instead of embedding its plaintext value in the pod manifest.

The HF token was injected as plaintext in the pod manifest, exposing it to potential leaks.

Show release-note lines
* Fix HF Token Vulnerability in Storage Initializer Container by @brettmthompson in https://github.com/kserve/kserve/pull/4677
github.com/kserve/kserve/releases/tag/v0.16.0

Breaking changes19

InferenceService services created without the default suffix

AffectsInferenceService components using the default suffix

These components were previously supported with the suffix.

Evidence“With this PR, the support for compatibility with the "default" suffix has been removed. Any InferenceService component that is still using the "default" suffix will now have a new service created without the suffix.” kserve/kserve/pull/4178 ↗
Show release-note lines
* chore: remove 'default' suffix compatibility by @sivanantha321 in https://github.com/kserve/kserve/pull/4178
github.com/kserve/kserve/releases/tag/v0.16.0

Default OpenTelemetryCollector resource requests and limits added

Affectsinstallations using the default OpenTelemetryCollector configuration for InferenceService autoscaling(Inferred)

The defaults set both CPU request and limit to 1, and both memory request and limit to 2Gi.

Show release-note lines
* add resource default for otel collector container limit in configmap by @andyi2it in https://github.com/kserve/kserve/pull/4633
github.com/kserve/kserve/releases/tag/v0.16.0

Quick-install script version pins updated

Affectsfresh kind-cluster installations using hack/quick_install.sh

On fresh kind clusters, hack/quick_install.sh -r failed amid Istio Helm schema-validation errors, and the KServe v0.16.0-rc0 release candidate was unavailable. The script now selects Istio 1.27.1 to address schema-validation compatibility and KServe v0.15.2, an available, supported release.

Show release-note lines
* fix: istio installation failures and update kserve to stable version  by @Ce11an in https://github.com/kserve/kserve/pull/4684
github.com/kserve/kserve/releases/tag/v0.16.0

SSL disabled in LLMInferenceService presets

AffectsLLMInferenceService deployments using the changed configuration presets

The presets previously enabled SSL with --enable-ssl-refresh and certificate and key arguments, and used HTTPS health probes. They now omit those SSL arguments and use HTTP health probes; the PR describes this as temporary while BackendTLSPolicy is not implemented.

Show release-note lines
* Temporarily disable SSL to unblock e2e tests by @Jooho in https://github.com/kserve/kserve/pull/4731
github.com/kserve/kserve/releases/tag/v0.16.0

LLMInferenceService presets set runAsNonRoot to false

AffectsLLMInferenceService deployments using the affected configuration presets(Inferred)

Before this release, the presets set runAsNonRoot to true.

Evidence“+ runAsNonRoot: false” kserve/kserve/pull/4731 ↗
Show release-note lines
* Temporarily disable SSL to unblock e2e tests by @Jooho in https://github.com/kserve/kserve/pull/4731
github.com/kserve/kserve/releases/tag/v0.16.0

Autoscaling metric quantities preserve their input representation

AffectsInferenceServices using autoscaling metric targets with value or averageValue, including KEDA compatibility paths.(Inferred)

Previously, JSON serialization could normalize value and averageValue, so the serialized representation differed from what was supplied. Serialization now uses the original raw input when available while retaining a parsed resource.Quantity for validation.

Evidence“// MarshalJSON implements custom JSON marshaling using the original raw value for KEDA compatibility” kserve/kserve/pull/4699 ↗
Show release-note lines
* Use a wrapper struct to accept resource.Quantity and keep the original input by @houshengbo in https://github.com/kserve/kserve/pull/4699
github.com/kserve/kserve/releases/tag/v0.16.0

S3 configuration sources and empty-value handling changed

AffectsS3 storage configured through service-account or Secret annotations, Secret data, or the inferenceservice-config ConfigMap

Previously, empty S3_ENDPOINT or S3_USE_HTTPS annotation values could still be set in the container environment. Now S3 values can also come from Secret data, with annotations taking precedence over Secret data and Secret data over the S3 config; empty strings from any source are omitted from the container environment.

Evidence“Regardless of where their value is set (annotation, secret data, or s3 config) they will always be set in the container env unless the value is an empty string.” kserve/kserve/pull/4727 ↗
Show release-note lines
* Add Support for Configuring S3 Storage via Secret Data by @brettmthompson in https://github.com/kserve/kserve/pull/4727
github.com/kserve/kserve/releases/tag/v0.16.0

llmisvc-resources chart adds inference service ConfigMap

Affectsthe llmisvc-resources Helm chart(Inferred)

The chart no longer includes gateway-inference-extension.yaml. Its new inferenceservice.yaml template defines the inferenceservice-config ConfigMap.

Evidence“+ name: inferenceservice-config” kserve/kserve/pull/4729 ↗
Show release-note lines
* Add llmd e2e tests by @andresllh in https://github.com/kserve/kserve/pull/4729
github.com/kserve/kserve/releases/tag/v0.16.0

LLMInferenceService CRDs and webhooks moved to dedicated Kustomize configuration

AffectsKustomize-based LLMInferenceService installations(Inferred)

The independent installation can be built with kustomize build config/overlay/llmisvc.

Show release-note lines
* refactor: separate llmisvc manifests from default kustomization by @Jooho in https://github.com/kserve/kserve/pull/4763
github.com/kserve/kserve/releases/tag/v0.16.0

KServe Python drops support for Pydantic v1

AffectsPython SDK setups using Pydantic v1(Inferred)

Previously, both Pydantic v1 and v2 were supported.

Evidence“Currently both v1 and v2 pydantic is supported, dropped support for v1 for the support overhead.” kserve/kserve/pull/4353 ↗
Show release-note lines
* chore: drop pydantic v1 support by @sivanantha321 in https://github.com/kserve/kserve/pull/4353
github.com/kserve/kserve/releases/tag/v0.16.0

Python Model class no longer exposes predictor_host or PredictorConfig

AffectsPython KServe SDK models using predictor_host or PredictorConfig on the Model class(Inferred)

Previously, the Python Model class exposed both attributes at model scope. PredictorConfig is now retrieved from context, and existing functionality is maintained without requiring it.

Evidence“Breaking: Remove predictor_host and PredictorConfig from the Model class” kserve/kserve/pull/4526 ↗
Show release-note lines
* Refactor KServe to use global context for PredictorConfig by @sivanantha321 in https://github.com/kserve/kserve/pull/4526
github.com/kserve/kserve/releases/tag/v0.16.0

Trained-model sharding Strategy interface removed

AffectsGo code that imports or refers to the trained-model sharding package’s Strategy interface(Inferred)

The interface declared GetOrAssignShard.

Show release-note lines
* Remove unused Strategy interface from sharding package by @houshengbo in https://github.com/kserve/kserve/pull/4590
github.com/kserve/kserve/releases/tag/v0.16.0

Creation blocked for standard predictors with a non-empty name

AffectsInferenceServices using standard predictors with a non-empty name field
Evidence“This PR adds validation to catch misconfiguration in the `name` field of standard predictor in creation time.” kserve/kserve/pull/4535 ↗
Show release-note lines
* disallow `name` field in standard predictor by @HutakiHare in https://github.com/kserve/kserve/pull/4535
github.com/kserve/kserve/releases/tag/v0.16.0

Parallelism CRD formats narrowed to int32

AffectsLLMInferenceService and LLMInferenceServiceConfig resources with parallelism.pipeline or parallelism.tensor values outside the int32 range(Inferred)
Evidence“+ format: int32” kserve/kserve/pull/4596 ↗
Show release-note lines
* update CRDs for LWS based multi node support by @VedantMahabaleshwarkar in https://github.com/kserve/kserve/pull/4596
github.com/kserve/kserve/releases/tag/v0.16.0

storage spec removed from LLMInferenceService CRDs and API types

AffectsLLMInferenceService or LLMInferenceServiceConfig manifests that set storage(Inferred)

The LLMInferenceService CRD previously included storage for compatibility with non-ModelMesh deployments.

Evidence“The Storage Spec was added to InferenceServices because this is the format that ModelMesh supports. For non-ModelMesh deployments, KServe added support for compatibility.” kserve/kserve/pull/4622 ↗
Show release-note lines
* Remove storage spec from LLMIsvc by @israel-hdez in https://github.com/kserve/kserve/pull/4622
github.com/kserve/kserve/releases/tag/v0.16.0

LLMInferenceService validation webhook added

AffectsLLMInferenceService resources with router routes or workload parallelism configured

It checks router cross-field constraints, including disallowing custom HTTPRoute references alongside an inline route spec, and validates workload parallelism constraints.

Show release-note lines
* feat(webhook): introduces Validating Webhook for LLMInferenceService by @VedantMahabaleshwarkar in https://github.com/kserve/kserve/pull/4631
github.com/kserve/kserve/releases/tag/v0.16.0

LLMInferenceService LoRA API now uses LLMModelSpec

Affectssetups using the LoRA API in the LLMInferenceService CRD

Previously, the LoRA API used the existing ModelSpec type.

Evidence“The LLMInferneceService CRD LoRA API wasn't meant to use the existing ModelSpec but the LLMModelSpec type” kserve/kserve/pull/4663 ↗
Show release-note lines
* [llmisvc] Improve config merge and update well-known presets by @pierDipi in https://github.com/kserve/kserve/pull/4663
github.com/kserve/kserve/releases/tag/v0.16.0

EnableDirectPvcVolumeMount flag removed

Affectsstorage-initializer configurations using EnableDirectPvcVolumeMount(Inferred)

The flag had defaulted to true for several releases. The PR describes its legacy false path as effectively dead, although its code and tests still existed.

Evidence“EnableDirectPvcVolumeMount has been defaulted to true for several releases, making the legacy false path effectively dead.” kserve/kserve/pull/4694 ↗
Show release-note lines
* deprecate: remove EnableDirectPvcVolumeMount flag by @anurags25 in https://github.com/kserve/kserve/pull/4694
github.com/kserve/kserve/releases/tag/v0.16.0

Validating webhooks added to the llmisvc-resources Helm chart

Affectsinstallations of the llmisvc-resources Helm chart managing LLMInferenceService or LLMInferenceServiceConfig resources(Inferred)

The validating webhooks cover CREATE and UPDATE operations.

Show release-note lines
* llmisvc dev & kustomize setup, add webhook config to helm chart  by @sivanantha321 in https://github.com/kserve/kserve/pull/4712
github.com/kserve/kserve/releases/tag/v0.16.0

Deprecations1

KServe deploy modes renamed to Standard and KNative

AffectsKServe configurations that set deploy-mode values or rely on the default mode
Removed innot stated

The default is now Standard. Legacy values remain as deprecated fields and are upconverted by the config constructor.

Show release-note lines
* refactor: 4602 - Refactor "RawDeployment" to "Standard" and "Serverless" to  "KNative" to clarify usage by @cjohannsen-cloudera in https://github.com/kserve/kserve/pull/4608
github.com/kserve/kserve/releases/tag/v0.16.0

Important fixes28

  • HuggingFace TextIteratorStreamer skips special tokens
    AffectsHuggingFace backend text-generation streaming

    With this functionality, streamed responses previously included special tokens such as <|im_end|> and <|endoftext|>.

    * fix: Update TextIteratorStreamer to skip special tokens by @sivanantha321 in https://github.com/kserve/kserve/pull/4490
    github.com/kserve/kserve/releases/tag/v0.16.0
  • InferenceGraph endpoint resolution honors the ServingRuntime protocol
    AffectsInferenceGraph workloads with an InferenceService using a V2-only ServingRuntime and no protocolVersion

    Previously, endpoint resolution returned a V1 endpoint in this case, misconfiguring the InferenceGraph router. The workload is now configured with the V2 endpoint.

    * Resolve inference endpoint using runtime protocol when applicable by @israel-hdez in https://github.com/kserve/kserve/pull/4527
    github.com/kserve/kserve/releases/tag/v0.16.0
  • KEDA OpenTelemetry scaling metadata corrected
    AffectsInferenceServices using KEDA with an OpenTelemetry metric trigger(Inferred)

    When KServe generated a KEDA OpenTelemetry metric trigger, it populated clampMin and clampMax with the InferenceService's replica bounds, even though those fields bound the scale itself. KServe-generated trigger metadata now omits both fields.

    * fix: unset clampMax and clampMin, since they are not for replicas by @houshengbo in https://github.com/kserve/kserve/pull/4556
    github.com/kserve/kserve/releases/tag/v0.16.0
  • HTTPRoute reconciliation requeue handling fixed
    AffectsInferenceServices using HTTPRoute ingress reconciliation(Inferred)

    When an HTTPRoute was missing or not ready during reconciliation, the reconciler could fail to requeue, and a not-ready route could be silently skipped. It now requeues for routes that are not ready or not found, supporting convergence to the desired state.

    * refactor: Enhance HTTPRoute readiness checks by @sivanantha321 in https://github.com/kserve/kserve/pull/4543
    github.com/kserve/kserve/releases/tag/v0.16.0
  • OpenTelemetryCollector autoscaling includes all specified metrics
    AffectsInferenceServices with multiple OpenTelemetryCollector autoscaling metrics(Inferred)

    Previously, only the last specified metric was included. Now all specified metrics are included for autoscaling.

    * Fix: Support multiple metrics in OpenTelemetryCollector for autoscaling by @houshengbo in https://github.com/kserve/kserve/pull/4591
    github.com/kserve/kserve/releases/tag/v0.16.0
  • KEDA autoscaling isolated by namespace and deployment
    AffectsInferenceService deployments using KEDA/OpenTelemetry autoscaling(Inferred)

    Metrics from different deployments or namespaces could be mixed, causing incorrect autoscaling. The OTelCollector now adds namespace and deployment labels, which KEDA queries use to filter metrics.

    * Fixed the issue of the same metrics across different deployments under different namespaces by @houshengbo in https://github.com/kserve/kserve/pull/4593
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Hugging Face classification probability handling fixed
    AffectsHugging Face classification tasks using return_probabilities=True(Inferred)

    With return_probabilities=True, sequence classification could return raw logits instead of probabilities, while fill-mask returned probabilities. Softmax is now applied consistently across task types.

    * feat: Add disable postprocessing option for raw logits by @sivanantha321 in https://github.com/kserve/kserve/pull/4566
    github.com/kserve/kserve/releases/tag/v0.16.0
  • InferenceService Stopped condition handling corrected
    AffectsInferenceServices reconciled by the stop/resume controller(Inferred)

    A missing Stopped condition is initialized to false. On resume, the condition is set to false only if it was previously true.

    * [Followup] Improve the stop/resume Inference Service tests and status by @hdefazio in https://github.com/kserve/kserve/pull/4636
    github.com/kserve/kserve/releases/tag/v0.16.0
  • GPU detection recognizes NVIDIA MIG resources
    Affectsworkloads with NVIDIA MIG GPU resource limits

    Resource names beginning with nvidia.com/mig identify workloads as GPU-enabled.

    * fix: Adds support for NVIDIA MIG GPU resource detection by @sivanantha321 in https://github.com/kserve/kserve/pull/4642
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Progressing and failed states reflected in component deployment status
    AffectsKServe component deployments(Inferred)
    * Last Deployment Status Should Reflect Deployment Status by @HotsauceLee in https://github.com/kserve/kserve/pull/4667
    github.com/kserve/kserve/releases/tag/v0.16.0
  • InferenceService model-copy status updated in all states
    AffectsInferenceServices with models in Pending, Loading, FailedToLoad, or Loaded states(Inferred)

    Previously, ModelCopies was updated only after a model reached Loaded, leaving totalCopies out of status while it was Pending, Loading, or FailedToLoad. Now, ModelCopies.TotalCopies is stored in every model state and counts ready pods.

    * Fix: Update ModelCopies.TotalCopies for all model states  by @hardik-menger in https://github.com/kserve/kserve/pull/4676
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Inference logging prioritizes populated embedded logger specs
    Affectsinference services with inference logging configured through an embedded logger spec and a global logger ConfigMap

    Previously, a populated embedded logger spec was ignored and the global ConfigMap was used instead. Now the embedded spec takes precedence, and the global configuration is used when the embedded spec is not populated.

    * 4739 - Fix: blob storage for inference logging recognizes embedded spec by @cjohannsen-cloudera in https://github.com/kserve/kserve/pull/4740
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Inference logging response metadata now includes request metadata headers
    Affectsinference logging requests with metadata headers(Inferred)

    Previously, requests carrying metadata headers had those headers omitted from the logged response metadata.

    * 4778 - Fix: apply metadata headers to inference logging response by @cjohannsen-cloudera in https://github.com/kserve/kserve/pull/4779
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Configurable router timeouts for InferenceGraph
    AffectsInferenceGraph resources needing custom router or service-client timeouts

    A short router server timeout could return EOF while graph InferenceServices were starting, for example during model downloads or Kubernetes node provisioning. Custom values under InferenceGraph.spec.routerTimeouts can avoid this; the source example sets serverRead, serverWrite, and serverIdle to 300 and reports a successful response.

    * Allow to set custom timeouts for `InferenceGraph` router by @lifo9 in https://github.com/kserve/kserve/pull/4218
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Standalone AWS_CA_BUNDLE paths honored for S3 SSL
    AffectsS3 storage with SSL verification enabled and AWS_CA_BUNDLE set to a local CA file(Inferred)

    Previously, the path was ignored during an S3 download with SSL verification enabled if CA_BUNDLE_CONFIGMAP_NAME was unset.

    * fix: Allow CA bundle path without config map by @fabiendupont in https://github.com/kserve/kserve/pull/4451
    github.com/kserve/kserve/releases/tag/v0.16.0
  • HuggingFaceServer FlashInfer build failure fixed
    AffectsHuggingFaceServer images running vLLM’s EngineCore with FlashInfer

    When EngineCore started and runtime JIT compilation ran, the Ninja build could fail with “RuntimeError: Ninja build failed.” The image now includes precompiled sampling kernels, avoiding that runtime build and failure.

    * Add AOT flashinfer build to huggingfaceserver dockerfile to precompil… by @AyushSawant18588 in https://github.com/kserve/kserve/pull/4567
    github.com/kserve/kserve/releases/tag/v0.16.0
  • LLMInferenceService types added to Python package exports
    AffectsPython SDK imports of LLMInferenceService or LLMInferenceServiceConfig(Inferred)

    These types were missing from Python imports.

    * fix(deps): adds missing LLMInferenceService types for python imports by @bartoszmajsak in https://github.com/kserve/kserve/pull/4604
    github.com/kserve/kserve/releases/tag/v0.16.0
  • ClusterServingRuntime image digest handling fixed
    AffectsClusterServingRuntime resources using image digests, especially in GPU clusters

    In GPU clusters, adding the -gpu postfix to an image digest could make the runtime fail. Digest-based image references are now left unchanged by image-tag updating.

    * fix: allow digests in runtimes' images by @tmvfb in https://github.com/kserve/kserve/pull/4653
    github.com/kserve/kserve/releases/tag/v0.16.0
  • LLMInferenceService controller RBAC permissions expanded
    Affectsclusters running the LLMInferenceService controller(Inferred)

    The added permissions support enhanced controller functionality and leader election.

    * llmisvc: fix RBAC, templating, and adds quick install script by @sivanantha321 in https://github.com/kserve/kserve/pull/4698
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Nil-pointer panic fixed in InferenceService reconciliation
    AffectsInferenceService deployment, HPA, or KEDA reconciliation(Inferred)

    When componentExt was nil during these reconciliation paths, InferenceService reconciliation could panic; it no longer does.

    * Fixed the panic nil pointer issue componentExt being nil by @houshengbo in https://github.com/kserve/kserve/pull/4704
    github.com/kserve/kserve/releases/tag/v0.16.0
  • llmisvc controller Dockerfile entrypoint corrected
    Affectsinstallations that build or run the llmisvc controller image from this Dockerfile(Inferred)

    The Dockerfile previously set the entrypoint to /llmisvc; it now sets it to /manager.

    * Fix incorrect entrypoint in llmisvc Dockerfile by @Jooho in https://github.com/kserve/kserve/pull/4730
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Reduced memory use for Hugging Face downloads
    AffectsHugging Face model downloads through the storage initializer

    Large-model downloads with the latest Hugging Face Hub Python library consistently ran out of memory. Storage initializer containers now receive HF_HUB_ENABLE_HF_TRANSFER=1, HF_XET_HIGH_PERFORMANCE=1, and HF_XET_NUM_CONCURRENT_RANGE_GETS=8; in the reported test, memory use fell from more than 2 GB to less than 0.5 GB while download time stayed consistent.

    * Configure HF Downloads to Lower Memory Usage by @brettmthompson in https://github.com/kserve/kserve/pull/4726
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Default Kustomize controller image reference corrected
    Affectsinstallations using the default Kustomize manager image patch(Inferred)

    The deployment failed with an image-pull back-off when it used kserve-controller:latest; it now uses kserve/kserve-controller:latest.

    * fix: wrong kserve controller image url by @Jooho in https://github.com/kserve/kserve/pull/4773
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Storage initializer duplicate environment variable fix
    Affectsstorage initialization with customized storage containers defining HuggingFace optimization or CA bundle environment variables(Inferred)

    Existing matching user-defined environment variables are retained without an automatically appended duplicate. Default HuggingFace optimization variables are added only when absent.

    * Fix the issue of Invalid value for env[1].valueFrom by @houshengbo in https://github.com/kserve/kserve/pull/4765
    github.com/kserve/kserve/releases/tag/v0.16.0
  • XGBoost loading of .ubj and .json models from modelcars fixed
    AffectsXGBoost server deployments loading .ubj or .json models from modelcars

    These loads failed because the model paths in modelcars are in a virtual filesystem. The server now passes file contents to XGBoost as a bytearray rather than having the library read the virtual path.

    * Fix XGBoost .ubj and .json models with modelcar by @balazsprehoda in https://github.com/kserve/kserve/pull/4746
    github.com/kserve/kserve/releases/tag/v0.16.0
  • CA bundle injection preserves existing environment variables
    AffectsLLMInferenceService workloads using CA bundle injection(Inferred)

    When a CA bundle ConfigMap-name or volume-mount-path environment variable already exists, injection leaves it in place rather than adding another; otherwise, it adds the configured or default value.

    * Fixed the CA bundle env var logic by @houshengbo in https://github.com/kserve/kserve/pull/4777
    github.com/kserve/kserve/releases/tag/v0.16.0
  • HuggingFace mean pooling runtime errors fixed
    AffectsHuggingFace runtime deployments whose models use mean pooling for sentence embeddings, including CPU-only and GPU-enabled deployments

    When attention_mask and token_embeddings were on different devices or had mismatched data types, mean pooling could raise runtime errors. It now aligns the mask with the embeddings' device and dtype, handles a missing mask, and clamps the denominator to prevent division by zero, supporting inference across CPU and GPU.

    * huggingfaceserver: fix mean_pooling device/dtype mismatch and add regression test by @JohnsonWang1015 in https://github.com/kserve/kserve/pull/4744
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Optional prediction-to-instance mapping for InferenceGraph steps
    AffectsInferenceGraph steps chaining V1 Inference Protocol services

    When an upstream service returned output under predictions but the downstream service expected instances, straightforward chaining was prevented. With mapPredictionsToInstances set to true, the router maps upstream predictions to downstream instances; without that setting, the mapping is not enabled.

    * Allow mapping predictions to instances in sequential inference graphs by @tmbochenski in https://github.com/kserve/kserve/pull/4738
    github.com/kserve/kserve/releases/tag/v0.16.0

Enhancements43

  • Per-runtime disable settings for built-in serving runtimes

    To use Set each runtime's disabled setting to true to disable it; it defaults to false.

    * Update kserve-resources helm chart to disable desired servingruntimes by @jmlaubach in https://github.com/kserve/kserve/pull/4485
    github.com/kserve/kserve/releases/tag/v0.16.0
  • KServe storage module released as a separate kserve-storage package
    * Initial segregation of the storage module from KServe SDK by @spolti in https://github.com/kserve/kserve/pull/4391
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Stop and resume Raw Deployment InferenceServices

    To use Set serving.kserve.io/stop to 'true' to stop a service and to 'false' to resume it.

    * Stop and resume a model [Raw Deployment] by @hdefazio in https://github.com/kserve/kserve/pull/4455
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Original model source URI injected into user containers

    To use Remote storage enabled

    * feat: support remote storage URI injection for serving runtimes by @de0725 in https://github.com/kserve/kserve/pull/4492
    github.com/kserve/kserve/releases/tag/v0.16.0
  • LLMInferenceService and LLMInferenceServiceConfig APIs and CRDs added
    * [API] Define LLMInferenceService and LLMInferenceServiceConfig types and CRDs by @pierDipi in https://github.com/kserve/kserve/pull/4522
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Transformer stop and resume support

    To use serving.kserve.io/stop annotation

    * Stop and Resume a transformer by @hdefazio in https://github.com/kserve/kserve/pull/4534
    github.com/kserve/kserve/releases/tag/v0.16.0
  • OCI storage validation enabled for multi-node/multi-GPU InferenceServices
    * Allow OCI for multi-node/multi-gpu by @israel-hdez in https://github.com/kserve/kserve/pull/4441
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Opt-in S3 storage for model request and response logs
    * 4380 - Inference logging to blob storage by @cjohannsen-cloudera in https://github.com/kserve/kserve/pull/4473
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Auto-update opt-out for ready InferenceServices
    * Auto-update annotation for isvc. by @andresllh in https://github.com/kserve/kserve/pull/4342
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Explainer stop and resume support

    To use constants.StopAnnotationKey on the InferenceService, set to true to stop or false to resume

    * Stop and resume an explainer by @hdefazio in https://github.com/kserve/kserve/pull/4546
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Storage initializer resource requests and limits configurable in the KServe Helm chart
    * feat: refactor storage initializer resources configuration by @takamai06 in https://github.com/kserve/kserve/pull/4411
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Initial LLMInferenceService controller scaffold and Helm charts
    * llmisvc: Initial controller scaffold and helm chart by @sivanantha321 in https://github.com/kserve/kserve/pull/4557
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Strategic merging for LLMInferenceServiceSpec
    * Add logic to merge specs for LLMInferenceService  by @VedantMahabaleshwarkar in https://github.com/kserve/kserve/pull/4563
    github.com/kserve/kserve/releases/tag/v0.16.0
  • KEDA external metric source authentication configuration
    * feat: support secure access to prometheus in keda by @andyi2it in https://github.com/kserve/kserve/pull/4384
    github.com/kserve/kserve/releases/tag/v0.16.0
  • utils collection helpers moved to collections.go
    * feat: improves collection helpers by @bartoszmajsak in https://github.com/kserve/kserve/pull/4579
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Hugging Face server raw-logit output

    To use The --return_raw_logits option, which is supported for sequence_classification, fill_mask, and token_classification and is mutually exclusive with return_probabilities.

    * feat: Add disable postprocessing option for raw logits by @sivanantha321 in https://github.com/kserve/kserve/pull/4566
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Stop and resume for serverless InferenceGraphs

    To use Set the serving.kserve.io/stop annotation to "true" to stop the graph or "false" to resume it.

    * Stop and resume an inference graph by @hdefazio in https://github.com/kserve/kserve/pull/4588
    github.com/kserve/kserve/releases/tag/v0.16.0
  • LWS-based multi-node fields added to LLMInferenceService and LLMInferenceServiceConfig CRDs
    * update CRDs for LWS based multi node support by @VedantMahabaleshwarkar in https://github.com/kserve/kserve/pull/4596
    github.com/kserve/kserve/releases/tag/v0.16.0
  • LLMInferenceServiceConfig base presets added
    * Add LLM InferenceService base configurations by @VedantMahabaleshwarkar in https://github.com/kserve/kserve/pull/4613
    github.com/kserve/kserve/releases/tag/v0.16.0
  • llm-d scheduler reconciliation for LLM load balancing
    * Adding Llm-d scheduler reconcilier by @andresllh in https://github.com/kserve/kserve/pull/4614
    github.com/kserve/kserve/releases/tag/v0.16.0
  • KEDA autoscaling behavior configuration
    * Added advanced config for ScaleUp and ScaleDown by @andyi2it in https://github.com/kserve/kserve/pull/4570
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Stop and resume raw-deployment-mode InferenceGraphs

    To use Set the InferenceGraph annotation serving.kserve.io/stop: "true" to stop it, or serving.kserve.io/stop: "false" to resume it.

    * Stop and resume an inference graph [Raw Deployment] by @hdefazio in https://github.com/kserve/kserve/pull/4637
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Single- and multi-node LLMInferenceServices workload reconcilers added
    * llm-d workload reconciliation by @brettmthompson in https://github.com/kserve/kserve/pull/4616
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Route reconciliation for LLMInferenceServices
    * Llm d http route reconciler by @andresllh in https://github.com/kserve/kserve/pull/4617
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Some LLMInferenceService controller logic added
    * Add logic for LLMISVC controller by @VedantMahabaleshwarkar in https://github.com/kserve/kserve/pull/4632
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Updated LLMInferenceServiceConfig well-known presets
    * [llmisvc] Improve config merge and update well-known presets by @pierDipi in https://github.com/kserve/kserve/pull/4663
    github.com/kserve/kserve/releases/tag/v0.16.0
  • LLMInferenceService configuration merging handles missing schedulers and external InferencePool objects
    * [llmisvc] Improve config merge and update well-known presets by @pierDipi in https://github.com/kserve/kserve/pull/4663
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Generic CRUD support for cluster-scoped resources and nil owners
    * [llmisvc] Support cluster-scoped objects in generic CRUD functions by @pierDipi in https://github.com/kserve/kserve/pull/4664
    github.com/kserve/kserve/releases/tag/v0.16.0
  • KServe Storage moved into a separate module
    * Promote new KServe Storage module by @spolti in https://github.com/kserve/kserve/pull/4625
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Configurable rollout strategies for raw and Standard deployments

    To use Configure a component’s deploymentStrategy or the ConfigMap’s deploymentRolloutStrategy or defaultRollout; ConfigMap defaults apply only when defaultDeploymentMode is "Standard".

    * Implement the progressive rollout for raw deployment by @houshengbo in https://github.com/kserve/kserve/pull/4623
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Published LLMInferenceService controller image and Helm chart
    * ci: publish llmisvc container image by @yuzisun in https://github.com/kserve/kserve/pull/4686
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Quick-install script for LLMInferenceService dependencies and components
    * llmisvc: fix RBAC, templating, and adds quick install script by @sivanantha321 in https://github.com/kserve/kserve/pull/4698
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Configurable OpenTelemetry and autoscaler settings in the kserve-resources Helm chart

    To use The kserve-resources Helm chart with the relevant kserve.opentelemetryCollector or kserve.autoscaler values set

    * Enabled the configuration options in Helm for opentelemetryCollector and autoscaler by @houshengbo in https://github.com/kserve/kserve/pull/4725
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Time-series forecasting endpoint and task integration
    * Time Series Forecast API Endpoint by @jinan-zhou in https://github.com/kserve/kserve/pull/4615
    github.com/kserve/kserve/releases/tag/v0.16.0
  • LLMInferenceService Kustomize deployment configuration
    * llmisvc dev & kustomize setup, add webhook config to helm chart  by @sivanantha321 in https://github.com/kserve/kserve/pull/4712
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Multiple storage locations for inference artifacts

    To use Kubernetes init-container feature flag for Knative deployments

    * Support Multiple Storage URIs for InferenceServices by @anurags25 in https://github.com/kserve/kserve/pull/4702
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Cert Manager installation added to the LLMInferenceService quick install script
    * Add Cert Manager installation to llmisvc quick install script by @sivanantha321 in https://github.com/kserve/kserve/pull/4733
    github.com/kserve/kserve/releases/tag/v0.16.0
  • CA bundles injected into LLMInferenceService S3 storage-init containers

    To use CA-bundle configuration defined in a service account, secret annotations, or the inferenceservice-config configmap

    * Injecting CA Bundle Into Storage Initializer Container for S3 Storage on LLMISVC Reconciliation by @brettmthompson in https://github.com/kserve/kserve/pull/4728
    github.com/kserve/kserve/releases/tag/v0.16.0
  • LLMInferenceService Kueue metadata propagation
    * Add metadata propagation for Kueue configurations to both Deployment and LeaderWorkerSet workloads by @hdefazio in https://github.com/kserve/kserve/pull/4747
    github.com/kserve/kserve/releases/tag/v0.16.0
  • GCS and Azure inference logging storage support
    * feature: 4553 - Support inference logging to GCS and Azure by @cjohannsen-cloudera in https://github.com/kserve/kserve/pull/4582
    github.com/kserve/kserve/releases/tag/v0.16.0
  • Modular setup scripts for KServe dependencies
    * Add centralized installation scripts for KServe dependencies by @Jooho in https://github.com/kserve/kserve/pull/4774
    github.com/kserve/kserve/releases/tag/v0.16.0
  • KServe v0.16.0 install manifests and Helm charts

    To use Helm --version v0.16.0

    * Prepare for 0.16.0 release by @houshengbo in https://github.com/kserve/kserve/pull/4786
    github.com/kserve/kserve/releases/tag/v0.16.0
  • New Helm chart for minimal LLMInferenceService CRDs

    To use Install the llmisvc-crd-minimal Helm chart at version v0.16.0 to deploy the minimal LLMInferenceService CRDs.

    * fix llmisvc-crd-minimal chart by @yuzisun in https://github.com/kserve/kserve/pull/4789
    github.com/kserve/kserve/releases/tag/v0.16.0

Dependency updates8

    8 more, none mentioning security
    • * Upgrade Torch to v2.6.0 everywhere by @ashahba in https://github.com/kserve/kserve/pull/4450
    • * upgrade vllm to v0.9.0 and Torch to v2.7.0 by @ashahba in https://github.com/kserve/kserve/pull/4501
    • * upgrade vllm to v0.9.0 and Torch to v2.7.0 by @ashahba in https://github.com/kserve/kserve/pull/4501
    • * Upgrade vLLM to v0.9.0.1 by @ashahba in https://github.com/kserve/kserve/pull/4507
    • * Upgrade vllm to v0.9.2 by @AyushSawant18588 in https://github.com/kserve/kserve/pull/4586
    • * Protobuf version bump by @karolpustelnik in https://github.com/kserve/kserve/pull/4634
    • * Update the kserve-storage module to the latest version by @spolti in https://github.com/kserve/kserve/pull/4754
    • * bump up istio version to 1.27.1 by @Jooho in https://github.com/kserve/kserve/pull/4775
    Other changes49

    Summaries are produced by AI, so they can be wrong, and they are provided without warranty.