dependabot[bot] opened a new pull request, #40883: URL: https://github.com/apache/beam/pull/40883
Bumps [vllm](https://github.com/vllm-project/vllm) from 0.10.1.1 to 0.32.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/vllm-project/vllm/releases";>vllm's releases</a>.</em></p> <blockquote> <h1>v0.32.0</h1> <h2>Highlights</h2> <p>This release features 814 commits from 363 contributors (121 new)!</p> <ul> <li><strong>Structured decisions and generic parsers</strong>: the new <code>POST /v1/systemone</code> endpoint answers <code>choice</code>, <code>noul</code> and <code>score</code> questions over a structured state (<a href="https://redirect.github.com/vllm-project/vllm/issues/59299";>#59299</a>), is served by default (<a href="https://redirect.github.com/vllm-project/vllm/issues/60677";>#60677</a>, <a href="https://redirect.github.com/vllm-project/vllm/issues/60696";>#60696</a>), and also runs on <code>Cloudflare/clef-omni</code> (<a href="https://redirect.github.com/vllm-project/vllm/issues/61064";>#61064</a>), <code>LiquidAI/d1-3B</code> (<a href="https://redirect.github.com/vllm-project/vllm/issues/60676";>#60676</a>) and DiffusionGemma (<a href="https://redirect.github.com/vllm-project/vllm/issues/60466";>#60466</a>). The new <code>hf</code> tool and reasoning parser (<code>--tool-call-parser hf</code>, <code>--reasoning-parser hf</code>) reads the checkpoint's <code>response_temp late</code>, so models that ship it need no model-specific parser (<a href="https://redirect.github.com/vllm-project/vllm/issues/58604";>#58604</a>, <a href="https://redirect.github.com/vllm-project/vllm/issues/58602";>#58602</a>), in both the Python and Rust frontends (<a href="https://redirect.github.com/vllm-project/vllm/issues/59005";>#59005</a>).</li> <li><strong>Low-precision KV cache everywhere</strong>: NVFP4 KV cache now runs on SM8x and SM12x (<a href="https://redirect.github.com/vllm-project/vllm/issues/46963";>#46963</a>), SM90 (<a href="https://redirect.github.com/vllm-project/vllm/issues/60620";>#60620</a>) and with XQA decode on SM12x (<a href="https://redirect.github.com/vllm-project/vllm/issues/60452";>#60452</a>), and on MiniMax-M3 sparse attention on SM100 (<a href="https://redirect.github.com/vllm-project/vllm/issues/59300";>#59300</a>); GLM-5.x defaults to an FP8 KV cache on SM100, halving KV memory per rank with 2.3-5.5% more throughput at a 7-11% TTFT cost in the PR's benchmark (<a href="https://redirect.github.com/vllm-project/vllm/issues/60140";>#60140</a>, <a href="https://redirect.github.com/vllm-project/vllm/issues/60281";>#60281</a>); <code>fp8_ds_mla</code> on SM90 gives GLM-5.3-Flash up to 31% more output throughput (<a href="https://redirect.github.com/vllm-project/vllm/issues/59246";>#59246</a>); ROCm gains the 4-bit <code>ultraquant_4bit</code> KV cache with about 2x the density of FP8 (<a href="https://redirect.github.com/vllm-project/vllm/issues/57057";>#57057</a>).</li> <li><strong>Model Runner V2</strong>: a GPU n-gram speculator with variable-length drafts (<a href="https://redirect.github.com/vllm-project/vllm/issues/40704";>#40704</a>), Elastic EP (<a href="https://redirect.github.com/vllm-project/vllm/issues/53934";>#53934</a>), pipeline parallelism with prefill context parallelism for MLA models (<a href="https://redirect.github.com/vllm-project/vllm/issues/59139";>#59139</a>), a stock torch.compile mode (<a href="https://redirect.github.com/vllm-project/vllm/issues/59079";>#59079</a>), and MRV2 is now the default on ROCm with the model-specific MRV1 exceptions removed (<a href="https://redirect.github.com/vllm-project/vllm/issues/56360";>#56360</a>).</li> <li><strong>Faster startup and serving</strong>: <code>vllm download-kernels</code> installs FlashInfer's precompiled kernels (<a href="https://redirect.github.com/vllm-project/vllm/issues/58765";>#58765</a>); <code>vllm preload --preload-autotune</code> keeps the FlashInfer autotune table warm across restarts (<a href="https://redirect.github.com/vllm-project/vllm/issues/60085";>#60085</a>); a shared-memory tensor arena broadcasts large multimodal tensors to workers with one copy (<a href="https://redirect.github.com/vllm-project/vllm/issues/51207";>#51207</a>); tokenizers and processors load from the local snapshot (<a href="https://redirect.github.com/vllm-project/vllm/issues/60825";>#60825</a>); and with <code>--api-server-count</code> above 1 each connection goes to the least-loaded API worker, for 43-85% more req/s on short requests (Qwen3-0.6B, 32 input / 1 output tokens) in the PR's benchmark (<a href="https://redirect.github.com/vllm-project/vllm/issues/60723";>#60723</a>).</l i> <li><strong>Model support</strong>: new EmbeddingGemma 2 omni-modal embeddings (<a href="https://redirect.github.com/vllm-project/vllm/issues/60254";>#60254</a>), MiniCPM-V 4.7 (<a href="https://redirect.github.com/vllm-project/vllm/issues/58674";>#58674</a>) and Nemotron 3.5 ASR (<a href="https://redirect.github.com/vllm-project/vllm/issues/59827";>#59827</a>); video input in the Transformers modeling backend (<a href="https://redirect.github.com/vllm-project/vllm/issues/57441";>#57441</a>); GLM-5.3 sparse MLA index conversions reused across layers (3.5-3.9x on that kernel, <a href="https://redirect.github.com/vllm-project/vllm/issues/59464";>#59464</a>) and fused multi-step draft decode (13.3% at concurrency 1, <a href="https://redirect.github.com/vllm-project/vllm/issues/57443";>#57443</a>); HiSparse per-request residency caching for up to 58% more throughput on GLM-5.3-NVFP4 on B200 with long, fully GPU-resident contexts at high concurrency (<a href="https://redirect.github.com/vllm -project/vllm/issues/60083">#60083</a>).</li> <li><strong>Security</strong>: fixes for GHSA-4hhp-h66f-j5j7 (chat template resource exhaustion, <a href="https://redirect.github.com/vllm-project/vllm/issues/50300";>#50300</a>), GHSA-q97c-p368-75c4 (tokenizer <code>max_token_id</code>, <a href="https://redirect.github.com/vllm-project/vllm/issues/59491";>#59491</a>) and GHSA-frvg-qw88-hww9 (P2P ZMQ socket exhaustion, <a href="https://redirect.github.com/vllm-project/vllm/issues/51504";>#51504</a>); CORS no longer allows every origin by default (<a href="https://redirect.github.com/vllm-project/vllm/issues/61009";>#61009</a>); untrusted images are limited to raster formats (<a href="https://redirect.github.com/vllm-project/vllm/issues/60022";>#60022</a>); shared-memory multimodal cache handles are signed (<a href="https://redirect.github.com/vllm-project/vllm/issues/59357";>#59357</a>); and JSON schemas nested deeper than 64 levels are rejected (<a href="https://redirect.github.com/vllm-project/vllm/issues/60036";>#60036</a>).</li> <li><strong>Breaking changes</strong>: Python 3.10 dropped (<a href="https://redirect.github.com/vllm-project/vllm/issues/60402";>#60402</a>); default CORS origins restricted (<a href="https://redirect.github.com/vllm-project/vllm/issues/61009";>#61009</a>); <code>transformers>=5.16.1,<5.20.0</code> and <code>tokenizers>=0.23.2</code> (<a href="https://redirect.github.com/vllm-project/vllm/issues/57387";>#57387</a>, <a href="https://redirect.github.com/vllm-project/vllm/issues/59796";>#59796</a>); torch.compile sequence parallelism and AsyncTP passes removed (<a href="https://redirect.github.com/vllm-project/vllm/issues/60517";>#60517</a>); cascade attention removed (<a href="https://redirect.github.com/vllm-project/vllm/issues/61126";>#61126</a>); <code>mamba_cache_mode="all"</code> removed (<a href="https://redirect.github.com/vllm-project/vllm/issues/58997";>#58997</a>); <code>max_tokens</code> no longer reserved from the context window (<a href="https://redirect.gi thub.com/vllm-project/vllm/issues/57035">#57035</a>); Responses API reasoning stream events renamed (<a href="https://redirect.github.com/vllm-project/vllm/issues/59652";>#59652</a>).</li> </ul> <h2>Release Artifacts</h2> <h3>Python Wheels</h3> <table> <thead> <tr> <th>Platform</th> <th>Install</th> </tr> </thead> <tbody> <tr> <td>PyPI (CUDA 13.0)</td> <td><code>pip install vllm</code></td> </tr> <tr> <td>PyPI (CUDA 13.0, uv)</td> <td><code>uv pip install vllm --torch-backend=auto</code></td> </tr> <tr> <td>ROCm</td> <td><code>pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.32.0/rocm723</code></td> </tr> <tr> <td>XPU</td> <td><code>uv pip install vllm --extra-index-url https://wheels.vllm.ai/0.32.0/xpu --extra-index-url https://download.pytorch.org/whl/xpu --index-strategy unsafe-best-match</code></td> </tr> </tbody> </table> <h3>Docker Images</h3> <table> <thead> <tr> <th>Platform</th> <th>Docker Image</th> </tr> </thead> <tbody> <tr> <td>CUDA 13.0 (Default)</td> <td><code>docker pull vllm/vllm-openai:v0.32.0</code></td> </tr> <tr> <td>CUDA 12.9</td> <td><code>docker pull vllm/vllm-openai:v0.32.0-cu129</code></td> </tr> <tr> <td>CUDA 13.0 + Ubuntu 24.04</td> <td><code>docker pull vllm/vllm-openai:v0.32.0-ubuntu2404</code></td> </tr> <tr> <td>CUDA 12.9 + Ubuntu 24.04</td> <td><code>docker pull vllm/vllm-openai:v0.32.0-cu129-ubuntu2404</code></td> </tr> <tr> <td>ROCm</td> <td><code>docker pull vllm/vllm-openai-rocm:v0.32.0</code></td> </tr> <tr> <td>CPU</td> <td><code>docker pull vllm/vllm-openai-cpu:v0.32.0</code></td> </tr> <tr> <td>XPU</td> <td><code>docker pull vllm/vllm-openai-xpu:v0.32.0</code></td> </tr> </tbody> </table> <h3>Other Artifacts</h3> <p>Pre-built release artifacts are available in the <strong>Assets</strong> section at the bottom of this page, including:</p> <ul> <li>Source distribution tarball</li> <li>CUDA 12.9 Python wheels for x86_64 and arm64</li> <li>CUDA 13.0 Python wheels for x86_64 and arm64</li> <li>CPU Python wheels for x86_64, arm64, and macOS</li> <li>XPU Python wheel for x86_64</li> </ul> <h2>Model Support</h2> <ul> <li><strong>New models</strong>: EmbeddingGemma 2 (<code>google/embeddinggemma-2</code>), an omni-modal text/image/audio/video embedding model, with unused modality encoders skipped through <code>--limit-mm-per-prompt</code> (<a href="https://redirect.github.com/vllm-project/vllm/issues/60254";>#60254</a>, <a href="https://redirect.github.com/vllm-project/vllm/issues/60289";>#60289</a>); MiniCPM-V 4.7 (<a href="https://redirect.github.com/vllm-project/vllm/issues/58674";>#58674</a>); Nemotron 3.5 ASR (<code>nvidia/nemotron-3.5-asr-streaming-0.6b</code>) complete-audio transcription; native realtime streaming is not yet supported (<a href="https://redirect.github.com/vllm-project/vllm/issues/59827";>#59827</a>); <code>Cloudflare/clef-omni</code> (<a href="https://redirect.github.com/vllm-project/vllm/issues/61064";>#61064</a>) and <code>LiquidAI/d1-3B</code> (<a href="https://redirect.github.com/vllm-project/vllm/issues/60676";>#60676</a>) on <code>/v1/systemone</code>; PLaMo 3 reasoning and tool parsers (<a href="https://redirect.github.com/vllm-project/vllm/issues/56575";>#56575</a>); and the zerank-2 reranker score template (<a href="https://redirect.github.com/vllm-project/vllm/issues/60415";>#60415</a>).</li> <li><strong>Transformers modeling backend</strong>: video inputs (<a href="https://redirect.github.com/vllm-project/vllm/issues/57441";>#57441</a>), and Glm, Arcee, CWM, Mellum (<a href="https://redirect.github.com/vllm-project/vllm/issues/59679";>#59679</a>), GPT-NeoX, Phi, Seed-OSS and Jais2 (<a href="https://redirect.github.com/vllm-project/vllm/issues/59701";>#59701</a>) now served through it.</li> <li><strong>New model capabilities</strong>: LoRA for BERT, RoBERTa and XLM-RoBERTa embedding models (<a href="https://redirect.github.com/vllm-project/vllm/issues/61033";>#61033</a>); Granite as an EAGLE3/DFlash target (<a href="https://redirect.github.com/vllm-project/vllm/issues/60855";>#60855</a>); Qwen3-ASR EAGLE3 (<a href="https://redirect.github.com/vllm-project/vllm/issues/52824";>#52824</a>); Sarvam MLA EAGLE3/DSpark with PP (<a href="https://redirect.github.com/vllm-project/vllm/issues/55902";>#55902</a>); GLM-5.3-Flash with the DFlash2 drafter (<a href="https://redirect.github.com/vllm-project/vllm/issues/56983";>#56983</a>); and MiniMax-M3 EAGLE3 at PP > 1 on MRV2 (<a href="https://redirect.github.com/vllm-project/vllm/issues/57197";>#57197</a>) with MTP sharing target embeddings under PP (<a href="https://redirect.github.com/vllm-project/vllm/issues/58648";>#58648</a>).</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/vllm-project/vllm/commit/5df63a7203cdb9328d824a27b682e77c7b9928be";><code>5df63a7</code></a> [Bugfix][Rust Frontend] Fix macOS build: read inherited listener family from ...</li> <li><a href="https://github.com/vllm-project/vllm/commit/82eb574d959031c670ecbc9930ed7b756a119bfe";><code>82eb574</code></a> [Bugfix] Fix NaN output of the Mamba2 chunk scan kernel on sm90 (<a href="https://redirect.github.com/vllm-project/vllm/issues/61718";>#61718</a>)</li> <li><a href="https://github.com/vllm-project/vllm/commit/1ae8f0868232c8185775209cf614208d6fa465d0";><code>1ae8f08</code></a> [Bugfix] Make Humming DeviceInfo lookups traceable under torch.compile (<a href="https://redirect.github.com/vllm-project/vllm/issues/61858";>#61858</a>)</li> <li><a href="https://github.com/vllm-project/vllm/commit/0c861334d8149740ce11e8d271d69fa1d7009430";><code>0c86133</code></a> [Frontend] Replace return_mm_kwargs with an mm_payload enum on render request...</li> <li><a href="https://github.com/vllm-project/vllm/commit/7a27e9b88788e704ad39fe0e002f174032cd6ccb";><code>7a27e9b</code></a> [CI] Run the Clef pooler test on GPU (<a href="https://redirect.github.com/vllm-project/vllm/issues/61409";>#61409</a>)</li> <li><a href="https://github.com/vllm-project/vllm/commit/d8751249ca19d37542526199077c102ea91533ce";><code>d875124</code></a> [Perf][Bugfix] Prefer MNNVL for single-node FlashInfer allreduce again (<a href="https://redirect.github.com/vllm-project/vllm/issues/61989";>#61989</a>)</li> <li><a href="https://github.com/vllm-project/vllm/commit/7960eeb9de293137b7fca35cd4ae92c4c50ea286";><code>7960eeb</code></a> [CI] Pin DeepSeek-V4.1-Flash-NVFP4 eval to its FP8-Engram revision (<a href="https://redirect.github.com/vllm-project/vllm/issues/61763";>#61763</a>)</li> <li><a href="https://github.com/vllm-project/vllm/commit/f5f79df2f0ec0ae091fce0f15f06e80df0a9302e";><code>f5f79df</code></a> [CI] Fix Unlimited-OCR loading after upstream <code>model_type</code> rename (<a href="https://redirect.github.com/vllm-project/vllm/issues/61590";>#61590</a>)</li> <li><a href="https://github.com/vllm-project/vllm/commit/ba899a91534b7223aa3706061093722764249564";><code>ba899a9</code></a> [XPU][MLA] Integrate XPU MLA support into the existing FLASH_ATTN_MLA backend...</li> <li><a href="https://github.com/vllm-project/vllm/commit/fc1411646e3bc0f0c5d7bfec36d6e1a48ef0a46a";><code>fc14116</code></a> [Dependency] Update Humming and remove NVFP4 scale workaround (<a href="https://redirect.github.com/vllm-project/vllm/issues/61556";>#61556</a>)</li> <li>Additional commits viewable in <a href="https://github.com/vllm-project/vllm/compare/v0.10.1.1...v0.32.0";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/beam/network/alerts). </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
