dependabot[bot] opened a new pull request, #3909: URL: https://github.com/apache/iceberg-python/pull/3909
Bumps [ray](https://github.com/ray-project/ray) from 2.57.0 to 2.58.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/ray-project/ray/releases";>ray's releases</a>.</em></p> <blockquote> <h2>Ray-2.58.0</h2> <h1>Highlights</h1> <ul> <li><strong>Ray Serve LLM:</strong> In this release we've completed KV cache and token aware request routing, which was previewed in 2.57. Tokenization now happens in-process on the <code>LLMRouter</code> ingress replica, the routing decision is made there, tokens are transmitted out-of-band so the engine does not re-tokenize, KV lifecycle events are broadcast to every ingress replica (<a href="https://redirect.github.com/ray-project/ray/issues/64642";>#64642</a>, <a href="https://redirect.github.com/ray-project/ray/issues/64920";>#64920</a>, <a href="https://redirect.github.com/ray-project/ray/issues/64949";>#64949</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65010";>#65010</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65095";>#65095</a>). KV cache and token aware routing is also aware of CPU KV caches, so offloaded KV cache blocks count toward a replica's cache hit (<a href="https://redirect.github.com/ray-project/ray/issues/65063";>#65063</a>).< /li> <li><strong>Ray Core:</strong> We enabled the capability to offload task events from. With <code>RAY_enable_task_events_to_dashboard_head</code> on, the task event buffer is replaced by the ray event recorder, events are exported from the aggregator agent to a task events head that keeps an in-memory store, and the state APIs and <code>ray.timeline</code> read from it (<a href="https://redirect.github.com/ray-project/ray/issues/64835";>#64835</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65028";>#65028</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65123";>#65123</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65160";>#65160</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65218";>#65218</a>). Enabling the feature removes task event ingestion and serving from the GCS hot path.</li> <li><strong>Ray Data:</strong> We’ve added Databricks integrations for writing to DeltaLake, with Catalog support. We’ve also shipped a new shuffle v2 backend, featuring improved performance for joins and aggregations.</li> <li><strong>Sandboxing:</strong> We've also added experimental Ray Sandbox, which runs task and actor code under gVisor and can run Docker-built images directly (<a href="https://redirect.github.com/ray-project/ray/issues/64964";>#64964</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65570";>#65570</a>).</li> <li><strong>TPU Support:</strong> Ray Train adds support for TorchTPU backend (<a href="https://redirect.github.com/ray-project/ray/issues/64796";>#64796</a>), and Ray Core adds <code>SubslicePlacementGroup</code> for gang scheduling on TPU subslices, single-host TPU support in <code>SlicePlacementGroup</code>, and resource accounting for <code>tpu7x</code> and multi-core chips (<a href="https://redirect.github.com/ray-project/ray/issues/64578";>#64578</a>, <a href="https://redirect.github.com/ray-project/ray/issues/64079";>#64079</a>, <a href="https://redirect.github.com/ray-project/ray/issues/64058";>#64058</a>). This lets TPU slices and subslices be reserved and trained on without external gang-scheduling glue.</li> </ul> <h1>Ray Data</h1> <h3>🎉 New Features</h3> <ul> <li>Add <code>Dataset.with_columns</code> for multi-column expression projection (<a href="https://redirect.github.com/ray-project/ray/issues/63858";>#63858</a>)</li> <li>Add <code>write_delta</code> for Delta Lake, with catalog support (<a href="https://redirect.github.com/ray-project/ray/issues/64923";>#64923</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65079";>#65079</a>)</li> <li>Add Torch inference API (<a href="https://redirect.github.com/ray-project/ray/issues/65157";>#65157</a>)</li> <li>Promote hash shuffle v2 to a selectable shuffle strategy, with aggregation support, vectorized aggregation, and block splitting during aggregation (<a href="https://redirect.github.com/ray-project/ray/issues/64953";>#64953</a>, <a href="https://redirect.github.com/ray-project/ray/issues/64652";>#64652</a>, <a href="https://redirect.github.com/ray-project/ray/issues/64956";>#64956</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65329";>#65329</a>, <a href="https://redirect.github.com/ray-project/ray/issues/64897";>#64897</a>)</li> <li>Add <code>ignore_missing_paths</code> and <code>skip_paths</code> to <code>read_parquet</code> on DatasourceV2 (<a href="https://redirect.github.com/ray-project/ray/issues/65118";>#65118</a>)</li> <li>Add <code>delta_timestamps</code> (temporal windows) to <code>read_lerobot</code> (<a href="https://redirect.github.com/ray-project/ray/issues/64877";>#64877</a>)</li> <li>Collect cluster usage metrics by sampling in background threads during execution (<a href="https://redirect.github.com/ray-project/ray/issues/64686";>#64686</a>)</li> <li>Tolerate actor deaths during init via <code>DataContext.max_consecutive_actor_init_deaths</code> (<a href="https://redirect.github.com/ray-project/ray/issues/64846";>#64846</a>)</li> <li>Expose <code>RAY_DATA_HASH_SHUFFLE_MAP_TASK_TARGET_INPUT_BYTES</code> in <code>DataContext</code> (<a href="https://redirect.github.com/ray-project/ray/issues/65103";>#65103</a>)</li> </ul> <h3>💫 Enhancements</h3> <ul> <li>Generate sortable, collision-resistant dataset IDs (<a href="https://redirect.github.com/ray-project/ray/issues/65075";>#65075</a>)</li> <li>Push <code>Limit</code> into <code>ReadFiles</code> when it sits directly on it, and extract <code>FileIndexer.list_file_infos</code> (<a href="https://redirect.github.com/ray-project/ray/issues/65167";>#65167</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65168";>#65168</a>)</li> <li>Use <code>BlockRefCounter</code> for object store memory estimation and remove <code>BlockRefCounter.clear()</code> (<a href="https://redirect.github.com/ray-project/ray/issues/64456";>#64456</a>, <a href="https://redirect.github.com/ray-project/ray/issues/64521";>#64521</a>)</li> <li>Fail execution if no operator makes progress within a timeout, instead of hanging (<a href="https://redirect.github.com/ray-project/ray/issues/65349";>#65349</a>)</li> <li>Improve <code>BatchIterator</code> and <code>iter_torch_batches</code>, and allow a custom <code>collate_fn</code> with a custom device (<a href="https://redirect.github.com/ray-project/ray/issues/64994";>#64994</a>, <a href="https://redirect.github.com/ray-project/ray/issues/64967";>#64967</a>)</li> <li>Skip downstream-capacity backpressure for eligible materializers (<a href="https://redirect.github.com/ray-project/ray/issues/64844";>#64844</a>)</li> <li>Shard exact download partitioning (<a href="https://redirect.github.com/ray-project/ray/issues/64066";>#64066</a>)</li> <li>Bump the <code>sort_reduce</code> memory multiplier to 3x and stop using <code>estimated_input_blocks</code> as the shuffle partition count (<a href="https://redirect.github.com/ray-project/ray/issues/65176";>#65176</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65296";>#65296</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65335";>#65335</a>)</li> <li>Disable cluster autoscaling when <code>PlacementGroupSchedulingStrategy</code> is in use (<a href="https://redirect.github.com/ray-project/ray/issues/64417";>#64417</a>)</li> <li>Propagate the operator name to shuffle tasks (<a href="https://redirect.github.com/ray-project/ray/issues/64802";>#64802</a>)</li> <li>Rename <code>reports_custom_op_stats</code> to <code>should_report_custom_op_stats</code> across all <code>MapTransformFn</code> variants (<a href="https://redirect.github.com/ray-project/ray/issues/64461";>#64461</a>, <a href="https://redirect.github.com/ray-project/ray/issues/64515";>#64515</a>)</li> <li>Rename <code>allocated</code> to <code>reserved</code> and add type aliases in the autoscaling coordinator (<a href="https://redirect.github.com/ray-project/ray/issues/64997";>#64997</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65096";>#65096</a>)</li> <li>Deprecate <code>ray_remote_args_fn</code> and <code>Dataset.zip</code> (<a href="https://redirect.github.com/ray-project/ray/issues/64963";>#64963</a>, <a href="https://redirect.github.com/ray-project/ray/issues/65111";>#65111</a>)</li> </ul> <h3>🔨 Fixes</h3> <ul> <li>Fix an RCE where <code>read_lance</code> or nested pickle objects could execute arbitrary code (<a href="https://redirect.github.com/ray-project/ray/issues/64881";>#64881</a>)</li> <li>Stop converting Arrow null columns to <code>null[pyarrow]</code> in <code>to_pandas</code> (<a href="https://redirect.github.com/ray-project/ray/issues/65187";>#65187</a>)</li> <li>Fix <code>iter_torch_batches</code> device resolution and typing (<a href="https://redirect.github.com/ray-project/ray/issues/65059";>#65059</a>, <a href="https://redirect.github.com/ray-project/ray/issues/64947";>#64947</a>)</li> <li>Propagate <code>isolate_read_workers</code> to <code>DatasourceV2</code> (<a href="https://redirect.github.com/ray-project/ray/issues/65191";>#65191</a>)</li> <li>Avoid signalling epoch end for failed attempts (<a href="https://redirect.github.com/ray-project/ray/issues/65082";>#65082</a>)</li> <li>Add <code>tf-keras</code> to the <code>text_embedding</code> pip packages (<a href="https://redirect.github.com/ray-project/ray/issues/64889";>#64889</a>)</li> </ul> <h3>📖 Documentation</h3> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/ray-project/ray/commit/01b49e4fa77b30bc28c69ed702dfe523fd6421d8";><code>01b49e4</code></a> [docker] Update latest Docker dependencies for 2.58.0 release (<a href="https://redirect.github.com/ray-project/ray/issues/65661";>#65661</a>)</li> <li><a href="https://github.com/ray-project/ray/commit/cec0a091b12675bbf101cf8a350c8706107baee1";><code>cec0a09</code></a> [cherry-pick][2.58.0][core][sandbox] Make Ray Sandbox run Docker-built images...</li> <li><a href="https://github.com/ray-project/ray/commit/0fba74d910a4d7ae569ffc06f307a994719a5d34";><code>0fba74d</code></a> [cherry-pick][2.58.0][Fix][Core] Give <code>canceled_tasks_</code> its own mutex to brea...</li> <li><a href="https://github.com/ray-project/ray/commit/d8277b53b0d4bbe6141654f6237fda5a73c95471";><code>d8277b5</code></a> [cherry-pick][2.58.0][Docs][KubeRay] Use RayJob sample YAML for History Serve...</li> <li><a href="https://github.com/ray-project/ray/commit/d178226cc01f47bc6512e8499a0eeb14cf0aa5b0";><code>d178226</code></a> [cherry-pick][2.58.0][docs] add initial documentation for Ray sandboxing (<a href="https://redirect.github.com/ray-project/ray/issues/65";>#65</a>...</li> <li><a href="https://github.com/ray-project/ray/commit/c6061c10b9775dfcc934a04111a8b8becc725072";><code>c6061c1</code></a> [cherry-pick][2.58.0][doc][History server] Update doc for history server for ...</li> <li><a href="https://github.com/ray-project/ray/commit/3421de4b35d29b5bc29121b95f220b64c7062a87";><code>3421de4</code></a> [cherry-pick][2.58.0][Data] Check object store utilization instead of any spi...</li> <li><a href="https://github.com/ray-project/ray/commit/bc8f786fe5256a30072cc93066b05fd512cd94b9";><code>bc8f786</code></a> [cherry-pick][2.58.0][serve][test] Wait for controller after chat-template te...</li> <li><a href="https://github.com/ray-project/ray/commit/f87adb6edd68fddfa1efbc130cbeecb0b99a3e0b";><code>f87adb6</code></a> [cherry-pick][2.58.0][core] Fix ray.timeline() parity test on windows (<a href="https://redirect.github.com/ray-project/ray/issues/65401";>#65401</a>...</li> <li><a href="https://github.com/ray-project/ray/commit/90489021a8f9ac22e58edfeb4bce56456b484c8a";><code>9048902</code></a> [cherry-pick][2.58.0][Fix][Core] Report shutdown from <code>check_signals</code> instead...</li> <li>Additional commits viewable in <a href="https://github.com/ray-project/ray/compare/ray-2.57.0...ray-2.58.0";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
