I agree with Curt that we should probably do an official vote to go ahead with the deprecation. That said, I'd be in favor of it
--Matt On Mon, Aug 31, 2026, 11:54 AM Curt Hagenlocher <[email protected]> wrote: > My vague expectation is that there should be a vote. From what I can > tell, the only complete implementation is the C++ one -- which means > that the feature wouldn't even meet today's bar for format changes. > > There is a recent request for sparse tensor support in Rust at > https://github.com/apache/arrow-rs/issues/10462 . > > On Mon, Aug 31, 2026 at 8:28 AM Antoine Pitrou <[email protected]> wrote: > > > > > > Hello, > > > > Just a ping on this for people who haven't seen the original discussion. > > > > Some questions: > > 1) is anybody opposed to such a deprecation? > > 2) do you think a formal vote is required? > > > > Regards > > > > Antoine. > > > > > > Le 01/07/2026 à 17:59, Antoine Pitrou a écrit : > > > > > > Hello, > > > > > > While the Arrow IPC format is primarily meant for RecordBatch > transport, > > > it also has dedicated messages for transporting N-dimensional Tensor > and > > > SparseTensor data. > > > > > > The Tensor message was added in 2017 > > > (https://github.com/apache/arrow/pull/435), while the SparseTensor > > > message was added in 2019 (https://github.com/apache/arrow/pull/3651). > > > > > > Both have not had any significant additions since 2020. Furthermore: > > > > > > * They are still marked experimental. > > > * They are not exercised by the the cross-implementation integration > > > test suite. > > > * Convenience IPC reading APIs are generally architected around > > > RecordBatch iteration, requiring more manual plumbing to read Tensor > and > > > SparseTensor messages from an IPC stream. > > > * They are generally not reachable from Flight RPC APIs, or other > > > high-level APIs such the C++ Dataset API. > > > * A code search on GitHub reveals no apparent third-party usage of > these > > > message types. > > > > > > Nowadays, Arrow has canonical extension types for fixed-shape and > > > variable-shaped tensors which allow passing (dense) N-dimensional > > > tensors as regular RecordBatch columns. We could also add canonical > > > extension types for the different kinds of sparse tensors, if there is > > > demand for it. > > > > > > Despite not seeing much usage, these message types carry their > > > maintenance baggage. Recently, we've had a number of security reports > > > around the lack of validation of untrusted Tensor and SparseTensor > > > messages in Arrow C++, requiring core developer intervention to analyze > > > and fix those issues. > > > > > > I would like to suggest that we start deprecating these messages types > > > in the IPC format, and that we recommend deprecating the corresponding > > > APIs in all major Arrow implementations. What do you think? > > > > > > Regards > > > > > > Antoine. > > >
