Is this buffer lengths buffer only present if the array type is Utf8View? Or are you suggesting that other types might want to adopt this as well?
On Thu, Oct 26, 2023 at 10:00 AM Dewey Dunnington <de...@voltrondata.com.invalid> wrote: > > I expect C code to not be much longer then this :-) > > nanoarrow's buffer-length-calculation and validation concepts are > (perhaps inadvisably) intertwined...even with both it is not that much > code (perhaps I was remembering how much time it took me to figure out > which 35 lines to write :-)) > > > That sounds a bit hackish to me. > > Including only *some* buffer sizes in array->buffers[array->n_buffers] > special-cased for only two types (or altering the number of buffers > required by the IPC format vs. the number of buffers required by the C > Data interface) seem equally hackish to me (not that I'm opposed to > either necessarily...the alternatives really are very bad). > > > How can you *not* care about buffer sizes, if you for example need to > send the buffers over IPC? > > I think IPC is the *only* operation that requires that information? > (Other than perhaps copying to another device?) I don't think there's > any barrier to accessing the content of all the array elements but I > could be mistaken. > > On Thu, Oct 26, 2023 at 1:04 PM Antoine Pitrou <anto...@python.org> wrote: > > > > > > Le 26/10/2023 à 17:45, Dewey Dunnington a écrit : > > > The lack of buffer sizes is something that has come up for me a few > > > times working with nanoarrow (which dedicates a significant amount of > > > code to calculating buffer sizes, which it uses to do validation and > > > more efficient copying). > > > > By the way, this is a bit surprising since it's really 35 lines of code > > in C++ currently: > > > > > https://github.com/apache/arrow/blob/57f643c2cecca729109daae18c7a64f3a37e76e4/cpp/src/arrow/c/bridge.cc#L1721-L1754 > > > > I expect C code to not be much longer then this :-) > > > > Regards > > > > Antoine. >
