adamreeve commented on issue #34736: URL: https://github.com/apache/arrow/issues/34736#issuecomment-2227511168
Hi @CurtHagenlocher, we've run into issues integrating with Polars, which always exports string data to Arrow as the LargeString type (see https://github.com/pola-rs/polars/issues/15047). We can work around this by casting to String first via PyArrow, but it would simplify things if there was LargeString support in .NET Arrow, even if it didn't yet support values buffers that were actually > 2GB. Would you be open to accepting a PR to add LargeString, LargeBinary and LargeList arrays? I'm hopeful I might eventually be able to help with adding support for IPC record batches and buffers > 2 GB too, but I think there is some value in having support for LargeString etc even if they don't actually support large buffers yet, and it makes sense to me to split this work out from adding support for large buffers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
