alamb commented on issue #8441: URL: https://github.com/apache/arrow-rs/issues/8441#issuecomment-3333815362
> How do you plan to solve 1a? How noticeable is it in Rust? Initially by being smarter about not allocating structs we don't need For for example, if you have 1000 columns with statistics, but your query is reading on 10 columns, or doesn't have predicates, (allocating and parsing) structures for statistics is wasteful. A custom thrift decoder makes it easier to avoid doing this Depending on how far that gets us, we can also use more efficient string representations if needed (e.g. ptr + offset + length, rather than individual allocations) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
