[ https://issues.apache.org/jira/browse/ARROW-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ben Kietzman resolved ARROW-7916. --------------------------------- Resolution: Fixed Issue resolved by pull request 6474 [https://github.com/apache/arrow/pull/6474] > [C++][Dataset] Project IPC record batches to materialized fields > ---------------------------------------------------------------- > > Key: ARROW-7916 > URL: https://issues.apache.org/jira/browse/ARROW-7916 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, C++ - Dataset > Affects Versions: 0.16.0 > Reporter: Ben Kietzman > Assignee: Ben Kietzman > Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > If batches mmaped from disk are projected before post filtering, unreferenced > columns will never be accessed (so the memory map shouldn't do I/O on them). > At the same time, it'd probably be wise to explicitly document that batches > yielded directly from fragments rather than from a Scanner will not be > filtered or projected (so they will not match the fragment's schema and will > include columns referenced by the filter even if they were not projected). -- This message was sent by Atlassian Jira (v8.3.4#803005)