Ben Kietzman created ARROW-7916: ----------------------------------- Summary: [C++][Dataset] Project IPC record batches to materialized fields Key: ARROW-7916 URL: https://issues.apache.org/jira/browse/ARROW-7916 Project: Apache Arrow Issue Type: Improvement Components: C++, C++ - Dataset Affects Versions: 0.16.0 Reporter: Ben Kietzman Assignee: Ben Kietzman Fix For: 1.0.0
If batches mmaped from disk are projected before post filtering, unreferenced columns will never be accessed (so the memory map shouldn't do I/O on them). At the same time, it'd probably be wise to explicitly document that batches yielded directly from fragments rather than from a Scanner will not be filtered or projected (so they will not match the fragment's schema and will include columns referenced by the filter even if they were not projected). -- This message was sent by Atlassian Jira (v8.3.4#803005)