[ https://issues.apache.org/jira/browse/ARROW-11981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander updated ARROW-11981: ------------------------------ Labels: dataset kernel pull-request-available query-engine (was: dataset pull-request-available query-engine) > [C++][Dataset][Compute] Replace UnionDataset with Union ExecNode > ---------------------------------------------------------------- > > Key: ARROW-11981 > URL: https://issues.apache.org/jira/browse/ARROW-11981 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ > Affects Versions: 3.0.0, 6.0.0 > Reporter: Ben Kietzman > Assignee: Alexander > Priority: Major > Labels: dataset, kernel, pull-request-available, query-engine > Time Spent: 20m > Remaining Estimate: 0h > > UnionDataset allows Fragments of multiple schemas and differing file formats > to be scanned together as a single Dataset. This is useful functionality but > makes the Dataset interface somewhat difficult to reason about since it must > be general enough to accommodate UnionDataset. > After ARROW-11928 it will probably be more natural to support unioning of > datasets through a subclass of ExecNode. Reconciliation of differing schemas > can then be trivially handled by a full ProjectNode. > Note this would obviate both ARROW-11001 and ARROW-11749. In addition, > Dataset could be simplified to a concrete class containing a set of > compatibly typed/formatted Fragments. -- This message was sent by Atlassian Jira (v8.3.4#803005)