[ https://issues.apache.org/jira/browse/ARROW-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056295#comment-17056295 ]
Ben Kietzman commented on ARROW-8065: ------------------------------------- https://github.com/apache/arrow/pull/6570#issuecomment-597260070 > [C++][Dataset] Untangle Dataset, Fragment and ScanOptions > --------------------------------------------------------- > > Key: ARROW-8065 > URL: https://issues.apache.org/jira/browse/ARROW-8065 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ - Dataset > Reporter: Francois Saint-Jacques > Priority: Major > > We should be able to list fragments without going through the > Scanner/ScanOptions hoops. This exposes a flaw with the current API where it > require a ScanOptions to create Fragment, this is also a problem for > ARROW-7824, i.e. why do we need a ScanOptions (read manifest) to write record > batches to a given path. > # Remove {{ScanOptions}} from Fragment's properties and move it into > {{Fragment::Scan}} parameters. > # Remove {{ScanOptions}} from {{Dataset::GetFragments}}, if required, we can > still provide an alternate signature, e.g. > {{Dataset::GetFragments(std::shared_ptr<Expression> predicate)}} for sub-tree > pruning in FileSystemDataset. > # Fragment constructor should take a schema (and store it as a property), > usually extracted from the Dataset schema. Update the schema() method > accordingly. -- This message was sent by Atlassian Jira (v8.3.4#803005)