Krisztian Szucs created ARROW-7965: -------------------------------------- Summary: [Python] Hold a reference to the dataset factory for later reuse Key: ARROW-7965 URL: https://issues.apache.org/jira/browse/ARROW-7965 Project: Apache Arrow Issue Type: Improvement Components: Python Reporter: Krisztian Szucs
Provide a more intuitive way to construct nested dataset: ```python # instead of using confusing factory function dataset([ factory("s3://old-taxi-data", format="parquet"), factory("local/path/to/new/data", format="csv") ]) # let the user to construct a new dataset directly from dataset objects dataset([ dataset("s3://old-taxi-data", format="parquet"), dataset("local/path/to/new/data", format="csv") ]) ``` In the future we might want to introduce a new Dataset class which wraps functionality of both the dataset actory and the materialized dataset enabling optimizations over rediscovery of already materialized datasets. -- This message was sent by Atlassian Jira (v8.3.4#803005)