[ https://issues.apache.org/jira/browse/SPARK-5947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng Lian updated SPARK-5947: ------------------------------ Priority: Blocker (was: Major) > First class partitioning support in data sources API > ---------------------------------------------------- > > Key: SPARK-5947 > URL: https://issues.apache.org/jira/browse/SPARK-5947 > Project: Spark > Issue Type: Improvement > Components: SQL > Reporter: Cheng Lian > Priority: Blocker > > For file system based data sources, implementing Hive style partitioning > support can be complex and error prone. To be specific, partitioning support > include: > # Partition discovery: Given a directory organized similar to Hive > partitions, discover the directory structure and partitioning information > automatically, including partition column names, data types, and values. > # Reading from partitioned tables > # Writing to partitioned tables > It would be good to have first class partitioning support in the data sources > API. For example, add a {{FileBasedScan}} trait with callbacks and default > implementations for these features. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org