Repository: spark Updated Branches: refs/heads/master fd60d4fa6 -> 9e451bcf3
[MINOR][DOC] Update `Partition Discovery` section to enumerate all available file sources ## What changes were proposed in this pull request? All built-in data sources support `Partition Discovery`. We had better update the document to give the users more benefit clearly. **AFTER** <img width="906" alt="1" src="https://user-images.githubusercontent.com/9700541/30083628-14278908-9244-11e7-98dc-9ad45fe233a9.png";> ## How was this patch tested? ``` SKIP_API=1 jekyll serve --watch ``` Author: Dongjoon Hyun <dongj...@apache.org> Closes #19139 from dongjoon-hyun/partitiondiscovery. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9e451bcf Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/9e451bcf Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/9e451bcf Branch: refs/heads/master Commit: 9e451bcf36151bf401f72dcd66001b9ceb079738 Parents: fd60d4f Author: Dongjoon Hyun <dongj...@apache.org> Authored: Tue Sep 5 14:35:09 2017 -0700 Committer: gatorsmile <gatorsm...@gmail.com> Committed: Tue Sep 5 14:35:09 2017 -0700 ---------------------------------------------------------------------- docs/sql-programming-guide.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/9e451bcf/docs/sql-programming-guide.md ---------------------------------------------------------------------- diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index ee231a9..032073b 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -733,8 +733,9 @@ SELECT * FROM parquetTable Table partitioning is a common optimization approach used in systems like Hive. In a partitioned table, data are usually stored in different directories, with partitioning column values encoded in -the path of each partition directory. The Parquet data source is now able to discover and infer -partitioning information automatically. For example, we can store all our previously used +the path of each partition directory. All built-in file sources (including Text/CSV/JSON/ORC/Parquet) +are able to discover and infer partitioning information automatically. +For example, we can store all our previously used population data into a partitioned table using the following directory structure, with two extra columns, `gender` and `country` as partitioning columns: --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
![https://user-images.githubusercontent.com/9700541/30083628-14278908-9244-11e7-98dc-9ad45fe233a9.png"](https://user-images.githubusercontent.com/9700541/30083628-14278908-9244-11e7-98dc-9ad45fe233a9.png"){kind=link}