[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/13623 LGTM. Merging to master and branch 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13623 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13623 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60497/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13623 **[Test build #60497 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60497/consoleFull)** for PR 13623 at commit [`b345d56`](https://github.com/apache/spark/commit/b345d56c38d90295db3957f198a766dbb6e58795). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13623 **[Test build #60497 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60497/consoleFull)** for PR 13623 at commit [`b345d56`](https://github.com/apache/spark/commit/b345d56c38d90295db3957f198a766dbb6e58795). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13623 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60430/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13623 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13623 **[Test build #60430 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60430/consoleFull)** for PR 13623 at commit [`ee438e4`](https://github.com/apache/spark/commit/ee438e466e9b5368f821e5cac580393ecf8921ef). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13623 @rxin Thanks. Consolidated all the underscore- and dot-files filtering logic. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13623 **[Test build #60430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60430/consoleFull)** for PR 13623 at commit [`ee438e4`](https://github.com/apache/spark/commit/ee438e466e9b5368f821e5cac580393ecf8921ef). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13623 In this file PartitioningAwareFileCatalog.scala, we have multiple places that we filter out files that start with underscore. Should we also filter dot in those places? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user clockfly commented on the issue: https://github.com/apache/spark/pull/13623 Looks good! +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user ssimeonov commented on the issue: https://github.com/apache/spark/pull/13623 ð --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13623 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60371/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13623 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13623 **[Test build #60371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60371/consoleFull)** for PR 13623 at commit [`91faf01`](https://github.com/apache/spark/commit/91faf0139f5c06e79bd92f637db078aa8ca0aa09). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13623 **[Test build #60371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60371/consoleFull)** for PR 13623 at commit [`91faf01`](https://github.com/apache/spark/commit/91faf0139f5c06e79bd92f637db078aa8ca0aa09). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13623 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13623 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60360/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13623 **[Test build #60360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60360/consoleFull)** for PR 13623 at commit [`5015fa0`](https://github.com/apache/spark/commit/5015fa0ee08b05a39261bef1ed0b152fedb5029e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13623 **[Test build #60360 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60360/consoleFull)** for PR 13623 at commit [`5015fa0`](https://github.com/apache/spark/commit/5015fa0ee08b05a39261bef1ed0b152fedb5029e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13623 @yhuai Unlike what we discussed before, we can't filter out metadata files [here][1]. Otherwise, Parquet read path may also skip all Parquet summary files, which isn't expected behavior. [1]: https://github.com/apache/spark/blob/9e204c62c6800e03759e04ef68268105d4b86bf2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala#L72 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
