[GitHub] spark pull request #15024: [SPARK-17470][SQL] unify path for data source tab...

2016-10-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15024#discussion_r85620456 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -784,6 +819,7 @@ object HiveExternalCatalog { val STAT

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67725 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67725/consoleFull)** for PR 15626 at commit [`bbc5a26`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15675 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67722/ Test FAILed. ---

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15675 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15675 **[Test build #67722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67722/consoleFull)** for PR 15675 at commit [`766c67d`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15148 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tcondie
Github user tcondie commented on the issue: https://github.com/apache/spark/pull/15626 All log are now uniformly marshall entries to JSON. Each log has a particular way of organizing its JSON entries: HDFSMetadataLog: Each Batch ID is a separate file. All log entries w

[GitHub] spark issue #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-28 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/15148 This LGTM now. Any other comments from other reviewers? I'll merge this, but we can follow up as needed. Thanks very much @Yunni for the PR and everyone else for helping to review!

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-28 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r85616890 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,339 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #15664: [SPARK-18123][SQL] Use db column names instead of...

2016-10-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15664#discussion_r85616485 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -112,7 +112,25 @@ object JdbcUtils extends Lo

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tcondie
Github user tcondie commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85615504 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/Offset.scala --- @@ -23,4 +23,16 @@ package org.apache.spark.sql.execution.stream

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tcondie
Github user tcondie commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85615514 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala --- @@ -44,9 +45,14 @@ import org.apache.spark.util.Uninter

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tcondie
Github user tcondie commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85615490 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLog.scala --- @@ -0,0 +1,62 @@ +/* +* Licensed to the Apache Sof

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tcondie
Github user tcondie commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85615458 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -713,6 +713,8 @@ class StreamExecution( }

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67724 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67724/consoleFull)** for PR 15626 at commit [`d62a877`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15148 **[Test build #67721 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67721/consoleFull)** for PR 15148 at commit [`3570845`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15148 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67721/ Test PASSed. ---

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/15626 Thanks for working on this! Could you include examples of the various logs, since we are committing to this specific JSON. --- If your project is set up for it, you can reply to this email and ha

[GitHub] spark issue #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15148 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tcondie
Github user tcondie commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85613884 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLogSuite.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tcondie
Github user tcondie commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85613714 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLogSuite.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tcondie
Github user tcondie commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85613616 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLogSuite.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-28 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r85611295 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -126,4 +140,59 @@ object FileSourceStrategy e

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-28 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r85610578 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -97,7 +99,19 @@ object FileSourceStrategy ext

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15513 I still found a general issue in the type description. `an expression of any type` appears 54 times in this PR. However, have you checked whether they can work well for the complex types

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-28 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84562093 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -126,4 +136,52 @@ object FileSourceStrategy e

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tcondie
Github user tcondie commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85609921 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CompactibleFileStreamLog.scala --- @@ -37,14 +39,19 @@ import org.apache.spark.sq

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tcondie
Github user tcondie commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85609758 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceOffsetSuite.scala --- @@ -36,4 +37,9 @@ class KafkaSourceOffsetSui

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67723 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67723/consoleFull)** for PR 15626 at commit [`4663d5e`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85602991 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/Offset.scala --- @@ -23,4 +23,16 @@ package org.apache.spark.sql.execution.streaming

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85605730 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLogSuite.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85606228 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLogSuite.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85603184 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLog.scala --- @@ -0,0 +1,62 @@ +/* +* Licensed to the Apache Softwa

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85605203 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLogSuite.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85604066 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala --- @@ -44,9 +45,14 @@ import org.apache.spark.util.Uninterrup

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85608065 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceOffset.scala --- @@ -51,4 +54,11 @@ private[kafka010] object KafkaSou

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85605189 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLogSuite.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85603294 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -713,6 +713,8 @@ class StreamExecution( }

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85603525 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryException.scala --- @@ -36,8 +36,8 @@ class StreamingQueryException private[sql]

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85605087 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLogSuite.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85602891 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala --- @@ -44,9 +45,14 @@ import org.apache.spark.util.Uninterrup

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85602329 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CompactibleFileStreamLog.scala --- @@ -37,14 +39,19 @@ import org.apache.spark.sql.S

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85604754 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryException.scala --- @@ -36,8 +36,8 @@ class StreamingQueryException private[sql]

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85603113 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLog.scala --- @@ -0,0 +1,62 @@ +/* +* Licensed to the Apache Softwa

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85607356 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala --- @@ -44,9 +45,14 @@ import org.apache.spark.util.Uninterrup

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15675 **[Test build #67722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67722/consoleFull)** for PR 15675 at commit [`766c67d`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #15675: [SPARK-18144][SQL] logging StreamingQueryListener...

2016-10-28 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/15675#discussion_r85607305 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingQueryListenerBus.scala --- @@ -39,12 +39,7 @@ class StreamingQueryList

[GitHub] spark issue #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15148 **[Test build #67721 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67721/consoleFull)** for PR 15148 at commit [`3570845`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #15675: [SPARK-18144][SQL] logging StreamingQueryListener...

2016-10-28 Thread CodingCat
GitHub user CodingCat opened a pull request: https://github.com/apache/spark/pull/15675 [SPARK-18144][SQL] logging StreamingQueryListener$QueryStartedEvent ## What changes were proposed in this pull request? The PR fixes the bug that the QueryStartedEvent is not logged

[GitHub] spark issue #15665: [SPARK-18146] [SQL] Avoid using Union to chain together ...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15665 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15665: [SPARK-18146] [SQL] Avoid using Union to chain together ...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15665 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67716/ Test PASSed. ---

[GitHub] spark issue #15665: [SPARK-18146] [SQL] Avoid using Union to chain together ...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15665 **[Test build #67716 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67716/consoleFull)** for PR 15665 at commit [`da2`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15626 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67711/ Test FAILed. ---

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15626 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67711/consoleFull)** for PR 15626 at commit [`b80b3ce`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15674: [SPARK-18164][SQL]ForeachSink should fail the Spark job ...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15674 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15674: [SPARK-18164][SQL]ForeachSink should fail the Spark job ...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15674 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67715/ Test PASSed. ---

[GitHub] spark issue #15674: [SPARK-18164][SQL]ForeachSink should fail the Spark job ...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15674 **[Test build #67715 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67715/consoleFull)** for PR 15674 at commit [`4466010`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15627: [SPARK-18099][YARN] Fail if same files added to distribu...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15627 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15627: [SPARK-18099][YARN] Fail if same files added to distribu...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15627 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67720/ Test PASSed. ---

[GitHub] spark issue #15627: [SPARK-18099][YARN] Fail if same files added to distribu...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15627 **[Test build #67720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67720/consoleFull)** for PR 15627 at commit [`33f95ab`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15626#discussion_r85602088 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceOffsetSuite.scala --- @@ -36,4 +37,9 @@ class KafkaSourceOffsetSuite

[GitHub] spark pull request #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls...

2016-10-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15417#discussion_r85600595 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala --- @@ -1016,6 +1016,8 @@ class FilterPushdownSui

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-28 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15553 Thanks for the comments. Specifically for our usecase, this is for automated full builds, where there is a container teardown after each (so that builds dont pollute each other : new $

[GitHub] spark issue #15627: [SPARK-18099][YARN] Fail if same files added to distribu...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15627 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15627: [SPARK-18099][YARN] Fail if same files added to distribu...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15627 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67719/ Test PASSed. ---

[GitHub] spark issue #15627: [SPARK-18099][YARN] Fail if same files added to distribu...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15627 **[Test build #67719 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67719/consoleFull)** for PR 15627 at commit [`a1dc858`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15627: [SPARK-18099][YARN] Fail if same files added to distribu...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15627 **[Test build #67720 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67720/consoleFull)** for PR 15627 at commit [`33f95ab`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #15618: [SPARK-14914][CORE] Fix Resource not closed after...

2016-10-28 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15618#discussion_r85436193 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -194,10 +194,13 @@ class ReceiverTracker(ssc: Streaming

[GitHub] spark pull request #15618: [SPARK-14914][CORE] Fix Resource not closed after...

2016-10-28 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15618#discussion_r85435956 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskResultGetterSuite.scala --- @@ -209,7 +209,8 @@ class TaskResultGetterSuite extends SparkFunSuit

[GitHub] spark issue #15627: [SPARK-18099][YARN] Fail if same files added to distribu...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15627 **[Test build #67719 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67719/consoleFull)** for PR 15627 at commit [`a1dc858`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15673 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67713/ Test PASSed. ---

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15673 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15673 **[Test build #67713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67713/consoleFull)** for PR 15673 at commit [`c62beda`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-28 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r85591762 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/MinHash.scala --- @@ -0,0 +1,194 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #15634: [SPARK-18103] [SQL] Rename *FileCatalog to *FileIndex

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15634 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15634: [SPARK-18103] [SQL] Rename *FileCatalog to *FileIndex

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15634 **[Test build #67718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67718/consoleFull)** for PR 15634 at commit [`ec6d4ee`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15634: [SPARK-18103] [SQL] Rename *FileCatalog to *FileIndex

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15634 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67718/ Test FAILed. ---

[GitHub] spark issue #15634: [SPARK-18103] [SQL] Rename *FileCatalog to *FileIndex

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15634 **[Test build #67718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67718/consoleFull)** for PR 15634 at commit [`ec6d4ee`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #15634: [SPARK-18103] [SQL] Rename *FileCatalog to *FileProvider

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15634 **[Test build #67717 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67717/consoleFull)** for PR 15634 at commit [`b6654f1`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15634: [SPARK-18103] [SQL] Rename *FileCatalog to *FileProvider

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15634 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67717/ Test FAILed. ---

[GitHub] spark issue #15634: [SPARK-18103] [SQL] Rename *FileCatalog to *FileProvider

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15634 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15634: [SPARK-18103] [SQL] Rename *FileCatalog to *FileProvider

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15634 **[Test build #67717 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67717/consoleFull)** for PR 15634 at commit [`b6654f1`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #15665: [SPARK-18146] [SQL] Avoid using Union to chain together ...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15665 **[Test build #67716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67716/consoleFull)** for PR 15665 at commit [`da2`](https://github.com/apache/spark/commit/d

[GitHub] spark pull request #15665: [SPARK-18146] [SQL] Avoid using Union to chain to...

2016-10-28 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15665#discussion_r85585863 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -388,16 +388,15 @@ final class DataFrameWriter[T] private[sql](ds: Dataset

[GitHub] spark pull request #15354: [SPARK-17764][SQL] Add `to_json` supporting to co...

2016-10-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/15354#discussion_r85583061 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -494,3 +495,46 @@ case class JsonToStruct(sche

[GitHub] spark pull request #15354: [SPARK-17764][SQL] Add `to_json` supporting to co...

2016-10-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/15354#discussion_r85583042 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2936,6 +2936,51 @@ object functions { def from_json(e: Column, schema:

[GitHub] spark issue #15674: [SPARK-18164][SQL]ForeachSink should fail the Spark job ...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15674 **[Test build #67715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67715/consoleFull)** for PR 15674 at commit [`4466010`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #15674: [SPARK-18164][SQL]ForeachSink should fail the Spark job ...

2016-10-28 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15674 cc @tdas --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[GitHub] spark pull request #15674: [SPARK-18164][SQL]ForeachSink should fail the Spa...

2016-10-28 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/15674 [SPARK-18164][SQL]ForeachSink should fail the Spark job if `process` throws exception ## What changes were proposed in this pull request? Fixed the issue that ForeachSink didn't rethrow th

[GitHub] spark issue #15665: [SPARK-18146] [SQL] Avoid using Union to chain together ...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15665 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15665: [SPARK-18146] [SQL] Avoid using Union to chain together ...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15665 **[Test build #67714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67714/consoleFull)** for PR 15665 at commit [`e8d28e1`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15665: [SPARK-18146] [SQL] Avoid using Union to chain together ...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15665 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67714/ Test FAILed. ---

[GitHub] spark issue #15665: [SPARK-18146] [SQL] Avoid using Union to chain together ...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15665 **[Test build #67712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67712/consoleFull)** for PR 15665 at commit [`dd01f3a`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15665: [SPARK-18146] [SQL] Avoid using Union to chain together ...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15665 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67712/ Test FAILed. ---

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15673 **[Test build #67713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67713/consoleFull)** for PR 15673 at commit [`c62beda`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #15665: [SPARK-18146] [SQL] Avoid using Union to chain together ...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15665 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15665: [SPARK-18146] [SQL] Avoid using Union to chain together ...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15665 **[Test build #67714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67714/consoleFull)** for PR 15665 at commit [`e8d28e1`](https://github.com/apache/spark/commit/e

[GitHub] spark pull request #15673: [SPARK-17992][SQL] Return all partitions from Hiv...

2016-10-28 Thread mallman
GitHub user mallman opened a pull request: https://github.com/apache/spark/pull/15673 [SPARK-17992][SQL] Return all partitions from HiveShim when Hive throws a metastore exception when attempting to fetch partitions by filter (Link to Jira issue: https://issues.apache.org/jira/brows

[GitHub] spark pull request #15665: [SPARK-18146] [SQL] Avoid using Union to chain to...

2016-10-28 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15665#discussion_r85577862 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -254,7 +254,8 @@ final class DataFrameWriter[T] private[sql](ds: Dataset[T

<    1   2   3   4   5   >