[GitHub] spark pull request #20631: [SPARK-23454][SS][DOCS] Added trigger information...

2018-02-17 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20631#discussion_r168941119 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1979,6 +2004,157 @@ which has methods that get called whenever there is a sequence of r

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20619 **[Test build #87534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87534/testReport)** for PR 20619 at commit [`8bd02d8`](https://github.com/apache/spark/commit/8b

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/953/ Test

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20619 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87533/ Test FAILed. ---

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20619 **[Test build #87533 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87533/testReport)** for PR 20619 at commit [`8bd02d8`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #20442: [SPARK-23265][ML]Update multi-column error handling logi...

2018-02-17 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/20442 Thanks for the comments. I am in China now for Chinese New Year. Will address the comments when I get back to work on 2/21. ---

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87532/ Test PASSed. ---

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20387 **[Test build #87532 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87532/testReport)** for PR 20387 at commit [`1a603db`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20619 **[Test build #87533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87533/testReport)** for PR 20619 at commit [`8bd02d8`](https://github.com/apache/spark/commit/8b

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/952/ Test

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168936300 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -640,4 +689,96 @@ private object RandomForestSuite { val

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168936292 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +292,41 @@ private[tree] class LearningNode( } } +

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 The final failure is irrelevant to this. ``` org.apache.spark.sql.sources.CreateTableAsSelectSuite.(It is not a test it is a sbt.testing.SuiteSelector) ``` --- --

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168936195 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -640,4 +689,96 @@ private object RandomForestSuite { val

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168936124 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +292,41 @@ private[tree] class LearningNode( } } +

[GitHub] spark pull request #20621: [SPARK-23436][SQL] Infer partition as Date only i...

2018-02-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20621#discussion_r168935309 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -407,6 +407,34 @@ object PartitioningUtils {

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20057 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87531/ Test PASSed. ---

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20057 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20057 **[Test build #87531 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87531/testReport)** for PR 20057 at commit [`0e646b2`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20387 **[Test build #87532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87532/testReport)** for PR 20387 at commit [`1a603db`](https://github.com/apache/spark/commit/1a

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-17 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20387 Thanks for the update! Enjoy your vacation, and thanks for letting me know. --- - To unsubscribe, e-mail: reviews-unsubscr...@spar

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/951/ Test

[GitHub] spark pull request #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable l...

2018-02-17 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r168933227 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/PushDownOperatorsToDataSource.scala --- @@ -17,130 +17,55 @@ pack

[GitHub] spark issue #20621: [SPARK-23436][SQL] Infer partition as Date only if it ca...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20621 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20621: [SPARK-23436][SQL] Infer partition as Date only if it ca...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20621 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87526/ Test PASSed. ---

[GitHub] spark issue #20621: [SPARK-23436][SQL] Infer partition as Date only if it ca...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20621 **[Test build #87526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87526/testReport)** for PR 20621 at commit [`8698f4d`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87528/ Test FAILed. ---

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20619 **[Test build #87528 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87528/testReport)** for PR 20619 at commit [`8bd02d8`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #20634: [SPARK-23456][SPARK-21783] Turn on `native` ORC impl and...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20634 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87525/ Test PASSed. ---

[GitHub] spark issue #20634: [SPARK-23456][SPARK-21783] Turn on `native` ORC impl and...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20634 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20634: [SPARK-23456][SPARK-21783] Turn on `native` ORC impl and...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20634 **[Test build #87525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87525/testReport)** for PR 20634 at commit [`bde6818`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87527/ Test FAILed. ---

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20619 **[Test build #87527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87527/testReport)** for PR 20619 at commit [`e08d06c`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20057 **[Test build #87531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87531/testReport)** for PR 20057 at commit [`0e646b2`](https://github.com/apache/spark/commit/0e

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168930195 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala --- @@ -85,15 +85,24 @@ private object PostgresDialect extends JdbcD

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20057 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20057 **[Test build #87530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87530/testReport)** for PR 20057 at commit [`7e09f90`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20057 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87530/ Test FAILed. ---

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20057 **[Test build #87530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87530/testReport)** for PR 20057 at commit [`7e09f90`](https://github.com/apache/spark/commit/7e

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread danielvdende
Github user danielvdende commented on the issue: https://github.com/apache/spark/pull/20057 Thanks again @dongjoon-hyun, much appreciated 👍. I've made changes accordingly, and added comments in this PR where appropriate. --- ---

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread danielvdende
Github user danielvdende commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168929874 --- Diff: docs/sql-programming-guide.md --- @@ -1372,6 +1372,13 @@ the following case-insensitive options: This is a JDBC writer related optio

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20057 **[Test build #87529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87529/testReport)** for PR 20057 at commit [`b79becd`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20057 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87529/ Test FAILed. ---

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20057 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread danielvdende
Github user danielvdende commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168929798 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala --- @@ -94,5 +94,21 @@ private case object OracleDialect extends JdbcDi

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread danielvdende
Github user danielvdende commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168929792 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/MsSqlServerDialect.scala --- @@ -42,4 +42,17 @@ private object MsSqlServerDialect extends J

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20057 **[Test build #87529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87529/testReport)** for PR 20057 at commit [`b79becd`](https://github.com/apache/spark/commit/b7

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20619 **[Test build #87528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87528/testReport)** for PR 20619 at commit [`8bd02d8`](https://github.com/apache/spark/commit/8b

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/950/ Test

[GitHub] spark pull request #20619: [SPARK-23457][SQL] Register task completion liste...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20619#discussion_r168929352 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -395,16 +395,21 @@ class ParquetF

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 Oh, @kiszk . The following meat really `comment` in the code. Sorry, I misunderstood. > Would it be worth to add this JIRA number in a comment as we did for ORC? --- ---

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20619 **[Test build #87527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87527/testReport)** for PR 20619 at commit [`e08d06c`](https://github.com/apache/spark/commit/e0

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/949/ Test

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 Thank you, @kiszk . I added SPARK-23390 in the PR description. > Would it be worth to add this JIRA number in a comment as we did for ORC? ---

[GitHub] spark issue #20621: [SPARK-23436][SQL] Infer partition as Date only if it ca...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20621 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/948/ Test

[GitHub] spark issue #20621: [SPARK-23436][SQL] Infer partition as Date only if it ca...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20621 **[Test build #87526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87526/testReport)** for PR 20621 at commit [`8698f4d`](https://github.com/apache/spark/commit/86

[GitHub] spark issue #20621: [SPARK-23436][SQL] Infer partition as Date only if it ca...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20621 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20619 LGTM with one minor comment --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 Thank you for review, @mgaido91 . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional command

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20057 I finished my second round. Could you update once more? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168928044 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala --- @@ -120,11 +121,13 @@ abstract class JdbcDialect extends Serializab

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927868 --- Diff: docs/sql-programming-guide.md --- @@ -1372,6 +1372,13 @@ the following case-insensitive options: This is a JDBC writer related opti

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927785 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala --- @@ -64,7 +64,16 @@ private class AggregatedDialect(dialects:

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927699 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala --- @@ -94,5 +94,21 @@ private case object OracleDialect extends JdbcD

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927651 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala --- @@ -94,5 +94,21 @@ private case object OracleDialect extends JdbcD

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927631 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/MsSqlServerDialect.scala --- @@ -42,4 +42,17 @@ private object MsSqlServerDialect extends

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927633 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/MySQLDialect.scala --- @@ -46,4 +46,17 @@ private case object MySQLDialect extends JdbcDia

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927608 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/MsSqlServerDialect.scala --- @@ -42,4 +42,17 @@ private object MsSqlServerDialect extends

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927597 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/MsSqlServerDialect.scala --- @@ -42,4 +42,17 @@ private object MsSqlServerDialect extends

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20619 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apac

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927546 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/DB2Dialect.scala --- @@ -49,4 +49,17 @@ private object DB2Dialect extends JdbcDialect {

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927495 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala --- @@ -120,11 +121,13 @@ abstract class JdbcDialect extends Serializab

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927408 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/DerbyDialect.scala --- @@ -41,4 +41,16 @@ private object DerbyDialect extends JdbcDialect

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927400 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -102,7 +102,12 @@ object JdbcUtils extends L

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927406 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/DerbyDialect.scala --- @@ -41,4 +41,16 @@ private object DerbyDialect extends JdbcDialect

[GitHub] spark issue #19775: [SPARK-22343][core] Add support for publishing Spark met...

2018-02-17 Thread stoader
Github user stoader commented on the issue: https://github.com/apache/spark/pull/19775 @erikerlandson we tested this on Kubernetes using https://github.com/prometheus/pushgateway/tree/v0.3.1 and https://github.com/prometheus/pushgateway/tree/v0.4.0 --- -

[GitHub] spark issue #20634: [SPARK-23456][SPARK-21783] Turn on `native` ORC impl and...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20634 **[Test build #87525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87525/testReport)** for PR 20634 at commit [`bde6818`](https://github.com/apache/spark/commit/bd

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 Yep. @kiszk . @mgaido91 also reports that, so I'm investigating that more. However, that doesn't mean this approach is not proper. You can see the manual test case example in previous

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 For the following, I'll. > Would it be worth to add this JIRA number in a comment as we did for ORC? --- - To unsubscri

[GitHub] spark issue #20634: [SPARK-23456][SPARK-21783] Turn on `native` ORC impl and...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20634 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/947/ Test

[GitHub] spark issue #20634: [SPARK-23456][SPARK-21783] Turn on `native` ORC impl and...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20634 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20634: [SPARK-23456][SPARK-21783] Turn on `native` ORC i...

2018-02-17 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/20634 [SPARK-23456][SPARK-21783] Turn on `native` ORC impl and PPD by default ## What changes were proposed in this pull request? Apache Spark 2.3 introduced `native` ORC supports with vect

[GitHub] spark pull request #20621: [SPARK-23436][SQL] Infer partition as Date only i...

2018-02-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20621#discussion_r168925882 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -407,6 +407,34 @@ object PartitioningUtils {

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20057 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20057 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87524/ Test FAILed. ---

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20057 **[Test build #87524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87524/testReport)** for PR 20057 at commit [`3921368`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #20621: [SPARK-23436][SQL] Infer partition as Date only i...

2018-02-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20621#discussion_r168925339 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -407,6 +407,34 @@ object PartitioningUtils {

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168925267 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -640,4 +689,96 @@ private object RandomForestSuite {

  1   2   >