[GitHub] spark pull request #16467: [SPARK-19017][SQL] NOT IN subquery with more than...

2017-01-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16467 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16467: [SPARK-19017][SQL] NOT IN subquery with more than one co...

2017-01-24 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16467 LGTM - merging to master/2.1. Thanks! Sorry for the long wait. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #16467: [SPARK-19017][SQL] NOT IN subquery with more than...

2017-01-24 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16467#discussion_r97667507 --- Diff: sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/not-in-multiple-columns.sql.out --- @@ -0,0 +1,59 @@ +-- Automatically

[GitHub] spark issue #16650: [SPARK-16554][CORE] Automatically Kill Executors and Nod...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16650 **[Test build #71947 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71947/testReport)** for PR 16650 at commit

[GitHub] spark issue #16650: [SPARK-16554][CORE] Automatically Kill Executors and Nod...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16650 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16650: [SPARK-16554][CORE] Automatically Kill Executors and Nod...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16650 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71947/ Test FAILed. ---

[GitHub] spark issue #16465: [SPARK-19064][PySpark]Fix pip installing of sub componen...

2017-01-24 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16465 Small follow up ping for @joshrosen to @davies maybe? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16694 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71948/ Test PASSed. ---

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16694 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16694 **[Test build #71948 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71948/testReport)** for PR 16694 at commit

[GitHub] spark issue #16688: [TESTS][SQL] Setup testdata at the beginning for tests t...

2017-01-24 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/16688 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16660: [SPARK-19311][SQL] fix UDT hierarchy issue

2017-01-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16660 LGTM except one comment. Thanks for working on this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16660: [SPARK-19311][SQL] fix UDT hierarchy issue

2017-01-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16660 The remaining comment is: https://github.com/apache/spark/pull/16660#discussion_r97427591 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #15768: [SPARK-18080][ML][PySpark] Locality Sensitive Hashing (L...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15768 **[Test build #3550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3550/consoleFull)** for PR 15768 at commit

[GitHub] spark issue #16650: [SPARK-16554][CORE] Automatically Kill Executors and Nod...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16650 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16650: [SPARK-16554][CORE] Automatically Kill Executors and Nod...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16650 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71944/ Test FAILed. ---

[GitHub] spark issue #16650: [SPARK-16554][CORE] Automatically Kill Executors and Nod...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16650 **[Test build #71944 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71944/testReport)** for PR 16650 at commit

[GitHub] spark issue #16329: [SPARK-16046][DOCS] Aggregations in the Spark SQL progra...

2017-01-24 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/16329 Sorry for the delay. This LGTM, but I'm currently away from my Apache SSH keys. Other committers should feel free to merge if you get there before I do. --- If your project is set up for it,

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16694 **[Test build #71948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71948/testReport)** for PR 16694 at commit

[GitHub] spark issue #16685: [SPARK-19335] Introduce insert, update, and upsert comma...

2017-01-24 Thread xwu0226
Github user xwu0226 commented on the issue: https://github.com/apache/spark/pull/16685 A few comments: 1. The mayor concern is that this solution need to pull in the whole target table data and do a join operation between the source dataframe and the target table to

[GitHub] spark issue #16685: [SPARK-19335] Introduce insert, update, and upsert comma...

2017-01-24 Thread ilganeli
Github user ilganeli commented on the issue: https://github.com/apache/spark/pull/16685 It sounds like you consider there to be too many errata and assumptions made in this patch for it to be a worthwhile code contribution. Given the numerous assumptions made in this PR, how would

[GitHub] spark issue #16650: [SPARK-16554][CORE] Automatically Kill Executors and Nod...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16650 **[Test build #71947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71947/testReport)** for PR 16650 at commit

[GitHub] spark issue #16685: [SPARK-19335] Introduce insert, update, and upsert comma...

2017-01-24 Thread ilganeli
Github user ilganeli commented on the issue: https://github.com/apache/spark/pull/16685 @gatorsmile That makes a lot of sense. Here is a code snippet that relies on the database to do the UPSERT: ``` /** * Generate the SQL statement to perform an upsert (UPDATE

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16694 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16694 **[Test build #71946 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71946/testReport)** for PR 16694 at commit

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16694 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71946/ Test FAILed. ---

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16694 **[Test build #71946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71946/testReport)** for PR 16694 at commit

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

2017-01-24 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/14702 jenkins retest please test failure from build HiveSparkSubmitSuite `set hive.metastore.warehouse.dir` is unrelated to the change --- If your project is set up for it, you can reply to

[GitHub] spark issue #16685: [SPARK-19335] Introduce insert, update, and upsert comma...

2017-01-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16685 Currently, I do not have a solution for supporting the parallel mass UPDATE, because the rows in the DataFrame might be out of order and a global transaction is missing. The solution posted in

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16694 **[Test build #71945 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71945/testReport)** for PR 16694 at commit

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16694 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71945/ Test FAILed. ---

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16694 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16694 **[Test build #71945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71945/testReport)** for PR 16694 at commit

[GitHub] spark issue #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/16694 cc @hhbyyh Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16694: [SPARK-19336][ML][Pyspark]: LinearSVC Python API

2017-01-24 Thread wangmiao1981
GitHub user wangmiao1981 opened a pull request: https://github.com/apache/spark/pull/16694 [SPARK-19336][ML][Pyspark]: LinearSVC Python API ## What changes were proposed in this pull request? Add Python API for the newly added LinearSVC algorithm. ## How was this

[GitHub] spark issue #16650: [SPARK-16554][CORE] Automatically Kill Executors and Nod...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16650 **[Test build #71944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71944/testReport)** for PR 16650 at commit

[GitHub] spark issue #16650: [SPARK-16554][CORE] Automatically Kill Executors and Nod...

2017-01-24 Thread jsoltren
Github user jsoltren commented on the issue: https://github.com/apache/spark/pull/16650 Working through some test failures in org.apache.spark.deploy.StandaloneDynamicAllocationSuite and org.apache.spark.HeartbeatReceiverSuite... --- If your project is set up for it, you can reply

[GitHub] spark issue #16605: [SPARK-18884][SQL] Support Array[_] in ScalaUDF

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16605 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71940/ Test FAILed. ---

[GitHub] spark issue #16605: [SPARK-18884][SQL] Support Array[_] in ScalaUDF

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16605 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16605: [SPARK-18884][SQL] Support Array[_] in ScalaUDF

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16605 **[Test build #71940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71940/testReport)** for PR 16605 at commit

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16686 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16686 **[Test build #71942 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71942/testReport)** for PR 16686 at commit

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16686 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71942/ Test PASSed. ---

[GitHub] spark issue #16685: [SPARK-19335] Introduce insert, update, and upsert comma...

2017-01-24 Thread ilganeli
Github user ilganeli commented on the issue: https://github.com/apache/spark/pull/16685 I recognize that this is not an optimal solution, but Spark has historically contained multiple sub-optimal operations that are nonetheless useful in certain contexts and it's left to the user to

[GitHub] spark issue #16685: [SPARK-19335] Introduce insert, update, and upsert comma...

2017-01-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16685 To support the UPSERT, this PR basically implements it by using SELECT, UPDATE and INSERT. It has to read the whole table from the JDBC-connected database, and process it in Spark. It does not

[GitHub] spark issue #16685: [SPARK-19335] Introduce insert, update, and upsert comma...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16685 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71943/ Test FAILed. ---

[GitHub] spark issue #16685: [SPARK-19335] Introduce insert, update, and upsert comma...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16685 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16685: [SPARK-19335] Introduce insert, update, and upsert comma...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16685 **[Test build #71943 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71943/testReport)** for PR 16685 at commit

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-24 Thread jsoltren
Github user jsoltren commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97625393 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -168,6 +169,21 @@ private[scheduler] class BlacklistTracker (

[GitHub] spark issue #16685: [SPARK-19335] Introduce insert, update, and upsert comma...

2017-01-24 Thread ilganeli
Github user ilganeli commented on the issue: https://github.com/apache/spark/pull/16685 @gatorsmile What is a "key" update and in what context would that sort of operation be needed? I don't think a secondary index on the table prevent this method from working, the primary

[GitHub] spark issue #16685: [SPARK-19335] Introduce insert, update, and upsert comma...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16685 **[Test build #71943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71943/testReport)** for PR 16685 at commit

[GitHub] spark issue #16685: [SPARK-19335] Introduce insert, update, and upsert comma...

2017-01-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16685 I still have a concern for Update, even if we do an update based on key values. Using this way for update statements, we are still facing non-deterministic results. For example, you are unable

[GitHub] spark pull request #16685: [SPARK-19335] Introduce insert, update, and upser...

2017-01-24 Thread ilganeli
Github user ilganeli commented on a diff in the pull request: https://github.com/apache/spark/pull/16685#discussion_r97623026 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -722,14 +724,246 @@ object JdbcUtils extends

[GitHub] spark issue #15768: [SPARK-18080][ML][PySpark] Locality Sensitive Hashing (L...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15768 **[Test build #3550 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3550/consoleFull)** for PR 15768 at commit

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-24 Thread jsoltren
Github user jsoltren commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97622417 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -719,7 +719,11 @@ private[spark] object TaskSchedulerImpl {

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-24 Thread jsoltren
Github user jsoltren commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97622210 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -429,7 +429,7 @@ class TaskSetManagerSuite extends SparkFunSuite

[GitHub] spark pull request #16685: [SPARK-19335] Introduce insert, update, and upser...

2017-01-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16685#discussion_r97622295 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -722,14 +724,246 @@ object JdbcUtils extends

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-24 Thread jsoltren
Github user jsoltren commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97622188 --- Diff: core/src/test/scala/org/apache/spark/scheduler/BlacklistTrackerSuite.scala --- @@ -43,7 +43,7 @@ class BlacklistTrackerSuite extends

[GitHub] spark pull request #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-24 Thread tcondie
Github user tcondie commented on a diff in the pull request: https://github.com/apache/spark/pull/16686#discussion_r97620911 --- Diff: external/kafka-0-10-sql/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister --- @@ -1 +1 @@

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16686 **[Test build #71942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71942/testReport)** for PR 16686 at commit

[GitHub] spark pull request #16521: [SPARK-19139][core] New auth mechanism for transp...

2017-01-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16521 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16685: [SPARK-19335] Introduce insert, update, and upser...

2017-01-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16685#discussion_r97619926 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -722,14 +724,246 @@ object JdbcUtils extends

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14702 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71941/ Test FAILed. ---

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14702 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16521: [SPARK-19139][core] New auth mechanism for transport lib...

2017-01-24 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16521 Thanks. Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14702 **[Test build #71941 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71941/testReport)** for PR 14702 at commit

[GitHub] spark issue #16377: [SPARK-18036][ML][MLLIB] Fixing decision trees handling ...

2017-01-24 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/16377 Thanks @jkbradley! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16690: [SPARK-19347] ReceiverSupervisorImpl can add block to Re...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16690 **[Test build #71938 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71938/testReport)** for PR 16690 at commit

[GitHub] spark pull request #16377: [SPARK-18036][ML][MLLIB] Fixing decision trees ha...

2017-01-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16377 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16690: [SPARK-19347] ReceiverSupervisorImpl can add block to Re...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16690 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16690: [SPARK-19347] ReceiverSupervisorImpl can add block to Re...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16690 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71938/ Test FAILed. ---

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-24 Thread tcondie
Github user tcondie commented on the issue: https://github.com/apache/spark/pull/16686 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16377: [SPARK-18036][ML][MLLIB] Fixing decision trees handling ...

2017-01-24 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16377 LGTM Thanks! Merging with master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15880: [SPARK-17913][SQL] compare atomic and string type column...

2017-01-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15880 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15880: [SPARK-17913][SQL] compare atomic and string type...

2017-01-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15880 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16605: [SPARK-18884][SQL] Support Array[_] in ScalaUDF

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16605 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16605: [SPARK-18884][SQL] Support Array[_] in ScalaUDF

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16605 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71939/ Test FAILed. ---

[GitHub] spark issue #16605: [SPARK-18884][SQL] Support Array[_] in ScalaUDF

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16605 **[Test build #71939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71939/testReport)** for PR 16605 at commit

[GitHub] spark issue #16355: [SPARK-16473][MLLIB] Fix BisectingKMeans Algorithm faili...

2017-01-24 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16355 Oh OK! Thanks @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16677: [WIP][SQL] Use map output statistices to improve global ...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16677 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71936/ Test PASSed. ---

[GitHub] spark issue #16677: [WIP][SQL] Use map output statistices to improve global ...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16677 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16677: [WIP][SQL] Use map output statistices to improve global ...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16677 **[Test build #71936 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71936/testReport)** for PR 16677 at commit

[GitHub] spark issue #16582: [SPARK-19220][UI] Make redirection to HTTPS apply to all...

2017-01-24 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16582 > Why not simply remove old redirect handler like collection.removeHandler ? I find it cleaner to just not do something that to do it then have to undo things when they fail. --- If your

[GitHub] spark pull request #16582: [SPARK-19220][UI] Make redirection to HTTPS apply...

2017-01-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16582#discussion_r97610826 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -274,25 +277,28 @@ private[spark] object JettyUtils extends Logging {

[GitHub] spark pull request #16582: [SPARK-19220][UI] Make redirection to HTTPS apply...

2017-01-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16582#discussion_r97610759 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -337,17 +350,20 @@ private[spark] object JettyUtils extends Logging {

[GitHub] spark issue #11867: [SPARK-14049] [CORE] Add functionality in spark history ...

2017-01-24 Thread paragpc
Github user paragpc commented on the issue: https://github.com/apache/spark/pull/11867 thanks @squito :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16689: [SPARK-19342][SPARKR] bug fixed in collect method for co...

2017-01-24 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16689 yes. but please see my other comment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16661: [SPARK-19313][ML][MLLIB] GaussianMixture should limit th...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16661 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71937/ Test PASSed. ---

[GitHub] spark issue #16661: [SPARK-19313][ML][MLLIB] GaussianMixture should limit th...

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16661 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16661: [SPARK-19313][ML][MLLIB] GaussianMixture should limit th...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16661 **[Test build #71937 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71937/testReport)** for PR 16661 at commit

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14702 **[Test build #71941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71941/testReport)** for PR 14702 at commit

[GitHub] spark issue #16688: [TESTS][SQL] Setup testdata at the beginning for tests t...

2017-01-24 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/16688 cc @cloud-fan @gatorsmile Could you please trigger a test for this. Not sure why the last run didn't succeed. Thanks a lot. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #16605: [SPARK-18884][SQL] Support Array[_] in ScalaUDF

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16605 **[Test build #71940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71940/testReport)** for PR 16605 at commit

[GitHub] spark issue #16387: [SPARK-18986][Core] ExternalAppendOnlyMap shouldn't fail...

2017-01-24 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16387 No the question is whether you tested without @viirya commit `b1ef9ec` (the last one that forces spills of in-memory maps), or just the very last version of the patch. --- If your project is set

[GitHub] spark issue #16605: [SPARK-18884][SQL] Support Array[_] in ScalaUDF

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16605 **[Test build #71939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71939/testReport)** for PR 16605 at commit

[GitHub] spark pull request #16329: [SPARK-16046][DOCS] Aggregations in the Spark SQL...

2017-01-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16329#discussion_r97590536 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaUserDefinedTypedAggregation.java --- @@ -0,0 +1,160 @@ +/* + * Licensed to the

[GitHub] spark pull request #16329: [SPARK-16046][DOCS] Aggregations in the Spark SQL...

2017-01-24 Thread aokolnychyi
Github user aokolnychyi commented on a diff in the pull request: https://github.com/apache/spark/pull/16329#discussion_r97589440 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaUserDefinedTypedAggregation.java --- @@ -0,0 +1,160 @@ +/* + * Licensed to

[GitHub] spark issue #16690: [SPARK-19347] ReceiverSupervisorImpl can add block to Re...

2017-01-24 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/16690 cc @vanzin @zsxwing (I was going to make a longer comment about removing `askWithRetry` and whether we need another method, but then saw the comments on the referenced PR -- I'll defer to

[GitHub] spark issue #16605: [SPARK-18884][SQL] Support Array[_] in ScalaUDF

2017-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16605 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71935/ Test FAILed. ---

[GitHub] spark issue #16690: [SPARK-19347] ReceiverSupervisorImpl can add block to Re...

2017-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16690 **[Test build #71938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71938/testReport)** for PR 16690 at commit

<    1   2   3   4   5   >