[GitHub] spark issue #16043: [SPARK-18601][SQL] Simplify Create/Get complex expressio...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16043 **[Test build #72209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72209/testReport)** for PR 16043 at commit

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-01-31 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/16740 @sethah Yes, we can directly compute the intercept easily. But I'm concerned that such special handling may not integrate well with other features or future changes. For example, we will need

[GitHub] spark pull request #16760: [SPARK-18872][SQL][TESTS] New test cases for EXIS...

2017-01-31 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/16760 [SPARK-18872][SQL][TESTS] New test cases for EXISTS subquery (Aggregate, Having, Orderby, Limit) ## What changes were proposed in this pull request? This PR adds the second set

[GitHub] spark issue #16760: [SPARK-18872][SQL][TESTS] New test cases for EXISTS subq...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16760 **[Test build #72208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72208/testReport)** for PR 16760 at commit

[GitHub] spark issue #16759: [SPARK-18871][SQL][TESTS] New test cases for IN/NOT IN s...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16759 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-31 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r98782219 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -187,6 +198,19 @@ private[scheduler] class BlacklistTracker (

[GitHub] spark issue #14725: [SPARK-17161] [PYSPARK][ML] Add PySpark-ML JavaWrapper c...

2017-01-31 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14725 LGTM - thanks for doing this - please ping me on the follow up PRs with the `CountVectorizerModel`. Before merge would you mind updating the PR description for how this was tested to remove the

[GitHub] spark pull request #16759: [SPARK-18871][SQL][TESTS] New test cases for IN/N...

2017-01-31 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/16759 [SPARK-18871][SQL][TESTS] New test cases for IN/NOT IN subquery 2nd batch ## What changes were proposed in this pull request? This is 2nd batch of test case for IN/NOT IN subquery. In

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-31 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r98781265 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -187,6 +198,19 @@ private[scheduler] class BlacklistTracker (

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-31 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r98781173 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -173,6 +174,16 @@ private[scheduler] class BlacklistTracker (

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98778359 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -313,6 +313,25 @@ abstract class SparkStrategies extends

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98776316 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -219,6 +219,160 @@ class KeyValueGroupedDataset[K, V]

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98774221 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -219,6 +219,160 @@ class KeyValueGroupedDataset[K, V]

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98779817 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulAggregate.scala --- @@ -235,3 +240,86 @@ case class StateStoreSaveExec(

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98778114 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/State.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98779663 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulAggregate.scala --- @@ -235,3 +240,86 @@ case class StateStoreSaveExec(

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98775275 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -219,6 +219,160 @@ class KeyValueGroupedDataset[K, V]

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98778548 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/StateImpl.scala --- @@ -0,0 +1,70 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98779439 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulAggregate.scala --- @@ -235,3 +240,86 @@ case class StateStoreSaveExec(

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98777503 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/State.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98780383 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/objects.scala --- @@ -144,6 +145,12 @@ object ObjectOperator { (i: InternalRow)

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98779599 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulAggregate.scala --- @@ -235,3 +240,86 @@ case class StateStoreSaveExec(

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98779015 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/IncrementalExecution.scala --- @@ -90,6 +93,14 @@ class IncrementalExecution(

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98775825 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -219,6 +219,160 @@ class KeyValueGroupedDataset[K, V]

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98780164 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulAggregate.scala --- @@ -184,7 +189,7 @@ case class StateStoreSaveExec(

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98778823 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/StateImpl.scala --- @@ -0,0 +1,70 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98779264 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulAggregate.scala --- @@ -54,6 +55,18 @@ trait StatefulOperator extends

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98777585 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/State.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98779142 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulAggregate.scala --- @@ -54,6 +55,18 @@ trait StatefulOperator extends

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98777860 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/State.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98778773 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/StateImpl.scala --- @@ -0,0 +1,70 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98776628 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -219,6 +219,160 @@ class KeyValueGroupedDataset[K, V]

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98777095 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -219,6 +219,160 @@ class KeyValueGroupedDataset[K, V]

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98776799 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -219,6 +219,160 @@ class KeyValueGroupedDataset[K, V]

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98776538 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -219,6 +219,160 @@ class KeyValueGroupedDataset[K, V]

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98778649 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/StateImpl.scala --- @@ -0,0 +1,70 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98777806 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/State.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98777964 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/State.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #14725: [SPARK-17161] [PYSPARK][ML] Add PySpark-ML JavaWr...

2017-01-31 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14725#discussion_r98780626 --- Diff: python/pyspark/ml/wrapper.py --- @@ -16,6 +16,10 @@ # from abc import ABCMeta, abstractmethod +import sys --- End diff --

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-01-31 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r98778763 --- Diff: mllib-local/src/test/scala/org/apache/spark/ml/util/TestingUtils.scala --- @@ -48,7 +48,7 @@ object TestingUtils { /** * Private

[GitHub] spark pull request #16696: [SPARK-19350] [SQL] Cardinality estimation of Lim...

2017-01-31 Thread ron8hu
Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16696#discussion_r98776491 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -727,37 +728,18 @@ case class

[GitHub] spark issue #16666: [SPARK-19319][SparkR]:SparkR Kmeans summary returns erro...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #72207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72207/testReport)** for PR 1 at commit

[GitHub] spark issue #16666: [SPARK-19319][SparkR]:SparkR Kmeans summary returns erro...

2017-01-31 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/1 ping @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16757: [SPARK-18609][SQL] Fix redundant Alias removal in the op...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16757 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16757: [SPARK-18609][SQL] Fix redundant Alias removal in the op...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16757 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72203/ Test PASSed. ---

[GitHub] spark issue #16757: [SPARK-18609][SQL] Fix redundant Alias removal in the op...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16757 **[Test build #72203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72203/testReport)** for PR 16757 at commit

[GitHub] spark pull request #16730: [SPARK-19395][SparkR]Convert coefficients in summ...

2017-01-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16730 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16730: [SPARK-19395][SparkR]Convert coefficients in summary to ...

2017-01-31 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16730 merged to master, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-01-31 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/16740 Allowing offset will only require a small change to the intercept calculation, won't it? scala val agg = data.agg(sum(w * (col("label") - col("label"))), sum(w)).first()

[GitHub] spark pull request #16729: [SPARK-19391][SparkR][ML] Tweedie GLM API for Spa...

2017-01-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16729#discussion_r98760363 --- Diff: R/pkg/R/mllib_regression.R --- @@ -53,12 +53,19 @@ setClass("IsotonicRegressionModel", representation(jobj = "jobj")) #'

[GitHub] spark pull request #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16686#discussion_r98749100 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceRDD.scala --- @@ -135,7 +136,24 @@ private[kafka010] class

[GitHub] spark pull request #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16686#discussion_r98746133 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala --- @@ -249,6 +249,24 @@ class KafkaTestUtils extends

[GitHub] spark pull request #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16686#discussion_r98746146 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala --- @@ -249,6 +249,24 @@ class KafkaTestUtils extends

[GitHub] spark pull request #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16686#discussion_r98746070 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala --- @@ -274,6 +292,11 @@ class KafkaTestUtils extends

[GitHub] spark pull request #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16686#discussion_r98756883 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala --- @@ -108,7 +113,14 @@ private[kafka010] class

[GitHub] spark pull request #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16686#discussion_r98749048 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceRDD.scala --- @@ -135,7 +136,24 @@ private[kafka010] class

[GitHub] spark issue #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrary state...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrary state...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72206/ Test FAILed. ---

[GitHub] spark issue #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrary state...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16758 **[Test build #72206 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72206/testReport)** for PR 16758 at commit

[GitHub] spark issue #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrary state...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16758 **[Test build #72206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72206/testReport)** for PR 16758 at commit

[GitHub] spark issue #16751: [SPARK-19409][BUILD] Bump parquet version to 1.8.2

2017-01-31 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16751 can you put rest of the cleanups in one place? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14866: [SPARK-17298][SQL] Require explicit CROSS join fo...

2017-01-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14866#discussion_r98751522 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -933,6 +936,47 @@ object CombineLimits extends

[GitHub] spark pull request #14866: [SPARK-17298][SQL] Require explicit CROSS join fo...

2017-01-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14866#discussion_r98751322 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -933,6 +936,47 @@ object CombineLimits extends

[GitHub] spark pull request #14866: [SPARK-17298][SQL] Require explicit CROSS join fo...

2017-01-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14866#discussion_r98750834 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -933,6 +936,47 @@ object CombineLimits extends

[GitHub] spark pull request #16746: [SPARK-15648][SQL] Add teradataDialect for JDBC c...

2017-01-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16746#discussion_r98751002 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/TeradataDialect.scala --- @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-01-31 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r98748723 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulAggregate.scala --- @@ -54,6 +55,18 @@ trait StatefulOperator extends

[GitHub] spark pull request #16746: [SPARK-15648][SQL] Add teradataDialect for JDBC c...

2017-01-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16746#discussion_r98750209 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/TeradataDialect.scala --- @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #15237: [SPARK-17663] [CORE] SchedulableBuilder should handle in...

2017-01-31 Thread erenavsarogullari
Github user erenavsarogullari commented on the issue: https://github.com/apache/spark/pull/15237 Hi @squito, Sorry, i am quite busy this week and happy to be merged this PR now if it is also ok for you. I plan to address fileName logging via separated jira. Also To

[GitHub] spark issue #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrary state...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72205/ Test FAILed. ---

[GitHub] spark issue #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrary state...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16758 **[Test build #72205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72205/testReport)** for PR 16758 at commit

[GitHub] spark issue #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrary state...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16690: [SPARK-19347] ReceiverSupervisorImpl can add bloc...

2017-01-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16690#discussion_r98747972 --- Diff: core/src/main/scala/org/apache/spark/rpc/RpcEndpointRef.scala --- @@ -75,10 +106,11 @@ private[spark] abstract class RpcEndpointRef(conf:

[GitHub] spark pull request #16690: [SPARK-19347] ReceiverSupervisorImpl can add bloc...

2017-01-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16690#discussion_r98748039 --- Diff: core/src/main/scala/org/apache/spark/rpc/RpcEndpointRef.scala --- @@ -19,6 +19,7 @@ package org.apache.spark.rpc import

[GitHub] spark pull request #16690: [SPARK-19347] ReceiverSupervisorImpl can add bloc...

2017-01-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16690#discussion_r98747902 --- Diff: core/src/main/scala/org/apache/spark/rpc/RpcEndpointRef.scala --- @@ -91,6 +123,7 @@ private[spark] abstract class RpcEndpointRef(conf: SparkConf)

[GitHub] spark issue #16751: [SPARK-19409][BUILD] Bump parquet version to 1.8.2

2017-01-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16751 Ya, @mallman . However, with the same reason, I conclude to put them away from here. Exactly, the opposite direction of your opinion. If we try to fix all of them in a single shot,

[GitHub] spark issue #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrary state...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16758 **[Test build #72205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72205/testReport)** for PR 16758 at commit

[GitHub] spark pull request #16758: Arbitrary stateful operations with MapGroupsWithS...

2017-01-31 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/16758 Arbitrary stateful operations with MapGroupsWithState ## What changes were proposed in this pull request? `mapGroupsWithState` is a new API for arbitrary stateful operations in Structured

[GitHub] spark pull request #16603: [SPARK-19244][Core] Sort MemoryConsumers accordin...

2017-01-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16603#discussion_r98743557 --- Diff: core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java --- @@ -144,23 +148,53 @@ public long acquireExecutionMemory(long required,

[GitHub] spark pull request #16603: [SPARK-19244][Core] Sort MemoryConsumers accordin...

2017-01-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16603#discussion_r98744044 --- Diff: core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java --- @@ -144,23 +148,53 @@ public long acquireExecutionMemory(long required,

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72204/ Test FAILed. ---

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16603: [SPARK-19244][Core] Sort MemoryConsumers accordin...

2017-01-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16603#discussion_r98743237 --- Diff: core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java --- @@ -144,23 +148,53 @@ public long acquireExecutionMemory(long required,

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #72204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72204/testReport)** for PR 15009 at commit

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #72204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72204/testReport)** for PR 15009 at commit

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72201/ Test PASSed. ---

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #72201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72201/consoleFull)** for PR 15009 at commit

[GitHub] spark issue #16751: [SPARK-19409][BUILD] Bump parquet version to 1.8.2

2017-01-31 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16751 FYI, there are at least two workarounds in the Spark codebase which can potentially be removed as a consequence of this upgrade. For example:

[GitHub] spark issue #16757: [SPARK-18609][SQL] Fix redundant Alias removal in the op...

2017-01-31 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16757 cc @cloud-fan @windpiger @sameeragarwal --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14725: [SPARK-17161] [PYSPARK][ML] Add PySpark-ML JavaWr...

2017-01-31 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/14725#discussion_r98736777 --- Diff: python/pyspark/ml/wrapper.py --- @@ -16,6 +16,10 @@ # from abc import ABCMeta, abstractmethod +import sys --- End diff

[GitHub] spark issue #16757: [SPARK-18609][SQL] Fix redundant Alias removal in the op...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16757 **[Test build #72203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72203/testReport)** for PR 16757 at commit

[GitHub] spark pull request #15120: [SPARK-4563][core] Allow driver to advertise a di...

2017-01-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15120#discussion_r98734873 --- Diff: core/src/main/scala/org/apache/spark/internal/config/ConfigProvider.scala --- @@ -66,7 +66,7 @@ private[spark] class SparkConfigProvider(conf:

[GitHub] spark issue #16757: [SPARK-18609][SQL] Fix redundant Alias removal in the op...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16757 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72202/ Test FAILed. ---

[GitHub] spark issue #16757: [SPARK-18609][SQL] Fix redundant Alias removal in the op...

2017-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16757 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16757: [SPARK-18609][SQL] Fix redundant Alias removal in the op...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16757 **[Test build #72202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72202/testReport)** for PR 16757 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-31 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Pinging @tdas on this-- looks like you're the committer who has contributed the most to kinesis-asl. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2017-01-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16281 Thank you for sharing that information, @mallman . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #16735: [SPARK-19228][SQL] Introduce tryParseDate method ...

2017-01-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16735#discussion_r98729386 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVInferSchema.scala --- @@ -140,12 +137,21 @@ private[csv] object

[GitHub] spark issue #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2017-01-31 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16281 FYI, we've been using 1.9.0 patched with a fix for https://issues.apache.org/jira/browse/PARQUET-783 without problem. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #16757: [SPARK-18609][SQL] Fix redundant Alias removal in the op...

2017-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16757 **[Test build #72202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72202/testReport)** for PR 16757 at commit

<    1   2   3   4   >