[GitHub] spark pull request #21465: [SPARK-24333][ML][PYTHON]Add fit with validation ...

2018-12-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21465#discussion_r239240113 --- Diff: python/pyspark/ml/regression.py --- @@ -705,12 +710,59 @@ def getNumTrees(self): return self.getOrDefault(self.numTrees)

[GitHub] spark pull request #21465: [SPARK-24333][ML][PYTHON]Add fit with validation ...

2018-12-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21465#discussion_r239243661 --- Diff: python/pyspark/ml/classification.py --- @@ -1174,9 +1165,31 @@ def trees(self): return [DecisionTreeClassificationModel(m) for m

[GitHub] spark pull request #21465: [SPARK-24333][ML][PYTHON]Add fit with validation ...

2018-12-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21465#discussion_r239242316 --- Diff: python/pyspark/ml/classification.py --- @@ -1174,9 +1165,31 @@ def trees(self): return [DecisionTreeClassificationModel(m) for m

[GitHub] spark pull request #21465: [SPARK-24333][ML][PYTHON]Add fit with validation ...

2018-12-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21465#discussion_r239243683 --- Diff: python/pyspark/ml/classification.py --- @@ -1174,9 +1165,31 @@ def trees(self): return [DecisionTreeClassificationModel(m) for m

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239238873 --- Diff: docs/ml-clustering.md --- @@ -265,3 +265,44 @@ Refer to the [R API docs](api/R/spark.gaussianMixture.html) for more details.

[GitHub] spark pull request #23241: [SPARK-26283][CORE]When zstd compression enabled,...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239232597 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23241 **[Test build #99742 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99742/testReport)** for PR 23241 at commit

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23240: [SPARK-26281][WebUI] Duration column of task table shoul...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23240 Thanks. I will update the PR title --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23241 **[Test build #99741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99741/testReport)** for PR 23241 at commit

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23241 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239228923 --- Diff: examples/src/main/r/ml/powerIterationClustering.R --- @@ -0,0 +1,37 @@ +# +# Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239224970 --- Diff: docs/ml-clustering.md --- @@ -265,3 +265,44 @@ Refer to the [R API docs](api/R/spark.gaussianMixture.html) for more details.

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23239 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239224498 --- Diff: docs/ml-clustering.md --- @@ -265,3 +265,44 @@ Refer to the [R API docs](api/R/spark.gaussianMixture.html) for more details.

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23239 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99737/ Test PASSed. ---

[GitHub] spark issue #23207: [SPARK-26193][SQL] Implement shuffle write metrics in SQ...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23207 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23207: [SPARK-26193][SQL] Implement shuffle write metrics in SQ...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23207 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99736/ Test PASSed. ---

[GitHub] spark issue #23207: [SPARK-26193][SQL] Implement shuffle write metrics in SQ...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23207 **[Test build #99736 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99736/testReport)** for PR 23207 at commit

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23239 **[Test build #99737 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99737/testReport)** for PR 23239 at commit

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23239 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23239 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99738/ Test FAILed. ---

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23239 **[Test build #99738 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99738/testReport)** for PR 23239 at commit

[GitHub] spark pull request #23241: [SPARK-26283][CORE]When zstd compression enabled,...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239219205 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark pull request #23241: [SPARK-26283][CORE]When zstd compression enabled,...

2018-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239218209 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark pull request #23241: [SPARK-26283][CORE]When zstd compression enabled,...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239216282 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark pull request #23228: [MINOR][DOC]The condition description of serializ...

2018-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23228#discussion_r239215561 --- Diff: core/src/main/scala/org/apache/spark/shuffle/sort/SortShuffleManager.scala --- @@ -33,10 +33,10 @@ import org.apache.spark.shuffle._ *

[GitHub] spark issue #23240: [SPARK-26281][WebUI] Duration column of task table shoul...

2018-12-05 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/23240 Oh, I see. Close this one now. Please change the title in #23160 --- - To unsubscribe, e-mail:

[GitHub] spark pull request #23240: [SPARK-26281][WebUI] Duration column of task tabl...

2018-12-05 Thread gengliangwang
Github user gengliangwang closed the pull request at: https://github.com/apache/spark/pull/23240 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21465: [SPARK-24333][ML][PYTHON]Add fit with validation ...

2018-12-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21465#discussion_r239211515 --- Diff: python/pyspark/ml/classification.py --- @@ -1174,9 +1165,31 @@ def trees(self): return [DecisionTreeClassificationModel(m) for m

[GitHub] spark issue #23159: [SPARK-26191][SQL] Control truncation of Spark plans via...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23159 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23159: [SPARK-26191][SQL] Control truncation of Spark plans via...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23159 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99735/ Test PASSed. ---

[GitHub] spark issue #23236: [SPARK-26275][PYTHON][ML] Increases timeout for Streamin...

2018-12-05 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/23236 Seems ok to me, but there are a few silly things with this test that might help also * why is the `stepSize` so low at 0.01? I think it would be fine at 0.1, but even conservatively at

[GitHub] spark issue #23159: [SPARK-26191][SQL] Control truncation of Spark plans via...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23159 **[Test build #99735 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99735/testReport)** for PR 23159 at commit

[GitHub] spark pull request #23241: [SPARK-26283][CORE]When zstd compression enabled,...

2018-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239208072 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark issue #23227: [SPARK-26271][FOLLOW-UP][SQL] remove unuse object SparkP...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23227 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99731/ Test FAILed. ---

[GitHub] spark issue #23227: [SPARK-26271][FOLLOW-UP][SQL] remove unuse object SparkP...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23227 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23196: [SPARK-26243][SQL] Use java.time API for parsing timesta...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23196 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99734/ Test PASSed. ---

[GitHub] spark issue #23196: [SPARK-26243][SQL] Use java.time API for parsing timesta...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23196 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23227: [SPARK-26271][FOLLOW-UP][SQL] remove unuse object SparkP...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23227 **[Test build #99731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99731/testReport)** for PR 23227 at commit

[GitHub] spark issue #23196: [SPARK-26243][SQL] Use java.time API for parsing timesta...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23196 **[Test build #99734 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99734/testReport)** for PR 23196 at commit

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239203950 --- Diff: R/pkg/R/mllib_clustering.R --- @@ -610,3 +616,58 @@ setMethod("write.ml", signature(object = "LDAModel", path = "character"),

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23241 **[Test build #99740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99740/testReport)** for PR 23241 at commit

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23241 cc @vanzin @srowen Kindly review --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239198848 --- Diff: R/pkg/tests/fulltests/test_mllib_clustering.R --- @@ -319,4 +319,18 @@ test_that("spark.posterior and spark.perplexity", {

[GitHub] spark pull request #23241: [SPARK-26283][CORE]When zstd compression enabled,...

2018-12-05 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/23241 [SPARK-26283][CORE]When zstd compression enabled, Inprogress application in the history server appUI showing finished job as running. ## What changes were proposed in this pull request?

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239197337 --- Diff: R/pkg/tests/fulltests/test_mllib_fpm.R --- @@ -84,19 +84,21 @@ test_that("spark.fpGrowth", { })

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239194803 --- Diff: R/pkg/R/mllib_clustering.R --- @@ -610,3 +616,58 @@ setMethod("write.ml", signature(object = "LDAModel", path = "character"),

[GitHub] spark issue #23206: [SPARK-26249][SQL] Add ability to inject a rule in order...

2018-12-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/23206 cc @viirya @maropu --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22612: [SPARK-24958] Add executors' process tree total m...

2018-12-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22612#discussion_r239185785 --- Diff: core/src/main/scala/org/apache/spark/executor/ProcfsMetricsGetter.scala --- @@ -0,0 +1,229 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22612 **[Test build #4454 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4454/testReport)** for PR 22612 at commit

[GitHub] spark issue #23236: [SPARK-26275][PYTHON][ML] Increases timeout for Streamin...

2018-12-05 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/23236 lgtm --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23223: [SPARK-26269][YARN]Yarnallocator should have same blackl...

2018-12-05 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/23223 @attilapiros > I mean if node blacklisting in Spark would be perfectly aligned to YARN then it would be just redundant to have it in Spark in the first place. I'm not super

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

2018-12-05 Thread rezasafi
Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/22612 The failure isn't related to my change. Jenkins retest this please --- - To unsubscribe, e-mail:

[GitHub] spark issue #23218: [SPARK-26266][BUILD] Update to Scala 2.12.8

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/23218 Thank you, @srowen , @HyukjinKwon , @shaneknapp ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #23218: [SPARK-26266][BUILD] Update to Scala 2.12.8

2018-12-05 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/23218 https://issues.apache.org/jira/browse/SPARK-26282 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #23195: [SPARK-26236][SS] Add kafka delegation token supp...

2018-12-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23195#discussion_r239177994 --- Diff: docs/structured-streaming-kafka-integration.md --- @@ -624,3 +624,199 @@ For experimenting on `spark-shell`, you can also use `--packages` to add

[GitHub] spark issue #23240: [SPARK-26281][WebUI] Duration column of task table shoul...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23240 Hi @gengliangwang , It seems, this was already handled in the PR, https://github.com/apache/spark/pull/23160 --- - To

[GitHub] spark pull request #23223: [SPARK-26269][YARN]Yarnallocator should have same...

2018-12-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/23223#discussion_r239174670 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -612,11 +612,14 @@ private[yarn] class

[GitHub] spark pull request #23223: [SPARK-26269][YARN]Yarnallocator should have same...

2018-12-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/23223#discussion_r239173889 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala --- @@ -417,4 +426,59 @@ class YarnAllocatorSuite

[GitHub] spark pull request #21465: [SPARK-24333][ML][PYTHON]Add fit with validation ...

2018-12-05 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/21465#discussion_r239173098 --- Diff: python/pyspark/ml/classification.py --- @@ -1174,9 +1165,31 @@ def trees(self): return [DecisionTreeClassificationModel(m) for m in

[GitHub] spark issue #23240: [SPARK-26281][WebUI] Duration column of task table shoul...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23240 **[Test build #99739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99739/testReport)** for PR 23240 at commit

[GitHub] spark issue #23240: [SPARK-26281][WebUI] Duration column of task table shoul...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23240 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23240: [SPARK-26281][WebUI] Duration column of task table shoul...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23240 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23228: [MINOR][DOC]The condition description of serialized shuf...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23228 **[Test build #4453 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4453/testReport)** for PR 23228 at commit

[GitHub] spark issue #23240: [SPARK-26281][WebUI] Duration column of task table shoul...

2018-12-05 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/23240 @shahidki31 @pgandhi999 @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23240: [SPARK-26281][WebUI] Duration column of task tabl...

2018-12-05 Thread gengliangwang
GitHub user gengliangwang opened a pull request: https://github.com/apache/spark/pull/23240 [SPARK-26281][WebUI] Duration column of task table should be executor run time instead of real duration ## What changes were proposed in this pull request? In PR

[GitHub] spark issue #23232: [SPARK-26233][SQL][BACKPORT-2.4] CheckOverflow when enco...

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/23232 I merged all three PRs (2.4/2.3/2.2). Please close the PRs. :) --- - To unsubscribe, e-mail:

[GitHub] spark issue #23207: [SPARK-26193][SQL] Implement shuffle write metrics in SQ...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23207 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99722/ Test PASSed. ---

[GitHub] spark issue #23207: [SPARK-26193][SQL] Implement shuffle write metrics in SQ...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23207 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23207: [SPARK-26193][SQL] Implement shuffle write metrics in SQ...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23207 **[Test build #99722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99722/testReport)** for PR 23207 at commit

[GitHub] spark issue #23222: [SPARK-20636] Add the rule TransposeWindow to the optimi...

2018-12-05 Thread ptkool
Github user ptkool commented on the issue: https://github.com/apache/spark/pull/23222 I'm not sure how this ended up being omitted. `TransposeWindowSuite` will be fine since it creates a simple optimizer from this rule and a few others. The new test added to

[GitHub] spark issue #23218: [SPARK-26266][BUILD] Update to Scala 2.12.8

2018-12-05 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/23218 shouldn't be too hard, but it will require some downtime. On Wed, Dec 5, 2018 at 5:41 AM Sean Owen wrote: > Ah OK, so all of them were a JVM crash. It would probably be a good

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23239 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23239 **[Test build #99738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99738/testReport)** for PR 23239 at commit

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23239 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23239 Yes, the 3 cases I pointed that need to handle NaN and -0.0 do not change the value in `UnsafeRow`. --- - To unsubscribe,

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread adoron
Github user adoron commented on the issue: https://github.com/apache/spark/pull/23239 @cloud-fan what about UnsafeRow::setDouble/Float? It doesn't go through the same flow. Is it not used? --- - To unsubscribe,

[GitHub] spark issue #23232: [SPARK-26233][SQL][BACKPORT-2.4] CheckOverflow when enco...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23232 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23239 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23232: [SPARK-26233][SQL][BACKPORT-2.4] CheckOverflow when enco...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23232 **[Test build #99716 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99716/testReport)** for PR 23232 at commit

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23239 **[Test build #99737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99737/testReport)** for PR 23239 at commit

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23239 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23239 cc @adoron @kiszk @viirya @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23239: [SPARK-26021][SQL][followup] only deal with NaN a...

2018-12-05 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/23239 [SPARK-26021][SQL][followup] only deal with NaN and -0.0 in UnsafeWriter ## What changes were proposed in this pull request? A followup of https://github.com/apache/spark/pull/23043

[GitHub] spark issue #22305: [SPARK-24561][SQL][Python] User-defined window aggregati...

2018-12-05 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/22305 Hi @BryanCutler @HyukjinKwon @ueshin , mind taking another look? I think this is in a good shape. Thanks! --- - To

[GitHub] spark pull request #23235: [SPARK-26151][SQL][FOLLOWUP] Return partial resul...

2018-12-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23235 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23231: [SPARK-26273][ML] Add OneHotEncoderEstimator as alias to...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99712/ Test PASSed. ---

[GitHub] spark issue #23231: [SPARK-26273][ML] Add OneHotEncoderEstimator as alias to...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23231 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23223: [SPARK-26269][YARN]Yarnallocator should have same blackl...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23223 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99733/ Test PASSed. ---

[GitHub] spark issue #23223: [SPARK-26269][YARN]Yarnallocator should have same blackl...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23223 **[Test build #99733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99733/testReport)** for PR 23223 at commit

[GitHub] spark issue #23235: [SPARK-26151][SQL][FOLLOWUP] Return partial results for ...

2018-12-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23235 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23227: [SPARK-26271][FOLLOW-UP][SQL] remove unuse object SparkP...

2018-12-05 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/23227 thanks,@cloud-fan,@HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #23223: [SPARK-26269][YARN]Yarnallocator should have same blackl...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23223 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   5   6   7   >