[GitHub] spark issue #20319: [SPARK-22884][ML][TESTS] ML test for StructuredStreaming...

2018-04-09 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20319 @WeichenXu123 thanks for checking it out. I've resolved the conflicts, build is green. --- - To unsubscribe, e-mail: reviews

[GitHub] spark pull request #20235: [Spark-22887][ML][TESTS][WIP] ML test for Structu...

2018-04-09 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20235#discussion_r180124798 --- Diff: mllib/src/test/scala/org/apache/spark/ml/fpm/FPGrowthSuite.scala --- @@ -34,86 +35,122 @@ class FPGrowthSuite extends SparkFunSuite

[GitHub] spark issue #20235: [Spark-22887][ML][TESTS][WIP] ML test for StructuredStre...

2018-04-09 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20235 @WeichenXu123 Executing `test("FPGrowth fit and transform with different data types")` with Byte data type produces an `org.apache.spark.sql.AnalysisException: cannot resolve 'C

[GitHub] spark issue #20045: [Spark-22360][SQL][TEST] Add unit tests for Window Speci...

2018-04-09 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20045 ping @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #20045: [Spark-22360][SQL][TEST] Add unit tests for Window Speci...

2018-03-14 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20045 ping @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-03-05 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r172122977 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/MultilayerPerceptronClassifierSuite.scala --- @@ -75,11 +71,9 @@ class

[GitHub] spark issue #20045: [Spark-22360][SQL][TEST] Add unit tests for Window Speci...

2018-03-05 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20045 I think I've addressed all your points @jiangxb1987. Do you think I need any more changes? --- - To unsubscribe, e-mail

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166886232 --- Diff: core/src/main/java/org/apache/spark/metrics/prometheus/client/exporter/TextFormatWithTimestamp.java --- @@ -0,0 +1,178

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166883609 --- Diff: core/src/main/java/org/apache/spark/metrics/prometheus/client/exporter/TextFormatWithTimestamp.java --- @@ -0,0 +1,178

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166883372 --- Diff: core/src/main/java/org/apache/spark/metrics/prometheus/client/exporter/PushGatewayWithTimestamp.java --- @@ -0,0 +1,320

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166892802 --- Diff: core/src/main/scala/org/apache/spark/metrics/sink/PrometheusSink.scala --- @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166889139 --- Diff: core/src/main/java/org/apache/spark/metrics/prometheus/client/exporter/TextFormatWithTimestamp.java --- @@ -0,0 +1,178

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166896270 --- Diff: core/src/main/java/org/apache/spark/metrics/prometheus/client/exporter/PushGatewayWithTimestamp.java --- @@ -0,0 +1,320

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166883368 --- Diff: core/src/main/java/org/apache/spark/metrics/prometheus/client/exporter/TextFormatWithTimestamp.java --- @@ -0,0 +1,178

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166881873 --- Diff: core/src/main/java/org/apache/spark/metrics/prometheus/client/exporter/PushGatewayWithTimestamp.java --- @@ -0,0 +1,320

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166886527 --- Diff: core/src/main/java/org/apache/spark/metrics/prometheus/client/exporter/TextFormatWithTimestamp.java --- @@ -0,0 +1,178

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166896081 --- Diff: core/src/main/java/org/apache/spark/metrics/prometheus/client/exporter/PushGatewayWithTimestamp.java --- @@ -0,0 +1,320

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166886839 --- Diff: core/src/main/java/org/apache/spark/metrics/prometheus/client/exporter/TextFormatWithTimestamp.java --- @@ -0,0 +1,178

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166894472 --- Diff: core/src/main/scala/org/apache/spark/metrics/sink/PrometheusSink.scala --- @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19775: [SPARK-22343][core] Add support for publishing Sp...

2018-02-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19775#discussion_r166894977 --- Diff: core/src/main/scala/org/apache/spark/metrics/sink/PrometheusSink.scala --- @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-02 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r165745445 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -599,8 +598,11 @@ class ALSSuite (ex, act

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-01 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r165304423 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -566,6 +565,7 @@ class ALSSuite test("read/

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-01 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r165308129 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -628,18 +635,24 @@ class ALSSuite } withClue

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-01 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r165305989 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -599,8 +599,15 @@ class ALSSuite (ex, act

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-01 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r165312057 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -662,28 +676,32 @@ class ALSSuite val knownItem

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-01 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r165312157 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -693,7 +711,9 @@ class ALSSuite val data = ratings.toDF

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-01 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r165308205 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -653,6 +666,7 @@ class ALSSuite test("ALS cold start

[GitHub] spark issue #20045: [Spark-22360][SQL][TEST] Add unit tests for Window Speci...

2018-01-29 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20045 Do you think I need to cover any other cases, @jiangxb1987 ? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #20235: [Spark-22887][ML][TESTS][WIP] ML test for Structu...

2018-01-29 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20235#discussion_r164427189 --- Diff: mllib/src/test/scala/org/apache/spark/ml/fpm/FPGrowthSuite.scala --- @@ -34,86 +35,122 @@ class FPGrowthSuite extends SparkFunSuite

[GitHub] spark issue #20330: [SPARK-23121][core] Fix for ui becoming unaccessible for...

2018-01-22 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20330 Thanks for your help @vanzin, @gengliangwang, @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20046: [SPARK-22362][SQL] Add unit test for Window Aggregate Fu...

2018-01-22 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20046 @jiangxb1987 how does your request to cover the sql interface relates to SPARK-23160? I assume it is to be covered in that issue, is that correct

[GitHub] spark pull request #20330: [SPARK-23121][core] Fix for ui becoming unaccessi...

2018-01-20 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20330#discussion_r162792383 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -1002,4 +1000,12 @@ private object ApiHelper

[GitHub] spark pull request #20330: [SPARK-23121][core] Fix for ui becoming unaccessi...

2018-01-19 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20330#discussion_r162759866 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -427,23 +435,21 @@ private[ui] class JobDataSource( val

[GitHub] spark issue #20319: [SPARK-22884][ML][TESTS] ML test for StructuredStreaming...

2018-01-19 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20319 @jkbradley could you check out this change, please? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20235: [Spark-22887][ML][TESTS][WIP] ML test for StructuredStre...

2018-01-19 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20235 @jkbradley could you check out this change, please? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #20330: [SPARK-23121][core] Fix for ui becoming unaccessi...

2018-01-19 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20330#discussion_r162712767 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -427,23 +435,21 @@ private[ui] class JobDataSource( val

[GitHub] spark pull request #20330: [SPARK-23121][core] Fix for ui becoming unaccessi...

2018-01-19 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20330#discussion_r162622469 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -65,10 +68,13 @@ private[ui] class AllJobsPage(parent: JobsTab, store

[GitHub] spark issue #20330: [SPARK-23121][core] Fix for ui becoming unaccessible for...

2018-01-19 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20330 cc @jiangxb1987, @srowen, @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #20330: [SPARK-23121][core] Fix for ui becoming unaccessible for...

2018-01-19 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20330 @guoxiaolongzte could you please check if this change fixes the issue you have observed? --- - To unsubscribe, e-mail

[GitHub] spark pull request #20330: [SPARK-23121][core] Fix for ui becoming unaccessi...

2018-01-19 Thread smurakozi
GitHub user smurakozi opened a pull request: https://github.com/apache/spark/pull/20330 [SPARK-23121][core] Fix for ui becoming unaccessible for long running streaming apps ## What changes were proposed in this pull request? The allJobs and the job pages attempt to use

[GitHub] spark pull request #20287: [SPARK-23121][WEB-UI] When the Spark Streaming ap...

2018-01-19 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20287#discussion_r162566289 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -427,17 +430,24 @@ private[ui] class JobDataSource( val

[GitHub] spark pull request #20287: [SPARK-23121][WEB-UI] When the Spark Streaming ap...

2018-01-19 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20287#discussion_r162566562 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobPage.scala --- @@ -335,9 +335,12 @@ private[ui] class JobPage(parent: JobsTab, store

[GitHub] spark pull request #20319: [SPARK-22884][ML][TESTS] ML test for StructuredSt...

2018-01-18 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20319#discussion_r162460705 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/Encoders.scala --- @@ -0,0 +1,25 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #20319: [SPARK-22884][ML][TESTS] ML test for StructuredSt...

2018-01-18 Thread smurakozi
GitHub user smurakozi opened a pull request: https://github.com/apache/spark/pull/20319 [SPARK-22884][ML][TESTS] ML test for StructuredStreaming: spark.ml.clustering ## What changes were proposed in this pull request? Converting clustering tests to also check code

[GitHub] spark issue #20045: [Spark-22360][SQL][TEST] Add unit tests for Window Speci...

2018-01-17 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20045 @gatorsmile @hvanhovell @jiangxb1987, could you have a look, please? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-01-17 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r162052224 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -29,15 +29,14 @@ import

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-01-17 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r162053757 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/MultilayerPerceptronClassifierSuite.scala --- @@ -75,11 +71,9 @@ class

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-01-17 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r162037613 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/GBTClassifierSuite.scala --- @@ -169,59 +171,28 @@ class GBTClassifierSuite extends

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-01-17 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r162058792 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/OneVsRestSuite.scala --- @@ -17,14 +17,13 @@ package

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-01-17 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r162051926 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/MultilayerPerceptronClassifierSuite.scala --- @@ -17,22 +17,18 @@ package

[GitHub] spark issue #20286: [SPARK-23119][SS] Minor fixes to V2 streaming APIs

2018-01-17 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20286 I think this change is OK, except the nits zsxwing already noted. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161763793 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/ConsoleWriter.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161760765 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/sources/ConsoleWriterSuite.scala --- @@ -0,0 +1,79

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161763865 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/sources/ConsoleWriterSuite.scala --- @@ -0,0 +1,79

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161749201 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/ConsoleWriter.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed

[GitHub] spark pull request #20251: [Spark-23051][core] Fix for broken job descriptio...

2018-01-12 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20251#discussion_r161293689 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -429,20 +429,40 @@ private[ui] class JobDataSource( val

[GitHub] spark pull request #20251: [Spark-23051][core] Fix for broken job descriptio...

2018-01-12 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20251#discussion_r161276636 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -429,20 +429,40 @@ private[ui] class JobDataSource( val

[GitHub] spark pull request #20251: [Spark-23051][core] Fix for broken job descriptio...

2018-01-12 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20251#discussion_r161273733 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -429,20 +429,40 @@ private[ui] class JobDataSource( val

[GitHub] spark pull request #20251: [Spark-23051][core] Fix for broken job descriptio...

2018-01-12 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20251#discussion_r161273851 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -429,20 +429,40 @@ private[ui] class JobDataSource( val

[GitHub] spark issue #20251: [Spark-23051][core] Fix for broken job description in Sp...

2018-01-12 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20251 Screenshots: [In 2.2](https://issues.apache.org/jira/secure/attachment/12905889/in-2.2.png) [In 2.3 before fix](https://issues.apache.org/jira/secure/attachment/12905887/Spark-23051

[GitHub] spark pull request #20251: [Spark-23051][core] Fix for broken job descriptio...

2018-01-12 Thread smurakozi
GitHub user smurakozi opened a pull request: https://github.com/apache/spark/pull/20251 [Spark-23051][core] Fix for broken job description in Spark UI ## What changes were proposed in this pull request? In 2.2, Spark UI displayed the stage description if the job description

[GitHub] spark pull request #20203: [SPARK-22577] [core] executor page blacklist stat...

2018-01-11 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20203#discussion_r161009520 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetBlacklistSuite.scala --- @@ -110,7 +131,13 @@ class TaskSetBlacklistSuite extends

[GitHub] spark pull request #20203: [SPARK-22577] [core] executor page blacklist stat...

2018-01-11 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20203#discussion_r161001141 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -211,6 +211,11 @@ private[spark] class AppStatusListener

[GitHub] spark pull request #20203: [SPARK-22577] [core] executor page blacklist stat...

2018-01-11 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20203#discussion_r161005667 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -223,6 +228,15 @@ private[spark] class AppStatusListener

[GitHub] spark pull request #20203: [SPARK-22577] [core] executor page blacklist stat...

2018-01-11 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20203#discussion_r161009908 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetBlacklistSuite.scala --- @@ -157,13 +187,19 @@ class TaskSetBlacklistSuite extends

[GitHub] spark pull request #20235: [Spark-22887][ML][TESTS][WIP] ML test for Structu...

2018-01-11 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20235#discussion_r160980529 --- Diff: mllib/src/test/scala/org/apache/spark/ml/fpm/FPGrowthSuite.scala --- @@ -34,86 +35,122 @@ class FPGrowthSuite extends SparkFunSuite

[GitHub] spark pull request #20235: [Spark-22887][ML][TESTS][WIP] ML test for Structu...

2018-01-11 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20235#discussion_r160969767 --- Diff: mllib/src/test/scala/org/apache/spark/ml/fpm/FPGrowthSuite.scala --- @@ -34,86 +35,122 @@ class FPGrowthSuite extends SparkFunSuite

[GitHub] spark pull request #20235: [Spark-22887][ML][TESTS][WIP] ML test for Structu...

2018-01-11 Thread smurakozi
GitHub user smurakozi opened a pull request: https://github.com/apache/spark/pull/20235 [Spark-22887][ML][TESTS][WIP] ML test for StructuredStreaming: spark.ml.fpm ## What changes were proposed in this pull request? Converting FPGrowth tests to also check code

[GitHub] spark issue #20045: [Spark-22360][SQL][TEST] Add unit tests for Window Speci...

2018-01-03 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20045 @gatorsmile @hvanhovell @jiangxb1987, could you have a look, please? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #20046: Adding unit tests for missing aggregate functions

2017-12-21 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20046#discussion_r158326073 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala --- @@ -154,6 +154,217 @@ class DataFrameWindowFunctionsSuite

[GitHub] spark pull request #20045: [Spark-22360][SQL] Add unit tests for Window Spec...

2017-12-21 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20045#discussion_r158304105 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala --- @@ -32,6 +32,217 @@ import

[GitHub] spark pull request #20045: [Spark-22360][SQL] Add unit tests for Window Spec...

2017-12-21 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20045#discussion_r158304013 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala --- @@ -246,13 +246,23 @@ class

[GitHub] spark pull request #20045: [Spark-22360][SQL] Add unit tests for Window Spec...

2017-12-21 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20045#discussion_r158303978 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala --- @@ -246,13 +246,23 @@ class

[GitHub] spark pull request #20045: [Spark-22360][SQL] Add unit tests for Window Spec...

2017-12-21 Thread smurakozi
GitHub user smurakozi opened a pull request: https://github.com/apache/spark/pull/20045 [Spark-22360][SQL] Add unit tests for Window Specifications ## What changes were proposed in this pull request? Improve the test coverage of window specifications. New tests

[GitHub] spark pull request #19599: [SPARK-22381] [ML] Add StringParam that supports ...

2017-12-13 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19599#discussion_r156609625 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -435,6 +435,43 @@ class BooleanParam(parent: String, name: String, doc

[GitHub] spark pull request #19599: [SPARK-22381] [ML] Add StringParam that supports ...

2017-12-13 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19599#discussion_r156609525 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -435,6 +435,43 @@ class BooleanParam(parent: String, name: String, doc

[GitHub] spark pull request #19599: [SPARK-22381] [ML] Add StringParam that supports ...

2017-12-13 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19599#discussion_r156605360 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -435,6 +435,43 @@ class BooleanParam(parent: String, name: String, doc

[GitHub] spark pull request #19599: [SPARK-22381] [ML] Add StringParam that supports ...

2017-12-12 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19599#discussion_r156339663 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -435,6 +435,43 @@ class BooleanParam(parent: String, name: String, doc

[GitHub] spark pull request #19599: [SPARK-22381] [ML] Add StringParam that supports ...

2017-12-12 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19599#discussion_r155782239 --- Diff: mllib/src/test/scala/org/apache/spark/ml/param/ParamsSuite.scala --- @@ -139,6 +139,17 @@ class ParamsSuite extends SparkFunSuite

[GitHub] spark pull request #19599: [SPARK-22381] [ML] Add StringParam that supports ...

2017-12-12 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19599#discussion_r156086625 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -435,6 +435,43 @@ class BooleanParam(parent: String, name: String, doc

[GitHub] spark pull request #19599: [SPARK-22381] [ML] Add StringParam that supports ...

2017-12-12 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19599#discussion_r156339373 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -224,8 +222,8 @@ class LinearRegression @Since("

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-09 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/19920 Thanks for your help @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155852268 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala --- @@ -88,4 +90,8 @@ private[history] class HistoryPage(parent

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-08 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155725705 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -24,27 +24,29 @@ import

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155718836 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala --- @@ -88,4 +90,8 @@ private[history] class HistoryPage(parent

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155657473 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -17,16 +17,15 @@ package org.apache.spark.status

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155657274 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala --- @@ -30,7 +31,8 @@ private[history] class HistoryPage(parent

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155657306 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala --- @@ -88,4 +90,9 @@ private[history] class HistoryPage(parent

[GitHub] spark issue #19906: [SPARK-22516][SQL] Bump up Univocity version to 2.5.9

2017-12-07 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/19906 Thanks for your help and reviews @HyukjinKwon, @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155632345 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -24,27 +24,32 @@ import

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155631788 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -25,6 +25,7 @@ import scala.collection.JavaConverters._ import

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155631687 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -29,6 +29,7 @@ import org.apache.spark.executor.TaskMetrics

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155631377 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala --- @@ -188,11 +188,11 @@ class HistoryServer

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155611738 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -24,27 +24,32 @@ import

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155610889 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -24,27 +24,32 @@ import

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155609528 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -29,6 +29,7 @@ import org.apache.spark.executor.TaskMetrics

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-07 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/19920 PR description was cleaned, thanks for catching it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
GitHub user smurakozi opened a pull request: https://github.com/apache/spark/pull/19920 [SPARK-21672][CORE] Remove SHS-specific application / attempt data … …structures ## What changes were proposed in this pull request? In general, the SHS pages now use

[GitHub] spark pull request #19906: [SPARK-22516][SQL] Bump up Univocity version to 2...

2017-12-06 Thread smurakozi
GitHub user smurakozi opened a pull request: https://github.com/apache/spark/pull/19906 [SPARK-22516][SQL] Bump up Univocity version to 2.5.9 ## What changes were proposed in this pull request? There was a bug in Univocity Parser that causes the issue in SPARK-22516

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-05 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/19893 Logging the leaked threads in a more grep friendly format would be nice, you could easily create a thread leak report. It would be also nice to see the leaks on the console

  1   2   >