[GitHub] spark issue #21212: [SPARK-24143] filter empty blocks when convert mapstatus...

2018-05-03 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21212 can you add a test in MapOutputTrackerSuite and update the pr description to include all the changes? but overall looks good. ---

[GitHub] spark pull request #21214: [SPARK-23775][TEST] Make DataFrameRangeSuite not ...

2018-05-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21214#discussion_r185834096 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameRangeSuite.scala --- @@ -153,23 +153,17 @@ class DataFrameRangeSuite extends QueryTest with

[GitHub] spark issue #21214: [SPARK-23775][TEST] Make DataFrameRangeSuite not flaky

2018-05-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21214 > Namely the test which killed threads passed and right after the upcoming test stuck in submitting a job. I think that indicates that there might be an issue with the code that cancels

[GitHub] spark issue #20261: [SPARK-22885][ML][TEST] ML test for StructuredStreaming:...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20261 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20261: [SPARK-22885][ML][TEST] ML test for StructuredStreaming:...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20261 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90134/ Test PASSed. ---

[GitHub] spark issue #20261: [SPARK-22885][ML][TEST] ML test for StructuredStreaming:...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20261 **[Test build #90134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90134/testReport)** for PR 20261 at commit

[GitHub] spark issue #21212: [SPARK-24143] filter empty blocks when convert mapstatus...

2018-05-03 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21212 If you have a heap dump, there are tools that can check for wasted space in ArrayBuffer. Eg. [jxray](http://www.jxray.com/) or [YourKit Memory

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2876/

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21163: [SPARK-24097][ML] Instrumentation improvements - RandomF...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21163 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21163: [SPARK-24097][ML] Instrumentation improvements - RandomF...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21163 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90130/ Test PASSed. ---

[GitHub] spark issue #21163: [SPARK-24097][ML] Instrumentation improvements - RandomF...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21163 **[Test build #90130 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90130/testReport)** for PR 21163 at commit

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21193 **[Test build #90139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90139/testReport)** for PR 21193 at commit

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-05-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21106 @hvanhovell If you don't have other comment, I will follow @cloud-fan's suggestion to add a sql conf for it. Thanks. --- - To

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90118/ Test PASSed. ---

[GitHub] spark pull request #21227: Backport [SPARK-24133][SQL] Check for integer ove...

2018-05-03 Thread ala
Github user ala closed the pull request at: https://github.com/apache/spark/pull/21227 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21021 **[Test build #90118 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90118/testReport)** for PR 21021 at commit

[GitHub] spark pull request #21182: [SPARK-24068] Propagating DataFrameReader's optio...

2018-05-03 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21182#discussion_r185827257 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -136,4 +138,6 @@ private[sql] class JSONOptions(

[GitHub] spark issue #21227: Backport [SPARK-24133][SQL] Check for integer overflows ...

2018-05-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/21227 LGTM - merging to 2.3. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21215: [SPARK-24148][SQL] Overloading array function to support...

2018-05-03 Thread mn-mikke
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/21215 @maropu Really nice idea to create typed empty arrays via an `Literal` expression! On the other hand, I feel that the end user shouldn't work with classes from Catalyst internals if we consider

[GitHub] spark issue #21227: Backport [SPARK-24133][SQL] Check for integer overflows ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21227 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90119/ Test PASSed. ---

[GitHub] spark issue #21227: Backport [SPARK-24133][SQL] Check for integer overflows ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21227 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21227: Backport [SPARK-24133][SQL] Check for integer overflows ...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21227 **[Test build #90119 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90119/testReport)** for PR 21227 at commit

[GitHub] spark issue #21217: [SPARK-24151][SQL] Fix CURRENT_DATE, CURRENT_TIMESTAMP t...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21217 **[Test build #90138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90138/testReport)** for PR 21217 at commit

[GitHub] spark pull request #21225: [SPARK-24168][SQL] WindowExec should not access S...

2018-05-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21225#discussion_r185821003 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExec.scala --- @@ -197,6 +198,7 @@ case class WindowExec(

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21173 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2875/

[GitHub] spark pull request #21182: [SPARK-24068] Propagating DataFrameReader's optio...

2018-05-03 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21182#discussion_r185820210 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -136,4 +138,6 @@ private[sql] class JSONOptions(

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21173 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21193 **[Test build #90135 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90135/testReport)** for PR 21193 at commit

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90135/ Test FAILed. ---

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21215: [SPARK-24148][SQL] Overloading array function to support...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21215 **[Test build #90137 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90137/testReport)** for PR 21215 at commit

[GitHub] spark issue #21217: [SPARK-24151][SQL] Fix CURRENT_DATE, CURRENT_TIMESTAMP t...

2018-05-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21217 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2874/

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21173 **[Test build #90136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90136/testReport)** for PR 21173 at commit

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21193 **[Test build #90135 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90135/testReport)** for PR 21193 at commit

[GitHub] spark issue #21225: [SPARK-24168][SQL] WindowExec should not access SQLConf ...

2018-05-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21225 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21193 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90120/ Test FAILed. ---

[GitHub] spark issue #21226: [SPARK-24169][SQL] JsonToStructs should not access SQLCo...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21226 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90114/ Test PASSed. ---

[GitHub] spark issue #21226: [SPARK-24169][SQL] JsonToStructs should not access SQLCo...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21226 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90117/ Test PASSed. ---

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21193 **[Test build #90120 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90120/testReport)** for PR 21193 at commit

[GitHub] spark issue #20261: [SPARK-22885][ML][TEST] ML test for StructuredStreaming:...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20261 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2873/

[GitHub] spark issue #20261: [SPARK-22885][ML][TEST] ML test for StructuredStreaming:...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20261 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21028 **[Test build #90117 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90117/testReport)** for PR 21028 at commit

[GitHub] spark issue #21226: [SPARK-24169][SQL] JsonToStructs should not access SQLCo...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21226 **[Test build #90114 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90114/testReport)** for PR 21226 at commit

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2872/

[GitHub] spark issue #21229: [SPARK-23697][CORE] LegacyAccumulatorWrapper should defi...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21229 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2871/

[GitHub] spark issue #21229: [SPARK-23697][CORE] LegacyAccumulatorWrapper should defi...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21229 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20261: [SPARK-22885][ML][TEST] ML test for StructuredStreaming:...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20261 **[Test build #90134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90134/testReport)** for PR 20261 at commit

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2870/

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20261: [SPARK-22885][ML][TEST] ML test for StructuredStreaming:...

2018-05-03 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/20261 Jenkins, test this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21163: [SPARK-24097][ML] Instrumentation improvements - RandomF...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21163 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2869/

[GitHub] spark issue #21229: [SPARK-23697][CORE] LegacyAccumulatorWrapper should defi...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21229 **[Test build #90132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90132/testReport)** for PR 21229 at commit

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21129 **[Test build #90133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90133/testReport)** for PR 21129 at commit

[GitHub] spark issue #21163: [SPARK-24097][ML] Instrumentation improvements - RandomF...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21163 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21129 **[Test build #90131 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90131/testReport)** for PR 21129 at commit

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90131/ Test FAILed. ---

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2868/

[GitHub] spark issue #21229: [SPARK-23697][CORE] LegacyAccumulatorWrapper should defi...

2018-05-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21229 cc @jiangxb1987 @rxin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21214: [SPARK-23775][TEST] Make DataFrameRangeSuite not flaky

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21214 **[Test build #90129 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90129/testReport)** for PR 21214 at commit

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21129 **[Test build #90131 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90131/testReport)** for PR 21129 at commit

[GitHub] spark issue #21163: [SPARK-24097][ML] Instrumentation improvements - RandomF...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21163 **[Test build #90130 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90130/testReport)** for PR 21163 at commit

[GitHub] spark pull request #21229: [SPARK-23697][CORE] LegacyAccumulatorWrapper shou...

2018-05-03 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/21229 [SPARK-23697][CORE] LegacyAccumulatorWrapper should define isZero correctly ## What changes were proposed in this pull request? It's possible that Accumulators of Spark 1.x may no longer

[GitHub] spark issue #21212: [SPARK-24143] filter empty blocks when convert mapstatus...

2018-05-03 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/21212 @squito @cloud-fan @jiangxb1987 Thanks a lot for review. > shall we also optimize the space usage for MapStatus @cloud-fan do you mean optimize space usage for MapStatus when

[GitHub] spark issue #21212: [SPARK-24143] filter empty blocks when convert mapstatus...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21212 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2867/

[GitHub] spark issue #21212: [SPARK-24143] filter empty blocks when convert mapstatus...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21212 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21028 **[Test build #90128 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90128/testReport)** for PR 21028 at commit

[GitHub] spark issue #21212: [SPARK-24143] filter empty blocks when convert mapstatus...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21212 **[Test build #90127 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90127/testReport)** for PR 21212 at commit

[GitHub] spark pull request #21227: Backport [SPARK-24133][SQL] Check for integer ove...

2018-05-03 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/21227#discussion_r185801594 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java --- @@ -81,7 +81,9 @@ public void close() {

[GitHub] spark issue #21225: [SPARK-24168][SQL] WindowExec should not access SQLConf ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21225 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21225: [SPARK-24168][SQL] WindowExec should not access SQLConf ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21225 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90113/ Test PASSed. ---

[GitHub] spark pull request #21054: [SPARK-23907][SQL] Add regr_* functions

2018-05-03 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21054#discussion_r185800917 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/regression.scala --- @@ -0,0 +1,190 @@ +/* + * Licensed

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2866/

[GitHub] spark issue #21225: [SPARK-24168][SQL] WindowExec should not access SQLConf ...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21225 **[Test build #90113 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90113/testReport)** for PR 21225 at commit

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21129 **[Test build #90126 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90126/testReport)** for PR 21129 at commit

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90126/ Test FAILed. ---

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21129 **[Test build #90126 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90126/testReport)** for PR 21129 at commit

[GitHub] spark pull request #21193: [SPARK-24121][SQL][WIP] Add API for handling expr...

2018-05-03 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r185794875 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -112,6 +112,102 @@ object JavaCode {

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21068 **[Test build #90125 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90125/testReport)** for PR 21068 at commit

[GitHub] spark issue #21214: [SPARK-23775][TEST] Make DataFrameRangeSuite not flaky

2018-05-03 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/21214 > Just curious how this happened? I've taken a look at 2 logs you've shown and seen the same pattern. Namely the test which killed threads passed and right after the upcoming test stuck in

[GitHub] spark pull request #21193: [SPARK-24121][SQL][WIP] Add API for handling expr...

2018-05-03 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r185794699 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -120,13 +216,15 @@ object JavaCode {

[GitHub] spark pull request #21225: [SPARK-24168][SQL] WindowExec should not access S...

2018-05-03 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/21225#discussion_r185793645 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExec.scala --- @@ -114,7 +114,8 @@ case class WindowExec( *

[GitHub] spark pull request #21054: [SPARK-23907][SQL] Add regr_* functions

2018-05-03 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21054#discussion_r185791203 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/regression.scala --- @@ -0,0 +1,190 @@ +/* + * Licensed

[GitHub] spark pull request #21228: [SPARK-24171] Adding a note for non-deterministic...

2018-05-03 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/21228#discussion_r185792840 --- Diff: R/pkg/R/functions.R --- @@ -3184,6 +3191,7 @@ setMethod("create_map", #' collect(select(df2, collect_list(df2$gear))) #'

[GitHub] spark pull request #21228: [SPARK-24171] Adding a note for non-deterministic...

2018-05-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21228#discussion_r185792266 --- Diff: R/pkg/R/functions.R --- @@ -963,6 +964,7 @@ setMethod("kurtosis", #' last(df$c, TRUE) #' } #' @note last since 1.4.0 +#'

[GitHub] spark pull request #21214: [SPARK-23775][TEST] Make DataFrameRangeSuite not ...

2018-05-03 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/21214#discussion_r185792166 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameRangeSuite.scala --- @@ -153,23 +153,17 @@ class DataFrameRangeSuite extends

[GitHub] spark pull request #21228: [SPARK-24171] Adding a note for non-deterministic...

2018-05-03 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/21228#discussion_r185791719 --- Diff: R/pkg/R/functions.R --- @@ -963,6 +964,7 @@ setMethod("kurtosis", #' last(df$c, TRUE) #' } #' @note last since 1.4.0

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-05-03 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r185791498 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTrackerSuite.scala --- @@ -0,0 +1,144 @@ +/*

<    1   2   3   4   5   6   7   8   >