[GitHub] spark issue #22967: [SPARK-25956] Make Scala 2.12 as default Scala version i...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22967 **[Test build #98813 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98813/testReport)** for PR 22967 at commit

[GitHub] spark issue #22967: [SPARK-25956] Make Scala 2.12 as default Scala version i...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22967 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22967: [SPARK-25956] Make Scala 2.12 as default Scala version i...

2018-11-13 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/22967 @dongjoon-hyun thanks for trigging the build. The python test script was only looking for scala 2.11 jars resulting python test failures. I just fixed it in the latest push. Let's see how it goes.

[GitHub] spark issue #22967: [SPARK-25956] Make Scala 2.12 as default Scala version i...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22967 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22967: [SPARK-25956] Make Scala 2.12 as default Scala version i...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22967 **[Test build #98812 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98812/testReport)** for PR 22967 at commit

[GitHub] spark pull request #22967: [SPARK-25956] Make Scala 2.12 as default Scala ve...

2018-11-13 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/22967#discussion_r22593 --- Diff: pom.xml --- @@ -2718,7 +2710,6 @@ *:*_2.11 -

[GitHub] spark pull request #23021: [SPARK-26032][PYTHON] Break large sql/tests.py fi...

2018-11-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23021 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23021 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23021 I am merging this in - maybe I am rushing it but please allow me to go ahead since it's going to block other PySpark PRs. At worst case, I am willing to revert and propose this again if

[GitHub] spark issue #23012: [SPARK-26014][R] Deprecate R prior to version 3.4 in Spa...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23012 Ah .. right makes sense to me. Thanks @shaneknapp. +1 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22966: [SPARK-25965][SQL][TEST] Add avro read benchmark

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22966 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98811/ Test PASSed. ---

[GitHub] spark issue #22966: [SPARK-25965][SQL][TEST] Add avro read benchmark

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22966 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22966: [SPARK-25965][SQL][TEST] Add avro read benchmark

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22966 **[Test build #98811 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98811/testReport)** for PR 22966 at commit

[GitHub] spark pull request #23029: [SPARK-26055][CORE] InterfaceStability annotation...

2018-11-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23029#discussion_r233322079 --- Diff: common/tags/src/main/java/org/apache/spark/annotation/InterfaceStability.java --- @@ -17,7 +17,7 @@ package

[GitHub] spark issue #22966: [SPARK-25965][SQL][TEST] Add avro read benchmark

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22966 **[Test build #98811 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98811/testReport)** for PR 22966 at commit

[GitHub] spark pull request #23029: [SPARK-26055][CORE] InterfaceStability annotation...

2018-11-13 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23029#discussion_r233321272 --- Diff: common/tags/src/main/java/org/apache/spark/annotation/InterfaceStability.java --- @@ -17,7 +17,7 @@ package

[GitHub] spark issue #23029: [SPARK-26055][CORE] InterfaceStability annotations shoul...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23029 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22966: [SPARK-25965][SQL][TEST] Add avro read benchmark

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22966 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23029: [SPARK-26055][CORE] InterfaceStability annotations shoul...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23029 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22966: [SPARK-25965][SQL][TEST] Add avro read benchmark

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22966 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22966: [SPARK-25965][SQL][TEST] Add avro read benchmark

2018-11-13 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/22966 @dongjoon-hyun sure. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23029: [SPARK-26055][CORE] InterfaceStability annotations shoul...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23029 **[Test build #98810 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98810/testReport)** for PR 23029 at commit

[GitHub] spark pull request #23029: [SPARK-26055][CORE] InterfaceStability annotation...

2018-11-13 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/23029 [SPARK-26055][CORE] InterfaceStability annotations should be retained at runtime ## What changes were proposed in this pull request? It's good to have annotations available at runtime,

[GitHub] spark issue #23029: [SPARK-26055][CORE] InterfaceStability annotations shoul...

2018-11-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23029 cc @rxin @srowen @vanzin @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22976: [SPARK-25974][SQL]Optimizes Generates bytecode for order...

2018-11-13 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22976 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23021 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23021 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98808/ Test PASSed. ---

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23021 **[Test build #98808 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98808/testReport)** for PR 23021 at commit

[GitHub] spark issue #22867: [SPARK-25778] WriteAheadLogBackedBlockRDD in YARN Cluste...

2018-11-13 Thread gss2002
Github user gss2002 commented on the issue: https://github.com/apache/spark/pull/22867 @vanzin can you review the latest PR. This seems like it works without issue. I ran this on my home cluster and my larger clusters at work where the issue was first discovered and it seems to solve

[GitHub] spark pull request #22989: [SPARK-25986][Build] Add rules to ban throw Error...

2018-11-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22989#discussion_r233318346 --- Diff: dev/checkstyle.xml --- @@ -180,5 +180,10 @@ + + + --- End

[GitHub] spark issue #23028: [SPARK-26053][SQL] Enhance LikeSimplification

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23028 **[Test build #98809 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98809/testReport)** for PR 23028 at commit

[GitHub] spark issue #23028: [SPARK-26053][SQL] Enhance LikeSimplification

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23028 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23028: [SPARK-26053][SQL] Enhance LikeSimplification

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23028 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98809/ Test FAILed. ---

[GitHub] spark pull request #22989: [SPARK-25986][Build] Add rules to ban throw Error...

2018-11-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22989#discussion_r233317742 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/clustering/KMeansSuite.scala --- @@ -331,7 +333,7 @@ class KMeansSuite extends SparkFunSuite with

[GitHub] spark pull request #23028: [SPARK-26053][SQL] Enhance LikeSimplification

2018-11-13 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/23028#discussion_r233317201 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -504,6 +504,19 @@ object LikeSimplification extends

[GitHub] spark pull request #23028: [SPARK-26053][SQL] Enhance LikeSimplification

2018-11-13 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/23028 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #23028: [SPARK-26053][SQL] Enhance LikeSimplification

2018-11-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23028#discussion_r233315756 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -504,6 +504,19 @@ object LikeSimplification extends

[GitHub] spark pull request #23028: [SPARK-26053][SQL] Enhance LikeSimplification

2018-11-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23028#discussion_r233315067 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -504,6 +504,19 @@ object LikeSimplification extends

[GitHub] spark pull request #23028: [SPARK-26053][SQL] Enhance LikeSimplification

2018-11-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23028#discussion_r233315192 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -504,6 +504,19 @@ object LikeSimplification extends

[GitHub] spark issue #22967: [SPARK-25956] Make Scala 2.12 as default Scala version i...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22967 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98807/ Test FAILed. ---

[GitHub] spark issue #22967: [SPARK-25956] Make Scala 2.12 as default Scala version i...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22967 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22967: [SPARK-25956] Make Scala 2.12 as default Scala version i...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22967 **[Test build #98807 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98807/testReport)** for PR 22967 at commit

[GitHub] spark issue #22504: [SPARK-25118][Submit] Persist Driver Logs in Client mode...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22504 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22504: [SPARK-25118][Submit] Persist Driver Logs in Client mode...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22504 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98805/ Test PASSed. ---

[GitHub] spark issue #22504: [SPARK-25118][Submit] Persist Driver Logs in Client mode...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22504 **[Test build #98805 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98805/testReport)** for PR 22504 at commit

[GitHub] spark issue #23027: [SPARK-26049][SQL][TEST] FilterPushdownBenchmark add InM...

2018-11-13 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/23027 cc @HyukjinKwon @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #23027: [SPARK-26049][SQL][TEST] FilterPushdownBenchmark add InM...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23027 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23027: [SPARK-26049][SQL][TEST] FilterPushdownBenchmark add InM...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23027 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98806/ Test PASSed. ---

[GitHub] spark issue #23027: [SPARK-26049][SQL][TEST] FilterPushdownBenchmark add InM...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23027 **[Test build #98806 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98806/testReport)** for PR 23027 at commit

[GitHub] spark issue #23028: [SPARK-26053][SQL] Enhance LikeSimplification

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23028 **[Test build #98809 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98809/testReport)** for PR 23028 at commit

[GitHub] spark issue #23028: [SPARK-26053][SQL] Enhance LikeSimplification

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23028 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23028: [SPARK-26053][SQL] Enhance LikeSimplification

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23028 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #23028: [SPARK-26053][SQL] Enhance LikeSimplification

2018-11-13 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/23028 [SPARK-26053][SQL] Enhance LikeSimplification ## What changes were proposed in this pull request? This PR enhance `LikeSimplification` in 2 cases: 1. null like col -> null 2.

[GitHub] spark issue #22993: [SPARK-24421][BUILD][CORE] Accessing sun.misc.Cleaner in...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22993 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98804/ Test PASSed. ---

[GitHub] spark issue #22993: [SPARK-24421][BUILD][CORE] Accessing sun.misc.Cleaner in...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22993 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22993: [SPARK-24421][BUILD][CORE] Accessing sun.misc.Cleaner in...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22993 **[Test build #98804 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98804/testReport)** for PR 22993 at commit

[GitHub] spark issue #23012: [SPARK-26014][R] Deprecate R prior to version 3.4 in Spa...

2018-11-13 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/23012 TL;DR: let's go w/deprecation. still TL;DR: if i never have to install or manage R again, i will be a happy person! @HyukjinKwon upgrading R is easy. getting the right mix of

[GitHub] spark issue #22974: [SPARK-22450][WIP][Core][MLLib][FollowUp] Safely registe...

2018-11-13 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/22974 @srowen Yes, this is the problem. I have to register `Param*` before any prediction model, but there are too many anonymous classes in `ParamValidators` and other places, and I have not found

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23021 adding @holdenk, @ueshin and @icexelloss as well. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23021 adding @icexelloss as well. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23021 > Did you test on python3 as well? Of course! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #23019: [SPARK-26025][k8s] Speed up docker image build on dev re...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23019 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98803/ Test PASSed. ---

[GitHub] spark issue #23019: [SPARK-26025][k8s] Speed up docker image build on dev re...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23019 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23019: [SPARK-26025][k8s] Speed up docker image build on dev re...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23019 **[Test build #98803 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98803/testReport)** for PR 23019 at commit

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r233292436 --- Diff: R/pkg/R/SQLContext.R --- @@ -189,19 +238,67 @@ createDataFrame <- function(data, schema = NULL, samplingRatio = 1.0, x

[GitHub] spark issue #23012: [SPARK-26014][R] Deprecate R prior to version 3.4 in Spa...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23012 @shaneknapp, do you roughly know how difficult it is (and do you have some time shortly) to upgrade R from 3.1 to 3.4? I am asking this because I had some difficulties when I tried to manually

[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23012#discussion_r233290797 --- Diff: docs/index.md --- @@ -31,7 +31,8 @@ Spark runs on both Windows and UNIX-like systems (e.g. Linux, Mac OS). It's easy locally on one

[GitHub] spark issue #23012: [SPARK-26014][R] Deprecate R prior to version 3.4 in Spa...

2018-11-13 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/23012 Hey shane I don’t think we are saying to test multiple R version at all. In fact quite the opposite, just the new(er) version at some point in the future. (We don’t have a better

[GitHub] spark pull request #21957: [SPARK-24994][SQL] When the data type of the fiel...

2018-11-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21957#discussion_r233287374 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -269,7 +269,8 @@ case class FileSourceScanExec(

[GitHub] spark issue #22962: [SPARK-25921][PySpark] Fix barrier task run without Barr...

2018-11-13 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22962 @HyukjinKwon No problem, I'll give a follow up PR to address all your comments and rewrite the UT in to a separate class. ---

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23021 **[Test build #98808 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98808/testReport)** for PR 23021 at commit

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23021 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23021 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23021 > Could you add some descriptions to run a single test file or a single test case if exists? Done! --- - To

[GitHub] spark issue #22994: [BUILD] refactor dev/lint-python in to something readabl...

2018-11-13 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/22994 nah, it's not urgent at all. i also tested this locally by breaking various python scripts and confirming that it caught errors as expected. On Tue, Nov 13, 2018 at 5:03 PM

[GitHub] spark pull request #22962: [SPARK-25921][PySpark] Fix barrier task run witho...

2018-11-13 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22962#discussion_r233283645 --- Diff: python/pyspark/taskcontext.py --- @@ -147,8 +147,8 @@ def __init__(self): @classmethod def _getOrCreate(cls):

[GitHub] spark issue #23024: [SPARK-26044][WEBUI]Aggregated Metrics table sort based ...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23024 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98800/ Test PASSed. ---

[GitHub] spark issue #23024: [SPARK-26044][WEBUI]Aggregated Metrics table sort based ...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23024 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23024: [SPARK-26044][WEBUI]Aggregated Metrics table sort based ...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23024 **[Test build #98800 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98800/testReport)** for PR 23024 at commit

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23021 Yup! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22994: [BUILD] refactor dev/lint-python in to something readabl...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22994 I haven't taken a look super closely but the idea looks itself okay. Is it urgent? if yes, yup. I don't object to go ahead right away. Otherwise, might be good to leave open for few days for

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/23021 One thing - I would put “pandas” right after test_ so you get the natural logical grouping with sorting by file name. On Tue, Nov 13, 2018 at 4:58 PM Hyukjin Kwon wrote:

[GitHub] spark issue #23019: [SPARK-26025][k8s] Speed up docker image build on dev re...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23019 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98797/ Test PASSed. ---

[GitHub] spark issue #23019: [SPARK-26025][k8s] Speed up docker image build on dev re...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23019 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23021: [SPARK-26032][PYTHON] Break large sql/tests.py files int...

2018-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23021 I am going to push after testing and double checking. The line counts would look like this ``` 54 ./test_utils.py 199 ./test_catalog.py 503

[GitHub] spark issue #23019: [SPARK-26025][k8s] Speed up docker image build on dev re...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23019 **[Test build #98797 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98797/testReport)** for PR 23019 at commit

[GitHub] spark issue #22967: [SPARK-25956] Make Scala 2.12 as default Scala version i...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22967 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22967: [SPARK-25956] Make Scala 2.12 as default Scala version i...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22967 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22967: [SPARK-25956] Make Scala 2.12 as default Scala version i...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22967 **[Test build #98807 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98807/testReport)** for PR 22967 at commit

[GitHub] spark issue #23024: [SPARK-26044][WEBUI]Aggregated Metrics table sort based ...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23024 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23024: [SPARK-26044][WEBUI]Aggregated Metrics table sort based ...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23024 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98798/ Test PASSed. ---

[GitHub] spark issue #22967: [SPARK-25956] Make Scala 2.12 as default Scala version i...

2018-11-13 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22967 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23024: [SPARK-26044][WEBUI]Aggregated Metrics table sort based ...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23024 **[Test build #98798 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98798/testReport)** for PR 23024 at commit

[GitHub] spark issue #23027: [SPARK-26049][SQL][TEST] Add InMemoryTable filter benchm...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23027 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23027: [SPARK-26049][SQL][TEST] Add InMemoryTable filter benchm...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23027 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23027: [SPARK-26049][SQL][TEST] Add InMemoryTable filter benchm...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23027 **[Test build #98806 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98806/testReport)** for PR 23027 at commit

[GitHub] spark pull request #23027: [SPARK-26049][SQL][TEST] Add InMemoryTable filter...

2018-11-13 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/23027 [SPARK-26049][SQL][TEST] Add InMemoryTable filter benchmark to FilterPushdownBenchmark ## What changes were proposed in this pull request? Add InMemoryTable filter benchmark to

[GitHub] spark issue #23009: [SPARK-26011][SPARK-SUBMIT] Yarn mode pyspark app withou...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23009 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23009: [SPARK-26011][SPARK-SUBMIT] Yarn mode pyspark app withou...

2018-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23009 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98801/ Test FAILed. ---

[GitHub] spark issue #23009: [SPARK-26011][SPARK-SUBMIT] Yarn mode pyspark app withou...

2018-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23009 **[Test build #98801 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98801/testReport)** for PR 23009 at commit

  1   2   3   4   5   6   >