[GitHub] spark issue #20929: [SPARK-23772][SQL] Provide an option to ignore column of...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20929 **[Test build #90978 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90978/testReport)** for PR 20929 at commit

[GitHub] spark issue #20997: [SPARK-19185] [DSTREAMS] Avoid concurrent use of cached ...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20997 **[Test build #90989 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90989/testReport)** for PR 20997 at commit

[GitHub] spark issue #20997: [SPARK-19185] [DSTREAMS] Avoid concurrent use of cached ...

2018-05-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20997 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20997: [SPARK-19185] [DSTREAMS] Avoid concurrent use of cached ...

2018-05-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20997 I'm fine with it. Unless Cody beats me to it or has more comments, I'll push this after the long weekend. --- - To unsubscribe,

[GitHub] spark issue #21394: [SPARK-24329][SQL] Test for skipping multi-space lines

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21394 **[Test build #90988 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90988/testReport)** for PR 21394 at commit

[GitHub] spark issue #21394: [SPARK-24329][SQL] Test for skipping multi-space lines

2018-05-22 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21394 jenkins, retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21311 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90981/ Test FAILed. ---

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21311 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21311 **[Test build #90981 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90981/testReport)** for PR 21311 at commit

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21311 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21311 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90970/ Test PASSed. ---

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21311 **[Test build #90970 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90970/testReport)** for PR 21311 at commit

[GitHub] spark pull request #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate th...

2018-05-22 Thread cxzl25
Github user cxzl25 commented on a diff in the pull request: https://github.com/apache/spark/pull/21311#discussion_r189997697 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/HashedRelationSuite.scala --- @@ -254,6 +254,30 @@ class HashedRelationSuite extends

[GitHub] spark pull request #21397: [SPARK-24334] Fix race condition in ArrowPythonRu...

2018-05-22 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21397#discussion_r189996870 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala --- @@ -94,8 +88,18 @@ class ArrowPythonRunner(

[GitHub] spark pull request #21045: [SPARK-23931][SQL] Adds zip function to sparksql

2018-05-22 Thread DylanGuedes
Github user DylanGuedes commented on a diff in the pull request: https://github.com/apache/spark/pull/21045#discussion_r189996412 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -127,6 +127,148 @@ case class

[GitHub] spark issue #21397: [SPARK-24334] Fix race condition in ArrowPythonRunner ca...

2018-05-22 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/21397 So far I have been using a local parquet file to test. Let me try if I can create one on the fly to reproduce this. --- -

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21193 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21193 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r189993896 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/maskExpressions.scala --- @@ -0,0 +1,569 @@ +/* + * Licensed to

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21311 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21311 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90967/ Test PASSed. ---

[GitHub] spark pull request #21380: [SPARK-24329][SQL] Remove comments filtering befo...

2018-05-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21380#discussion_r189991484 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -300,14 +302,11 @@ private[csv] object

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21311 **[Test build #90967 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90967/testReport)** for PR 21311 at commit

[GitHub] spark issue #21394: [SPARK-24329][SQL] Test for skipping multi-space lines

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21394 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90971/ Test FAILed. ---

[GitHub] spark issue #21394: [SPARK-24329][SQL] Test for skipping multi-space lines

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21394 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21394: [SPARK-24329][SQL] Test for skipping multi-space lines

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21394 **[Test build #90971 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90971/testReport)** for PR 21394 at commit

[GitHub] spark issue #21387: Correct reference to Offset class

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21387 **[Test build #90987 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90987/testReport)** for PR 21387 at commit

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-05-22 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r189982571 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala --- @@ -54,7 +54,8 @@

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-05-22 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r189986135 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/bindings/python/Dockerfile --- @@ -0,0 +1,34 @@ +# +# Licensed to the

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-05-22 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r189981496 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -154,6 +176,24 @@ private[spark] object Config

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90965/ Test PASSed. ---

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21106 **[Test build #90965 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90965/testReport)** for PR 21106 at commit

[GitHub] spark issue #21387: Correct reference to Offset class

2018-05-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21387 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21395: [SPARK-24348][SQL] "element_at" error fix

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21395 **[Test build #90986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90986/testReport)** for PR 21395 at commit

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90964/ Test PASSed. ---

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21193 **[Test build #90964 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90964/testReport)** for PR 21193 at commit

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21069 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3474/

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21069 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3473/

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21399 **[Test build #90982 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90982/testReport)** for PR 21399 at commit

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90982/ Test FAILed. ---

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21069 **[Test build #90985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90985/testReport)** for PR 21069 at commit

[GitHub] spark issue #21398: [SPARK-24338][SQL] Fixed Hive CREATETABLE error in Sentr...

2018-05-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21398 Oh man, this thing. I didn't post this upstream because I think this is kinda hacky, and I also consider this primarily a Sentry bug. But if others are ok with it... Just make sure you have

[GitHub] spark pull request #21069: [SPARK-23920][SQL]add array_remove to remove all ...

2018-05-22 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/21069#discussion_r189978900 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1882,3 +1882,98 @@ case class

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90983/ Test FAILed. ---

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21399 **[Test build #90983 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90983/testReport)** for PR 21399 at commit

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3472/

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask and Use `whi...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21381 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90961/ Test PASSed. ---

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21399 **[Test build #90984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90984/testReport)** for PR 21399 at commit

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask and Use `whi...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21381 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask and Use `whi...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21381 **[Test build #90961 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90961/testReport)** for PR 21381 at commit

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3471/

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90973/ Test FAILed. ---

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21399 **[Test build #90983 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90983/testReport)** for PR 21399 at commit

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21397: [SPARK-24334] Fix race condition in ArrowPythonRunner ca...

2018-05-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21397 Yea, I was wondering about it too. It should be nicer if we have some steps in the PR description. --- - To unsubscribe,

[GitHub] spark pull request #21396: [SPARK-24349][SQL] Ignore setting token if using ...

2018-05-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21396#discussion_r189974620 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HiveDelegationTokenProvider.scala --- @@ -85,7 +85,10 @@ private[spark] class

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21399 **[Test build #90982 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90982/testReport)** for PR 21399 at commit

[GitHub] spark pull request #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate th...

2018-05-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21311#discussion_r189974407 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/HashedRelationSuite.scala --- @@ -254,6 +254,30 @@ class HashedRelationSuite

[GitHub] spark pull request #21397: [SPARK-24334] Fix race condition in ArrowPythonRu...

2018-05-22 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/21397#discussion_r189973566 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala --- @@ -94,8 +88,18 @@ class ArrowPythonRunner(

[GitHub] spark issue #21397: [SPARK-24334] Fix race condition in ArrowPythonRunner ca...

2018-05-22 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21397 I'm guessing making a reliable test is too difficult? is it possible to provide some steps to reproduce? --- - To

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21319 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21319 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90959/ Test PASSed. ---

[GitHub] spark issue #21387: Correct reference to Offset class

2018-05-22 Thread mojodna
Github user mojodna commented on the issue: https://github.com/apache/spark/pull/21387 I don't see any more mentions of the incorrect package. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21319 **[Test build #90959 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90959/testReport)** for PR 21319 at commit

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21399 Let me push some more commits in main code and test code too. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21311 **[Test build #90981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90981/testReport)** for PR 21311 at commit

[GitHub] spark pull request #21384: [SPARK-23416][SS] Add a specific stop method for ...

2018-05-22 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21384#discussion_r189970144 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -356,6 +356,22 @@ class

[GitHub] spark pull request #21399: [SPARK-22269][BUILD] Run Java linter via SBT for ...

2018-05-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21399#discussion_r189968751 --- Diff: project/plugins.sbt --- @@ -1,3 +1,11 @@ +addSbtPlugin("com.etsy" % "sbt-checkstyle-plugin" % "3.1.1") + +// sbt-checkstyle-plugin

[GitHub] spark pull request #21399: [SPARK-22269][BUILD] Run Java linter via SBT for ...

2018-05-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21399#discussion_r189968502 --- Diff: project/plugins.sbt --- @@ -1,3 +1,11 @@ +addSbtPlugin("com.etsy" % "sbt-checkstyle-plugin" % "3.1.1") + +// sbt-checkstyle-plugin uses

[GitHub] spark issue #18826: LogisticRegressionModel.toString should summarize model

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18826 **[Test build #4187 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4187/testReport)** for PR 18826 at commit

[GitHub] spark issue #17697: [SPARK-20414][MLLIB] avoid creating only 16 reducers whe...

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17697 **[Test build #4186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4186/testReport)** for PR 17697 at commit

[GitHub] spark issue #21401: [SPARK-24350][SQL] "array_position" error fix

2018-05-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21401 @ueshin can you trigger tests? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21395: [SPARK-24348][SQL] "element_at" error fix

2018-05-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21395#discussion_r189967468 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -756,6 +756,10 @@ class DataFrameFunctionsSuite extends

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r189967162 --- Diff: python/pyspark/shuffle.py --- @@ -67,6 +67,19 @@ def get_used_memory(): return 0 +def safe_iter(f): +"""

[GitHub] spark issue #21401: [SPARK-24350][SQL] "array_position" error fix

2018-05-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21401 Please describe more in the description; you'd be better to copy the description there. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3470/

[GitHub] spark pull request #21384: [SPARK-23416][SS] Add a specific stop method for ...

2018-05-22 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21384#discussion_r189965983 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -356,6 +356,22 @@ class

[GitHub] spark pull request #21401: [SPARK-24350][SQL] "array_position" error fix

2018-05-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21401#discussion_r189966177 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -708,6 +708,10 @@ class DataFrameFunctionsSuite extends

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-05-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r189964737 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2408,4 +2409,53 @@ class JsonSuite extends

[GitHub] spark pull request #21368: [SPARK-16451][repl] Fail shell if SparkSession fa...

2018-05-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21368#discussion_r189964634 --- Diff: python/pyspark/sql/session.py --- @@ -547,6 +547,40 @@ def _create_from_pandas_with_arrow(self, pdf, schema, timezone): df._schema =

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-05-22 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r189964186 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2408,4 +2409,53 @@ class JsonSuite extends

[GitHub] spark issue #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks when spi...

2018-05-22 Thread advancedxy
Github user advancedxy commented on the issue: https://github.com/apache/spark/pull/21369 > @advancedxy , using jvisualvm+heap dump I could see that the second introduced test case ("drop all references to the underlying map once the iterator is exhausted") eliminated all references

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21399 **[Test build #90980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90980/testReport)** for PR 21399 at commit

[GitHub] spark pull request #21399: [SPARK-22269][BUILD] Run Java linter via SBT for ...

2018-05-22 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/21399#discussion_r189963174 --- Diff: project/SparkBuild.scala --- @@ -740,6 +741,16 @@ object Unidoc { ) } +object CheckStyle { --- End diff --

[GitHub] spark pull request #21399: [SPARK-22269][BUILD] Run Java linter via SBT for ...

2018-05-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21399#discussion_r189962904 --- Diff: project/SparkBuild.scala --- @@ -740,6 +741,16 @@ object Unidoc { ) } +object CheckStyle { --- End diff --

[GitHub] spark issue #20929: [SPARK-23772][SQL] Provide an option to ignore column of...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20929 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3469/

[GitHub] spark issue #20929: [SPARK-23772][SQL] Provide an option to ignore column of...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20929 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3468/

[GitHub] spark issue #20929: [SPARK-23772][SQL] Provide an option to ignore column of...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20929 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20929: [SPARK-23772][SQL] Provide an option to ignore column of...

2018-05-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20929 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21354: [Web UI] Do not skip cells in Tasks table on Stage page ...

2018-05-22 Thread artemrd
Github user artemrd commented on the issue: https://github.com/apache/spark/pull/21354 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-05-22 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r189961165 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1256,8 +1256,18 @@ object SQLConf { "issues.

<    1   2   3   4   5   6   7   8   9   >