[GitHub] spark issue #19492: [SPARK-22228][SQL] Add support for array...

2018-01-23 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19492 Kindly ping @viirya --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #19978: [SPARK-22784][CORE][WIP] Configure reading buffer size i...

2018-01-23 Thread MikhailErofeev
Github user MikhailErofeev commented on the issue: https://github.com/apache/spark/pull/19978 @srowen, yes, the processing is no longer IO-bound after backporting SPARK-20923 --- - To unsubscribe, e-mail: reviews-u

[GitHub] spark pull request #19978: [SPARK-22784][CORE][WIP] Configure reading buffer...

2018-01-23 Thread MikhailErofeev
Github user MikhailErofeev closed the pull request at: https://github.com/apache/spark/pull/19978 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r163472676 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1313,6 +1331,9 @@ object CodeGenerato

[GitHub] spark issue #20376: [SPARK-23020][CORE][FOLLOWUP] Fix Java style check issue...

2018-01-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20376 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #20377: [SPARK-17088] [FOLLOW-UP] Fix 'sharesHadoopClasses' opti...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20377 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86562/ Test PASSed. ---

[GitHub] spark issue #20377: [SPARK-17088] [FOLLOW-UP] Fix 'sharesHadoopClasses' opti...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20377 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20376: [SPARK-23020][CORE][FOLLOWUP] Fix Java style check issue...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20376 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/179/ Test

[GitHub] spark issue #20376: [SPARK-23020][CORE][FOLLOWUP] Fix Java style check issue...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20376 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20377: [SPARK-17088] [FOLLOW-UP] Fix 'sharesHadoopClasses' opti...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20377 **[Test build #86562 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86562/testReport)** for PR 20377 at commit [`0574ec7`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #20376: [SPARK-23020][CORE][FOLLOWUP] Fix Java style check issue...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20376 **[Test build #86561 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86561/testReport)** for PR 20376 at commit [`dfd828b`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #20376: [SPARK-23020][CORE][FOLLOWUP] Fix Java style check issue...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20376 **[Test build #86574 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86574/testReport)** for PR 20376 at commit [`dfd828b`](https://github.com/apache/spark/commit/df

[GitHub] spark issue #20376: [SPARK-23020][CORE][FOLLOWUP] Fix Java style check issue...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20376 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20376: [SPARK-23020][CORE][FOLLOWUP] Fix Java style check issue...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20376 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86561/ Test FAILed. ---

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17702 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/178/ Test

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17702 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2018-01-23 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/17702 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17702 **[Test build #86573 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86573/testReport)** for PR 17702 at commit [`dc373ae`](https://github.com/apache/spark/commit/dc

[GitHub] spark issue #20380: [SPARK-20906][SPARKR] Add API doc example for Constraine...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20380 **[Test build #86572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86572/testReport)** for PR 20380 at commit [`f56b00d`](https://github.com/apache/spark/commit/f5

[GitHub] spark issue #20379: [SPARK-23177][SQL][PySpark][Backport-2.3] Extract zero-p...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20379 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/177/ Test

[GitHub] spark issue #20380: [SPARK-20906][SPARKR] Add API doc example for Constraine...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20380 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20379: [SPARK-23177][SQL][PySpark][Backport-2.3] Extract zero-p...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20379 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20380: [SPARK-20906][SPARKR] Add API doc example for Constraine...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20380 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/176/ Test

[GitHub] spark pull request #20380: [SPARK-20906][SPARKR] Add API doc example for Con...

2018-01-23 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/20380 [SPARK-20906][SPARKR] Add API doc example for Constrained Logistic Regression ## What changes were proposed in this pull request? doc only changes ## How was this patch tested

[GitHub] spark issue #20379: [SPARK-23177][SQL][PySpark][Backport-2.3] Extract zero-p...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20379 **[Test build #86571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86571/testReport)** for PR 20379 at commit [`a66b5e0`](https://github.com/apache/spark/commit/a6

[GitHub] spark issue #20379: [SPARK-23177][SQL][PySpark][Backport-2.3] Extract zero-p...

2018-01-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20379 cc @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@

[GitHub] spark pull request #20379: [SPARK-23177][SQL][PySpark][Backport-2.3] Extract...

2018-01-23 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/20379 [SPARK-23177][SQL][PySpark][Backport-2.3] Extract zero-parameter UDFs from aggregate ## What changes were proposed in this pull request? We extract Python UDFs in logical aggregate which de

[GitHub] spark pull request #20338: [SPARK-23174][BUILD][PYTHON] python code style ch...

2018-01-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20338#discussion_r163467168 --- Diff: dev/tox.ini --- @@ -13,7 +13,7 @@ # See the License for the specific language governing permissions and # limitations under the Licen

[GitHub] spark pull request #20338: [SPARK-23174][BUILD][PYTHON] python code style ch...

2018-01-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20338#discussion_r163465854 --- Diff: dev/tox.ini --- @@ -13,7 +13,7 @@ # See the License for the specific language governing permissions and # limitations under the Licen

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread ash211
Github user ash211 commented on the issue: https://github.com/apache/spark/pull/20372 Please fix the scala style checks -- ``` Running Scala style checks Scalastyle checks failed at following o

[GitHub] spark pull request #20367: [SPARK-23166][ML] Add maxDF Parameter to CountVec...

2018-01-23 Thread ymazari
Github user ymazari commented on a diff in the pull request: https://github.com/apache/spark/pull/20367#discussion_r163465302 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/CountVectorizerSuite.scala --- @@ -119,6 +119,41 @@ class CountVectorizerSuite extends SparkFunSu

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #86570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86570/testReport)** for PR 18931 at commit [`79d0106`](https://github.com/apache/spark/commit/79

[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20338 **[Test build #86569 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86569/testReport)** for PR 20338 at commit [`34a8590`](https://github.com/apache/spark/commit/34

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/175/ Test

[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20338 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20338 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/174/ Test

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20372 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20372 **[Test build #86567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86567/testReport)** for PR 20372 at commit [`ef04de9`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20372 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86567/ Test FAILed. ---

[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

2018-01-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20338 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20372 **[Test build #86567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86567/testReport)** for PR 20372 at commit [`ef04de9`](https://github.com/apache/spark/commit/ef

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #86568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86568/testReport)** for PR 18931 at commit [`9f0d1da`](https://github.com/apache/spark/commit/9f

[GitHub] spark pull request #20372: Improved block merging logic for partitions

2018-01-23 Thread ash211
Github user ash211 commented on a diff in the pull request: https://github.com/apache/spark/pull/20372#discussion_r163464207 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -445,16 +445,25 @@ case class FileSourceScanExec(

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/173/ Test

[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20338 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20338 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86560/ Test FAILed. ---

[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20338 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86559/ Test FAILed. ---

[GitHub] spark issue #20368: [SPARK-23195] [SQL] Keep the Hint of Cached Data

2018-01-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20368 Thanks! revert it from master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20338 **[Test build #86560 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86560/testReport)** for PR 20338 at commit [`34a8590`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20338 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20338 **[Test build #86559 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86559/testReport)** for PR 20338 at commit [`d10fbb4`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread ash211
Github user ash211 commented on the issue: https://github.com/apache/spark/pull/20372 Jenkins, this is ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark pull request #20037: [SPARK-22849] ivy.retrieve pattern should also co...

2018-01-23 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/20037#discussion_r163463718 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -1271,7 +1271,7 @@ private[spark] object SparkSubmitUtils { // retr

[GitHub] spark issue #20352: [SPARK-21727][R] Allow multi-element atomic vector as co...

2018-01-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20352 merged to master/2.3. we could revisit migration guide if necessary. thanks! --- - To unsubscribe, e-mail: reviews-unsubs

[GitHub] spark issue #20368: [SPARK-23195] [SQL] Keep the Hint of Cached Data

2018-01-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20368 Oh, thank you so much for fast recovery, @gatorsmile . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org F

[GitHub] spark pull request #20352: [SPARK-21727][R] Allow multi-element atomic vecto...

2018-01-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20352 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #86566 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86566/testReport)** for PR 18931 at commit [`2f2d1fd`](https://github.com/apache/spark/commit/2f

[GitHub] spark issue #20368: [SPARK-23195] [SQL] Keep the Hint of Cached Data

2018-01-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20368 I am reverting this PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/172/ Test

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2018-01-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r163462731 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,13 +149,100 @@ trait CodegenSupport extends SparkPl

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2018-01-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r163462688 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,13 +149,100 @@ trait CodegenSupport extends SparkPl

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2018-01-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r163462698 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,13 +149,100 @@ trait CodegenSupport extends SparkPl

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2018-01-23 Thread lvdongr
Github user lvdongr commented on the issue: https://github.com/apache/spark/pull/18756 Execute me ,has the concept of default value been introduce to schema in master branch? @gatorsmile thank you. --- - To unsubsc

[GitHub] spark issue #20224: [SPARK-23032][SQL] Add a per-query codegenStageId to Who...

2018-01-23 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/20224 BTW, inspired by @cloud-fan 's comment, here's an example of the codegen stage IDs when scalar subqueries are involved: ```scala val sub = "(select sum(id) from range(5))" val df

[GitHub] spark pull request #19285: [SPARK-22068][CORE]Reduce the duplicate code betw...

2018-01-23 Thread ConeyLiu
Github user ConeyLiu commented on a diff in the pull request: https://github.com/apache/spark/pull/19285#discussion_r163462053 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -233,17 +235,13 @@ private[spark] class MemoryStore( }

[GitHub] spark pull request #20224: [SPARK-23032][SQL] Add a per-query codegenStageId...

2018-01-23 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/20224#discussion_r163461361 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -365,6 +388,26 @@ case class WholeStageCodegenExec(

[GitHub] spark issue #19285: [SPARK-22068][CORE]Reduce the duplicate code between put...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19285 **[Test build #86565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86565/testReport)** for PR 19285 at commit [`f392217`](https://github.com/apache/spark/commit/f3

[GitHub] spark issue #20355: [SPARK-23148][SQL] Allow pathnames with special characte...

2018-01-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20355 Please fix the PR title. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark issue #20368: [SPARK-23195] [SQL] Keep the Hint of Cached Data

2018-01-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20368 As a result, this seems to block SparkPullRequestBuilder, too. - https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/SparkPullRequestBuilder/86557/ --- --

[GitHub] spark issue #20368: [SPARK-23195] [SQL] Keep the Hint of Cached Data

2018-01-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20368 Hi, All. This seems to break both `master` and `branch-2.3`. It might be correlated to the sister PR, [SPARK-23192](https://github.com/apache/spark/commit/613c290336e382664c24319f

[GitHub] spark issue #20355: [SPARK-23148][SQL] Allow pathnames with special characte...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20355 **[Test build #86564 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86564/testReport)** for PR 20355 at commit [`51b0db5`](https://github.com/apache/spark/commit/51

[GitHub] spark issue #20355: [SPARK-23148][SQL] Allow pathnames with special characte...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20355 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20355: [SPARK-23148][SQL] Allow pathnames with special characte...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/171/ Test

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17702 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17702 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86558/ Test FAILed. ---

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17702 **[Test build #86558 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86558/testReport)** for PR 17702 at commit [`dc373ae`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #20169: [SPARK-17088][hive] Fix 'sharesHadoopClasses' option whe...

2018-01-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20169 To be clear, I think I personally usually cc related guys. Yup, if I were fixing this codes, I would have cc'ed you @gatorsmile because I agree that it's good to do. Sure, more eyes are better.

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20378 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20378 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86563/ Test FAILed. ---

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20378 **[Test build #86563 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86563/testReport)** for PR 20378 at commit [`e69827e`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20378 **[Test build #86563 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86563/testReport)** for PR 20378 at commit [`e69827e`](https://github.com/apache/spark/commit/e6

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20378 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20378 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/170/ Test

[GitHub] spark pull request #20224: [SPARK-23032][SQL] Add a per-query codegenStageId...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20224#discussion_r163456017 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala --- @@ -228,4 +229,21 @@ class WholeStageCodegenSuite exten

[GitHub] spark pull request #20378: [SPARK-11222][Build][Python] Python document styl...

2018-01-23 Thread rekhajoshm
GitHub user rekhajoshm opened a pull request: https://github.com/apache/spark/pull/20378 [SPARK-11222][Build][Python] Python document style checker added ## What changes were proposed in this pull request? Using pydocstyle for python document style checker https://github.com/

[GitHub] spark pull request #20355: SPARK-23148: [SQL] Allow pathnames with special c...

2018-01-23 Thread henryr
Github user henryr commented on a diff in the pull request: https://github.com/apache/spark/pull/20355#discussion_r163455727 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -78,4 +78,20 @@ class FileBasedDataSourceSuite extends QueryTest

[GitHub] spark issue #20377: [SPARK-17088] [FOLLOW-UP] Fix 'sharesHadoopClasses' opti...

2018-01-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20377 cc @vanzin @cloud-fan @felixcheung @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additi

[GitHub] spark pull request #19285: [SPARK-22068][CORE]Reduce the duplicate code betw...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19285#discussion_r163455326 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -702,6 +645,76 @@ private[spark] class MemoryStore( } }

[GitHub] spark issue #20169: [SPARK-17088][hive] Fix 'sharesHadoopClasses' option whe...

2018-01-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20169 Let me explain my point here. I am not trying to offend anybody. Spark is becoming more and more complex. Peer review and test cases are the major methods we used for quality control. Spark is an

[GitHub] spark issue #20377: [SPARK-17088] [FOLLOW-UP] Fix 'sharesHadoopClasses' opti...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20377 **[Test build #86562 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86562/testReport)** for PR 20377 at commit [`0574ec7`](https://github.com/apache/spark/commit/05

[GitHub] spark issue #20377: [SPARK-17088] [FOLLOW-UP] Fix 'sharesHadoopClasses' opti...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20377 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20377: [SPARK-17088] [FOLLOW-UP] Fix 'sharesHadoopClasses' opti...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20377 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/169/ Test

[GitHub] spark pull request #20377: [SPARK-17088] [FOLLOW-UP] Fix 'sharesHadoopClasse...

2018-01-23 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/20377 [SPARK-17088] [FOLLOW-UP] Fix 'sharesHadoopClasses' option when creating client ## What changes were proposed in this pull request? This PR is to remove useless `warehouseDir`, which is alr

[GitHub] spark issue #19285: [SPARK-22068][CORE]Reduce the duplicate code between put...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19285 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86557/ Test FAILed. ---

[GitHub] spark issue #19285: [SPARK-22068][CORE]Reduce the duplicate code between put...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19285 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19285: [SPARK-22068][CORE]Reduce the duplicate code between put...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19285 **[Test build #86557 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86557/testReport)** for PR 19285 at commit [`c988762`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #20224: [SPARK-23032][SQL] Add a per-query codegenStageId...

2018-01-23 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/20224#discussion_r163453639 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala --- @@ -228,4 +229,21 @@ class WholeStageCodegenSuite ext

  1   2   3   4   5   6   >