[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152912084 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -485,4 +485,13 @@ package object config { "array in the

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152911936 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -485,4 +485,13 @@ package object config { "array in the

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152911829 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -485,4 +485,13 @@ package object config { "array in the

[GitHub] spark issue #19810: [SPARK-22599][SQL] In-Memory Table Pruning without Extra...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19810 **[Test build #84152 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84152/testReport)** for PR 19810 at commit

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152911325 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -485,4 +485,13 @@ package object config { "array in the

[GitHub] spark issue #19809: [SPARK-17920][SQL] [FOLLOWUP] Backport PR 19779 to branc...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19809 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84150/ Test PASSed. ---

[GitHub] spark issue #19809: [SPARK-17920][SQL] [FOLLOWUP] Backport PR 19779 to branc...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19809 **[Test build #84150 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84150/testReport)** for PR 19809 at commit

[GitHub] spark issue #19809: [SPARK-17920][SQL] [FOLLOWUP] Backport PR 19779 to branc...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19809 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19810: [SQL][SPARK-22599] In-Memory Table Pruning without Extra...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19810 **[Test build #84151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84151/testReport)** for PR 19810 at commit

[GitHub] spark pull request #19810: Partition level pruning 2

2017-11-23 Thread CodingCat
GitHub user CodingCat opened a pull request: https://github.com/apache/spark/pull/19810 Partition level pruning 2 ## What changes were proposed in this pull request? In the current implementation of Spark, InMemoryTableExec read all data in a cached table, filter

[GitHub] spark issue #19518: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-11-23 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19518 @kiszk I meant that `janinoc` creates a slightly different constant pool from `javac`. I am not sure about performances, but the number of constant pool entries is definitely different. For

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152908363 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -485,4 +485,13 @@ package object config { "array in the

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152907606 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -485,4 +485,13 @@ package object config { "array in the

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152907079 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -485,4 +485,13 @@ package object config { "array in the

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152906960 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -472,15 +475,66 @@ private[spark] class MapOutputTrackerMaster(

[GitHub] spark issue #19809: [SPARK-17920][SQL] [FOLLOWUP] Backport PR 19779 to branc...

2017-11-23 Thread vinodkc
Github user vinodkc commented on the issue: https://github.com/apache/spark/pull/19809 ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19809: [SPARK-17920][SQL] [FOLLOWUP] Backport PR 19779 to branc...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19809 **[Test build #84150 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84150/testReport)** for PR 19809 at commit

[GitHub] spark pull request #19809: [SPARK-17920][SQL] [FOLLOWUP] Backport PR 19779 t...

2017-11-23 Thread vinodkc
GitHub user vinodkc opened a pull request: https://github.com/apache/spark/pull/19809 [SPARK-17920][SQL] [FOLLOWUP] Backport PR 19779 to branch-2.2 ## What changes were proposed in this pull request? A followup of > https://github.com/apache/spark/pull/19795

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19808 **[Test build #84149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84149/testReport)** for PR 19808 at commit

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19808 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19808 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19808 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84147/ Test FAILed. ---

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19808 **[Test build #84147 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84147/testReport)** for PR 19808 at commit

[GitHub] spark pull request #19803: [SPARK-22596][SQL] set ctx.currentVars in Codegen...

2017-11-23 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19803#discussion_r152899229 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -108,20 +108,22 @@ trait CodegenSupport extends

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19788 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84148/ Test FAILed. ---

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19788 **[Test build #84148 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84148/testReport)** for PR 19788 at commit

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19788 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19788 **[Test build #84148 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84148/testReport)** for PR 19788 at commit

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152896879 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -472,15 +475,66 @@ private[spark] class MapOutputTrackerMaster(

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152896399 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -485,4 +485,13 @@ package object config { "array in the

[GitHub] spark issue #19518: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-11-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19518 I like the latest @kiszk hybrid idea in terms of performance and readability. Also, this is a corner case, so I don't want affect most regular small queries. ---

[GitHub] spark pull request #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsa...

2017-11-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19789#discussion_r152895550 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaRDD.scala --- @@ -211,8 +211,8 @@ private[spark] class

[GitHub] spark issue #19775: [SPARK-22343][core] Add support for publishing Spark met...

2017-11-23 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19775 Do we have to put this in Spark, is it a necessary part of k8s? I think if we pull in that PR(https://github.com/apache/spark/pull/11994), then this can be stayed out of Spark as a package. Even

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19803 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19803 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84146/ Test PASSed. ---

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19803 **[Test build #84146 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84146/testReport)** for PR 19803 at commit

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19803 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84145/ Test PASSed. ---

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19803 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19803 **[Test build #84145 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84145/testReport)** for PR 19803 at commit

[GitHub] spark issue #19802: [WIP][SPARK-22594][CORE] Handling spark-submit and maste...

2017-11-23 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19802 Can you please explain more, and how to reproduce this issue? Spark's RPC is not designed for version compatible. --- - To

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-11-23 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r152891920 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -196,12 +196,14 @@ private[spark] class

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-11-23 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r152891792 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -812,10 +812,13 @@ private[spark] object MapOutputTracker extends Logging {

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-11-23 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r152891172 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -196,12 +196,14 @@ private[spark] class

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-11-23 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r152891438 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -196,12 +196,14 @@ private[spark] class

[GitHub] spark pull request #19808: [SPARK-22597][SQL] Add spark-sql cmd script for W...

2017-11-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19808#discussion_r152891440 --- Diff: bin/find-spark-home.cmd --- @@ -32,7 +32,7 @@ if not "x%PYSPARK_PYTHON%"=="x" ( ) rem If there is python installed, trying to

[GitHub] spark pull request #19808: [SPARK-22597][SQL] Add spark-sql cmd script for W...

2017-11-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19808#discussion_r152891405 --- Diff: bin/spark-sql.cmd --- @@ -0,0 +1,25 @@ +@echo off + +rem +rem Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19808 **[Test build #84147 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84147/testReport)** for PR 19808 at commit

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19808 cc @cloud-fan, @felixcheung, @jsnowacki and @srowen who I could think are probably interested in this. --- - To

[GitHub] spark pull request #19808: [SPARK-22597][SQL] Add spark-sql cmd script for W...

2017-11-23 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/19808 [SPARK-22597][SQL] Add spark-sql cmd script for Windows users ## What changes were proposed in this pull request? This PR proposes to add cmd scripts so that Windows users can also run

[GitHub] spark pull request #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer t...

2017-11-23 Thread tengpeng
Github user tengpeng commented on a diff in the pull request: https://github.com/apache/spark/pull/17819#discussion_r152891005 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Bucketizer.scala --- @@ -108,26 +164,53 @@ final class Bucketizer @Since("1.4.0")

[GitHub] spark issue #19800: [SPARK-22591][SQL] GenerateOrdering shouldn't change Cod...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19800 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84144/ Test PASSed. ---

[GitHub] spark issue #19800: [SPARK-22591][SQL] GenerateOrdering shouldn't change Cod...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19800 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19800: [SPARK-22591][SQL] GenerateOrdering shouldn't change Cod...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19800 **[Test build #84144 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84144/testReport)** for PR 19800 at commit

[GitHub] spark issue #19518: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-11-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19518 @mgaido91 Thank you for your questions. 1. I am using `javac` as shown. I am sorry that I cannot understand what you are pointing out. In this benchmark, what are differences between `javac` and

[GitHub] spark issue #19518: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-11-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19518 Based on performance results and usage of constant pool entry, I would like to use hybrid approach with flat global variable and array. For example, first 500 variables are stored into flat

[GitHub] spark issue #19518: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-11-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19518 I created and ran another synthetic benchmark program for comparing flat global variables, inner global variables, and array. In summary, the followings are performance results (**small number is

[GitHub] spark pull request #19807: [SPARK-22495] Fix setup of SPARK_HOME variable on...

2017-11-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19807#discussion_r152889444 --- Diff: bin/find-spark-home.cmd --- @@ -0,0 +1,60 @@ +@echo off + +rem +rem Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152888380 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -472,15 +475,66 @@ private[spark] class MapOutputTrackerMaster(

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152888257 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -472,15 +475,66 @@ private[spark] class MapOutputTrackerMaster(

[GitHub] spark pull request #19806: [SPARK-22595][SQL] fix flaky test: CastSuite.SPAR...

2017-11-23 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19806#discussion_r152886393 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala --- @@ -829,7 +829,7 @@ class CastSuite extends

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19803 **[Test build #84146 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84146/testReport)** for PR 19803 at commit

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19803 **[Test build #84145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84145/testReport)** for PR 19803 at commit

[GitHub] spark pull request #19803: [SPARK-22596][SQL] set ctx.currentVars in Codegen...

2017-11-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19803#discussion_r152881282 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala --- @@ -60,20 +60,23 @@ case class

[GitHub] spark pull request #19803: [SPARK-22596][SQL] set ctx.currentVars in Codegen...

2017-11-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19803#discussion_r152881786 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala --- @@ -60,20 +60,23 @@ case class

[GitHub] spark pull request #19803: [SPARK-22596][SQL] set ctx.currentVars in Codegen...

2017-11-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19803#discussion_r152883588 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -355,19 +355,12 @@ case class FileSourceScanExec(

[GitHub] spark pull request #19803: [SPARK-22596][SQL] set ctx.currentVars in Codegen...

2017-11-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19803#discussion_r152883562 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -56,9 +56,7 @@ case class ProjectExec(projectList:

[GitHub] spark pull request #19803: [SPARK-22596][SQL] set ctx.currentVars in Codegen...

2017-11-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19803#discussion_r152882655 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -355,19 +355,12 @@ case class FileSourceScanExec(

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19803 Two comments, LGTM otherwise. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19803: [SPARK-22596][SQL] set ctx.currentVars in Codegen...

2017-11-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19803#discussion_r152882876 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -56,9 +56,7 @@ case class ProjectExec(projectList:

[GitHub] spark issue #19807: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19807 LGTM pending AppVeyor tests. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19807: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19807 Build started: [SparkR] `ALL`

[GitHub] spark issue #19800: [SPARK-22591][SQL] GenerateOrdering shouldn't change Cod...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19800 **[Test build #84144 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84144/testReport)** for PR 19800 at commit

[GitHub] spark pull request #19803: [SPARK-22596][SQL] set ctx.currentVars in Codegen...

2017-11-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19803#discussion_r152881704 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala --- @@ -60,20 +60,23 @@ case class

[GitHub] spark pull request #19800: [SPARK-22591][SQL] GenerateOrdering shouldn't cha...

2017-11-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19800#discussion_r152881639 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala --- @@ -149,10 +151,14 @@ object

[GitHub] spark pull request #19801: [SPARK-22592][SQL] cleanup filter converting for ...

2017-11-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19801 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19801: [SPARK-22592][SQL] cleanup filter converting for hive

2017-11-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19801 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19801: [SPARK-22592][SQL] cleanup filter converting for hive

2017-11-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19801 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19801: [SPARK-22592][SQL] cleanup filter converting for ...

2017-11-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19801#discussion_r152880052 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -660,40 +626,68 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request #19801: [SPARK-22592][SQL] cleanup filter converting for ...

2017-11-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19801#discussion_r152880027 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -643,9 +607,11 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request #19806: [SPARK-22595][SQL] fix flaky test: CastSuite.SPAR...

2017-11-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19806#discussion_r152879501 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala --- @@ -829,7 +829,7 @@ class CastSuite extends

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19789 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19789 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84143/ Test PASSed. ---

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19789 **[Test build #84143 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84143/testReport)** for PR 19789 at commit

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19789 **[Test build #84143 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84143/testReport)** for PR 19789 at commit

[GitHub] spark issue #19807: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19807 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19807: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19807 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84142/ Test PASSed. ---

[GitHub] spark issue #19807: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19807 **[Test build #84142 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84142/testReport)** for PR 19807 at commit

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2017-11-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19789 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19806: [SPARK-22595][SQL] fix flaky test: CastSuite.SPARK-22500...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19806 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19806: [SPARK-22595][SQL] fix flaky test: CastSuite.SPARK-22500...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19806 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84141/ Test PASSed. ---

[GitHub] spark issue #19806: [SPARK-22595][SQL] fix flaky test: CastSuite.SPARK-22500...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19806 **[Test build #84141 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84141/testReport)** for PR 19806 at commit

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r152870781 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -472,15 +475,66 @@ private[spark] class MapOutputTrackerMaster(

[GitHub] spark issue #19763: [SPARK-22537][core] Aggregation of map output statistics...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19763 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19763: [SPARK-22537][core] Aggregation of map output statistics...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19763 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84134/ Test PASSed. ---

[GitHub] spark issue #19763: [SPARK-22537][core] Aggregation of map output statistics...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19763 **[Test build #84134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84134/testReport)** for PR 19763 at commit

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19803 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84137/ Test PASSed. ---

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19803 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19803 **[Test build #84137 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84137/testReport)** for PR 19803 at commit

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19803 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19803: [SPARK-22596][SQL] set ctx.currentVars in CodegenSupport...

2017-11-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19803 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84138/ Test PASSed. ---

  1   2   3   >