[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-21 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19294 IMO it should be fine to not provide output directory if you are not using absolute output paths, I also don't think we should always create absolute output paths in

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-21 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/19270 ![screen shot 2017-09-21 at 1 55 19 pm](https://user-images.githubusercontent.com/13952758/30718357-8e9ee2c0-9ed4-11e7-9f70-31153c4e88f1.png) ---

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r140358379 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -63,9 +74,22 @@ private[parquet]

[GitHub] spark pull request #19269: [SPARK-22026][SQL][WIP] data source v2 write path

2017-09-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/19269#discussion_r140371372 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceV2Writer.java --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19311: [SPARK-22083][CORE] Release locks in MemoryStore.evictBl...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19311 **[Test build #82046 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82046/testReport)** for PR 19311 at commit

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16578 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16578 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82047/ Test PASSed. ---

[GitHub] spark issue #19271: [SPARK-22053][SS] Stream-stream inner join in Append Mod...

2017-09-21 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/19271 @brkyvz 1. we can add this test later. 2. as discussed offline, this method is only to generate the expression and is not responsible for checking whether the expression has watermarks.

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 > Oh crap. I know what happened here. I've been updating some of the unit tests locally, and I've been running changes against these modified tests. I'll fix this and push a commit. This

[GitHub] spark issue #19309: [SPARK-19558][sql] Add config key to register QueryExecu...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19309 **[Test build #82041 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82041/testReport)** for PR 19309 at commit

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r140357267 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/JoinFieldExtractionPushdown.scala --- @@ -0,0 +1,66 @@ +/* + *

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18659 **[Test build #82042 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82042/testReport)** for PR 18659 at commit

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82042/ Test PASSed. ---

[GitHub] spark issue #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluation for...

2017-09-21 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19122 @BryanCutler Do you have more comments? I can check it out now but don't want to review at the same time. --- - To

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16578 **[Test build #82047 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82047/testReport)** for PR 16578 at commit

[GitHub] spark pull request #19278: [SPARK-22060][ML] Fix CrossValidator/TrainValidat...

2017-09-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19278#discussion_r140374060 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -303,16 +302,17 @@ object CrossValidatorModel extends

[GitHub] spark pull request #19278: [SPARK-22060][ML] Fix CrossValidator/TrainValidat...

2017-09-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19278#discussion_r140373098 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tuning/CrossValidatorSuite.scala --- @@ -159,12 +159,15 @@ class CrossValidatorSuite

[GitHub] spark pull request #19278: [SPARK-22060][ML] Fix CrossValidator/TrainValidat...

2017-09-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19278#discussion_r140373060 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tuning/TrainValidationSplitSuite.scala --- @@ -160,11 +160,13 @@ class TrainValidationSplitSuite

[GitHub] spark pull request #19278: [SPARK-22060][ML] Fix CrossValidator/TrainValidat...

2017-09-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19278#discussion_r140373382 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -396,17 +396,24 @@ private[ml] object DefaultParamsReader {

[GitHub] spark pull request #19278: [SPARK-22060][ML] Fix CrossValidator/TrainValidat...

2017-09-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19278#discussion_r140373339 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -396,17 +396,24 @@ private[ml] object DefaultParamsReader {

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17298 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82050/ Test FAILed. ---

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17298 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17298 **[Test build #82050 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82050/testReport)** for PR 17298 at commit

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-21 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r140377904 --- Diff: python/pyspark/ml/tuning.py --- @@ -255,18 +257,30 @@ def _fit(self, dataset): randCol = self.uid + "_rand" df =

[GitHub] spark issue #19313: [SPARK-21928][CORE] Set classloader on SerializerManager...

2017-09-21 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19313 thanks @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-21 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r140373348 --- Diff: python/pyspark/ml/tuning.py --- @@ -14,15 +14,16 @@ # See the License for the specific language governing permissions and #

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-21 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r140376994 --- Diff: python/pyspark/ml/tuning.py --- @@ -255,18 +257,30 @@ def _fit(self, dataset): randCol = self.uid + "_rand" df =

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-21 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r140375935 --- Diff: python/pyspark/ml/tests.py --- @@ -836,6 +836,27 @@ def test_save_load_simple_estimator(self): loadedModel =

[GitHub] spark pull request #19313: [SPARK-21928][CORE] Set classloader on Serializer...

2017-09-21 Thread squito
Github user squito closed the pull request at: https://github.com/apache/spark/pull/19313 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r140375921 --- Diff: python/pyspark/ml/tuning.py --- @@ -14,15 +14,16 @@ # See the License for the specific language governing permissions and #

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r140375759 --- Diff: python/pyspark/ml/tests.py --- @@ -836,6 +836,27 @@ def test_save_load_simple_estimator(self): loadedModel =

[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...

2017-09-21 Thread maryannxue
Github user maryannxue commented on the issue: https://github.com/apache/spark/pull/19281 Thank you, @wzhfy, very much for the comments! I have made corresponding changes in my latest commit. --- - To unsubscribe,

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r140375849 --- Diff: python/pyspark/ml/tests.py --- @@ -986,6 +1007,25 @@ def test_save_load_simple_estimator(self): loadedModel =

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-21 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/19294 +CC @weiqingy Can you try this PR with SHC and see if it works ? That is, remove your current workaround for SPARK-21549 from SHC and try writing to hbase with a spark version patched with

[GitHub] spark pull request #19314: [SPARK-22094][SS]processAllAvailable should check...

2017-09-21 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/19314 [SPARK-22094][SS]processAllAvailable should check the query state ## What changes were proposed in this pull request? `processAllAvailable` should also check the query state and if the

[GitHub] spark issue #19194: [SPARK-20589] Allow limiting task concurrency per stage

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19194 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19312: [SPARK-22072][SPARK-22071][BUILD]Improve release build s...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19312 **[Test build #82056 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82056/testReport)** for PR 19312 at commit

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18659 **[Test build #82053 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82053/testReport)** for PR 18659 at commit

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140361162 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-21 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/19270 For me it's definitely the UI that doesn't work and the SHS that does, I'' see if I can recreate and screenshot the js error I'm getting for you ---

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16578 **[Test build #82051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82051/testReport)** for PR 16578 at commit

[GitHub] spark issue #19194: [SPARK-20589] Allow limiting task concurrency per stage

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19194 **[Test build #82052 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82052/testReport)** for PR 19194 at commit

[GitHub] spark issue #19311: [SPARK-22083][CORE] Release locks in MemoryStore.evictBl...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19311 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19311: [SPARK-22083][CORE] Release locks in MemoryStore.evictBl...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19311 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82046/ Test PASSed. ---

[GitHub] spark issue #19312: [SPARK-22072][SPARK-22071][BUILD]Improve release build s...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19312 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82048/ Test FAILed. ---

[GitHub] spark issue #19312: [SPARK-22072][SPARK-22071][BUILD]Improve release build s...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19312 **[Test build #82048 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82048/testReport)** for PR 19312 at commit

[GitHub] spark issue #19312: [SPARK-22072][SPARK-22071][BUILD]Improve release build s...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19312 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19313: [SPARK-21928][CORE] Set classloader on SerializerManager...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19313 **[Test build #82049 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82049/testReport)** for PR 19313 at commit

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82044/ Test FAILed. ---

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #82044 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82044/testReport)** for PR 19222 at commit

[GitHub] spark pull request #19311: [SPARK-22083][CORE] Release locks in MemoryStore....

2017-09-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19311#discussion_r140379012 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -544,20 +544,39 @@ private[spark] class MemoryStore( }

[GitHub] spark pull request #19269: [SPARK-22026][SQL][WIP] data source v2 write path

2017-09-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/19269#discussion_r140379971 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceV2Writer.java --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 >Test build #82051 has finished for PR 16578 at commit 00ab80c. > > * This patch fails Spark unit tests. > * This patch merges cleanly. > * This patch adds no public classes.

[GitHub] spark issue #19312: [SPARK-22072][SPARK-22071][BUILD]Improve release build s...

2017-09-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19312 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #9207: [SPARK-11171][SPARK-11237][SPARK-11241][ML] Try adding PM...

2017-09-21 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/9207 @MLNick: Do you have the bandwith to revisit this? I'm open to refactoring to a more plug-gable approach if we've got the review bandwidth for it. ---

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r140351256 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -63,9 +74,22 @@ private[parquet]

[GitHub] spark pull request #19271: [SPARK-22053][SS] Stream-stream inner join in App...

2017-09-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19271 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19271: [SPARK-22053][SS] Stream-stream inner join in Append Mod...

2017-09-21 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/19271 @brkyvz @zsxwing thank you for reviewing. I have merged it to master! --- - To unsubscribe, e-mail:

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17298 **[Test build #82050 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82050/testReport)** for PR 17298 at commit

[GitHub] spark issue #19313: [SPARK-21928][CORE] Set classloader on SerializerManager...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19313 **[Test build #82049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82049/testReport)** for PR 19313 at commit

[GitHub] spark issue #18994: [SPARK-21784][SQL] Adds support for defining information...

2017-09-21 Thread sureshthalamati
Github user sureshthalamati commented on the issue: https://github.com/apache/spark/pull/18994 ping @gatorsmile @cloud-fan @rxin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16578 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r140366890 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/AggregateFieldExtractionPushdown.scala --- @@ -0,0 +1,77 @@ +/* + *

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16578 **[Test build #82051 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82051/testReport)** for PR 16578 at commit

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16578 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82051/ Test FAILed. ---

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16578 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19314: [SPARK-22094][SS]processAllAvailable should check the qu...

2017-09-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19314 cc @marmbrus --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19314: [SPARK-22094][SS]processAllAvailable should check the qu...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19314 **[Test build #82055 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82055/testReport)** for PR 19314 at commit

[GitHub] spark pull request #19269: [SPARK-22026][SQL][WIP] data source v2 write path

2017-09-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/19269#discussion_r140389681 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceV2Writer.java --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDF...

2017-09-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18659#discussion_r140389432 --- Diff: python/pyspark/sql/functions.py --- @@ -2142,18 +2159,26 @@ def udf(f=None, returnType=StringType()): | 8| JOHN DOE|

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82053/ Test PASSed. ---

[GitHub] spark issue #19301: [SPARK-22084][SQL] Fix performance regression in aggrega...

2017-09-21 Thread stanzhai
Github user stanzhai commented on the issue: https://github.com/apache/spark/pull/19301 @cenyuhai This is an optimize for physical plan, and your case can be optimized. ```SQL select dt, geohash_of_latlng, sum(mt_cnt), sum(ele_cnt), round(sum(mt_cnt) *

[GitHub] spark pull request #19313: [SPARK-21928][CORE] Set classloader on Serializer...

2017-09-21 Thread squito
GitHub user squito opened a pull request: https://github.com/apache/spark/pull/19313 [SPARK-21928][CORE] Set classloader on SerializerManager's private kryo ## What changes were proposed in this pull request? We have to make sure that SerializerManager's private instance of

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-21 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/19294 @szhem Did you try this patch with sql ? A cursory look at `org.apache.spark.sql.execution.datasources.FileFormatWriter` and

[GitHub] spark issue #19309: [SPARK-19558][sql] Add config key to register QueryExecu...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19309 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82041/ Test PASSed. ---

[GitHub] spark issue #19309: [SPARK-19558][sql] Add config key to register QueryExecu...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19309 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #82043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82043/testReport)** for PR 18747 at commit

[GitHub] spark issue #19271: [SPARK-22053][SS] Stream-stream inner join in Append Mod...

2017-09-21 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/19271 LGTM: Would love to see the following things a bit more buttoned up in a follow up PR: 1. Tests on join + aggregation and aggregation + join 2. A better, safer, more understandable

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16578 **[Test build #82045 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82045/testReport)** for PR 16578 at commit

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r140368054 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala --- @@ -77,20 +77,21 @@ trait

[GitHub] spark issue #17359: [SPARK-20028][SQL] Add aggreagate expression nGrams

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17359 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19313: [SPARK-21928][CORE] Set classloader on SerializerManager...

2017-09-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19313 Merging to 2.1. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19313: [SPARK-21928][CORE] Set classloader on SerializerManager...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19313 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82049/ Test PASSed. ---

[GitHub] spark issue #19313: [SPARK-21928][CORE] Set classloader on SerializerManager...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19313 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19281 **[Test build #82054 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82054/testReport)** for PR 19281 at commit

[GitHub] spark pull request #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calc...

2017-09-21 Thread maryannxue
Github user maryannxue commented on a diff in the pull request: https://github.com/apache/spark/pull/19281#discussion_r140381057 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -96,6 +96,24 @@ object SortOrder {

[GitHub] spark issue #19278: [SPARK-22060][ML] Fix CrossValidator/TrainValidationSpli...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19278 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19278: [SPARK-22060][ML] Fix CrossValidator/TrainValidationSpli...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19278 **[Test build #82057 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82057/testReport)** for PR 19278 at commit

[GitHub] spark issue #19278: [SPARK-22060][ML] Fix CrossValidator/TrainValidationSpli...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19278 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82057/ Test PASSed. ---

[GitHub] spark issue #19317: [SPARK-22098][CORE] Add new method aggregateByKeyLocally...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19317 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19317: [SPARK-22098][CORE] Add new method aggregateByKeyLocally...

2017-09-21 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/19317 cc @VinceShieh --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18945: [SPARK-21766][SQL] Convert nullable int columns to float...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18945 **[Test build #82060 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82060/testReport)** for PR 18945 at commit

[GitHub] spark pull request #18945: [SPARK-21766][SQL] Convert nullable int columns t...

2017-09-21 Thread logannc
Github user logannc commented on a diff in the pull request: https://github.com/apache/spark/pull/18945#discussion_r140413579 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1761,12 +1761,37 @@ def toPandas(self): raise ImportError("%s\n%s" % (e.message,

[GitHub] spark pull request #18945: [SPARK-21766][SQL] Convert nullable int columns t...

2017-09-21 Thread logannc
Github user logannc commented on a diff in the pull request: https://github.com/apache/spark/pull/18945#discussion_r140414783 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1761,12 +1761,37 @@ def toPandas(self): raise ImportError("%s\n%s" % (e.message,

[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19281 **[Test build #82054 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82054/testReport)** for PR 19281 at commit

[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19281 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82054/ Test PASSed. ---

[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19281 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   3   4   5   >