[GitHub] spark pull request: [SQL] Prevents per row dynamic dispatching and...

2014-09-29 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/2592 [SQL] Prevents per row dynamic dispatching and pattern matching when inserting Hive values Builds all wrappers at first according to object inspector types to avoid per row costs. TODO:

[GitHub] spark pull request: [SPARK-3613] Record only average block size in...

2014-09-29 Thread Ishiihara
Github user Ishiihara commented on the pull request: https://github.com/apache/spark/pull/2470#issuecomment-57274044 @rxin I looked through Roaring bitmap and that is a highly compressed bitmap compared with other bitmap implementations. I will start working on this and keep you updat

[GitHub] spark pull request: [SPARK-3495] Block replication fails continuou...

2014-09-29 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2366#discussion_r18201286 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -787,31 +791,110 @@ private[spark] class BlockManager( } /**

[GitHub] spark pull request: [SPARK-3495] Block replication fails continuou...

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2366#discussion_r18201230 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -787,31 +791,110 @@ private[spark] class BlockManager( } /**

[GitHub] spark pull request: [SPARK-3495] Block replication fails continuou...

2014-09-29 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2366#discussion_r18201208 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -787,31 +791,110 @@ private[spark] class BlockManager( } /**

[GitHub] spark pull request: [SPARK-3495] Block replication fails continuou...

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2366#discussion_r18201177 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -787,31 +791,110 @@ private[spark] class BlockManager( } /**

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r18201113 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/python/PythonDStream.scala --- @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r18201102 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/python/PythonDStream.scala --- @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r1820 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/python/PythonDStream.scala --- @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r18201096 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/python/PythonDStream.scala --- @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r18201085 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/python/PythonDStream.scala --- @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request: [SPARK-2706][SQL] Enable Spark to support Hive...

2014-09-29 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2241#issuecomment-57273157 Hey @zhzhan I've published a modified version of Hive 0.13 that we can link against. A few benefits is: 1. I fixed the hive-exec jar so it only contains hive pac

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r18201011 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/python/PythonDStream.scala --- @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...

2014-09-29 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2432#discussion_r18200976 --- Diff: core/src/main/java/org/apache/spark/ApplicationId.java --- @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r18200937 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/python/PythonDStream.scala --- @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request: [SPARK-3734] DriverRunner should not read SPAR...

2014-09-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2586 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-3412][SQL]add missing row api

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2529#issuecomment-57272803 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/207/consoleFull) for PR 2529 at commit [`4c18c29`](https://github.com/

[GitHub] spark pull request: [SPARK-3734] DriverRunner should not read SPAR...

2014-09-29 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2586#issuecomment-57272762 LGTM. Merging into master and 1.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r18200872 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/python/PythonDStream.scala --- @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2330#issuecomment-57272680 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21025/consoleFull) for PR 2330 at commit [`0dae310`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3713][SQL] Uses JSON to serialize DataT...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2563#issuecomment-57272666 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/210/consoleFull) for PR 2563 at commit [`03da3ec`](https://github.com/a

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2591#issuecomment-57272501 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57272306 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21020/consoleFull) for PR 2432 at commit [`f6af132`](https://github.com/a

[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...

2014-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57272310 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3654][SQL] Implement all extended HiveQ...

2014-09-29 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2590#issuecomment-57272298 Awesome! I just started working on this last weekend and you've already got done :) Left some minor comments. This generally LGTM. --- If your project is set up for it

[GitHub] spark pull request: [SPARK-3654][SQL] Implement all extended HiveQ...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2590#issuecomment-57272266 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/209/consoleFull) for PR 2590 at commit [`ba26cd1`](https://github.com/a

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57272186 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21022/consoleFull) for PR 2588 at commit [`6dab2e3`](https://github.com/a

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57272188 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57272107 It worked because askSlaves was true and the driver always queries the slaves in your afterUnpersist test. The problem is with regard to reporting, not whether the block its

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2591#issuecomment-57271920 We should also cherrypick this into branch-1.0. Master branch has been fixed in https://github.com/apache/spark/pull/2588 --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57271851 Actually, I took a look, it does test that. So I am not sure how it was passing earlier some of the times. --- If your project is set up for it, you can reply to this emai

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/2591 [SPARK-3709] Executors don't always report broadcast block removal properly back to the driver (for branch-1.1) You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] spark pull request: [SPARK-3654][SQL] Implement all extended HiveQ...

2014-09-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2590#discussion_r18200542 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSqlParser.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark pull request: [SPARK-3166]: Allow custom serialiser to be sh...

2014-09-29 Thread ypwais
Github user ypwais commented on the pull request: https://github.com/apache/spark/pull/1890#issuecomment-57271841 Any chance this might make it into v1.2? I'd love to use custom {Input,Output}Formats (e.g. Parquet) and I personally spent almost a day after getting bitten by this clas

[GitHub] spark pull request: SPARK-1830 Deploy failover, Make Persistence e...

2014-09-29 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/771#issuecomment-57271768 Because they were private spark. It is very inconvenient for someone to write his/her own recovery mode with all that private spark. + This felt like developer facing A

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2588 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57271603 Yes - can you submit one? I'm going to merge this because it has been blocking a lot of other patches. --- If your project is set up for it, you can reply to this email an

[GitHub] spark pull request: [SPARK-3654][SQL] Implement all extended HiveQ...

2014-09-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2590#discussion_r18200443 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSqlParser.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark pull request: [SPARK-3412][SQL]add missing row api

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2529#issuecomment-57271368 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/208/consoleFull) for PR 2529 at commit [`4c18c29`](https://github.com/a

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57271331 Isnt there a way to augment the existing tests to make sure that the state in the driver (blockmanagermaster) is cleared after removing tests? --- If your project is set up

[GitHub] spark pull request: [SPARK-3654][SQL] Implement all extended HiveQ...

2014-09-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2590#discussion_r18200397 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala --- @@ -75,6 +75,9 @@ class LocalHiveContext(sc: SparkContext) extends HiveCo

[GitHub] spark pull request: [SPARK-3495] Block replication fails continuou...

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2366#discussion_r18200328 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -787,31 +791,110 @@ private[spark] class BlockManager( } /**

[GitHub] spark pull request: [SPARK-3654][SQL] Implement all extended HiveQ...

2014-09-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2590#discussion_r18200330 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSqlParser.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark pull request: [SPARK-3654][SQL] Implement all extended HiveQ...

2014-09-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2590#discussion_r18200319 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSqlParser.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark pull request: [SPARK-3412][SQL]add missing row api

2014-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2529#issuecomment-57270939 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3654][SQL] Implement all extended HiveQ...

2014-09-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2590#discussion_r18200306 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSqlParser.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark pull request: [SPARK-3495] Block replication fails continuou...

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2366#discussion_r18200286 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -111,6 +112,9 @@ private[spark] class BlockManager( MetadataCleanerType

[GitHub] spark pull request: [SPARK-3495] Block replication fails continuou...

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2366#discussion_r18200248 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -787,31 +791,110 @@ private[spark] class BlockManager( } /**

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r18200194 --- Diff: python/pyspark/serializers.py --- @@ -114,6 +114,9 @@ def __ne__(self, other): def __repr__(self): return "<%s object>" % self.__c

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r18200181 --- Diff: python/pyspark/accumulators.py --- @@ -256,3 +256,8 @@ def _start_update_server(): thread.daemon = True thread.start() return

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r18200173 --- Diff: bin/pyspark --- @@ -87,11 +87,7 @@ export PYSPARK_SUBMIT_ARGS if [[ -n "$SPARK_TESTING" ]]; then unset YARN_CONF_DIR unset HADOOP_CON

[GitHub] spark pull request: [SPARK-3613] Record only average block size in...

2014-09-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2470#issuecomment-57270495 I also filed a new jira for the compressed bitmap thing: https://issues.apache.org/jira/browse/SPARK-3740 --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-3613] Record only average block size in...

2014-09-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2470 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-3613] Record only average block size in...

2014-09-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2470#issuecomment-57270302 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r18200124 --- Diff: python/pyspark/accumulators.py --- @@ -256,3 +256,8 @@ def _start_update_server(): thread.daemon = True thread.start() retu

[GitHub] spark pull request: [SPARK-3412][SQL]add missing row api

2014-09-29 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/2529#issuecomment-57270205 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57270130 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57270125 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21019/consoleFull) for PR 2588 at commit [`f430686`](https://github.com/a

[GitHub] spark pull request: [SPARK-3613] Record only average block size in...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2470#issuecomment-57270082 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/206/consoleFull) for PR 2470 at commit [`822ff54`](https://github.com/

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57270069 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/205/consoleFull) for PR 2588 at commit [`f430686`](https://github.com/

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-09-29 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2538#discussion_r18200031 --- Diff: bin/pyspark --- @@ -87,11 +87,7 @@ export PYSPARK_SUBMIT_ARGS if [[ -n "$SPARK_TESTING" ]]; then unset YARN_CONF_DIR unset HADOOP_C

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-09-29 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/2576#issuecomment-57269751 @cloud-fan, has not since #2475 has not merged to master. @marmbrus can you take a look at this? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-3412][SQL]add missing row api

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2529#issuecomment-57269637 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/207/consoleFull) for PR 2529 at commit [`4c18c29`](https://github.com/a

[GitHub] spark pull request: [SPARK-3412][SQL]add missing row api

2014-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2529#issuecomment-57269335 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57269222 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21022/consoleFull) for PR 2588 at commit [`6dab2e3`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3707] [SQL] Fix bug of type coercion in...

2014-09-29 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/2559#discussion_r18199560 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala --- @@ -74,7 +81,7 @@ class AnalysisSuite extends FunSui

[GitHub] spark pull request: [SPARK-3412][SQL]add missing row api

2014-09-29 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/2529#issuecomment-57268536 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57268365 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21020/consoleFull) for PR 2432 at commit [`f6af132`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3654][SQL] Implement all extended HiveQ...

2014-09-29 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2590#issuecomment-57268296 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark pull request: Minor cleanup of code.

2014-09-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2581 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: Minor cleanup of code.

2014-09-29 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2581#issuecomment-57267027 LGTM. Merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3709] [WIP] Executors don't always repo...

2014-09-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57266871 @nchammas - would you be interested in submitting a pr to change the qa script so that the timeout and failure message already prints the commit hash? --- If your project i

[GitHub] spark pull request: [SPARK-3709] [WIP] Executors don't always repo...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57266877 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/204/consoleFull)** after a configured wait of `120m`. --- If your project

[GitHub] spark pull request: [SPARK-3709] [WIP] Executors don't always repo...

2014-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57266831 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3709] [WIP] Executors don't always repo...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57266826 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21014/consoleFull)** after a configured wait of `120m`. --- If your project i

[GitHub] spark pull request: [SPARK-3654][SQL] Implement all extended HiveQ...

2014-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2590#issuecomment-57266684 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-3613] Record only average block size in...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2470#issuecomment-57266700 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/206/consoleFull) for PR 2470 at commit [`822ff54`](https://github.com/a

[GitHub] spark pull request: [SPARK-3654][SQL] Implement all extended HiveQ...

2014-09-29 Thread ravipesala
GitHub user ravipesala opened a pull request: https://github.com/apache/spark/pull/2590 [SPARK-3654][SQL] Implement all extended HiveQL statements/commands with a separate parser combinator Created separate parser for hql. It preparses the commands like cache,uncache,add jar etc..

[GitHub] spark pull request: Added debug logging ... [DO NOT MERGE]

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57266321 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/205/consoleFull) for PR 2588 at commit [`f430686`](https://github.com/a

[GitHub] spark pull request: Added debug logging ... [DO NOT MERGE]

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57266312 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21019/consoleFull) for PR 2588 at commit [`f430686`](https://github.com/ap

[GitHub] spark pull request: Added debug logging ... [DO NOT MERGE]

2014-09-29 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2588#discussion_r18198828 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerSlaveActor.scala --- @@ -58,9 +58,9 @@ class BlockManagerSlaveActor( SparkEnv.get

[GitHub] spark pull request: Added debug logging ... [DO NOT MERGE]

2014-09-29 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57266259 @aarondav I already merged this actually... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: [SPARK-3709] Fix the flaky test in BroadcastSu...

2014-09-29 Thread rxin
Github user rxin closed the pull request at: https://github.com/apache/spark/pull/2585 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[GitHub] spark pull request: [SPARK-3709] Fix the flaky test in BroadcastSu...

2014-09-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2585#issuecomment-57266183 Replaced by https://github.com/apache/spark/pull/2588 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Added debug logging ... [DO NOT MERGE]

2014-09-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57265739 Make sure you backport in 0.3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: Added debug logging ... [DO NOT MERGE]

2014-09-29 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57265715 LGTM. Merging into master, branch-1.1, and branch-1.0. Should I also backport to branch-0.9? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-3707] [SQL] Fix bug of type coercion in...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2559#issuecomment-57265653 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21018/consoleFull) for PR 2559 at commit [`199a85d`](https://github.com/a

[GitHub] spark pull request: [SPARK-3739] [SQL] Update the split num base o...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2589#issuecomment-57265671 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21017/consoleFull) for PR 2589 at commit [`5f0d75b`](https://github.com/a

[GitHub] spark pull request: [SPARK-3707] [SQL] Fix bug of type coercion in...

2014-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2559#issuecomment-57265657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3739] [SQL] Update the split num base o...

2014-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2589#issuecomment-57265675 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-2706][SQL] Enable Spark to support Hive...

2014-09-29 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/2241#issuecomment-57265563 I meant those implicit classes and methods. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-2706][SQL] Enable Spark to support Hive...

2014-09-29 Thread zhzhan
Github user zhzhan commented on a diff in the pull request: https://github.com/apache/spark/pull/2241#discussion_r18198152 --- Diff: sql/hive/pom.xml --- @@ -119,6 +83,74 @@ + hive-default + + + !hive.version --

[GitHub] spark pull request: [SPARK-2706][SQL] Enable Spark to support Hive...

2014-09-29 Thread zhzhan
Github user zhzhan commented on the pull request: https://github.com/apache/spark/pull/2241#issuecomment-57264575 @yhuai Can you be more specific regarding the comments: "I think those implicits are not necessary. Can you change those?" --- If your project is set up for it, you can r

[GitHub] spark pull request: [SPARK-3739] [SQL] Update the split num base o...

2014-09-29 Thread chenghao-intel
GitHub user chenghao-intel opened a pull request: https://github.com/apache/spark/pull/2589 [SPARK-3739] [SQL] Update the split num base on block size for table scanning Source file input split is probably better based on block size of HDFS, while scanning Hive table, other than th

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-09-29 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/2576#issuecomment-57261596 Have you considered cooperate with https://github.com/apache/spark/pull/2475? --- If your project is set up for it, you can reply to this email and have your reply app

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...

2014-09-29 Thread huifeidemaer
Github user huifeidemaer commented on a diff in the pull request: https://github.com/apache/spark/pull/2388#discussion_r18197041 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/TopicModeling.scala --- @@ -0,0 +1,818 @@ +/* + * Licensed to the Apache Softwar

[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-57261085 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21016/consoleFull) for PR 2570 at commit [`4ea462c`](https://github.com/ap

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...

2014-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-57260813 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-57260810 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21015/consoleFull) for PR 2388 at commit [`298c720`](https://github.com/a

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-57260729 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21015/consoleFull) for PR 2388 at commit [`298c720`](https://github.com/ap

  1   2   3   4   5   >