[GitHub] spark pull request: [SPARK-2808][Streaming][Kafka] update kafka to...

2015-04-08 Thread zzcclp
Github user zzcclp commented on the pull request: https://github.com/apache/spark/pull/4537#issuecomment-91130762 @koeninger , I merge this RP into master and build, it is OK. then run tests as follows: >mvn -Pscala-2.10 -Phadoop-2.3 -Pyarn -Dyarn.version=2.3.0-cdh5.1.2 -Dhadoop.v

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5236#issuecomment-91130309 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5236#issuecomment-91130301 [Test build #29924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29924/consoleFull) for PR 5236 at commit [`642a06d`](https://gith

[GitHub] spark pull request: [SPARK-6715][SQL] Eliminate duplicate filters ...

2015-04-08 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/5369#issuecomment-91129184 Okay. However, I think it is still important to make the query plan simpler by removing redundant parts even those parts are fast operations. E.g., for this pr, th

[GitHub] spark pull request: [SPARK-6715][SQL] Eliminate duplicate filters ...

2015-04-08 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/5369 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6691][Streaming][WIP] Add a dynamic Rat...

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5385#issuecomment-91128786 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6691][Streaming][WIP] Add a dynamic Rat...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5385#issuecomment-91128782 [Test build #29926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29926/consoleFull) for PR 5385 at commit [`82ae607`](https://gith

[GitHub] spark pull request: [SQL][minor] move `resolveGetField` into a obj...

2015-04-08 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/5435#issuecomment-91128109 cc @marmbrus, it's a follow-up of https://github.com/apache/spark/pull/5304. --- If your project is set up for it, you can reply to this email and have your reply appea

[GitHub] spark pull request: [SQL][minor] move `resolveGetField` into a obj...

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5435#issuecomment-91127640 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SQL][minor] move `resolveGetField` into a obj...

2015-04-08 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/5435 [SQL][minor] move `resolveGetField` into a object The method `resolveGetField` isn't belong to `LogicalPlan` logically and didn't access any members of it. You can merge this pull request into a

[GitHub] spark pull request: [SPARK-6691][Streaming][WIP] Add a dynamic Rat...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5385#issuecomment-91127152 [Test build #29926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29926/consoleFull) for PR 5385 at commit [`82ae607`](https://githu

[GitHub] spark pull request: [SPARK-6691][Streaming][WIP] Add a dynamic Rat...

2015-04-08 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/5385#issuecomment-91126900 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-6766][Streaming] Fix issue about Stream...

2015-04-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/5414#discussion_r28037692 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingJobProgressListener.scala --- @@ -33,7 +33,7 @@ private[streaming] class StreamingJob

[GitHub] spark pull request: [SPARK-6343] Doc driver-worker network reqs

2015-04-08 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5382#issuecomment-91125785 Great LTM - @WangTaoTheTonic does that look okay to you? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-6528][ML] Add IDF transformer

2015-04-08 Thread yinxusen
Github user yinxusen commented on a diff in the pull request: https://github.com/apache/spark/pull/5266#discussion_r28037378 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/IDF.scala --- @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] spark pull request: [SPARK-5972] [MLlib] Cache residuals and gradi...

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5330#issuecomment-91124464 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-5972] [MLlib] Cache residuals and gradi...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5330#issuecomment-91124457 [Test build #29923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29923/consoleFull) for PR 5330 at commit [`67b5311`](https://gith

[GitHub] spark pull request: [SPARK-5654] Integrate SparkR

2015-04-08 Thread concretevitamin
Github user concretevitamin commented on the pull request: https://github.com/apache/spark/pull/5096#issuecomment-91123979 :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-08 Thread ilganeli
Github user ilganeli commented on a diff in the pull request: https://github.com/apache/spark/pull/5236#discussion_r28037051 --- Diff: network/common/src/main/java/org/apache/spark/network/util/JavaUtils.java --- @@ -121,4 +125,69 @@ private static boolean isSymlink(File file) thro

[GitHub] spark pull request: [SPARK-6796][Streaming][WebUI] Add "Active Bat...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5434#issuecomment-91122700 [Test build #29922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29922/consoleFull) for PR 5434 at commit [`8286a26`](https://gith

[GitHub] spark pull request: [SPARK-6796][Streaming][WebUI] Add "Active Bat...

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5434#issuecomment-91122724 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-3074] [PySpark] support groupByKey() wi...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1977#issuecomment-91121784 [Test build #29925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29925/consoleFull) for PR 1977 at commit [`67772dd`](https://githu

[GitHub] spark pull request: [SPARK-5654] Integrate SparkR

2015-04-08 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/5096#issuecomment-91121381 Thanks @andrewor14 @pwendell for the reviews. Now that Jenkins is happy I am going merge this in and I'll file follow up issues for things like YARN cluster mode which w

[GitHub] spark pull request: [SPARK-5654] Integrate SparkR

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5096#issuecomment-91121091 [Test build #29919 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29919/consoleFull) for PR 5096 at commit [`da64742`](https://gith

[GitHub] spark pull request: [SPARK-5654] Integrate SparkR

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5096#issuecomment-91121106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5236#issuecomment-91119705 [Test build #29924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29924/consoleFull) for PR 5236 at commit [`642a06d`](https://githu

[GitHub] spark pull request: [SPARK-5886][ML] Add label indexer

2015-04-08 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4735#discussion_r28035557 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LabelIndexer.scala --- @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-6343] Doc driver-worker network reqs

2015-04-08 Thread parente
Github user parente commented on the pull request: https://github.com/apache/spark/pull/5382#issuecomment-91116274 @pwendell @WangTaoTheTonic Good catch. I misconnected the upper line. It was not intentional. I've updated the PR with the diagram fix. --- If your project is s

[GitHub] spark pull request: [SPARK-3074] [PySpark] support groupByKey() wi...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1977#issuecomment-91115066 [Test build #29921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29921/consoleFull) for PR 1977 at commit [`e78c15c`](https://gith

[GitHub] spark pull request: [SPARK-3074] [PySpark] support groupByKey() wi...

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1977#issuecomment-91115081 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SQL] [SPARK-6794] Use kryo-based SparkSqlSeri...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5433#issuecomment-91114821 [Test build #29920 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29920/consoleFull) for PR 5433 at commit [`527eac6`](https://gith

[GitHub] spark pull request: [SQL] [SPARK-6794] Use kryo-based SparkSqlSeri...

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5433#issuecomment-91114827 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-3468][WebUI] Timeline-View feature

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2342#issuecomment-91114206 [Test build #29918 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29918/consoleFull) for PR 2342 at commit [`dec85db`](https://gith

[GitHub] spark pull request: [SPARK-3468][WebUI] Timeline-View feature

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2342#issuecomment-91114215 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-5972] [MLlib] Cache residuals and gradi...

2015-04-08 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/5330#issuecomment-91114211 @jkbradley I have fixed up your comment. So the idea is to broadcast each tree across all data, right? Sorry for the mess up. I got confused a bit, by your previous co

[GitHub] spark pull request: [SPARK-5972] [MLlib] Cache residuals and gradi...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5330#issuecomment-91114090 [Test build #29923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29923/consoleFull) for PR 5330 at commit [`67b5311`](https://githu

[GitHub] spark pull request: [SPARK-6796][Streaming][WebUI] Add "Active Bat...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5434#issuecomment-91113182 [Test build #29922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29922/consoleFull) for PR 5434 at commit [`8286a26`](https://githu

[GitHub] spark pull request: [SPARK-6796][Streaming][WebUI] Add "Active Bat...

2015-04-08 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/5434#issuecomment-91113024 cc @tdas and @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-6796][Streaming][WebUI] Add "Active Bat...

2015-04-08 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/5434 [SPARK-6796][Streaming][WebUI] Add "Active Batches" and "Completed Batches" lists to StreamingPage This PR adds two lists, `Active Batches` and `Completed Batches`. Here is the screenshot:

[GitHub] spark pull request: [SPARK-4705] Handle multiple app attempts even...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5432#issuecomment-9161 [Test build #29917 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29917/consoleFull) for PR 5432 at commit [`3a14503`](https://gith

[GitHub] spark pull request: [SPARK-4705] Handle multiple app attempts even...

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5432#issuecomment-9181 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-5794] [SQL] fix add jar

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4586#issuecomment-91110437 [Test build #29916 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29916/consoleFull) for PR 4586 at commit [`efdd602`](https://gith

[GitHub] spark pull request: [SPARK-5794] [SQL] fix add jar

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4586#issuecomment-91110458 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5236#discussion_r28033893 --- Diff: network/common/src/main/java/org/apache/spark/network/util/JavaUtils.java --- @@ -121,4 +125,69 @@ private static boolean isSymlink(File file) throws

[GitHub] spark pull request: [SPARK-6479][Block Manager]Create off-heap blo...

2015-04-08 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/5430#discussion_r28033633 --- Diff: core/src/main/scala/org/apache/spark/storage/OffHeapStore.scala --- @@ -0,0 +1,142 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SQL] SPARK-6548: Adding stddev to DataFrame f...

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5357#issuecomment-91108793 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SQL] SPARK-6548: Adding stddev to DataFrame f...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5357#issuecomment-91108787 [Test build #29915 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29915/consoleFull) for PR 5357 at commit [`a012fc3`](https://gith

[GitHub] spark pull request: [SPARK-3074] [PySpark] support groupByKey() wi...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1977#issuecomment-91108728 [Test build #29921 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29921/consoleFull) for PR 1977 at commit [`e78c15c`](https://githu

[GitHub] spark pull request: [SPARK-6479][Block Manager]Create off-heap blo...

2015-04-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5430#issuecomment-91108743 Thanks - let's put the Tachyon patch with this. In this case, I think it will be easier to review and understand the API semantics. --- If your project is set up for it, yo

[GitHub] spark pull request: [SPARK-3074] [PySpark] support groupByKey() wi...

2015-04-08 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/1977#discussion_r28033500 --- Diff: python/pyspark/shuffle.py --- @@ -529,6 +522,322 @@ def sorted(self, iterator, key=None, reverse=False): return heapq.merge(chunks, key=k

[GitHub] spark pull request: [SPARK-3074] [PySpark] support groupByKey() wi...

2015-04-08 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/1977#issuecomment-91108456 @JoshRosen Thanks for the comments, it looks better now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-6765] Fix test code style for SQL

2015-04-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5412 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6765] Fix test code style for SQL

2015-04-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5412#issuecomment-91105998 Ok I've merged it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SQL] [SPARK-6794] Use kryo-based SparkSqlSeri...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5433#issuecomment-91105448 [Test build #29920 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29920/consoleFull) for PR 5433 at commit [`527eac6`](https://githu

[GitHub] spark pull request: [SPARK-3074] [PySpark] support groupByKey() wi...

2015-04-08 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/1977#discussion_r28032966 --- Diff: python/pyspark/shuffle.py --- @@ -529,6 +522,322 @@ def sorted(self, iterator, key=None, reverse=False): return heapq.merge(chunks, key=k

[GitHub] spark pull request: [SQL] [SPARK-6794] Use kryo-based SparkSqlSeri...

2015-04-08 Thread vlyubin
GitHub user vlyubin opened a pull request: https://github.com/apache/spark/pull/5433 [SQL] [SPARK-6794] Use kryo-based SparkSqlSerializer for GeneralHashedRelation Benchmarking results: http://pastie.org/private/1dneo1mta5zpsw6gmsoeq You can merge this pull request into a Git repos

[GitHub] spark pull request: [SPARK-3074] [PySpark] support groupByKey() wi...

2015-04-08 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/1977#discussion_r28032910 --- Diff: python/pyspark/shuffle.py --- @@ -529,6 +522,322 @@ def sorted(self, iterator, key=None, reverse=False): return heapq.merge(chunks, key=k

[GitHub] spark pull request: [SPARK-6577] [MLlib] SparseMatrix should be su...

2015-04-08 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/5355#issuecomment-91104754 @mengxr @davies Can some initial comments be given on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark pull request: [SPARK-5654] Integrate SparkR

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5096#issuecomment-91104742 [Test build #29919 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29919/consoleFull) for PR 5096 at commit [`da64742`](https://githu

[GitHub] spark pull request: [SPARK-3468][WebUI] Timeline-View feature

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2342#issuecomment-91104394 [Test build #29918 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29918/consoleFull) for PR 2342 at commit [`dec85db`](https://githu

[GitHub] spark pull request: [SPARK-4705] Handle multiple app attempts even...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5432#issuecomment-91103821 [Test build #29917 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29917/consoleFull) for PR 5432 at commit [`3a14503`](https://githu

[GitHub] spark pull request: [SPARK-3468][WebUI] Timeline-View feature

2015-04-08 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/2342#issuecomment-91103750 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-6006][SQL]: Optimize count distinct for...

2015-04-08 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4764#issuecomment-91093315 As a very rough sketch (this is totally untested and I'm probably missing cases), I'd hope the solution could look something like the following: ```scala obj

[GitHub] spark pull request: [CORE] The newest log will be loaded many time...

2015-04-08 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/5386#issuecomment-91093107 ok. thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: [CORE] The newest log will be loaded many time...

2015-04-08 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei closed the pull request at: https://github.com/apache/spark/pull/5386 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [CORE] The newest log will be loaded many time...

2015-04-08 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/5386#issuecomment-91093099 ok. thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: [SPARK-6343] Doc driver-worker network reqs

2015-04-08 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/5382#issuecomment-91092834 I have same question as Patrick too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark pull request: [SPARK-6479][Block Manager]Create off-heap blo...

2015-04-08 Thread viper-kun
Github user viper-kun commented on a diff in the pull request: https://github.com/apache/spark/pull/5430#discussion_r28030832 --- Diff: core/src/main/scala/org/apache/spark/storage/OffHeapStore.scala --- @@ -0,0 +1,142 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] spark pull request: [SPARK-6479][Block Manager]Create off-heap blo...

2015-04-08 Thread viper-kun
Github user viper-kun commented on a diff in the pull request: https://github.com/apache/spark/pull/5430#discussion_r28030751 --- Diff: core/src/main/scala/org/apache/spark/storage/OffHeapBlockManager.scala --- @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Found

[GitHub] spark pull request: [SPARK-6006][SQL]: Optimize count distinct for...

2015-04-08 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4764#issuecomment-91089650 Thanks for working ont his and sorry for the delay in reviewing it. My high level feedback is that I think we should optimize handling of distinct aggregation, but ther

[GitHub] spark pull request: [SPARK-5654] Integrate SparkR

2015-04-08 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/5096#issuecomment-91088926 @pwendell Its around 2 minutes on my laptop. Here is the output on my machine ``` time ./run-tests.sh ./run-tests.sh 1:56.96 total ```

[GitHub] spark pull request: [SPARK-6765] Fix test code style for SQL

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5412#issuecomment-91088775 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6765] Fix test code style for SQL

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5412#issuecomment-91088769 [Test build #29904 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29904/consoleFull) for PR 5412 at commit [`9098a31`](https://gith

[GitHub] spark pull request: [Spark-5682] Add spark encrypted shuffle by us...

2015-04-08 Thread kellyzly
Github user kellyzly commented on a diff in the pull request: https://github.com/apache/spark/pull/5307#discussion_r28030065 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala --- @@ -123,12 +130,34 @@ private[spark] class DiskBlockObjectWriter(

[GitHub] spark pull request: [SQL] [WIP] Blacklists several Hive 0.13.1 spe...

2015-04-08 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4851#issuecomment-91088528 Okay, lets close this issue then. I think we are on track to do the metadata / execution split for hive in 1.4 --- If your project is set up for it, you can reply to t

[GitHub] spark pull request: [SPARK-6696] [SQL] Adds HiveContext.refreshTab...

2015-04-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5349 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6696] [SQL] Adds HiveContext.refreshTab...

2015-04-08 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/5349#issuecomment-91088466 Thanks, I merged this to master. Would be great to add a TestHive wrapper in a followup PR. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-6343] Doc driver-worker network reqs

2015-04-08 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5382#issuecomment-91088355 Great to have an improvement here. One thing I don't understand, there are two curved arrows from the SparkContext to the Executors/Workers. However, in the upper arrow

[GitHub] spark pull request: [SPARK-5794] [SQL] fix add jar

2015-04-08 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/4586#discussion_r28029928 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala --- @@ -793,6 +793,20 @@ class HiveQuerySuite extends HiveCompari

[GitHub] spark pull request: [SPARK-6762]Fix potential resource leaks in Ch...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5407#issuecomment-91087655 [Test build #29914 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29914/consoleFull) for PR 5407 at commit [`a754adc`](https://gith

[GitHub] spark pull request: [SPARK-6762]Fix potential resource leaks in Ch...

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5407#issuecomment-91087668 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SQL] Faster Scala row conversion

2015-04-08 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/5419#discussion_r28029875 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -95,10 +95,14 @@ trait ScalaReflection { }

[GitHub] spark pull request: [SQL] Faster Scala row conversion

2015-04-08 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/5419#discussion_r28029884 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -150,7 +150,7 @@ abstract class QueryPlan[PlanType <: Tree

[GitHub] spark pull request: [SPARK-6451][SQL] supported code generation fo...

2015-04-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5138 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6451][SQL] supported code generation fo...

2015-04-08 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/5138#issuecomment-91087439 Thanks, merged to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-6762]Fix potential resource leaks in Ch...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5407#issuecomment-91086071 [Test build #29914 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29914/consoleFull) for PR 5407 at commit [`a754adc`](https://githu

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5236#issuecomment-91085117 [Test build #29913 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29913/consoleFull) for PR 5236 at commit [`8927e66`](https://githu

[GitHub] spark pull request: [SPARK-5338][MESOS] Add cluster mode support f...

2015-04-08 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/5144#issuecomment-91085082 @tnachen Thanks for following up on the reviews. This is much better than before. However, I still find the code in `MesosClusterScheduler` very difficult to follow fo

[GitHub] spark pull request: [Spark-5682] Add spark encrypted shuffle by us...

2015-04-08 Thread kellyzly
Github user kellyzly commented on a diff in the pull request: https://github.com/apache/spark/pull/5307#discussion_r28029243 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala --- @@ -123,12 +130,34 @@ private[spark] class DiskBlockObjectWriter(

[GitHub] spark pull request: [SPARK-5654] Integrate SparkR

2015-04-08 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5096#issuecomment-91084853 @shivaram - hey one thing I forgot to ask, how much time do the SparkR tests add to the overall Spark tests? --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [Spark-6693][MLlib]add tostring with max lines...

2015-04-08 Thread hhbyyh
Github user hhbyyh commented on the pull request: https://github.com/apache/spark/pull/5344#issuecomment-91084759 Let me know if more cases should be added. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4961#issuecomment-91084699 [Test build #29912 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29912/consoleFull) for PR 4961 at commit [`76ace9a`](https://githu

[GitHub] spark pull request: [SPARK-5338][MESOS] Add cluster mode support f...

2015-04-08 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5144#discussion_r28029140 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -0,0 +1,546 @@ +/* + * Licensed to the A

[GitHub] spark pull request: [SPARK-6638] [SQL] Improve performance of Stri...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5350#issuecomment-91084587 [Test build #642 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/642/consoleFull) for PR 5350 at commit [`1314a37`](https://githu

[GitHub] spark pull request: [SPARK-6638] [SQL] Improve performance of Stri...

2015-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5350#issuecomment-91084318 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6638] [SQL] Improve performance of Stri...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5350#issuecomment-91084313 [Test build #29901 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29901/consoleFull) for PR 5350 at commit [`1314a37`](https://gith

[GitHub] spark pull request: [SPARK-2808][Streaming][Kafka] update kafka to...

2015-04-08 Thread zzcclp
Github user zzcclp commented on the pull request: https://github.com/apache/spark/pull/4537#issuecomment-91084285 there is not any error, why? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-08 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/4961#issuecomment-91084207 Hi @tdas , would you please take a look at this again, so I can move on to another PR which is depending on this, thanks a lot and appreciate your time :). --- If your

[GitHub] spark pull request: [SPARK-4925] Publish Spark SQL hive-thriftserv...

2015-04-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5429#issuecomment-91084117 [Test build #29911 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29911/consoleFull) for PR 5429 at commit [`c00aa17`](https://githu

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-08 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/4961#issuecomment-91084100 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

  1   2   3   4   5   6   7   >