[GitHub] spark pull request: [SPARK-3481][SQL] removes the evil MINOR HACK

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2505#issuecomment-56480427 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20690/consoleFull) for PR 2505 at commit [`4874ec8`](https://github.com/ap

[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...

2014-09-22 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-56480392 Are we proposing to introduce hdfs caching tags/idioms directly into TaskSetManager in this pr ? That does not look right. We need to generalize this so that any rdd c

[GitHub] spark pull request: Modify default YARN memory_overhead-- from an ...

2014-09-22 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/2485#issuecomment-56480298 This looks good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [SPARK-3481][SQL] removes the evil MINOR HACK

2014-09-22 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2505#issuecomment-56480117 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark pull request: [SPARK-3481][SQL] removes the evil MINOR HACK

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2505#issuecomment-56480056 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [SPARK-3481][SQL] removes the evil MINOR HACK

2014-09-22 Thread scwf
GitHub user scwf opened a pull request: https://github.com/apache/spark/pull/2505 [SPARK-3481][SQL] removes the evil MINOR HACK a follow up of https://github.com/apache/spark/pull/2377 and https://github.com/apache/spark/pull/2352, see detail there. You can merge this pull request

[GitHub] spark pull request: SPARK-3172 and SPARK-3577

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2504#issuecomment-56479827 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20689/consoleFull) for PR 2504 at commit [`c854514`](https://github.com/a

[GitHub] spark pull request: SPARK-3172 and SPARK-3577

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2504#issuecomment-56479829 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20689/

[GitHub] spark pull request: SPARK-3172 and SPARK-3577

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2504#issuecomment-56479669 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20689/consoleFull) for PR 2504 at commit [`c854514`](https://github.com/ap

[GitHub] spark pull request: SPARK-3172 and SPARK-3577

2014-09-22 Thread sryza
GitHub user sryza opened a pull request: https://github.com/apache/spark/pull/2504 SPARK-3172 and SPARK-3577 The posted patch addresses both SPARK-3172 and SPARK-3577. It renames ShuffleWriteMetrics to WriteMetrics and uses it for tracking all three of shuffle write, spilling on t

[GitHub] spark pull request: [SPARK-3649] Remove GraphX custom serializers

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2503#issuecomment-56478252 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20688/consoleFull) for PR 2503 at commit [`a49c2ad`](https://github.com/a

[GitHub] spark pull request: [SPARK-3649] Remove GraphX custom serializers

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2503#issuecomment-56478260 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20688/

[GitHub] spark pull request: add a util method for changing the log level w...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2433#issuecomment-56478099 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20687/consoleFull) for PR 2433 at commit [`cdb3bfc`](https://github.com/a

[GitHub] spark pull request: [SPARK-2017] [SPARK-2016] Web UI responsivenes...

2014-09-22 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/1682#issuecomment-56478100 I've opened [SPARK-3644](https://issues.apache.org/jira/browse/SPARK-3644) as a forum for discussing the design of a REST API; sorry for the delay (got busy with other

[GitHub] spark pull request: add a util method for changing the log level w...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2433#issuecomment-56478104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20687/

[GitHub] spark pull request: [SPARK-3634] [PySpark] User's module should ta...

2014-09-22 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2492#issuecomment-56477603 > Understood, this side-effect is bit dangerous. The third-package could appear in sys.path in any order Are you worried about a user adding a Python module who

[GitHub] spark pull request: [SPARK-3649] Remove GraphX custom serializers

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2503#issuecomment-56476058 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20688/consoleFull) for PR 2503 at commit [`a49c2ad`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3649] Remove GraphX custom serializers

2014-09-22 Thread ankurdave
GitHub user ankurdave opened a pull request: https://github.com/apache/spark/pull/2503 [SPARK-3649] Remove GraphX custom serializers As [reported][1] on the mailing list, GraphX throws ``` java.lang.ClassCastException: java.lang.Long cannot be cast to scala.Tuple2

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-56475771 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20686/

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-56475768 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20686/consoleFull) for PR 2388 at commit [`bf84e7b`](https://github.com/a

[GitHub] spark pull request: add a util method for changing the log level w...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2433#issuecomment-56475233 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20687/consoleFull) for PR 2433 at commit [`cdb3bfc`](https://github.com/ap

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56473764 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20684/consoleFull) for PR 1290 at commit [`a28aa4a`](https://github.com/a

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56473769 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20684/

[GitHub] spark pull request: [SPARK-3007][SQL]Add "Dynamic Partition" suppo...

2014-09-22 Thread baishuo
Github user baishuo commented on the pull request: https://github.com/apache/spark/pull/2226#issuecomment-56473456 thanks a lot to @liancheng :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56473120 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20683/

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56473117 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20683/consoleFull) for PR 1290 at commit [`b3531d6`](https://github.com/a

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-56472988 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20686/consoleFull) for PR 2388 at commit [`bf84e7b`](https://github.com/ap

[GitHub] spark pull request: [WIP][SPARK-3212][SQL] Use logical plan matchi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2501#issuecomment-56472896 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20685/

[GitHub] spark pull request: [WIP][SPARK-3212][SQL] Use logical plan matchi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2501#issuecomment-56472893 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20685/consoleFull) for PR 2501 at commit [`80f26ac`](https://github.com/a

[GitHub] spark pull request: [SPARK-3007][SQL]Add "Dynamic Partition" suppo...

2014-09-22 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2226#issuecomment-56472453 LGTM @marmbrus This is finally good to go :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2497#issuecomment-56472123 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20680/consoleFull) for PR 2497 at commit [`b3b3e50`](https://github.com/a

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2497#issuecomment-56472127 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20680/

[GitHub] spark pull request: [WIP][SPARK-3212][SQL] Use logical plan matchi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2501#issuecomment-56471636 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20685/consoleFull) for PR 2501 at commit [`80f26ac`](https://github.com/ap

[GitHub] spark pull request: Merge pull request #1 from apache/master

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2502#issuecomment-56471527 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [SPARK-1720][SPARK-1719] Add the value of LD_L...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-56471514 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20681/consoleFull) for PR 1031 at commit [`f44c221`](https://github.com/a

[GitHub] spark pull request: [SPARK-1720][SPARK-1719] Add the value of LD_L...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-56471518 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20681/

[GitHub] spark pull request: Merge pull request #1 from apache/master

2014-09-22 Thread ceys
GitHub user ceys opened a pull request: https://github.com/apache/spark/pull/2502 Merge pull request #1 from apache/master Update from original You can merge this pull request into a Git repository by running: $ git pull https://github.com/ceys/spark master Alternatively you c

[GitHub] spark pull request: [WIP][SPARK-3212][SQL] Use logical plan matchi...

2014-09-22 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/2501 [WIP][SPARK-3212][SQL] Use logical plan matching instead of temporary tables for table caching _Also addresses: SPARK-1379 and SPARK-3641_ This PR introduces a new trait, `CacheManger`, wh

[GitHub] spark pull request: Adds json api for stages, storage and executor...

2014-09-22 Thread praveenr019
Github user praveenr019 closed the pull request at: https://github.com/apache/spark/pull/882 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Adds json api for stages, storage and executor...

2014-09-22 Thread praveenr019
Github user praveenr019 commented on the pull request: https://github.com/apache/spark/pull/882#issuecomment-56470825 Closing this pull request since its committed on a old branch. Thanks @JoshRosen, would be glad to see this feature in Spark. --- If your project is set up fo

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56470743 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20684/consoleFull) for PR 1290 at commit [`a28aa4a`](https://github.com/ap

[GitHub] spark pull request: [SPARK-1545] [mllib] Add Random Forests

2014-09-22 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2435#discussion_r17889907 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/RandomForest.scala --- @@ -0,0 +1,430 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-56470414 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20678/

[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-56470411 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20678/consoleFull) for PR 1486 at commit [`9c4933c`](https://github.com/a

[GitHub] spark pull request: stop, start and destroy require the EC2_REGION

2014-09-22 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2473#discussion_r17889871 --- Diff: docs/ec2-scripts.md --- @@ -137,11 +146,11 @@ cost you any EC2 cycles, but ***will*** continue to cost money for EBS storage. - To

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56470205 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20682/

[GitHub] spark pull request: stop, start and destroy require the EC2_REGION

2014-09-22 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2473#discussion_r17889800 --- Diff: docs/ec2-scripts.md --- @@ -137,11 +146,11 @@ cost you any EC2 cycles, but ***will*** continue to cost money for EBS storage. - To stop

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56470085 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20683/consoleFull) for PR 1290 at commit [`b3531d6`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3007][SQL]Add "Dynamic Partition" suppo...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2226#issuecomment-56470082 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20679/consoleFull) for PR 2226 at commit [`e69ce88`](https://github.com/a

[GitHub] spark pull request: [SPARK-3007][SQL]Add "Dynamic Partition" suppo...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2226#issuecomment-56470087 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20679/

[GitHub] spark pull request: [SPARK-1545] [mllib] Add Random Forests

2014-09-22 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2435#discussion_r17889650 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/RandomForest.scala --- @@ -0,0 +1,430 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] spark pull request: [SPARK-1545] [mllib] Add Random Forests

2014-09-22 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2435#discussion_r17889641 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/RandomForest.scala --- @@ -0,0 +1,430 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] spark pull request: [SPARK-3653] Respect SPARK_*_MEMORY for cluste...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2500#issuecomment-56469714 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20677/consoleFull) for PR 2500 at commit [`6217b38`](https://github.com/a

[GitHub] spark pull request: [SPARK-3653] Respect SPARK_*_MEMORY for cluste...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2500#issuecomment-56469716 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20677/

[GitHub] spark pull request: stop, start and destroy require the EC2_REGION

2014-09-22 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2473#discussion_r17889468 --- Diff: docs/ec2-scripts.md --- @@ -48,6 +48,15 @@ by looking for the "Name" tag of the instance in the Amazon EC2 Console. key pair, `` is the nu

[GitHub] spark pull request: stop, start and destroy require the EC2_REGION

2014-09-22 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2473#discussion_r17889459 --- Diff: docs/ec2-scripts.md --- @@ -137,11 +146,11 @@ cost you any EC2 cycles, but ***will*** continue to cost money for EBS storage. - To

[GitHub] spark pull request: [SPARK-1720][SPARK-1719] Add the value of LD_L...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-56468425 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20681/consoleFull) for PR 1031 at commit [`f44c221`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2497#issuecomment-56468434 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20680/consoleFull) for PR 2497 at commit [`b3b3e50`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-09-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2497#issuecomment-56468116 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-09-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2469#issuecomment-56468138 Thanks for fixing this @vanzin. I will look at it shortly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark pull request: [SPARK-3652] [SQL] upgrade spark sql hive vers...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2499#issuecomment-56468081 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/142/consoleFull) for PR 2499 at commit [`6d5d071`](https://github.com/

[GitHub] spark pull request: [SPARK-3652] [SQL] upgrade spark sql hive vers...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2499#issuecomment-56468023 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/142/consoleFull) for PR 2499 at commit [`6d5d071`](https://github.com/a

[GitHub] spark pull request: [SPARK-3614][MLLIB] Add minimumOccurence filte...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2494#issuecomment-56468001 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20676/

[GitHub] spark pull request: [SPARK-3614][MLLIB] Add minimumOccurence filte...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2494#issuecomment-56467997 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20676/consoleFull) for PR 2494 at commit [`1801fd2`](https://github.com/a

[GitHub] spark pull request: [SPARK-3007][SQL]Add "Dynamic Partition" suppo...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2226#issuecomment-56467600 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20679/consoleFull) for PR 2226 at commit [`e69ce88`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2497#issuecomment-56467574 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20673/consoleFull) for PR 2497 at commit [`b3b3e50`](https://github.com/a

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2497#issuecomment-56467583 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20673/

[GitHub] spark pull request: [SPARK-3007][SQL]Add "Dynamic Partition" suppo...

2014-09-22 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2226#discussion_r1720 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala --- @@ -522,6 +523,52 @@ class HiveQuerySuite extends HiveCompar

[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-56467045 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20678/consoleFull) for PR 1486 at commit [`9c4933c`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3653] Respect SPARK_*_MEMORY for cluste...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2500#issuecomment-56466221 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20677/consoleFull) for PR 2500 at commit [`6217b38`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3653] Respect SPARK_*_MEMORY for cluste...

2014-09-22 Thread andrewor14
GitHub user andrewor14 opened a pull request: https://github.com/apache/spark/pull/2500 [SPARK-3653] Respect SPARK_*_MEMORY for cluster mode `SPARK_DRIVER_MEMORY` was only used to start the `SparkSubmit` JVM, which becomes the driver only in client mode but not cluster mode. In clus

[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-56465510 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20674/consoleFull) for PR 1486 at commit [`8f9c5d6`](https://github.com/a

[GitHub] spark pull request: [SPARK-3652] [SQL] upgrade spark sql hive vers...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2499#issuecomment-56465504 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-56465520 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20674/

[GitHub] spark pull request: [SPARK-3652] [SQL] upgrade spark sql hive vers...

2014-09-22 Thread scwf
GitHub user scwf opened a pull request: https://github.com/apache/spark/pull/2499 [SPARK-3652] [SQL] upgrade spark sql hive version to 0.13.1 Now spark sql hive version is 0.12.0 and do not support 0.13.1 because of some api level changes in hive new version. Since hive has bac

[GitHub] spark pull request: [SPARK-3614][MLLIB] Add minimumOccurence filte...

2014-09-22 Thread rnowling
Github user rnowling commented on the pull request: https://github.com/apache/spark/pull/2494#issuecomment-56464302 @Ishiihara Thanks for pointing out the style check -- I found and fixed the style error in IDF.scala. Thanks for mentioning options for the mimimumOccurence memb

[GitHub] spark pull request: [SPARK-3614][MLLIB] Add minimumOccurence filte...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2494#issuecomment-56464144 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20676/consoleFull) for PR 2494 at commit [`1801fd2`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3614][MLLIB] Add minimumOccurence filte...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2494#issuecomment-56463906 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20675/

[GitHub] spark pull request: [SPARK-3614][MLLIB] Add minimumOccurence filte...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2494#issuecomment-56463904 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20675/consoleFull) for PR 2494 at commit [`6897252`](https://github.com/a

[GitHub] spark pull request: [SPARK-3614][MLLIB] Add minimumOccurence filte...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2494#issuecomment-56463855 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20675/consoleFull) for PR 2494 at commit [`6897252`](https://github.com/ap

[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-56463517 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20674/consoleFull) for PR 1486 at commit [`8f9c5d6`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3634] [PySpark] User's module should ta...

2014-09-22 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/2492#issuecomment-56463455 > Maybe my JIRA was misleadingly named; my motivation here is allowing users to specify versions of packages that take precedence over other versions of that same package

[GitHub] spark pull request: WHITESPACE CHANGE DO NOT MERGE

2014-09-22 Thread shaneknapp
Github user shaneknapp closed the pull request at: https://github.com/apache/spark/pull/2498 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2497#issuecomment-56463199 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20673/consoleFull) for PR 2497 at commit [`b3b3e50`](https://github.com/ap

[GitHub] spark pull request: WHITESPACE CHANGE DO NOT MERGE

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2498#issuecomment-56463163 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20672/

[GitHub] spark pull request: WHITESPACE CHANGE DO NOT MERGE

2014-09-22 Thread shaneknapp
Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/2498#issuecomment-56463049 jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: WHITESPACE CHANGE DO NOT MERGE

2014-09-22 Thread shaneknapp
GitHub user shaneknapp opened a pull request: https://github.com/apache/spark/pull/2498 WHITESPACE CHANGE DO NOT MERGE WHITESPACE CHANGE DO NOT MERGE You can merge this pull request into a Git repository by running: $ git pull https://github.com/shaneknapp/spark sknapptest Alt

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-09-22 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/2497#issuecomment-56462789 Backport of #2469 to branch-1.1. Sending now to speed up the review process, since the original PR doesn't merge cleanly into this branch. --- If your project is set up f

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-09-22 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/2497 [SPARK-3606] [yarn] Correctly configure AmIpFilter for Yarn HA (1.1 vers... ...ion). This is a backport of SPARK-3606 to branch-1.1. Some of the code had to be duplicated since branch-1.1

[GitHub] spark pull request: [SPARK-3614][MLLIB] Add minimumOccurence filte...

2014-09-22 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2494#discussion_r17886499 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/IDF.scala --- @@ -30,9 +30,20 @@ import org.apache.spark.rdd.RDD * Inverse document frequ

[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...

2014-09-22 Thread cmccabe
Github user cmccabe commented on a diff in the pull request: https://github.com/apache/spark/pull/1486#discussion_r17886353 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -309,4 +323,42 @@ private[spark] object HadoopRDD { f(inputSplit, firstPar

[GitHub] spark pull request: SPARK-2621. Update task InputMetrics increment...

2014-09-22 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/2087#issuecomment-56461580 MapReduce doesn't use getPos, but it does look like it might be helpful in some situations. One caveat is that pos only means # bytes for file input formats. For example,

[GitHub] spark pull request: [SPARK-3614][MLLIB] Add minimumOccurence filte...

2014-09-22 Thread Ishiihara
Github user Ishiihara commented on the pull request: https://github.com/apache/spark/pull/2494#issuecomment-56461303 @rnowling Please run sbt/sbt scalastyle on your local machine to clear out style issues. --- If your project is set up for it, you can reply to this email and have yo

[GitHub] spark pull request: [SPARK-3650] Fix TriangleCount handling of rev...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2495#issuecomment-56461187 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20670/

[GitHub] spark pull request: [SPARK-3650] Fix TriangleCount handling of rev...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2495#issuecomment-56461185 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20670/consoleFull) for PR 2495 at commit [`d054d33`](https://github.com/a

[GitHub] spark pull request: [SPARK-3614][MLLIB] Add minimumOccurence filte...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2494#issuecomment-56461090 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20671/

[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...

2014-09-22 Thread cmccabe
Github user cmccabe commented on a diff in the pull request: https://github.com/apache/spark/pull/1486#discussion_r17886029 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskLocation.scala --- @@ -22,13 +22,35 @@ package org.apache.spark.scheduler * In the latter c

[GitHub] spark pull request: [SPARK-3614][MLLIB] Add minimumOccurence filte...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2494#issuecomment-56461087 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20671/consoleFull) for PR 2494 at commit [`a200bab`](https://github.com/a

[GitHub] spark pull request: [SPARK-3614][MLLIB] Add minimumOccurence filte...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2494#issuecomment-56461020 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20671/consoleFull) for PR 2494 at commit [`a200bab`](https://github.com/ap

[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...

2014-09-22 Thread cmccabe
Github user cmccabe commented on a diff in the pull request: https://github.com/apache/spark/pull/1486#discussion_r17886024 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskLocation.scala --- @@ -22,13 +22,35 @@ package org.apache.spark.scheduler * In the latter c

  1   2   3   >