[GitHub] spark pull request: [SPARK-1331] Added graceful shutdown to Spark ...

2014-04-04 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/247#discussion_r11283933 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala --- @@ -124,84 +143,109 @@ abstract class NetworkReceiver[T:

[GitHub] spark pull request: [SPARK-1331] Added graceful shutdown to Spark ...

2014-04-04 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/247#discussion_r11284020 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/RecurringTimer.scala --- @@ -17,44 +17,80 @@ package

[GitHub] spark pull request: [SPARK-1331] Added graceful shutdown to Spark ...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/247#issuecomment-39534647 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1331] Added graceful shutdown to Spark ...

2014-04-04 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/247#issuecomment-39534715 Hey @pwendell I did a pass as well. Some of the network receiver related changes I have punted for now as those have actually changed with #300 and its best to analyze them

[GitHub] spark pull request: [SPARK-1331] Added graceful shutdown to Spark ...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/247#issuecomment-39534741 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1259] Make RDD locally iterable

2014-04-04 Thread epahomov
Github user epahomov commented on the pull request: https://github.com/apache/spark/pull/156#issuecomment-39538362 Doc's changed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-1305: Support persisting RDD's directly ...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/158#issuecomment-39540222 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1305: Support persisting RDD's directly ...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/158#issuecomment-39540229 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1403] Move the class loader creation ba...

2014-04-04 Thread manku-timma
Github user manku-timma commented on the pull request: https://github.com/apache/spark/pull/322#issuecomment-39541428 I can confirm that the third line is needed. Without that line I see the same failure as earlier. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-04-04 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-39541500 I think I like reference tracking. Its more evident what it means. Automatic is a little too generic. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-1331] Added graceful shutdown to Spark ...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/247#issuecomment-39541530 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1331] Added graceful shutdown to Spark ...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/247#issuecomment-39541536 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1403] Move the class loader creation ba...

2014-04-04 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/322#issuecomment-39543041 Which exact failure is it? Could you post the stack trace? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-1403] Move the class loader creation ba...

2014-04-04 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/322#issuecomment-39543185 Hi, I think `MesosExecutorBackend` should have own `ContextClassLoader` and it should be set to the class loader of `MesosExecutorBackend` class before it creates

[GitHub] spark pull request: [SPARK-1403] Move the class loader creation ba...

2014-04-04 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/322#issuecomment-39543328 @ueshin - ah cool. Do you mind giving a code snippet of what that would look like? Then @manku-timma can see if it fixes the problem. Probably is a better solution...

[GitHub] spark pull request: [SPARK-1403] Move the class loader creation ba...

2014-04-04 Thread manku-timma
Github user manku-timma commented on the pull request: https://github.com/apache/spark/pull/322#issuecomment-39543590 java.lang.ClassNotFoundException: org/apache/spark/serializer/JavaSerializer at java.lang.Class.forName0(Native Method) at

[GitHub] spark pull request: SPARK-1305: Support persisting RDD's directly ...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/158#issuecomment-39543610 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13755/ --- If your project

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-39543917 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-39544120 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Don't create SparkContext in JobProgressListen...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/324#issuecomment-39544264 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Don't create SparkContext in JobProgressListen...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/324#issuecomment-39544278 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Fix SPARK-1413: Parquet messes up stdout and s...

2014-04-04 Thread witgo
GitHub user witgo opened a pull request: https://github.com/apache/spark/pull/325 Fix SPARK-1413: Parquet messes up stdout and stdin when used in Spark REPL You can merge this pull request into a Git repository by running: $ git pull https://github.com/witgo/spark SPARK-1413

[GitHub] spark pull request: Fix SPARK-1413: Parquet messes up stdout and s...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/325#issuecomment-39545021 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-1331] Added graceful shutdown to Spark ...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/247#issuecomment-39545197 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13756/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-1331] Added graceful shutdown to Spark ...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/247#issuecomment-39545196 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1403] Move the class loader creation ba...

2014-04-04 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/322#issuecomment-39546729 Put the following line at the beginning of `registered` method (at [line

[GitHub] spark pull request: Don't create SparkContext in JobProgressListen...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/324#issuecomment-39548477 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13758/ --- If your project

[GitHub] spark pull request: Don't create SparkContext in JobProgressListen...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/324#issuecomment-39548476 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-1403] Move the class loader creation ba...

2014-04-04 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/322#issuecomment-39550942 @manku-timma Yes, the standalone backend `org.apache.spark.executor.CoarseGrainedExecutorBackend` and the local-mode backend `org.apache.spark.scheduler.local.LocalActor`

[GitHub] spark pull request: [SPARK-1403] Move the class loader creation ba...

2014-04-04 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/322#issuecomment-39551604 BTW, we might have to restore the context class loader of `MesosExecutorBackend` to bootstrap class loader. We should restore if there are 2 or more threads to handle

[GitHub] spark pull request: [SPARK-1403] Move the class loader creation ba...

2014-04-04 Thread manku-timma
Github user manku-timma commented on the pull request: https://github.com/apache/spark/pull/322#issuecomment-39552826 @ueshin, your one line change works for me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-1212, Part II] Support sparse data in M...

2014-04-04 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/245#issuecomment-39553980 Right now, there is still use of jblas in addition to breeze in the code base and in APIs. In theory there are no more API changes before 1.0 now. This seems pretty

[GitHub] spark pull request: [SPARK-1403] Move the class loader creation ba...

2014-04-04 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/322#issuecomment-39560562 @manku-timma, I just wondered that we should restore the class loader in `registered` method like: ```scala val cl = Thread.currentThread.getContextClassLoader

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-39561674 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-39561682 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1350. Always use JAVA_HOME to run execut...

2014-04-04 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/313#issuecomment-39567024 I committed this. Thanks Sandy! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-39567830 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13759/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-39567828 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-732: eliminate duplicate update of the a...

2014-04-04 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/228#issuecomment-39570964 @kayousterhout any feedback? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1350. Always use JAVA_HOME to run execut...

2014-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/313 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: HOTFIX for broken CI, by SPARK-1336

2014-04-04 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/321#issuecomment-39586434 @marmbrus does this look okay to you? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: Remove the getStageInfo() method from SparkCon...

2014-04-04 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/308#issuecomment-39586502 I think this is just an omission that should have been private. If needed we can make it public later on. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: SPARK-1216. Add a OneHotEncoder for handling c...

2014-04-04 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/304#issuecomment-39587209 The test failures appear unrelated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-39588422 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-39588439 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1337] Garbage collect the oldest stages...

2014-04-04 Thread dgshep
Github user dgshep commented on the pull request: https://github.com/apache/spark/pull/323#issuecomment-39590424 I did. Patrick beat me to the punch :P --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-1337] Garbage collect the oldest stages...

2014-04-04 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/323#issuecomment-39591428 Hey @dgshep thanks for reporting this - we did the exact same fix :) Didn't realize you were planning to submit a patch. --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-1337] Garbage collect the oldest stages...

2014-04-04 Thread dgshep
Github user dgshep commented on the pull request: https://github.com/apache/spark/pull/323#issuecomment-39591578 Hey no worries! Glad it is fixed. How far back is it back ported? We could really use this in our production deployment. --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...

2014-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/310 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-1216. Add a OneHotEncoder for handling c...

2014-04-04 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/304#issuecomment-39591730 Are you sure it's not related - just wondering because it's also in the ML code... ``` org.apache.spark.SparkException: Job aborted: Task 0.0:0 failed 1

[GitHub] spark pull request: SPARK-1252. On YARN, use container-log4j.prope...

2014-04-04 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/148#issuecomment-39591912 @tgravescs mind taking a look at this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-1375. Additional spark-submit cleanup

2014-04-04 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/278#issuecomment-39592148 Jenkins, test this please. @sryza Looks good to me, pending tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [WIP] Fix SPARK-1413: Parquet messes up stdout...

2014-04-04 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/325#issuecomment-39592160 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-1337] Garbage collect the oldest stages...

2014-04-04 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/323#issuecomment-39592465 I back ported it into 0.9, but we just cut 0.9.1 so it won't be around until 0.9.2 what version are you using? --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-1337] Garbage collect the oldest stages...

2014-04-04 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/323#issuecomment-39592504 If you guys are okay to do your own build it should be easy to back port it into older versions --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: SPARK-1375. Additional spark-submit cleanup

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/278#issuecomment-39592599 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: HOTFIX for broken CI, by SPARK-1336

2014-04-04 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/321#issuecomment-39592848 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-1337] Garbage collect the oldest stages...

2014-04-04 Thread dgshep
Github user dgshep commented on the pull request: https://github.com/apache/spark/pull/323#issuecomment-39592929 We are on 0.9 atm. Will definitely consider rolling our own in the interim. Also, I had some other concerns with the cleanup in JobProgressListener. It may not clean all

[GitHub] spark pull request: Don't create SparkContext in JobProgressListen...

2014-04-04 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/324#issuecomment-39593765 Yay for test cleanup! :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP] Fix SPARK-1413: Parquet messes up stdout...

2014-04-04 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/325#discussion_r11306151 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala --- @@ -81,11 +81,14 @@ private[sql] case class ParquetRelation(val path:

[GitHub] spark pull request: [WIP] Fix SPARK-1413: Parquet messes up stdout...

2014-04-04 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/325#issuecomment-39593934 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Set spark.executor.uri from environment variab...

2014-04-04 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/311#issuecomment-39594034 Jenkins, test this please. Good catch! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [WIP] Fix SPARK-1413: Parquet messes up stdout...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/325#issuecomment-39594128 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP] Fix SPARK-1413: Parquet messes up stdout...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/325#issuecomment-39594136 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Spark 939 allow user jars to take precedence o...

2014-04-04 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/217#issuecomment-39594472 @pwendell are we allowed to have JARs and .class files in our repo, even for testing? If not we'll need to modify the tests here to build them. --- If your project is set

[GitHub] spark pull request: [SPARK-1133] Add whole text files reader in ML...

2014-04-04 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/252#issuecomment-39594639 Thanks Xusen, I've merged this in. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP] Fix SPARK-1413: Parquet messes up stdout...

2014-04-04 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/325#issuecomment-39595364 There is a problem I do not know what the reason is cause import org.apache.spark.SparkContext import org.apache.spark.sql.SQLContext case class

[GitHub] spark pull request: SPARK-1230: [WIP] Enable SparkContext.addJars(...

2014-04-04 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/119#discussion_r11307071 --- Diff: core/src/test/scala/org/apache/spark/TestUtils.scala --- @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request: Spark 1271 (1320) cogroup and groupby should p...

2014-04-04 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/242#issuecomment-39595898 @mateiz what about pulling the test utility in defined here? https://github.com/apache/spark/pull/119/files#diff-80212629295b483ab30a3436a653e97aR65 Some other

[GitHub] spark pull request: [SPARK-1198] Allow pipes tasks to run in diffe...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/128#issuecomment-39596974 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1198] Allow pipes tasks to run in diffe...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/128#issuecomment-39596991 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1198] Allow pipes tasks to run in diffe...

2014-04-04 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/128#issuecomment-39597145 I upmerged this to latest master. I changed it to not user apache commons-io since that dependency was removed. To do this I had to modify some of the Utils

[GitHub] spark pull request: [SPARK-1337] Garbage collect the oldest stages...

2014-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/323 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Add test utility for generating Jar files with...

2014-04-04 Thread pwendell
GitHub user pwendell opened a pull request: https://github.com/apache/spark/pull/326 Add test utility for generating Jar files with compiled classes. This was requested by a few different people and may be generally useful, so I'd like to contribute this and not block on a

[GitHub] spark pull request: [SPARK-1133] Add whole text files reader in ML...

2014-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/252 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Add test utility for generating Jar files with...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/326#issuecomment-39597490 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1252. On YARN, use container-log4j.prope...

2014-04-04 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/148#discussion_r11307961 --- Diff: yarn/alpha/src/main/resources/log4j-spark-container.properties --- @@ -0,0 +1,24 @@ +# +# Licensed under the Apache License, Version 2.0

[GitHub] spark pull request: [SPARK-1198] Allow pipes tasks to run in diffe...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/128#issuecomment-39597492 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1198] Allow pipes tasks to run in diffe...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/128#issuecomment-39597507 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Add test utility for generating Jar files with...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/326#issuecomment-39597496 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP] Fix SPARK-1413: Parquet messes up stdout...

2014-04-04 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/325#issuecomment-39597999 @AndreSchumacher, any thoughts? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SQL] SPARK-1366 Consistent sql function acros...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/319#issuecomment-39598069 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1375. Additional spark-submit cleanup

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/278#issuecomment-39598083 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SQL] SPARK-1366 Consistent sql function acros...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/319#issuecomment-39598033 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1198] Allow pipes tasks to run in diffe...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/128#issuecomment-39598067 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1198] Allow pipes tasks to run in diffe...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/128#issuecomment-39598068 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13763/ --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-1375. Additional spark-submit cleanup

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/278#issuecomment-39598085 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13761/ --- If your project

[GitHub] spark pull request: [WIP] Fix SPARK-1413: Parquet messes up stdout...

2014-04-04 Thread AndreSchumacher
Github user AndreSchumacher commented on the pull request: https://github.com/apache/spark/pull/325#issuecomment-39598624 Ouch. OK, I'll have a look at this. The default j.u.l. logger writes to stdout. Maybe the log forwarding doesn't work as intended. --- If your project is set up

[GitHub] spark pull request: [SQL] SPARK-1366 Consistent sql function acros...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/319#issuecomment-39599138 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13766/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SQL] SPARK-1366 Consistent sql function acros...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/319#issuecomment-39599137 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SQL] Minor fixes.

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/315#issuecomment-39599638 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SQL] Minor fixes.

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/315#issuecomment-39599658 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SQL] SPARK-1371 Hash Aggregation Improvements

2014-04-04 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/295#issuecomment-39599677 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1305: Support persisting RDD's directly ...

2014-04-04 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/158#discussion_r11308914 --- Diff: core/src/main/scala/org/apache/spark/storage/TachyonStore.scala --- @@ -0,0 +1,142 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SQL] SPARK-1366 Consistent sql function acros...

2014-04-04 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/319#issuecomment-39599848 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SQL] SPARK-1366 Consistent sql function acros...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/319#issuecomment-39600176 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SQL] SPARK-1371 Hash Aggregation Improvements

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/295#issuecomment-39600180 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SQL] SPARK-1371 Hash Aggregation Improvements

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/295#issuecomment-39600197 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SQL] SPARK-1366 Consistent sql function acros...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/319#issuecomment-39600195 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP] Fix SPARK-1413: Parquet messes up stdout...

2014-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/325#issuecomment-39600312 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

  1   2   3   4   >