[GitHub] spark pull request: [WIP][SPARK-1875]:NoClassDefFoundError: String...

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/820#issuecomment-43468538 Thanks @witgo for the quick turnaround -- will take a look at it. One thing that concerns me slightly now is that we always need to specify a Maven profile. Is there a

[GitHub] spark pull request: [WIP][SPARK-1875]:NoClassDefFoundError: String...

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/820#issuecomment-43468668 Basically one specific problem here is that when we publish to Maven Central, and people link to our POM, they will get a dependency on Spark *without* any profiles

[GitHub] spark pull request: [WIP][SPARK-1875]:NoClassDefFoundError: String...

2014-05-19 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/820#issuecomment-43468949 @mateiz This problem only occurs in spark-assembly_2.10,will not affect user testing. ``` [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @

[GitHub] spark pull request: [WIP][SPARK-1875]:NoClassDefFoundError: String...

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/820#issuecomment-43469224 So if a user adds a dependency in their project on spark-core 1.0.0, spark-hive 1.0.0 and a hadoop-client that Hive supports, will things work? I'll try this too but it's

[GitHub] spark pull request: [WIP][SPARK-1875]:NoClassDefFoundError: String...

2014-05-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/820#issuecomment-43470003 I disagree with this change. At least please see comment at https://issues.apache.org/jira/browse/SPARK-1875 first. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/819#discussion_r12780712 --- Diff: bin/compute-classpath.cmd --- @@ -42,6 +42,21 @@ if exist %FWDIR%RELEASE ( set CLASSPATH=%CLASSPATH%;%ASSEMBLY_JAR% +rem When

[GitHub] spark pull request: SPARK-1877: ClassNotFoundException when loadin...

2014-05-19 Thread ghidi
GitHub user ghidi opened a pull request: https://github.com/apache/spark/pull/821 SPARK-1877: ClassNotFoundException when loading RDD with serialized obje... When I load a RDD that has custom serialized objects, Spark throws ClassNotFoundException. This happens only when Spark is

[GitHub] spark pull request: SPARK-1877: ClassNotFoundException when loadin...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/821#issuecomment-43470344 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-05-19 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/807#issuecomment-43470341 Not sure why the build failed with a dependency resolution issue. It seems to work locally when I do sbt assembly locally. Does the order of modules specified in

[GitHub] spark pull request: [WIP][SPARK-1875]:NoClassDefFoundError: String...

2014-05-19 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/820#issuecomment-43470508 @mateiz `spark-examples_2.10` is consistent with the situation you say ``` [INFO] [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @

[GitHub] spark pull request: [WIP][SPARK-1875]:NoClassDefFoundError: String...

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/820#issuecomment-43470503 The fix suggested there does seem better, namely don't exclude commons-lang. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-05-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/807#discussion_r12781197 --- Diff: external/flume-sink/pom.xml --- @@ -0,0 +1,82 @@ +?xml version=1.0 encoding=UTF-8? --- End diff -- Pardon for jumping in with

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-05-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/807#discussion_r12781207 --- Diff: external/flume-sink/src/main/scala/org/apache/spark/flume/sink/SparkSink.scala --- @@ -0,0 +1,392 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-05-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/807#discussion_r12781257 --- Diff: external/flume-sink/src/main/scala/org/apache/spark/flume/sink/SparkSink.scala --- @@ -0,0 +1,392 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-1878: Fix the incorrect initialization o...

2014-05-19 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/822 SPARK-1878: Fix the incorrect initialization order You can merge this pull request into a Git repository by running: $ git pull https://github.com/zsxwing/spark SPARK-1878 Alternatively you

[GitHub] spark pull request: SPARK-1878: Fix the incorrect initialization o...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/822#issuecomment-43471640 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [WIP][SPARK-1875]:NoClassDefFoundError: String...

2014-05-19 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/820#issuecomment-43471608 @srowen I agree with what @mateiz said. We should not exclude commons-lang. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: SPARK-1879. Increase MaxPermSize since some of...

2014-05-19 Thread mateiz
GitHub user mateiz opened a pull request: https://github.com/apache/spark/pull/823 SPARK-1879. Increase MaxPermSize since some of our builds have many classes See https://issues.apache.org/jira/browse/SPARK-1879 -- builds with Hadoop2 and Hive ran out of PermGen space in

[GitHub] spark pull request: SPARK-1879. Increase MaxPermSize since some of...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/823#issuecomment-43473503 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1875:NoClassDefFoundError: StringUtils w...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/824#issuecomment-43473830 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1875:NoClassDefFoundError: StringUtils w...

2014-05-19 Thread witgo
GitHub user witgo opened a pull request: https://github.com/apache/spark/pull/824 SPARK-1875:NoClassDefFoundError: StringUtils when building against Hadoo... ...p 1 You can merge this pull request into a Git repository by running: $ git pull https://github.com/witgo/spark

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/819#discussion_r12782348 --- Diff: README.md --- @@ -41,9 +42,9 @@ And run the following command, which should also return 1000: Spark also comes with several sample programs in

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43474498 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43474514 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43474576 Thanks for the review, @andrewor14! I think I've dealt with all the comments (modulo a few I replied to above). The enabledelayedexpansion thing was very weird; if you

[GitHub] spark pull request: [SPARK-1875]NoClassDefFoundError: StringUtils ...

2014-05-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/824#issuecomment-43475009 I don't see a strong reason to force Commons Lang 2.6 for build profiles where nothing uses 2.6. Other than that yes it is the same as the change I am currently testing.

[GitHub] spark pull request: [SPARK-1875]NoClassDefFoundError: StringUtils ...

2014-05-19 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/824#issuecomment-43475257 @srowen The following code in line with your thoughts? https://github.com/witgo/spark/compare/SPARK-1875_new --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43475552 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43475544 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1880] [SQL] Eliminate unnecessary job e...

2014-05-19 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/825 [SPARK-1880] [SQL] Eliminate unnecessary job executions. There are unnecessary job executions in `BroadcastNestedLoopJoin`. When `Innner` or `LeftOuter` join, preparation of

[GitHub] spark pull request: [SPARK-1875]NoClassDefFoundError: StringUtils ...

2014-05-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/824#issuecomment-43476257 No, I think the only change that's needed is un-excluding commons-lang. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: SPARK-1879. Increase MaxPermSize since some of...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/823#issuecomment-43476292 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15076/ --- If your project

[GitHub] spark pull request: SPARK-1879. Increase MaxPermSize since some of...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/823#issuecomment-43476291 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-1880] [SQL] Eliminate unnecessary job e...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/825#issuecomment-43476612 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1880] [SQL] Eliminate unnecessary job e...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/825#issuecomment-43476605 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1875]NoClassDefFoundError: StringUtils ...

2014-05-19 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/824#issuecomment-43476568 Has been modified. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43477472 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15077/ --- If your project

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43477471 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-1875]NoClassDefFoundError: StringUtils ...

2014-05-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/824#issuecomment-43477986 Looks correct to me. The dependency graph looks good, spark-shell works with or without Hive, and commons-lang is in the assembly with this change. --- If your project is

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43478312 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15078/ --- If your project

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43478310 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-1880] [SQL] Eliminate unnecessary job e...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/825#issuecomment-43482897 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15079/ --- If your project

[GitHub] spark pull request: Fix doc about NetworkWordCount/JavaNetworkWord...

2014-05-19 Thread jmu
GitHub user jmu opened a pull request: https://github.com/apache/spark/pull/826 Fix doc about NetworkWordCount/JavaNetworkWordCount usage of spark streaming Usage: NetworkWordCount master hostname port -- Usage: NetworkWordCount hostname port Usage:

[GitHub] spark pull request: Fix doc about NetworkWordCount/JavaNetworkWord...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/826#issuecomment-43483480 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: improve performance of MemoryStore#tryToPut by...

2014-05-19 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/791#issuecomment-43484083 As far as I know, reasons for task failure may be: exception happens during task execution, Executor lost and relaunch, stage cancelled by user. But I'm not sure if I

[GitHub] spark pull request: improve performance of MemoryStore#tryToPut by...

2014-05-19 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/791#issuecomment-43485552 It should read MT safe - phone autocorrected it, sigh. There could be any number of reasons for dropping block to fail (including disk issues, etc). When it

[GitHub] spark pull request: SPARK-1830 Deploy failover, Make Persistence e...

2014-05-19 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/771#issuecomment-43488669 @aarondav Mind taking a look ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP][SPARK-1776] Have Spark's SBT build read ...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/772#issuecomment-43489835 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP][SPARK-1776] Have Spark's SBT build read ...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/772#issuecomment-43489825 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: bugfix: check executor id existence when execu...

2014-05-19 Thread zhpengg
GitHub user zhpengg opened a pull request: https://github.com/apache/spark/pull/827 bugfix: check executor id existence when executor exit You can merge this pull request into a Git repository by running: $ git pull https://github.com/zhpengg/spark

[GitHub] spark pull request: bugfix: check executor id existence when execu...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/827#issuecomment-43493518 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [WIP][SPARK-1776] Have Spark's SBT build read ...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/772#issuecomment-43496674 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15080/ --- If your project is set up for it, you can

[GitHub] spark pull request: [WIP][SPARK-1776] Have Spark's SBT build read ...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/772#issuecomment-43496673 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1563 Add package-info.java and package.s...

2014-05-19 Thread ScrapCodes
Github user ScrapCodes closed the pull request at: https://github.com/apache/spark/pull/599 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [WIP][SPARK-1776] Have Spark's SBT build read ...

2014-05-19 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/772#issuecomment-43497055 @pwendell Haven't worked on backward compatibility, other things you pointed out should work fine. Also the bug with tests dependencies in streaming got fixed. Now

[GitHub] spark pull request: [WIP]Improve ALS resource usage

2014-05-19 Thread witgo
GitHub user witgo opened a pull request: https://github.com/apache/spark/pull/828 [WIP]Improve ALS resource usage Now,In ALS algorithm, RDD can not be cleaned You can merge this pull request into a Git repository by running: $ git pull https://github.com/witgo/spark

[GitHub] spark pull request: [WIP]Improve ALS algorithm resource usage

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/828#issuecomment-43524803 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-05-19 Thread harishreedharan
Github user harishreedharan commented on a diff in the pull request: https://github.com/apache/spark/pull/807#discussion_r12802518 --- Diff: external/flume-sink/src/main/scala/org/apache/spark/flume/sink/SparkSink.scala --- @@ -0,0 +1,392 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-05-19 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/807#issuecomment-43524979 Thanks for the comments @srowen! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-05-19 Thread harishreedharan
Github user harishreedharan commented on a diff in the pull request: https://github.com/apache/spark/pull/807#discussion_r12802607 --- Diff: external/flume-sink/pom.xml --- @@ -0,0 +1,82 @@ +?xml version=1.0 encoding=UTF-8? --- End diff -- The sink will be deployed

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-05-19 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/807#issuecomment-43526213 The latest commit should fix the build issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-05-19 Thread harishreedharan
Github user harishreedharan commented on a diff in the pull request: https://github.com/apache/spark/pull/807#discussion_r12802646 --- Diff: external/flume-sink/src/main/scala/org/apache/spark/flume/sink/SparkSink.scala --- @@ -0,0 +1,392 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-1817] RDD.zip() should verify partition...

2014-05-19 Thread kanzhang
Github user kanzhang commented on the pull request: https://github.com/apache/spark/pull/760#issuecomment-43528643 @witgo I'm not sure what you are trying to say in above comment, but if you meant only string formatting is different, the underlying floating-point representations are

[GitHub] spark pull request: SPARK-1879. Increase MaxPermSize since some of...

2014-05-19 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/823#issuecomment-43529752 LGTM. I ran into this while testing, so this is very welcome. :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: bugfix: check executor id existence when execu...

2014-05-19 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/827#issuecomment-43534385 Jenkins, test this please. Thanks for the fix, would you mind creating a JIRA here: https://issues.apache.org/jira/browse/SPARK and prepending the jira

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/819#discussion_r12807150 --- Diff: bin/compute-classpath.cmd --- @@ -20,6 +20,13 @@ rem rem This script computes Spark's classpath and prints it to stdout; it's used by both

[GitHub] spark pull request: SPARK-1830 Deploy failover, Make Persistence e...

2014-05-19 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/771#issuecomment-43535796 @ScrapCodes, I definitely like the idea. I think we need to update the API a little before it's ready to be publicly accessible, though. First, LeaderElectionAgent and

[GitHub] spark pull request: SPARK-1830 Deploy failover, Make Persistence e...

2014-05-19 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/771#discussion_r12807670 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -143,6 +146,9 @@ private[spark] class Master( leaderElectionAgent =

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/819#discussion_r12808084 --- Diff: bin/compute-classpath.cmd --- @@ -20,6 +20,13 @@ rem rem This script computes Spark's classpath and prints it to stdout; it's used by both

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43550903 I tested this again on Windows building with and without hive and I can verify that it works as we expect. I think this is ready to go. --- If your project is set up

[GitHub] spark pull request: [SPARK-1669 SPARK-1379][SQL][WIP] Made Schem...

2014-05-19 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/829 [SPARK-1669 SPARK-1379][SQL][WIP] Made SchemaRDD.cache() to leverage InMemoryColumnarTableScan Corresponding JIRA issues: [SPARK-1669](https://issues.apache.org/jira/browse/SPARK-1669),

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43562033 @mateiz Is this good to merge? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1669 SPARK-1379][SQL][WIP] Made Schem...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/829#issuecomment-43562334 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1669 SPARK-1379][SQL][WIP] Made Schem...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/829#issuecomment-43562341 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1368][SQL] Optimized HiveTableScan

2014-05-19 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/758#issuecomment-43562306 @marmbrus Updated `HiveComparisonTest` and removed `SortedOperation`, how about this version? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: Fix doc about NetworkWordCount/JavaNetworkWord...

2014-05-19 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/826#issuecomment-43563509 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: Small updates to Streaming Programming Guide

2014-05-19 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/830 Small updates to Streaming Programming Guide Please merge. More update will come soon. You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: Fix doc about NetworkWordCount/JavaNetworkWord...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/826#issuecomment-43563791 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Fix doc about NetworkWordCount/JavaNetworkWord...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/826#issuecomment-43563784 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43564080 @tdas sure, go for it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/819#issuecomment-43564151 Great! Thanks! Merging it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1875]NoClassDefFoundError: StringUtils ...

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/824#issuecomment-43564210 Jenkins, add to whitelist and test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Small updates to Streaming Programming Guide

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/830#issuecomment-43564252 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-1875]NoClassDefFoundError: StringUtils ...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/824#issuecomment-43564255 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1875]NoClassDefFoundError: StringUtils ...

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/824#issuecomment-43564271 This looks good to me conceptually as well, will have to try it when built :). --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-1875]NoClassDefFoundError: StringUtils ...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/824#issuecomment-43564270 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1876] Windows fixes to deal with latest...

2014-05-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/819 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [WIP]Improve ALS algorithm resource usage

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/828#issuecomment-43564458 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1878: Fix the incorrect initialization o...

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/822#issuecomment-43564511 Jenkins, this is ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP]Improve ALS algorithm resource usage

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/828#issuecomment-43564725 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1878: Fix the incorrect initialization o...

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/822#issuecomment-43564559 @zsxwing are you sure this is a bug? Do we have any tests or example programs that fail because of it? --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: SPARK-1878: Fix the incorrect initialization o...

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/822#issuecomment-43564676 Oh I see, we just don't normally use sc() by default, but we expose it to users. That should indeed be fixed. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [WIP]Improve ALS algorithm resource usage

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/828#issuecomment-43564707 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1878: Fix the incorrect initialization o...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/822#issuecomment-43564726 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1878: Fix the incorrect initialization o...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/822#issuecomment-43564710 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1877: ClassNotFoundException when loadin...

2014-05-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/821#issuecomment-43566794 Jenkins, this is ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1877: ClassNotFoundException when loadin...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/821#issuecomment-43566839 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1877: ClassNotFoundException when loadin...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/821#issuecomment-43566851 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1877: ClassNotFoundException when loadin...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/821#issuecomment-43567000 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15086/ --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-1877: ClassNotFoundException when loadin...

2014-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/821#issuecomment-43566999 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

  1   2   3   >