[GitHub] spark pull request: [SPARK-2643] Stages web ui has ERROR when pool...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1854#issuecomment-51881015 QA tests have started for PR 1854. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18364/consoleFull --- If

[GitHub] spark pull request: [SPARK-2650][SQL] More precise initial buffer ...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1901#issuecomment-51881563 QA results for PR 1901:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [MLlib] Correctly set vectorSize and alpha

2014-08-12 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1900#issuecomment-51882028 LGTM. Merged into both master and branch-1.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark pull request: [MLlib] Correctly set vectorSize and alpha

2014-08-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1900 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2979][MLlib] Improve the convergence ra...

2014-08-12 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1897#discussion_r16099150 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/GeneralizedLinearAlgorithm.scala --- @@ -137,11 +154,45 @@ abstract class GeneralizedLinearAl

[GitHub] spark pull request: [SPARK-2969][SQL] Make ScalaReflection be able...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1889#issuecomment-51882621 QA results for PR 1889:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2979][MLlib] Improve the convergence ra...

2014-08-12 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1897#discussion_r16099170 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/classification/LogisticRegressionSuite.scala --- @@ -185,6 +185,58 @@ class LogisticRegressionSuite exte

[GitHub] spark pull request: [SPARK-2979][MLlib] Improve the convergence ra...

2014-08-12 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1897#discussion_r16099253 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/GeneralizedLinearAlgorithm.scala --- @@ -137,11 +154,45 @@ abstract class GeneralizedLinearAl

[GitHub] spark pull request: [SPARK-2970] [SQL] spark-sql script ends with ...

2014-08-12 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/1891#issuecomment-51883319 The build failure was caused by PySpark. (Let me try again...) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-2970] [SQL] spark-sql script ends with ...

2014-08-12 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/1891#issuecomment-51883335 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: [SPARK-2970] [SQL] spark-sql script ends with ...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1891#issuecomment-51883703 QA tests have started for PR 1891. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18365/consoleFull --- If

[GitHub] spark pull request: [SPARK-2924] remove default args to overloaded...

2014-08-12 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1704#issuecomment-51884393 Hm - seems like it's either out of data again or it didn't get merged. If you bring it up to date I _promise_ I'll merge it quickly (or I'll do a manual resolution if ne

[GitHub] spark pull request: [SPARK-2643] Stages web ui has ERROR when pool...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1854#issuecomment-51884838 QA results for PR 1854:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2970] [SQL] spark-sql script ends with ...

2014-08-12 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/1891#issuecomment-51885042 Thanks a lot for triggering test... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: [SPARK-2969][SQL] Make ScalaReflection be able...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1889#issuecomment-51885640 QA tests have started for PR 1889. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18366/consoleFull --- If

[GitHub] spark pull request: [SPARK-1777] Prevent OOMs from single partitio...

2014-08-12 Thread liyezhang556520
Github user liyezhang556520 commented on a diff in the pull request: https://github.com/apache/spark/pull/1165#discussion_r16100690 --- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala --- @@ -141,6 +193,93 @@ private class MemoryStore(blockManager: BlockManager,

[GitHub] spark pull request: [SPARK-2947] DAGScheduler resubmit the stage i...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1877#issuecomment-51887280 QA tests have started for PR 1877. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18367/consoleFull --- If

[GitHub] spark pull request: [SPARK-2062][GraphX] VertexRDD.apply does not ...

2014-08-12 Thread larryxiao
GitHub user larryxiao opened a pull request: https://github.com/apache/spark/pull/1903 [SPARK-2062][GraphX] VertexRDD.apply does not use the mergeFunc create verticesDeduplicate with reduceByKey, using mergeFunc then proceed with verticesDedup But this is not tested and

[GitHub] spark pull request: [SPARK-2062][GraphX] VertexRDD.apply does not ...

2014-08-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1903#issuecomment-51887950 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-2878]: Fix custom spark.kryo.registrato...

2014-08-12 Thread GrahamDennis
Github user GrahamDennis commented on the pull request: https://github.com/apache/spark/pull/1890#issuecomment-51888165 I've updated my PR, and now instead of getting the Executor process to download jars before registering itself with the application driver, the Worker process downlo

[GitHub] spark pull request: [SPARK-2970] [SQL] spark-sql script ends with ...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1891#issuecomment-51889573 QA results for PR 1891:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2986] [SQL] fixed: setting properties d...

2014-08-12 Thread guowei2
GitHub user guowei2 opened a pull request: https://github.com/apache/spark/pull/1904 [SPARK-2986] [SQL] fixed: setting properties does not effect it seems that set command does not run by SparkSQLDriver. it runs on hive api. user can not change reduce number by setting spark.sql

[GitHub] spark pull request: [SPARK-2924] remove default args to overloaded...

2014-08-12 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/1704#discussion_r16103132 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala --- @@ -98,9 +98,11 @@ class StreamingContext private[streaming] (

[GitHub] spark pull request: [SPARK-2986] [SQL] fixed: setting properties d...

2014-08-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1904#issuecomment-51891698 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-2924] remove default args to overloaded...

2014-08-12 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/1704#discussion_r16103171 --- Diff: project/MimaExcludes.scala --- @@ -133,7 +136,10 @@ object MimaExcludes { MimaBuild.excludeSparkClass("mllib.optimization.SquaredG

[GitHub] spark pull request: [SPARK-2969][SQL] Make ScalaReflection be able...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1889#issuecomment-51891864 QA results for PR 1889:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2947] DAGScheduler resubmit the stage i...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1877#issuecomment-51892023 QA results for PR 1877:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2970] [SQL] spark-sql script ends with ...

2014-08-12 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/1891#issuecomment-51896285 Ah... It's strange. I think there are no test using SparkSQLCLIDriver right? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-08-12 Thread hunggpham
Github user hunggpham commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-51898209 Yes i forked your repository and can see the codes now.One question: I don't see backprop codes.Will that be added later? Thanks. Sent from cell p

[GitHub] spark pull request: [SPARK-2969][SQL] Make ScalaReflection be able...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1889#issuecomment-51898757 QA tests have started for PR 1889. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18368/consoleFull --- If

[GitHub] spark pull request: [SPARK-2987] - Adjust build system and tests t...

2014-08-12 Thread ScrapCodes
GitHub user ScrapCodes opened a pull request: https://github.com/apache/spark/pull/1905 [SPARK-2987] - Adjust build system and tests to work with scala 2.11 This patch assumes that we are on akka 2.3.4. So it does not deal with that. You can merge this pull request into a Git repo

[GitHub] spark pull request: [SPARK-2987] - Adjust build system and tests t...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1905#issuecomment-51900317 QA tests have started for PR 1905. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18369/consoleFull --- If

[GitHub] spark pull request: [SPARK-2987] - Adjust build system and tests t...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1905#issuecomment-51900326 QA results for PR 1905:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2987] - Adjust build system and tests t...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1905#issuecomment-51900706 QA tests have started for PR 1905. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18370/consoleFull --- If

[GitHub] spark pull request: [SPARK-2987] - Adjust build system and tests t...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1905#issuecomment-51900759 QA results for PR 1905:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2987] - Adjust build system and tests t...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1905#issuecomment-51901484 QA tests have started for PR 1905. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18371/consoleFull --- If

[GitHub] spark pull request: [SPARK-2987] - Adjust build system and tests t...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1905#issuecomment-51903449 QA tests have started for PR 1905. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18372/consoleFull --- If

[GitHub] spark pull request: [SPARK-2969][SQL] Make ScalaReflection be able...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1889#issuecomment-51904098 QA results for PR 1889:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2987] - Adjust build system and tests t...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1905#issuecomment-51906898 QA results for PR 1905:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2987] - Adjust build system and tests t...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1905#issuecomment-51909284 QA results for PR 1905:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2969][SQL] Make ScalaReflection be able...

2014-08-12 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/1889#issuecomment-51918543 @ueshin If we are changing the default value of `ArrayType.containsNull` to `true`, we may also need to add logic to set that value based on data sources. In https://githu

[GitHub] spark pull request: [WIP][SPARK-1720]use LD_LIBRARY_PATH instead o...

2014-08-12 Thread witgo
GitHub user witgo reopened a pull request: https://github.com/apache/spark/pull/1031 [WIP][SPARK-1720]use LD_LIBRARY_PATH instead of -Djava.library.path - [ ] Standalone - [ ] Mesos - [ ] YARN - [ ] Amazon EC2 You can merge this pull request into a Git repository by r

[GitHub] spark pull request: [WIP][SPARK-1720]use LD_LIBRARY_PATH instead o...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-51920856 QA tests have started for PR 1031. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18373/consoleFull --- If

[GitHub] spark pull request: [WIP][SPARK-1720]use LD_LIBRARY_PATH instead o...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-51921344 QA results for PR 1031:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [WIP][SPARK-1720] Add LD_LIBRARY_PATH environm...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-51923013 QA tests have started for PR 1031. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18374/consoleFull --- If

[GitHub] spark pull request: [WIP][SPARK-1720] Add the value of LD_LIBRARY_...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-51931255 QA results for PR 1031:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-1766] sorted functions to meet pedantic...

2014-08-12 Thread copester
GitHub user copester opened a pull request: https://github.com/apache/spark/pull/1906 [SPARK-1766] sorted functions to meet pedantic requirements You can merge this pull request into a Git repository by running: $ git pull https://github.com/ResilientScience/spark master Alte

[GitHub] spark pull request: [SPARK-1766] sorted functions to meet pedantic...

2014-08-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1906#issuecomment-51932732 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-2140] Enabling YARN memory overhead cal...

2014-08-12 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/1906#discussion_r16121764 --- Diff: yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -113,11 +113,12 @@ class Client(clientArgs: ClientArguments, hadoopConf:

[GitHub] spark pull request: SPARK-1297 Upgrade HBase dependency to 0.98

2014-08-12 Thread tmalaska
Github user tmalaska commented on the pull request: https://github.com/apache/spark/pull/1893#issuecomment-51936244 +1 I'm all for the update to 0.98 HBase. Just make sure we address everything that Sean O is asking. We need this to be able to build with Hadoop1 or Hadoop

[GitHub] spark pull request: [SPARK-2987][WIP] - Adjust build system and te...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1905#issuecomment-5193 QA tests have started for PR 1905. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18375/consoleFull --- If

[GitHub] spark pull request: [SPARK-2924] remove default args to overloaded...

2014-08-12 Thread avati
Github user avati commented on the pull request: https://github.com/apache/spark/pull/1704#issuecomment-51942842 Had conflict with recent merge of 9038d9. Addressed other code comments as well. --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request: [WIP][SPARK-1720] Add the value of LD_LIBRARY_...

2014-08-12 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-51944309 Does this work cross-platform? I think OSX uses `DYLD_LIBRARY_PATH`, and Windows probably uses something else. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-2969][SQL] Make ScalaReflection be able...

2014-08-12 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1889#issuecomment-51944634 It would be great to get this change in for 1.1 since its kinda of an API change. However, for now I'm okay if trying to store nulls into parquet arrays throws exception

[GitHub] spark pull request: [WIP][SPARK-1720] Add the value of LD_LIBRARY_...

2014-08-12 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-51945400 It's definitely not cross-platform. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark pull request: [WIP][SPARK-1720] Add the value of LD_LIBRARY_...

2014-08-12 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-51946148 Yes,I only took into account the linux and windows. Yarn only distinguish windows and non-windows at [ApplicationConstants.java#L210]( https://github.com/apache/had

[GitHub] spark pull request: [SPARK-2987][WIP] - Adjust build system and te...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1905#issuecomment-51948261 QA results for PR 1905:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: Vida/ec2 reuse security group

2014-08-12 Thread vidaha
Github user vidaha commented on the pull request: https://github.com/apache/spark/pull/1899#issuecomment-51949860 Hi Josh, IMHO, it's best not to require a Spark cluster name and the security group to be the same. While you can reuse an existing security group to launch anot

[GitHub] spark pull request: Vida/ec2 reuse security group

2014-08-12 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/1899#issuecomment-51953660 After a closer look, it looks like the existing `spark-ec2` already names instances after `cluster_name`, so it's safe and backwards-compatible to use that to identify

[GitHub] spark pull request: SPARK-2333

2014-08-12 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/1899#issuecomment-51954787 Jenkins, this is ok to test. Test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Add unit test to spark_ec2 script

2014-08-12 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/134#issuecomment-51955535 This PR is a pretty neat idea; it would be great to get more testing of the Spark EC2 scripts, especially as we add new features (like #1899). Does anyone want to adopt

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/1907 [WIP] [SPARK-2468] Netty based block server / client module This is a rough draft, but it can't be much worse than the old Netty module since the old one didn't really work :) Compared with th

[GitHub] spark pull request: SPARK-2333

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1899#issuecomment-51955708 QA tests have started for PR 1899. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18376/consoleFull --- If

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1907#issuecomment-51956457 QA tests have started for PR 1907. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18377/consoleFull --- If

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1907#issuecomment-51956530 QA results for PR 1907:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/1907#issuecomment-51957070 @rxin Very nice ! Do you have any benchmarks of how fast things are ? In terms of say % of network bandwidth we can use ? --- If your project is set up for it, you can

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1907#issuecomment-51957157 QA tests have started for PR 1907. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18378/consoleFull --- If

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1907#discussion_r16132782 --- Diff: core/src/main/scala/org/apache/spark/network/netty/server.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1907#issuecomment-51957233 That's part of the TODO :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1907#issuecomment-51957274 QA results for PR 1907:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: SPARK-2333

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1899#issuecomment-51957963 QA tests have started for PR 1899. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18379/consoleFull --- If

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1907#issuecomment-51958686 QA tests have started for PR 1907. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18380/consoleFull --- If

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1907#issuecomment-51958814 QA results for PR 1907:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread dcapwell
Github user dcapwell commented on a diff in the pull request: https://github.com/apache/spark/pull/1907#discussion_r16133840 --- Diff: core/src/main/scala/org/apache/spark/network/netty/ReferenceCountedBuffer.scala --- @@ -17,15 +17,25 @@ package org.apache.spark.networ

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread coderplay
Github user coderplay commented on a diff in the pull request: https://github.com/apache/spark/pull/1907#discussion_r16134006 --- Diff: core/src/main/scala/org/apache/spark/network/netty/server.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread dcapwell
Github user dcapwell commented on a diff in the pull request: https://github.com/apache/spark/pull/1907#discussion_r16134103 --- Diff: core/src/main/scala/org/apache/spark/network/netty/client.scala --- @@ -0,0 +1,276 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: SPARK-2425 Don't kill a still-running Applicat...

2014-08-12 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/1360#issuecomment-51959659 @pwendell Still should go into 1.1.0... The change is fairly small, and the unpatched behavior is pretty nasty for long-running applications. --- If your project

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread coderplay
Github user coderplay commented on a diff in the pull request: https://github.com/apache/spark/pull/1907#discussion_r16134163 --- Diff: core/src/main/scala/org/apache/spark/network/netty/server.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] spark pull request: SPARK-2830

2014-08-12 Thread atalwalkar
GitHub user atalwalkar opened a pull request: https://github.com/apache/spark/pull/1908 SPARK-2830 As per discussions with Xiangrui, I've reorganized and edited the mllib documentation. You can merge this pull request into a Git repository by running: $ git pull https://github

[GitHub] spark pull request: SPARK-2425 Don't kill a still-running Applicat...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1360#issuecomment-51960128 QA tests have started for PR 1360. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18382/consoleFull --- If

[GitHub] spark pull request: SPARK-2830

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1908#issuecomment-51960126 QA tests have started for PR 1908. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18381/consoleFull --- If

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread coderplay
Github user coderplay commented on a diff in the pull request: https://github.com/apache/spark/pull/1907#discussion_r16134481 --- Diff: core/src/main/scala/org/apache/spark/network/netty/server.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread coderplay
Github user coderplay commented on a diff in the pull request: https://github.com/apache/spark/pull/1907#discussion_r16134600 --- Diff: core/src/main/scala/org/apache/spark/network/netty/server.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1907#discussion_r16134921 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockFetcherIterator.scala --- @@ -262,77 +271,49 @@ object BlockFetcherIterator { readMetrics:

[GitHub] spark pull request: SPARK-2830

2014-08-12 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1908#issuecomment-51961773 @atalwalkar Could you add `[MLLIB] re-organize mllib documentation` to the title? --- If your project is set up for it, you can reply to this email and have your reply ap

[GitHub] spark pull request: SPARK-2830

2014-08-12 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/1908#issuecomment-51962565 nit: That's not really an adequate title for this PR, Ameet. It should include enough description so that we can tell what it is about in the corresponding subject o

[GitHub] spark pull request: [WIP][SPARK-2935][SQL]Fix parquet predicate pu...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1863#issuecomment-51962859 QA tests have started for PR 1863. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18383/consoleFull --- If

[GitHub] spark pull request: [WIP] [SPARK-2468] Netty based block server / ...

2014-08-12 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1907#discussion_r16135674 --- Diff: core/src/main/scala/org/apache/spark/network/netty/client.scala --- @@ -0,0 +1,276 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [WIP][SPARK-2961][SQL] Use statistics to skip ...

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1883#issuecomment-51962881 QA tests have started for PR 1883. This patch DID NOT merge cleanly! View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18384/consoleFull

[GitHub] spark pull request: [SPARK-2358][MLLIB] Add an option to include n...

2014-08-12 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/1295#issuecomment-51962886 It would be great to mention this option in the MLlib documentation. --- If your project is set up for it, you can reply to this email and have your reply appear on Git

[GitHub] spark pull request: [SPARK-2991] Implement RDD lazy transforms for...

2014-08-12 Thread erikerlandson
GitHub user erikerlandson opened a pull request: https://github.com/apache/spark/pull/1909 [SPARK-2991] Implement RDD lazy transforms for scanLeft and scan Discussion of implementations: http://erikerlandson.github.io/blog/2014/08/09/implementing-an-rdd-scanleft-transform-with-c

[GitHub] spark pull request: SPARK-2333

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1899#issuecomment-51963248 QA results for PR 1899:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2991] Implement RDD lazy transforms for...

2014-08-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1909#issuecomment-51963308 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-2986] [SQL] fixed: setting properties d...

2014-08-12 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1904#issuecomment-51963827 Why do you want use hive code for configuration? We have our own configuration that already mirrors into HiveConf. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-2848] Shade Guava in uber-jars.

2014-08-12 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/1813#issuecomment-51964431 Updated to remove forked code. I'm not too proud of the code to package things in the sbt build, but it's how I got things to work without having to expose Guava in the "c

[GitHub] spark pull request: [SPARK-2848] Shade Guava in uber-jars.

2014-08-12 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/1813#issuecomment-51964451 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2848] Shade Guava in uber-jars.

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1813#issuecomment-51964832 QA tests have started for PR 1813. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18386/consoleFull --- If

[GitHub] spark pull request: SPARK-2333

2014-08-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1899#issuecomment-51965003 QA results for PR 1899:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2991] Implement RDD lazy transforms for...

2014-08-12 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/1909#issuecomment-51965072 Erik, you've been doing some great work on making non-lazy transforms lazy! I haven't had time to thoroughly review your recent PRs, but can you do some checks and p

[GitHub] spark pull request: [SPARK-2991] Implement RDD lazy transforms for...

2014-08-12 Thread erikerlandson
Github user erikerlandson commented on the pull request: https://github.com/apache/spark/pull/1909#issuecomment-51965848 Good point, I will look into those. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

  1   2   3   >