[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11520055 --- Diff: core/src/main/scala/org/apache/spark/ui/WebUI.scala --- @@ -17,53 +17,134 @@ package org.apache.spark.ui -import

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40173436 This is a really nice re-factoring! Unfortunately I just had time to look at some of it due to running into a weird performance issue. For some reason the pages were

[GitHub] spark pull request: PySpark API for SparkSQL

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/363#issuecomment-40173668 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: PySpark API for SparkSQL

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/363#issuecomment-40173673 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP] SPARK-1430: Support sparse data in Pytho...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/341#issuecomment-40174483 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40174456 @tdas there are some binary compatiblity checks here that need to be fixed. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [WIP] SPARK-1430: Support sparse data in Pytho...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/341#issuecomment-40174484 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14049/ --- If your project

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11520596 --- Diff: core/src/main/scala/org/apache/spark/ui/WebUI.scala --- @@ -17,53 +17,134 @@ package org.apache.spark.ui -import

[GitHub] spark pull request: [SPARK-1157][MLlib] L-BFGS Optimizer based on ...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/353#issuecomment-40174782 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1157][MLlib] L-BFGS Optimizer based on ...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/353#issuecomment-40174791 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1157][MLlib] L-BFGS Optimizer based on ...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/353#issuecomment-40174857 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14051/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-1157][MLlib] L-BFGS Optimizer based on ...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/353#issuecomment-40174856 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: update spark.default.parallelism

2014-04-11 Thread CrazyJvm
GitHub user CrazyJvm opened a pull request: https://github.com/apache/spark/pull/389 update spark.default.parallelism actually, the value 8 is only valid in mesos fine-grained mode : code override def defaultParallelism() = sc.conf.getInt(spark.default.parallelism, 8)

[GitHub] spark pull request: update spark.default.parallelism

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/389#issuecomment-40175755 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-1157][MLlib] L-BFGS Optimizer based on ...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/353#issuecomment-40177717 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1157][MLlib] L-BFGS Optimizer based on ...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/353#issuecomment-40177723 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1157][MLlib] L-BFGS Optimizer based on ...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/353#issuecomment-40177788 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14052/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-1157][MLlib] L-BFGS Optimizer based on ...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/353#issuecomment-40177786 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: PySpark API for SparkSQL

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/363#issuecomment-40177915 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: PySpark API for SparkSQL

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/363#issuecomment-40177916 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14050/ --- If your project

[GitHub] spark pull request: SPARK-1417: Spark on Yarn - spark UI link from...

2014-04-11 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/344#issuecomment-40178755 Merging this into master. @tgravescs, @pwendell if this is relevant to 1.0, please do merge into that branch too. --- If your project is set up for it, you can reply

[GitHub] spark pull request: SPARK-1417: Spark on Yarn - spark UI link from...

2014-04-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/344 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Update WindowedDStream.scala

2014-04-11 Thread baishuo
GitHub user baishuo opened a pull request: https://github.com/apache/spark/pull/390 Update WindowedDStream.scala update the content of Exception when windowDuration is not multiple of parent.slideDuration You can merge this pull request into a Git repository by running: $ git

[GitHub] spark pull request: SPARK-1426: Make MLlib work with NumPy version...

2014-04-11 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/391 SPARK-1426: Make MLlib work with NumPy versions older than 1.7 Currently it requires NumPy 1.7 due to using the copyto method

[GitHub] spark pull request: SPARK-1426: Make MLlib work with NumPy version...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/391#issuecomment-40190567 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: pyspark need Py2 to work, graceful and helping...

2014-04-11 Thread abhishekkr
GitHub user abhishekkr opened a pull request: https://github.com/apache/spark/pull/392 pyspark need Py2 to work, graceful and helping fail if system default is Py3 it failed on mine thus noticed You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: pyspark need Py2 to work, graceful and helping...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/392#issuecomment-40193401 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-1157][MLlib] L-BFGS Optimizer based on ...

2014-04-11 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/353#discussion_r11528081 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/LBFGS.scala --- @@ -0,0 +1,259 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-1157][MLlib] L-BFGS Optimizer based on ...

2014-04-11 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/353#discussion_r11528087 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/optimization/LBFGSSuite.scala --- @@ -0,0 +1,203 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-1157][MLlib] L-BFGS Optimizer based on ...

2014-04-11 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/353#discussion_r11528239 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/optimization/LBFGSSuite.scala --- @@ -0,0 +1,203 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-1426: Make MLlib work with NumPy version...

2014-04-11 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/391#issuecomment-40196304 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1426: Make MLlib work with NumPy version...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/391#issuecomment-40196347 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1426: Make MLlib work with NumPy version...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/391#issuecomment-40196355 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1426: Make MLlib work with NumPy version...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/391#issuecomment-40198985 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14053/ --- If your project

[GitHub] spark pull request: SPARK-1426: Make MLlib work with NumPy version...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/391#issuecomment-40198984 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: SPARK-1143: refactor TaskSchedulerImplSuite us...

2014-04-11 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/339#issuecomment-40199289 Hi, @kayousterhout , mind reviewing this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1426: Make MLlib work with NumPy version...

2014-04-11 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/391#issuecomment-40206521 @techaddict Is it the only place that requires numpy 1.7? If so, please also remove the check in `/Users/meng/src/spark-mengxr/python/pyspark/mllib/__init__.py` and run

[GitHub] spark pull request: SPARK-1408 Modify Spark on Yarn to point to th...

2014-04-11 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/362#discussion_r11536291 --- Diff: yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -347,8 +347,8 @@ class ApplicationMaster(args:

[GitHub] spark pull request: SPARK-1408 Modify Spark on Yarn to point to th...

2014-04-11 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/362#discussion_r11536569 --- Diff: docs/configuration.md --- @@ -627,6 +627,13 @@ Apart from these, the following properties are also available, and may be useful Number of

[GitHub] spark pull request: SPARK-1417: Spark on Yarn - spark UI link from...

2014-04-11 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/344#issuecomment-40216406 I pulled this into branch-1.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1470: Spark logger moving to use scala-l...

2014-04-11 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/332#issuecomment-40221509 How to let Jenkins to run the test? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1395] Allow local: URIs to work on Ya...

2014-04-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/303#discussion_r11539966 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala --- @@ -424,29 +408,110 @@ object ClientBase { } }

[GitHub] spark pull request: [WIP] Add the lifecycle interface

2014-04-11 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/379#issuecomment-40224441 @andrewor14 I don't speak english well. On weekends, I will write a document. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: SPARK-1143: refactor TaskSchedulerImplSuite us...

2014-04-11 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/339#issuecomment-40224832 sure, take your time I notified you just because you're the original reporter of the issue and I want to ensure I understand the issue correctly --- If your

[GitHub] spark pull request: [BUGFIX] In-memory columnar storage bug fixes

2014-04-11 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/374#issuecomment-40227452 @marmbrus I think we'd better remove either `HiveInMemoryCompatabilitySuite` or `HiveCompatabilitySuite`. Travis complains that the build time is too long ( 50 min). I

[GitHub] spark pull request: [BUGFIX] In-memory columnar storage bug fixes

2014-04-11 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/374#issuecomment-40229263 Okay, that's a good point. Why don't you just move the before and after functions that turn on caching into hive compatibility and remove the in memory sub class.

[GitHub] spark pull request: SPARK-1057 (alternative) Remove fastutil

2014-04-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/266#issuecomment-40232250 Ah looks like this is not merging cleanly. Any chance you could bring it up to date? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: SPARK-1057 (alternative) Remove fastutil

2014-04-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/266#issuecomment-40232019 Okay this looks good, thanks @srowen! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: Update WindowedDStream.scala

2014-04-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/390#issuecomment-40232810 good catch --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: Update WindowedDStream.scala

2014-04-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/390#issuecomment-40232681 Jenkins test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Update WindowedDStream.scala

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/390#issuecomment-40233622 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Update WindowedDStream.scala

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/390#issuecomment-40233557 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11543991 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11544206 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-1459] Use local path (and not complete ...

2014-04-11 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/375#issuecomment-40238010 Ok, maybe it's not a bug per se. But I think it somewhat breaks the user's expectations. Let's say that given a user's configuration, the default fs is hdfs; I'd

[GitHub] spark pull request: [BUGFIX] In-memory columnar storage bug fixes

2014-04-11 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/374#issuecomment-40238090 OK, merged these two giant suites, hope both Travis and Jenkins are happy with this. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [BUGFIX] In-memory columnar storage bug fixes

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/374#issuecomment-40238344 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [BUGFIX] In-memory columnar storage bug fixes

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/374#issuecomment-40238325 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11545457 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [BUGFIX] In-memory columnar storage bug fixes

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/374#issuecomment-40238429 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11545501 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingJobProgressListener.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed to the

[GitHub] spark pull request: [BUGFIX] In-memory columnar storage bug fixes

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/374#issuecomment-40238430 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14055/ --- If your project is set up for it, you can

[GitHub] spark pull request: Update WindowedDStream.scala

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/390#issuecomment-40238669 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14054/ --- If your project

[GitHub] spark pull request: SPARK-1293 [SQL] [WIP] Parquet support for nes...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/360#issuecomment-40238811 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11545788 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingJobProgressListener.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed to the

[GitHub] spark pull request: PySpark API for SparkSQL

2014-04-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/363#issuecomment-40239515 @ahirreddy could you make a spark JIRA for this? Seems like a large enough feature that we'd want to track it. Add the component as SQL. --- If your project is set up

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11546093 --- Diff: core/src/main/scala/org/apache/spark/ui/UIUtils.scala --- @@ -67,19 +154,23 @@ private[spark] object UIUtils { type=text/css /

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11546163 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11546175 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Some clean up in build/docs

2014-04-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/382 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [BUGFIX] In-memory columnar storage bug fixes

2014-04-11 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/374#issuecomment-40241016 Hmm, this failure seems to be related to the new Python Spark SQL API (which isn't even part of this PR). I will investigate. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11546692 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11546746 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingJobProgressListener.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: PySpark API for SparkSQL

2014-04-11 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/363#issuecomment-40241301 There already is a JIRA :) https://issues.apache.org/jira/browse/SPARK-1374 --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/290#discussion_r11546882 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingJobProgressListener.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: PySpark API for SparkSQL

2014-04-11 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/363#issuecomment-40241427 Also, I think this PR is breaking jenkins by leaving files around that are making RAT fail. I don't think its the fault of this PR, but a problem with the RAT code.

[GitHub] spark pull request: [SQL] SPARK-1424 Generalize insertIntoTable fu...

2014-04-11 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/354#issuecomment-40241700 @rxin @mateiz PTAL --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: Remove extendedDebugInfo option in test build ...

2014-04-11 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/346#issuecomment-40241825 @haosdent Mind closing this PR if we aren't going to merge it? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SQL] SPARK-1424 Generalize insertIntoTable fu...

2014-04-11 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/354#discussion_r11547314 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala --- @@ -62,4 +65,43 @@ trait SchemaRDDLike { def registerAsTable(tableName:

[GitHub] spark pull request: [SPARK-1225, 1241] [MLLIB] Add AreaUnderCurve ...

2014-04-11 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/364#issuecomment-40242267 Thanks Xiangrui! Merged into both master and branch-1.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40242411 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40242398 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40242484 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40242487 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14057/ --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-1293 [SQL] [WIP] Parquet support for nes...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/360#issuecomment-40242736 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14056/ --- If your project

[GitHub] spark pull request: SPARK-1293 [SQL] [WIP] Parquet support for nes...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/360#issuecomment-40242735 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40242857 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40242872 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40243023 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40243025 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14058/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40243853 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40243869 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1386] Web UI for Spark Streaming

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/290#issuecomment-40247442 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14059/ --- If your project

[GitHub] spark pull request: HOTFIX: Ignore python metastore files in RAT c...

2014-04-11 Thread pwendell
GitHub user pwendell opened a pull request: https://github.com/apache/spark/pull/393 HOTFIX: Ignore python metastore files in RAT checks. This was causing some errors with pull request tests. You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [BUGFIX] In-memory columnar storage bug fixes

2014-04-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/374#issuecomment-40248604 @marmbrus thanks for catching this I've submitted #393 to fix it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: Yarn: do not set local IP in remote process en...

2014-04-11 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/394 Yarn: do not set local IP in remote process environment. Without this, remote processes won't launch when you set SPARK_LOCAL_IP for your driver. You can merge this pull request into a Git

[GitHub] spark pull request: [SPARK-1459] Use local path (and not complete ...

2014-04-11 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/375#issuecomment-40248807 Commit that implements what I'm suggesting, if you're interested: https://github.com/vanzin/spark/commit/15c4d1e7961d2ea4f89f889f2caa5211bbe7d0de I haven't

[GitHub] spark pull request: Yarn: do not set local IP in remote process en...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/394#issuecomment-40248929 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: HOTFIX: Ignore python metastore files in RAT c...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/393#issuecomment-40248931 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: HOTFIX: Ignore python metastore files in RAT c...

2014-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/393#issuecomment-40248944 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

  1   2   3   >