[GitHub] spark pull request: [SPARK-2254] [SQL] ScalaRefection should mark ...

2014-06-25 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/1193#issuecomment-47193596 @rxin Thanks! I will fix soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-2254] [SQL] ScalaRefection should mark ...

2014-06-25 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1193#issuecomment-47193347 Merging this one so you can fix the conflict with this other. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-2254] [SQL] ScalaRefection should mark ...

2014-06-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1193 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2287] [SQL] Make ScalaReflection be abl...

2014-06-25 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/1226#issuecomment-47192918 This will cause merge conflict with #1193, but I can fix it soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-2287] [SQL] Make ScalaReflection be abl...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1226#issuecomment-47192751 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2286][UI] Report exception/errors for f...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1225#issuecomment-47192746 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-2286][UI] Report exception/errors for f...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1225#issuecomment-47192750 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2287] [SQL] Make ScalaReflection be abl...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1226#issuecomment-47192744 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-2286][UI] Report exception/errors for f...

2014-06-25 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/1225#discussion_r14227125 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -58,10 +98,19 @@ case class ExceptionFailure( * it was fetched. */

[GitHub] spark pull request: [SPARK-2287] [SQL] Make ScalaReflection be abl...

2014-06-25 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/1226 [SPARK-2287] [SQL] Make ScalaReflection be able to handle Generic case classes. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ueshin/apache-sp

[GitHub] spark pull request: [SPARK-2286][UI] Report exception/errors for f...

2014-06-25 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1225#discussion_r14226907 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -30,27 +30,67 @@ import org.apache.spark.storage.BlockManagerId @DeveloperApi

[GitHub] spark pull request: [SPARK-2286][UI] Report exception/errors for f...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1225#issuecomment-47191933 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47191952 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47191953 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16138/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-2286][UI] Report exception/errors for f...

2014-06-25 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/1225#discussion_r14226878 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -30,27 +30,67 @@ import org.apache.spark.storage.BlockManagerId @DeveloperApi

[GitHub] spark pull request: [SPARK-2286][UI] Report exception/errors for f...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1225#issuecomment-47191925 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: SPARK-2186: Spark SQL DSL support for simple a...

2014-06-25 Thread edrevo
Github user edrevo commented on the pull request: https://github.com/apache/spark/pull/1211#issuecomment-47191917 I have added DSL support (`avg()`, `count(distinct())`...) following @rxin's suggestion. I'm happy to change the DSL style if you end up favoring the other approach. ---

[GitHub] spark pull request: [SPARK-2286][UI] Report exception/errors for f...

2014-06-25 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/1225 [SPARK-2286][UI] Report exception/errors for failed tasks that are not ExceptionFailure Also added inline doc for each TaskEndReason. You can merge this pull request into a Git repository by runn

[GitHub] spark pull request: [SPARK-2172] PySpark cannot import mllib modul...

2014-06-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1223 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47190739 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47190746 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-06-25 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47190546 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-06-25 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47190528 No, just want to see Jenkins happy. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1224#discussion_r14226148 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -185,12 +185,15 @@ class JobProgressListener(conf: SparkConf) extends Sp

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1224#discussion_r14226156 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -185,12 +185,15 @@ class JobProgressListener(conf: SparkConf) extends Sp

[GitHub] spark pull request: [SPARK-1946] Submit tasks after (configured ra...

2014-06-25 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/900#discussion_r14225947 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -244,6 +255,17 @@ class CoarseGrainedSchedulerBac

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1224 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-1946] Submit tasks after (configured ra...

2014-06-25 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/900#issuecomment-47189335 That seems like a good plan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1224#issuecomment-47189318 Ok I'm going to merge this in master & branch-1.0. Thanks for taking a look. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/1224#discussion_r14225916 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -185,12 +185,15 @@ class JobProgressListener(conf: SparkConf) ex

[GitHub] spark pull request: [SPARK-1946] Submit tasks after (configured ra...

2014-06-25 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/900#discussion_r14225862 --- Diff: yarn/common/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterSchedulerBackend.scala --- @@ -0,0 +1,51 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1224#issuecomment-47188905 LGTM, feel free to merge. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-1946] Submit tasks after (configured ra...

2014-06-25 Thread li-zhihui
Github user li-zhihui commented on the pull request: https://github.com/apache/spark/pull/900#issuecomment-47188350 @tgravescs @kayousterhout How about move waitBackendReady to TaskSchedulerImpl.start. It will be called only once at spark initialization. --- If your project is s

[GitHub] spark pull request: [SPARK-1946] Submit tasks after (configured ra...

2014-06-25 Thread li-zhihui
Github user li-zhihui commented on a diff in the pull request: https://github.com/apache/spark/pull/900#discussion_r14225520 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala --- @@ -95,6 +95,7 @@ private[spark] class SparkDeploySch

[GitHub] spark pull request: [SPARK-1946] Submit tasks after (configured ra...

2014-06-25 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/900#discussion_r14225485 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -244,6 +255,17 @@ class CoarseGrainedSchedul

[GitHub] spark pull request: [SPARK-1946] Submit tasks after (configured ra...

2014-06-25 Thread li-zhihui
Github user li-zhihui commented on a diff in the pull request: https://github.com/apache/spark/pull/900#discussion_r14225397 --- Diff: yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -164,6 +164,7 @@ class ApplicationMaster(args: Application

[GitHub] spark pull request: [SPARK-1946] Submit tasks after (configured ra...

2014-06-25 Thread li-zhihui
Github user li-zhihui commented on a diff in the pull request: https://github.com/apache/spark/pull/900#discussion_r14225319 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -244,6 +255,17 @@ class CoarseGrainedSchedulerBa

[GitHub] spark pull request: [SPARK-2172] PySpark cannot import mllib modul...

2014-06-25 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1223#issuecomment-47187655 LGTM and tested with `mvn install`. Thanks for fixing it! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1224#issuecomment-47187503 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16137/ --- If your project

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1224#issuecomment-47187500 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: [SQL]Extract the joinkeys from join condition

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1190#issuecomment-47187501 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: [SQL]Extract the joinkeys from join condition

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1190#issuecomment-47187502 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16136/ --- If your project

[GitHub] spark pull request: [SPARK-1946] Submit tasks after (configured ra...

2014-06-25 Thread li-zhihui
Github user li-zhihui commented on a diff in the pull request: https://github.com/apache/spark/pull/900#discussion_r14225172 --- Diff: yarn/common/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterSchedulerBackend.scala --- @@ -0,0 +1,51 @@ +/* + * Licensed to t

[GitHub] spark pull request: [SPARK-1749] Job cancellation when SchedulerBa...

2014-06-25 Thread berngp
Github user berngp commented on the pull request: https://github.com/apache/spark/pull/1219#issuecomment-47186363 Thank you, this is fantastic. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1749] Job cancellation when SchedulerBa...

2014-06-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1219 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-1749] Job cancellation when SchedulerBa...

2014-06-25 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1219#issuecomment-47185485 Thanks Mark and Kay. I've merged this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark pull request: SPARK-2150: Provide direct link to finished ap...

2014-06-25 Thread rahulsinghaliitd
Github user rahulsinghaliitd commented on the pull request: https://github.com/apache/spark/pull/1094#issuecomment-47184407 Am I supposed to resolve the conflicts or will they be resolved by the admin who merges this change? --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1224#issuecomment-47184179 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1224#issuecomment-47184168 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1224#discussion_r14223909 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -185,12 +185,15 @@ class JobProgressListener(conf: SparkConf) extends Sp

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/1224 [SPARK-2284][UI] Mark all failed tasks as failures. Previously only tasks failed with ExceptionFailure reason was marked as failure. You can merge this pull request into a Git repository by running:

[GitHub] spark pull request: [SPARK-2284][UI] Mark all failed tasks as fail...

2014-06-25 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1224#issuecomment-47183970 @pwendell If still possible, this should go into 1.0.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark pull request: [SQL]Extract the joinkeys from join condition

2014-06-25 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1190#issuecomment-47183791 Thank you @marmbrus , updated, let's see the testing result. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SQL]Extract the joinkeys from join condition

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1190#issuecomment-47183743 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SQL]Extract the joinkeys from join condition

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1190#issuecomment-47183740 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: Improve ALS algorithm resource usage

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/929#issuecomment-47182435 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: Improve ALS algorithm resource usage

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/929#issuecomment-47182436 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16135/ --- If your project

[GitHub] spark pull request: SPARK-2281 [MLlib] Simplify the duplicate code...

2014-06-25 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1215#discussion_r14223272 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/Gradient.scala --- @@ -37,7 +37,11 @@ abstract class Gradient extends Serializable {

[GitHub] spark pull request: [SPARK-2172] PySpark cannot import mllib modul...

2014-06-25 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1223#discussion_r14223221 --- Diff: mllib/pom.xml --- @@ -76,5 +76,16 @@ scalatest-maven-plugin + + +src/main/resources

[GitHub] spark pull request: [SQL] SPARK-1800 Add broadcast hash join opera...

2014-06-25 Thread concretevitamin
Github user concretevitamin commented on the pull request: https://github.com/apache/spark/pull/1163#issuecomment-47181980 Sounds good. On Wednesday, June 25, 2014, Michael Armbrust wrote: > Thanks, I've merged this into master. I did not merge this into 1.0 as it

[GitHub] spark pull request: [SPARK-1749] Job cancellation when SchedulerBa...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1219#issuecomment-47180598 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16134/ --- If your project

[GitHub] spark pull request: [SPARK-1749] Job cancellation when SchedulerBa...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1219#issuecomment-47180596 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: Improve ALS algorithm resource usage

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/929#issuecomment-47180380 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: Improve ALS algorithm resource usage

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/929#issuecomment-47180387 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SQL]Extract the joinkeys from join condition

2014-06-25 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1190#issuecomment-47179857 and please rebase to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SQL]Extract the joinkeys from join condition

2014-06-25 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1190#issuecomment-47179834 Okay, you've convinced me with the outer join argument. Remove HashFilteredJoin as its pretty redundant with your pattern. --- If your project is set up for it, you ca

[GitHub] spark pull request: [SPARK-2283][SQL] Reset test environment befor...

2014-06-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1221 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2283][SQL] Reset test environment befor...

2014-06-25 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1221#issuecomment-47179540 Thanks, merged into master and 1.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: SPARK-2099. Report progress while task is runn...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1056#issuecomment-47179388 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: [SQL] SPARK-1800 Add broadcast hash join opera...

2014-06-25 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1163#issuecomment-47179384 Thanks, I've merged this into master. I did not merge this into 1.0 as it has this hint that we are not sure we want to support long term, so I'd like to avoid having i

[GitHub] spark pull request: SPARK-2099. Report progress while task is runn...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1056#issuecomment-47179389 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16131/ --- If your project

[GitHub] spark pull request: [SPARK-2283][SQL] Reset test environment befor...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1221#issuecomment-47178632 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: [SPARK-2283][SQL] Reset test environment befor...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1221#issuecomment-47178633 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16130/ --- If your project

[GitHub] spark pull request: [SPARK-1749] Job cancellation when SchedulerBa...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1219#issuecomment-47178411 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1749] Job cancellation when SchedulerBa...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1219#issuecomment-47178404 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-2172] PySpark cannot import mllib modul...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1223#issuecomment-47178146 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-1749] Job cancellation when SchedulerBa...

2014-06-25 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1219#issuecomment-47178169 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-2172] PySpark cannot import mllib modul...

2014-06-25 Thread piotrszul
GitHub user piotrszul opened a pull request: https://github.com/apache/spark/pull/1223 [SPARK-2172] PySpark cannot import mllib modules in YARN-client mode Include pyspark/mllib python sources as resources in the mllib.jar. This way they will be included in the final assembly Yo

[GitHub] spark pull request: [WIP][SPARK-2097][SQL] UDF Support

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1063#issuecomment-47178002 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16133/ --- If your project is set up for it, you can

[GitHub] spark pull request: [WIP][SPARK-2097][SQL] UDF Support

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1063#issuecomment-47178001 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [WIP][SPARK-2097][SQL] UDF Support

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1063#issuecomment-47177906 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: [SQL]Extract the joinkeys from join condition

2014-06-25 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1190#issuecomment-47177931 BTW, if I follow the current implementation pattern, which also mean I have to add filter for the outer join, too, which may make the code duplicated(with the opti

[GitHub] spark pull request: [WIP][SPARK-2097][SQL] UDF Support

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1063#issuecomment-47177900 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-2283][SQL] Reset test environment befor...

2014-06-25 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/1221#issuecomment-47177834 @marmbrus Done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: [SQL] SPARK-1800 Add broadcast hash join opera...

2014-06-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1163 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: Introducing an Improved Pregel API

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1217#issuecomment-47177729 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: Introducing an Improved Pregel API

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1217#issuecomment-47177730 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16132/ --- If your project is set up for it, you can

[GitHub] spark pull request: Introducing an Improved Pregel API

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1217#issuecomment-47177656 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Introducing an Improved Pregel API

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1217#issuecomment-47177654 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SQL]Extract the joinkeys from join condition

2014-06-25 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1190#issuecomment-47177647 The `join/where` predicate push down has been done in `PushPredicateThroughJoin` of the logical plan optimizer, I don't think we really need to do it again here. H

[GitHub] spark pull request: SPARK-2099. Report progress while task is runn...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1056#issuecomment-47177081 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: SPARK-2099. Report progress while task is runn...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1056#issuecomment-47177088 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SQL]Extract the joinkeys from join condition

2014-06-25 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1190#issuecomment-47176944 I'm not sure what the point of this change is. It is only serving to make the planner more brittle and tied to the specifics of the current implementation of the optimi

[GitHub] spark pull request: [SPARK-1749] Job cancellation when SchedulerBa...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1219#issuecomment-47173831 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: [SQL] SPARK-1800 Add broadcast hash join opera...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1163#issuecomment-47173837 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16125/ --- If your project

[GitHub] spark pull request: [SPARK-1749] Job cancellation when SchedulerBa...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1219#issuecomment-47173834 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: SPARK-2282: Reuse PySpark Accumulator sockets ...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1220#issuecomment-47173836 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16129/ --- If your project

[GitHub] spark pull request: [SPARK-1749] Job cancellation when SchedulerBa...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1219#issuecomment-47173839 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16127/ --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-2283: reset test environment before runn...

2014-06-25 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1221#issuecomment-47173847 Add SQL to the title please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SQL] SPARK-1800 Add broadcast hash join opera...

2014-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1163#issuecomment-47173833 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

  1   2   3   4   >