[GitHub] spark pull request: SPARK-1627: Support external aggregation by us...

2014-06-06 Thread lamuguo
Github user lamuguo commented on a diff in the pull request: https://github.com/apache/spark/pull/867#discussion_r13477507 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Aggregate.scala --- @@ -155,48 +155,60 @@ case class Aggregate( } } else

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/993 [WIP][SPARK-2054][SQL] Code Generation for Expression Evaluation This PR drops the aggregation code generation, which will be added back in a follow-up PR. The following remains to be done: -

[GitHub] spark pull request: SPARK-2043: ExternalAppendOnlyMap doesn't alwa...

2014-06-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/986 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-1830 Deploy failover, Make Persistence e...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/771#issuecomment-45305638 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/993#issuecomment-45305818 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2050][SQL] LIKE, RLIKE and IN in HQL sh...

2014-06-06 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/989#issuecomment-45306348 We probably should do for those as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-2050][SQL] LIKE, RLIKE and IN in HQL sh...

2014-06-06 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/989#issuecomment-45306351 Merging this into master branch-1.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-2050][SQL] LIKE, RLIKE and IN in HQL sh...

2014-06-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/989 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/993#issuecomment-45306506 One more to do is maven build ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1508][SQL] Add SQLConf to SQLContext.

2014-06-06 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/956#issuecomment-45306552 @marmbrus I'm thinking we can merge this one first and address rest of the comments later so we can start building on top of it. What do you think? --- If your project is

[GitHub] spark pull request: Set RDD name to input path

2014-06-06 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/992#issuecomment-45306770 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: Set RDD name to input path

2014-06-06 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/992#issuecomment-45306769 Do you mind creating a JIRA ticket for this change and update the title to include the JIRA ticket number, like other pull requests do? Thanks!

[GitHub] spark pull request: [SPARK-2050 - 2][SQL] DIV and BETWEEN should n...

2014-06-06 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/994 [SPARK-2050 - 2][SQL] DIV and BETWEEN should not be case sensitive. Followup: #989 You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-2050 - 2][SQL] DIV and BETWEEN should n...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/994#issuecomment-45306927 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Set RDD name to input path

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/992#issuecomment-45306929 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1552] Fix type comparison bug in {map,o...

2014-06-06 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/967#issuecomment-45306941 Thanks. Merging this in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1508][SQL] Add SQLConf to SQLContext.

2014-06-06 Thread concretevitamin
Github user concretevitamin commented on a diff in the pull request: https://github.com/apache/spark/pull/956#discussion_r13477966 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-1552] Fix type comparison bug in {map,o...

2014-06-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/967 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1508][SQL] Add SQLConf to SQLContext.

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/956#issuecomment-45307202 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2050 - 2][SQL] DIV and BETWEEN should n...

2014-06-06 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/994#issuecomment-45307172 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread concretevitamin
Github user concretevitamin commented on the pull request: https://github.com/apache/spark/pull/993#issuecomment-45307236 Another TODO might be to beef up IN's code gen semantics (recall NULL in NULL and the alike cases). --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-1841]: update scalatest to version 2.1....

2014-06-06 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/713#issuecomment-45307330 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1841]: update scalatest to version 2.1....

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/713#issuecomment-45307463 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP][SPARK-1477]: Add the lifecycle interface...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/991#issuecomment-45308170 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2052] [SQL] Add optimization for CaseCo...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/990#issuecomment-45308169 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [WIP][SPARK-1477]: Add the lifecycle interface...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/991#issuecomment-45308173 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15497/ --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-1830 Deploy failover, Make Persistence e...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/771#issuecomment-45308174 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2052] [SQL] Add optimization for CaseCo...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/990#issuecomment-45308172 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15495/ --- If your project

[GitHub] spark pull request: [SPARK-1969][MLlib] Public available online su...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/955#issuecomment-45309135 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1508][SQL] Add SQLConf to SQLContext.

2014-06-06 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/956#discussion_r13478757 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -177,8 +177,7 @@ private[sql] abstract class SparkStrategies

[GitHub] spark pull request: [SPARK-1508][SQL] Add SQLConf to SQLContext.

2014-06-06 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/956#discussion_r13479054 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/commands.scala --- @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-1508][SQL] Add SQLConf to SQLContext.

2014-06-06 Thread concretevitamin
Github user concretevitamin commented on a diff in the pull request: https://github.com/apache/spark/pull/956#discussion_r13479071 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -177,8 +177,7 @@ private[sql] abstract class

[GitHub] spark pull request: SPARK-1830 Deploy failover, Make Persistence e...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/771#issuecomment-45310507 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: SPARK-1830 Deploy failover, Make Persistence e...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/771#issuecomment-45310508 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15498/ --- If your project

[GitHub] spark pull request: [SPARK-1841]: update scalatest to version 2.1....

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/713#issuecomment-45310662 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/993#issuecomment-45311891 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13479600 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala --- @@ -48,14 +48,19 @@ class Projection(expressions:

[GitHub] spark pull request: [SPARK-2050 - 2][SQL] DIV and BETWEEN should n...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/994#issuecomment-45311973 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/993#issuecomment-45311970 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/993#issuecomment-45311971 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15499/ --- If your project is set up for it, you can

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13479648 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala --- @@ -23,6 +23,16 @@ import

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13479655 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala --- @@ -23,6 +23,16 @@ import

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13479686 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala --- @@ -72,7 +72,9 @@ abstract class RuleExecutor[TreeType :

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13479742 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/types/dataTypes.scala --- @@ -72,6 +79,13 @@ abstract class NumericType extends NativeType {

[GitHub] spark pull request: SPARK-2056 Set RDD name to input path

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/992#issuecomment-45312220 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13479735 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/types/dataTypes.scala --- @@ -33,6 +33,13 @@ abstract class DataType { case

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13479764 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/types/dataTypes.scala --- @@ -72,6 +79,13 @@ abstract class NumericType extends NativeType {

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13479850 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregates.scala --- @@ -0,0 +1,175 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13479917 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/AnonymousRowSuite.scala --- @@ -0,0 +1,32 @@ +package org.apache.spark.sql.catalyst.expressions +

[GitHub] spark pull request: [SQL] SPARK-1800 Add broadcast hash join opera...

2014-06-06 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/734#issuecomment-45312913 Should this go in now? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13480009 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/GeneratedRow.scala --- @@ -0,0 +1,833 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13480043 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/GeneratedRow.scala --- @@ -0,0 +1,833 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13480098 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/GeneratedRow.scala --- @@ -0,0 +1,833 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13480084 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/GeneratedRow.scala --- @@ -0,0 +1,833 @@ +/* --- End diff --

[GitHub] spark pull request: Update run-example

2014-06-06 Thread maji2014
Github user maji2014 commented on the pull request: https://github.com/apache/spark/pull/988#issuecomment-45315985 OK, SPARK-2057 is opened for tracing this issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-2056 Set RDD name to input path

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/992#issuecomment-45317252 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-1841]: update scalatest to version 2.1....

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/713#issuecomment-45317260 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1508][SQL] Add SQLConf to SQLContext.

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/956#issuecomment-45317258 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2050 - 2][SQL] DIV and BETWEEN should n...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/994#issuecomment-45317255 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15500/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-2050 - 2][SQL] DIV and BETWEEN should n...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/994#issuecomment-45317251 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP][SPARK-1930] The Container is running bey...

2014-06-06 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-45317843 I meant, I was expecting the default 3xx mb to be overhead : while this ended up making it 2.5gig or so : causing master to fail due to inadequate memory to run the job.

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-06 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-45318235 What is the impact on memory footprint for this change ? We have jobs which result in 40k stages, and we have jobs which have stages with 50k tasks. I would assume

[GitHub] spark pull request: [WIP][SPARK-1930] The Container is running bey...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-45319395 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP][SPARK-1930] The Container is running bey...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-45319768 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1416: PySpark support for SequenceFile a...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/455#issuecomment-45320861 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-06 Thread darabos
Github user darabos commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-45321679 Wow, 40k stages? :open_mouth: To estimate the memory use, I guess a large stack trace could be ~10 kB, so it would be 400 MB total. Would that be noticeable compared to

[GitHub] spark pull request: [SPARK-1841]: update scalatest to version 2.1....

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/713#issuecomment-45322836 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15503/ --- If your project

[GitHub] spark pull request: [SPARK-1969][MLlib] Public available online su...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/955#issuecomment-45322839 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1841]: update scalatest to version 2.1....

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/713#issuecomment-45322834 Build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-1841]: update scalatest to version 2.1....

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/713#issuecomment-45322841 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1508][SQL] Add SQLConf to SQLContext.

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/956#issuecomment-45322835 Build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-1508][SQL] Add SQLConf to SQLContext.

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/956#issuecomment-45322837 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15502/ --- If your project

[GitHub] spark pull request: [SPARK-1308] Add getNumPartitions to pyspark R...

2014-06-06 Thread syedhashmi
GitHub user syedhashmi opened a pull request: https://github.com/apache/spark/pull/995 [SPARK-1308] Add getNumPartitions to pyspark RDD Add getNumPartitions to pyspark RDD to provide an intuitive way to get number of partitions in RDD like we can do in scala today. You can merge

[GitHub] spark pull request: [SPARK-1308] Add getNumPartitions to pyspark R...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/995#issuecomment-45324006 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-1812][wip]

2014-06-06 Thread ScrapCodes
GitHub user ScrapCodes opened a pull request: https://github.com/apache/spark/pull/996 [SPARK-1812][wip] Was just trying out akka 2.3.3 (theoretically it should all work.) You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-1812][wip]

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/996#issuecomment-45327565 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1969][MLlib] Public available online su...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/955#issuecomment-45327623 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15504/ --- If your project

[GitHub] spark pull request: [SPARK-1841]: update scalatest to version 2.1....

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/713#issuecomment-45327670 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-1841]: update scalatest to version 2.1....

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/713#issuecomment-45327671 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15505/ --- If your project

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-06 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-45328207 I dont have solutions, particularly since I have not looked much into ui code in spark. Having hit memory issues form ui recently (and since it is too important to

[GitHub] spark pull request: [WIP][SPARK-1930] The Container is running bey...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-45333612 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-2058: Overriding config from SPARK_HOME ...

2014-06-06 Thread EugenCepoi
GitHub user EugenCepoi opened a pull request: https://github.com/apache/spark/pull/997 SPARK-2058: Overriding config from SPARK_HOME with SPARK_CONF_DIR Fixes https://issues.apache.org/jira/browse/SPARK-2058 The general idea is to use SPARK_CONF_DIR content instead of

[GitHub] spark pull request: SPARK-2058: Overriding config from SPARK_HOME ...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/997#issuecomment-45336582 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [WIP][SPARK-1930] The Container is running bey...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-45337362 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [WIP][SPARK-1930] The Container is running bey...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-45337364 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15506/ --- If your project

[GitHub] spark pull request: [WIP][SPARK-1930] The Container is running bey...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-45337367 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-06-06 Thread coderh
Github user coderh commented on the pull request: https://github.com/apache/spark/pull/597#issuecomment-45338790 Just a question on the result. ``` implicitPref rank numInterations lambda - rmse true 30 40 1.0 - 0.5776665087027969 ``` Here,

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-06-06 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/597#issuecomment-45339833 Simple RMSE is not a great metric for this model, because it treats all errors equally when the model itself does not at all. 1s are much more important than 0s. The

[GitHub] spark pull request: [WIP][SPARK-1930] The Container is running bey...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-45341590 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [WIP][SPARK-1930] The Container is running bey...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-45341591 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15507/ --- If your project

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-06 Thread darabos
Github user darabos commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-45342809 @ankurdave: Cool! For some reason I didn't wire up RDDs, only stages. Your change should complement this nicely. @rxin: I went with details instead of call stack

[GitHub] spark pull request: SPARK-1416: PySpark support for SequenceFile a...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/455#issuecomment-45344727 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1416: PySpark support for SequenceFile a...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/455#issuecomment-45344852 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15508/ --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-1416: PySpark support for SequenceFile a...

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/455#issuecomment-45344851 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread hsaputra
Github user hsaputra commented on the pull request: https://github.com/apache/spark/pull/993#issuecomment-45349273 HI @marmbrus, one generic comment, could you add object or class header comment to describe why each of them needed and the context why they are used. It should be very

[GitHub] spark pull request: [WIP][SPARK-2054][SQL] Code Generation for Exp...

2014-06-06 Thread hsaputra
Github user hsaputra commented on a diff in the pull request: https://github.com/apache/spark/pull/993#discussion_r13495163 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/GeneratedRow.scala --- @@ -0,0 +1,833 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-1508][SQL] Add SQLConf to SQLContext.

2014-06-06 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/956#discussion_r13495842 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -177,8 +177,7 @@ private[sql] abstract class SparkStrategies

[GitHub] spark pull request: [SPARK-1508][SQL] Add SQLConf to SQLContext.

2014-06-06 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/956#issuecomment-45351772 It may be better to define all properties used by Spark SQL in a single place. Also, instead of plain String (e.g. spark.sql.shufflePartitions), using constant String val

[GitHub] spark pull request: [SPARK-1508][SQL] Add SQLConf to SQLContext.

2014-06-06 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/956#discussion_r13496356 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/commands.scala --- @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-1812][wip]

2014-06-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/996#issuecomment-45353489 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

  1   2   3   >