[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12720#discussion_r61376668 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala --- @@ -337,6 +337,16 @@ case class PrettyAttribute(

[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12720#discussion_r61376609 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala --- @@ -337,6 +337,16 @@ case class PrettyAttribute(

[GitHub] spark pull request: [SPARK-13902][SCHEDULER] Make DAGScheduler.get...

2016-04-27 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/12655#issuecomment-215317033 I saw the PR #8427 now. Both the #8427 approach and @markhamstra's approach (should we use `getOrElseUpdate` instead of `getOrElse`?) seem like the simplest way to

[GitHub] spark pull request: [SPARK-14972] Improve performance of JSON sche...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12750#issuecomment-215316922 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14972] Improve performance of JSON sche...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12750#issuecomment-215316921 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14972] Improve performance of JSON sche...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12750#issuecomment-215316797 **[Test build #57215 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57215/consoleFull)** for PR 12750 at commit

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215316563 **[Test build #57222 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57222/consoleFull)** for PR 12718 at commit

[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12720#discussion_r61376302 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -866,71 +867,189 @@ class Analyzer( * Note: CTEs

[GitHub] spark pull request: [SPARK-10001][Core] Don't short-circuit action...

2016-04-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12745 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-10001][Core] Don't short-circuit action...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12745#issuecomment-215315836 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10001][Core] Don't short-circuit action...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12745#issuecomment-215315834 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10001][Core] Don't short-circuit action...

2016-04-27 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12745#issuecomment-215315768 Merging in master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10001][Core] Don't short-circuit action...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12745#issuecomment-215315711 **[Test build #57213 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57213/consoleFull)** for PR 12745 at commit

[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12720#discussion_r61375897 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -866,71 +867,189 @@ class Analyzer( * Note:

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215315114 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215315097 **[Test build #57218 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57218/consoleFull)** for PR 12683 at commit

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215315113 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12720#discussion_r61375586 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -866,71 +867,189 @@ class Analyzer( * Note:

[GitHub] spark pull request: [SPARK-12660] [SPARK-14967] [SQL] Implement Ex...

2016-04-27 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12736#issuecomment-215314522 LGTM. A unrelated question, how do we express the EXCEPT ALL semantic? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-10001][Core] Don't short-circuit action...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12745#issuecomment-215314462 **[Test build #2898 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2898/consoleFull)** for PR 12745 at commit

[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12720#discussion_r61375441 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -75,76 +77,63 @@ case class ScalarSubquery(

[GitHub] spark pull request: [SPARK-14783] [SPARK-14786] [BRANCH-1.6] Prese...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12724#issuecomment-215314235 **[Test build #57221 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57221/consoleFull)** for PR 12724 at commit

[GitHub] spark pull request: [SPARK-14850][ML] specialize array data for Ve...

2016-04-27 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/12640#discussion_r61375365 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayData.scala --- @@ -29,6 +29,82 @@ abstract class ArrayData extends

[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12720#issuecomment-215314230 **[Test build #2899 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2899/consoleFull)** for PR 12720 at commit

[GitHub] spark pull request: [SPARK-14850][ML] specialize array data for Ve...

2016-04-27 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/12640#issuecomment-215314086 @cloud-fan This is still much slower than 1.4 and adding more subclasses of ArrayData may prevent JIT inline methods like `getInt` and `getDouble`. Is it easy to

[GitHub] spark pull request: [SPARK-14783] [SPARK-14786] [BRANCH-1.6] Prese...

2016-04-27 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/12724#issuecomment-215314116 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12660] [SPARK-14967] [SQL] Implement Ex...

2016-04-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12736#discussion_r61375198 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -398,6 +398,66 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark pull request: [SPARK-12660] [SPARK-14967] [SQL] Implement Ex...

2016-04-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12736#discussion_r61375067 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercionSuite.scala --- @@ -488,14 +488,6 @@ class

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61374828 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -175,124 +172,143 @@ class TaskMetrics private[spark] () extends

[GitHub] spark pull request: [SPARK-12660] [SPARK-14967] [SQL] Implement Ex...

2016-04-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12736#discussion_r61374712 --- Diff: sql/core/src/test/java/test/org/apache/spark/sql/JavaDatasetSuite.java --- @@ -291,7 +291,7 @@ public void testSetOperation() {

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61374697 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,391 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215312526 **[Test build #57220 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57220/consoleFull)** for PR 12612 at commit

[GitHub] spark pull request: [SPARK-14487][SQL] User Defined Type registrat...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12259#issuecomment-215311747 **[Test build #57219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57219/consoleFull)** for PR 12259 at commit

[GitHub] spark pull request: [SPARK-12235][SPARKR] Enhance mutate() to supp...

2016-04-27 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10220#discussion_r61374122 --- Diff: R/pkg/R/DataFrame.R --- @@ -1451,17 +1451,54 @@ setMethod("mutate", function(.data, ...) { x <- .data

[GitHub] spark pull request: [SPARK-14487][SQL] User Defined Type registrat...

2016-04-27 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/12259#issuecomment-215310697 LGTM pending Jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-14487][SQL] User Defined Type registrat...

2016-04-27 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/12259#discussion_r61373781 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -117,6 +117,7 @@ class SQLContext private[sql]( * * @since

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215310647 **[Test build #57218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57218/consoleFull)** for PR 12683 at commit

[GitHub] spark pull request: [SPARK-14346][SQL] Add PARTITIONED BY and CLUS...

2016-04-27 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/12734#issuecomment-215310529 @jodersky Oh sorry, pasted the JIRA ticket summary to the PR title but forgot to add the tags. Updated! --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-14729][Scheduler] Refactored YARN sched...

2016-04-27 Thread hbhanawat
Github user hbhanawat commented on the pull request: https://github.com/apache/spark/pull/12641#issuecomment-215310532 Hmm. @vanzin I think you have a point. There are few things that can be done but not sure if they will simplify without reducing the flexibility. I will

[GitHub] spark pull request: [SPARK-14487][SQL] User Defined Type registrat...

2016-04-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/12259#discussion_r61373671 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/util/MLlibTestSparkContext.scala --- @@ -24,14 +24,18 @@ import org.scalatest.Suite import

[GitHub] spark pull request: [SPARK-14972] Improve performance of JSON sche...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12750#issuecomment-215310239 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13568] [ML] Create feature transformer ...

2016-04-27 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/11601#discussion_r61373605 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala --- @@ -0,0 +1,219 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-13568] [ML] Create feature transformer ...

2016-04-27 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/11601#discussion_r61373554 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala --- @@ -0,0 +1,219 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-14972] Improve performance of JSON sche...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12750#issuecomment-215310152 **[Test build #57212 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57212/consoleFull)** for PR 12750 at commit

[GitHub] spark pull request: [SPARK-14972] Improve performance of JSON sche...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12750#issuecomment-215310236 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13568] [ML] Create feature transformer ...

2016-04-27 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/11601#discussion_r61373571 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala --- @@ -0,0 +1,219 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-14487][SQL] User Defined Type registrat...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12259#issuecomment-215310117 **[Test build #57217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57217/consoleFull)** for PR 12259 at commit

[GitHub] spark pull request: [SPARK-14706][ML][PySpark] Python ML persisten...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12604#issuecomment-215309644 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14706][ML][PySpark] Python ML persisten...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12604#issuecomment-215309599 **[Test build #57216 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57216/consoleFull)** for PR 12604 at commit

[GitHub] spark pull request: [SPARK-14706][ML][PySpark] Python ML persisten...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12604#issuecomment-215309643 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13961][ML] spark.ml ChiSqSelector and R...

2016-04-27 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/12467#issuecomment-215308986 Look good overall, I have my last inline comment. After that, it should be ready to go. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-12660] [SPARK-14967] [SQL] Implement Ex...

2016-04-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12736#discussion_r61372903 --- Diff: sql/core/src/test/java/test/org/apache/spark/sql/JavaDatasetSuite.java --- @@ -291,7 +291,7 @@ public void testSetOperation() {

[GitHub] spark pull request: [SPARK-14706][ML][PySpark] Python ML persisten...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12604#issuecomment-215308604 **[Test build #57216 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57216/consoleFull)** for PR 12604 at commit

[GitHub] spark pull request: [SPARK-14972] Improve performance of JSON sche...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12750#issuecomment-215308136 **[Test build #57215 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57215/consoleFull)** for PR 12750 at commit

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

2016-04-27 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215308071 @GayathriMurali You should modify [```RWrappers.load```](https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/r/RWrappers.scala#L44)

[GitHub] spark pull request: [SPARK-13961][ML] spark.ml ChiSqSelector and R...

2016-04-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/12467#discussion_r61372578 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/RFormulaSuite.scala --- @@ -290,4 +291,18 @@ class RFormulaSuite extends SparkFunSuite with

[GitHub] spark pull request: [SPARK-14487][SQL] User Defined Type registrat...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12259#issuecomment-215307961 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14487][SQL] User Defined Type registrat...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12259#issuecomment-215307962 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14487][SQL] User Defined Type registrat...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12259#issuecomment-215307877 **[Test build #57207 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57207/consoleFull)** for PR 12259 at commit

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215307044 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215307046 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215307026 **[Test build #57211 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57211/consoleFull)** for PR 12683 at commit

[GitHub] spark pull request: [SPARK-14961] Build HashedRelation larger than...

2016-04-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12740 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-14961] Build HashedRelation larger than...

2016-04-27 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12740#issuecomment-215306302 When profiling the performance for BytesToBytesMap, it does have difference. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-14961] Build HashedRelation larger than...

2016-04-27 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12740#issuecomment-215306341 Merging this into master, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-10780][ML][WIP] Add initial model to km...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9#issuecomment-215305404 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10780][ML][WIP] Add initial model to km...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9#issuecomment-215305403 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61371630 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/SortBasedAggregateExec.scala --- @@ -46,7 +46,7 @@ case class

[GitHub] spark pull request: [SPARK-10780][ML][WIP] Add initial model to km...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9#issuecomment-215305352 **[Test build #57210 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57210/consoleFull)** for PR 9 at commit

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61371318 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/SortBasedAggregateExec.scala --- @@ -46,7 +46,7 @@ case class

[GitHub] spark pull request: [SPARK-14935][CORE] DistributedSuite "local-cl...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12744#issuecomment-215304506 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14935][CORE] DistributedSuite "local-cl...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12744#issuecomment-215304507 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14935][CORE] DistributedSuite "local-cl...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12744#issuecomment-215304423 **[Test build #57202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57202/consoleFull)** for PR 12744 at commit

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215304306 This looks pretty good to me. We should get it to pass tests and then merge it asap. Some of the comments can be addressed later. --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-14783] [SPARK-14786] [BRANCH-1.6] Prese...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12724#issuecomment-215304302 **[Test build #57214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57214/consoleFull)** for PR 12724 at commit

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61371102 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/SortBasedAggregateExec.scala --- @@ -46,7 +46,7 @@ case class

[GitHub] spark pull request: [SPARK-14783] [SPARK-14786] [BRANCH-1.6] Prese...

2016-04-27 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/12724#issuecomment-215303991 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61371062 --- Diff: project/MimaExcludes.scala --- @@ -674,6 +674,19 @@ object MimaExcludes { ) ++ Seq( // [SPARK-4452][Core]Shuffle data

[GitHub] spark pull request: [SPARK-14970][SQL] Prevent DataSource from enu...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12748#issuecomment-215303740 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14346][SQL] Show Create Table (Native)

2016-04-27 Thread xwu0226
Github user xwu0226 commented on the pull request: https://github.com/apache/spark/pull/12579#issuecomment-215303758 @yhuai @liancheng , I see PR [#12734](https://github.com/apache/spark/pull/12734) takes care of the PARTITIONED BY and CLUSTERED BY (with SORTED BY) clause for CTAS

[GitHub] spark pull request: [SPARK-14970][SQL] Prevent DataSource from enu...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12748#issuecomment-215303742 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14970][SQL] Prevent DataSource from enu...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12748#issuecomment-215303655 **[Test build #57205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57205/consoleFull)** for PR 12748 at commit

[GitHub] spark pull request: [SPARK-10001][Core] Don't short-circuit action...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12745#issuecomment-215303593 **[Test build #57213 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57213/consoleFull)** for PR 12745 at commit

[GitHub] spark pull request: [SPARK-12919][SPARKR] Implement dapply() on Da...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12493#issuecomment-215303408 **[Test build #57201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57201/consoleFull)** for PR 12493 at commit

[GitHub] spark pull request: [SPARK-12919][SPARKR] Implement dapply() on Da...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12493#issuecomment-215303502 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12919][SPARKR] Implement dapply() on Da...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12493#issuecomment-215303503 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10780][ML][WIP] Add initial model to km...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9#issuecomment-215303378 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10780][ML][WIP] Add initial model to km...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9#issuecomment-215303382 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10780][ML][WIP] Add initial model to km...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9#issuecomment-215303315 **[Test build #57209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57209/consoleFull)** for PR 9 at commit

[GitHub] spark pull request: [SPARK-14972] Improve performance of JSON sche...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12750#issuecomment-215303075 **[Test build #57212 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57212/consoleFull)** for PR 12750 at commit

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61370631 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: [SPARK-10001][Core] Don't short-circuit action...

2016-04-27 Thread jodersky
Github user jodersky commented on a diff in the pull request: https://github.com/apache/spark/pull/12745#discussion_r61370492 --- Diff: core/src/main/scala/org/apache/spark/util/SignalUtils.scala --- @@ -94,7 +94,7 @@ private[spark] object SignalUtils extends Logging {

[GitHub] spark pull request: [SPARK-14972] Improve performance of JSON sche...

2016-04-27 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/12750 [SPARK-14972] Improve performance of JSON schema inference's compatibleType method This patch improves the performance of `InferSchema.compatibleType` and `inferField`. The net result of this

[GitHub] spark pull request: [SPARK-14783] [SPARK-14786] [BRANCH-1.6] Prese...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12724#issuecomment-215302424 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14783] [SPARK-14786] [BRANCH-1.6] Prese...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12724#issuecomment-215302427 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14783] [SPARK-14786] [BRANCH-1.6] Prese...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12724#issuecomment-215302234 **[Test build #57193 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57193/consoleFull)** for PR 12724 at commit

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12683#issuecomment-215302215 **[Test build #57211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57211/consoleFull)** for PR 12683 at commit

[GitHub] spark pull request: [SPARK-10001][Core] Don't short-circuit action...

2016-04-27 Thread jodersky
Github user jodersky commented on the pull request: https://github.com/apache/spark/pull/12745#issuecomment-215302197 removed the label, sorry about that --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10001][Core][Hotfix] Don't short-circui...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12745#issuecomment-215302102 **[Test build #2898 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2898/consoleFull)** for PR 12745 at commit

[GitHub] spark pull request: [SPARK-10001][Core][Hotfix] Don't short-circui...

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12745#discussion_r61370370 --- Diff: core/src/main/scala/org/apache/spark/util/SignalUtils.scala --- @@ -94,7 +94,7 @@ private[spark] object SignalUtils extends Logging {

  1   2   3   4   5   6   7   8   9   10   >