[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55482347 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -380,6 +380,9 @@ object ColumnPruning extends

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/11596#issuecomment-194167614 @cloud-fan Thanks !! Actually i remember exploring this option. There is a generate syntax for outer lateral view like following - ```SQL SELECT * FROM src

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55482082 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -380,6 +380,9 @@ object ColumnPruning extends

[GitHub] spark pull request: [SPARK-13327][SPARKR] Added parameter validati...

2016-03-08 Thread olarayej
Github user olarayej commented on a diff in the pull request: https://github.com/apache/spark/pull/11220#discussion_r55481841 --- Diff: R/pkg/R/DataFrame.R --- @@ -303,8 +303,28 @@ setMethod("colnames", #' @rdname columns #' @name colnames<- setMethod("colnames<-",

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55481794 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -380,6 +380,9 @@ object ColumnPruning extends

[GitHub] spark pull request: Update Java Doc in Spark Submit

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11600#issuecomment-194164345 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55481593 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -380,6 +380,9 @@ object ColumnPruning extends

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55481539 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -380,6 +380,9 @@ object ColumnPruning extends

[GitHub] spark pull request: Update Java Doc in Spark Submit

2016-03-08 Thread AhmedKamal
GitHub user AhmedKamal opened a pull request: https://github.com/apache/spark/pull/11600 Update Java Doc in Spark Submit ## What changes were proposed in this pull request? JIRA : https://issues.apache.org/jira/browse/SPARK-13769 The java doc here

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55481375 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -380,6 +380,9 @@ object ColumnPruning extends

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55481341 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -380,6 +380,9 @@ object ColumnPruning extends

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55481255 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -380,6 +380,9 @@ object ColumnPruning extends

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55480958 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -380,6 +380,9 @@ object ColumnPruning extends

[GitHub] spark pull request: [SPARK-13706] [ML] Add Python Example for Trai...

2016-03-08 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/11547#issuecomment-194153088 Made a few minor comments, pending those LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11596#issuecomment-194153085 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11596#issuecomment-194153084 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13706] [ML] Add Python Example for Trai...

2016-03-08 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/11547#discussion_r55480608 --- Diff: examples/src/main/python/ml/train_validation_split.py --- @@ -0,0 +1,69 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request: [SPARK-13706] [ML] Add Python Example for Trai...

2016-03-08 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/11547#discussion_r55480665 --- Diff: examples/src/main/python/ml/train_validation_split.py --- @@ -0,0 +1,69 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11596#issuecomment-194152932 **[Test build #52725 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52725/consoleFull)** for PR 11596 at commit

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55480611 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -380,6 +380,9 @@ object ColumnPruning extends

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/11596#issuecomment-194152170 I'm not sure which one is better, it depends on which one is easier to understand and implement. I'm ok to generate verbose SQL string, but the generator itself

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55480282 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -380,6 +380,9 @@ object ColumnPruning extends

[GitHub] spark pull request: [SPARK-13706] [ML] Add Python Example for Trai...

2016-03-08 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/11547#discussion_r55479803 --- Diff: examples/src/main/python/ml/train_validation_split.py --- @@ -0,0 +1,69 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request: [SPARK-13636][SQL] Directly consume UnsafeRow ...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11484#issuecomment-194146675 **[Test build #52733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52733/consoleFull)** for PR 11484 at commit

[GitHub] spark pull request: [SPARK-13636][SQL] Directly consume UnsafeRow ...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11484#issuecomment-194145448 **[Test build #52732 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52732/consoleFull)** for PR 11484 at commit

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/11599#issuecomment-194143635 @cloud-fan Added another two cases. Feel free to let me know if you want me to add more cases. Thanks! --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-13759][SQL] Add IsNotNull constraints f...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11594#issuecomment-194142858 **[Test build #52731 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52731/consoleFull)** for PR 11594 at commit

[GitHub] spark pull request: [SPARK-13759][SQL] Add IsNotNull constraints f...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11594#issuecomment-194141605 **[Test build #52730 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52730/consoleFull)** for PR 11594 at commit

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11599#issuecomment-194141048 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11599#issuecomment-194141046 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11599#issuecomment-194140806 **[Test build #52724 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52724/consoleFull)** for PR 11599 at commit

[GitHub] spark pull request: [SPARK-12792][SPARKR] Refactor RRDD to support...

2016-03-08 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10947#discussion_r55477889 --- Diff: core/src/main/scala/org/apache/spark/api/r/RRunner.scala --- @@ -0,0 +1,367 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-12792][SPARKR] Refactor RRDD to support...

2016-03-08 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10947#discussion_r55477830 --- Diff: core/src/main/scala/org/apache/spark/api/r/RRunner.scala --- @@ -0,0 +1,367 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-12792][SPARKR] Refactor RRDD to support...

2016-03-08 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10947#discussion_r55477700 --- Diff: core/src/main/scala/org/apache/spark/api/r/RRunner.scala --- @@ -0,0 +1,367 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-12893][YARN] Fix history URL redirect e...

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10821#issuecomment-194140043 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13759][SQL] Add IsNotNull constraints f...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11594#issuecomment-194140033 **[Test build #52729 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52729/consoleFull)** for PR 11594 at commit

[GitHub] spark pull request: [SPARK-12893][YARN] Fix history URL redirect e...

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10821#issuecomment-194140046 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12792][SPARKR] Refactor RRDD to support...

2016-03-08 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10947#discussion_r55477584 --- Diff: core/src/main/scala/org/apache/spark/api/r/RRunner.scala --- @@ -0,0 +1,367 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-12893][YARN] Fix history URL redirect e...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10821#issuecomment-194139901 **[Test build #52722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52722/consoleFull)** for PR 10821 at commit

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/11596#issuecomment-194139575 @cloud-fan Yeah.. that should be possible, You are thinking to represent each Generate as a sub select clause ? --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-12718][SPARK-13720][SQL] SQL generation...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11555#issuecomment-194139422 **[Test build #52728 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52728/consoleFull)** for PR 11555 at commit

[GitHub] spark pull request: [SPARK-13320] [SQL] Support Star in CreateStru...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11208#issuecomment-194138373 **[Test build #52727 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52727/consoleFull)** for PR 11208 at commit

[GitHub] spark pull request: [SPARK-13320] [SQL] Support Star in CreateStru...

2016-03-08 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/11208#issuecomment-194136402 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12792][SPARKR] Refactor RRDD to support...

2016-03-08 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10947#discussion_r55476565 --- Diff: core/src/main/scala/org/apache/spark/api/r/RRunner.scala --- @@ -0,0 +1,367 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/11596#issuecomment-194133518 But a `Generate` operator only has one `generator`, so I think we can convert any kind of `Generate` to SELECT? --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-12792][SPARKR] Refactor RRDD to support...

2016-03-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10947#issuecomment-194133921 cc @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-13696] Remove BlockStore class & simpli...

2016-03-08 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/11534#discussion_r55476210 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -789,117 +752,193 @@ private[spark] class BlockManager( //

[GitHub] spark pull request: [SPARK-13696] Remove BlockStore class & simpli...

2016-03-08 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/11534#discussion_r55476161 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -789,117 +752,193 @@ private[spark] class BlockManager( //

[GitHub] spark pull request: [SPARK-12792][SPARKR] Refactor RRDD to support...

2016-03-08 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10947#discussion_r55476044 --- Diff: core/src/main/scala/org/apache/spark/api/r/RRunner.scala --- @@ -0,0 +1,367 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/11596#discussion_r55475930 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/LogicalPlanToSQLSuite.scala --- @@ -445,4 +461,98 @@ class LogicalPlanToSQLSuite extends

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/11596#discussion_r55475917 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/SQLBuilder.scala --- @@ -297,6 +303,60 @@ class SQLBuilder(logicalPlan: LogicalPlan,

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/11596#discussion_r55475944 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/LogicalPlanToSQLSuite.scala --- @@ -445,4 +461,98 @@ class LogicalPlanToSQLSuite extends

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/11596#discussion_r55475915 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/SQLBuilder.scala --- @@ -94,6 +94,12 @@ class SQLBuilder(logicalPlan: LogicalPlan,

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/11596#issuecomment-194130778 @cloud-fan Thank you. 1. Can any Generate operator be converted to a LATERAL VIEW? Does the plan tree have to match some kind of pattern? Yes, based on

[GitHub] spark pull request: [SPARK-13674][SQL] Add wholestage codegen supp...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11517#issuecomment-194128908 **[Test build #52726 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52726/consoleFull)** for PR 11517 at commit

[GitHub] spark pull request: [SPARK-13674][SQL] Add wholestage codegen supp...

2016-03-08 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/11517#issuecomment-194128858 cc @davies @nongli --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-13674][SQL] Add wholestage codegen supp...

2016-03-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/11517#discussion_r55475400 --- Diff: core/src/test/scala/org/apache/spark/rdd/PartitionwiseSampledRDDSuite.scala --- @@ -29,6 +29,8 @@ class MockSampler extends RandomSampler[Long,

[GitHub] spark pull request: [SPARK-13674][SQL] Add wholestage codegen supp...

2016-03-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/11517#discussion_r55475367 --- Diff: core/src/main/scala/org/apache/spark/util/random/RandomSampler.scala --- @@ -41,6 +41,12 @@ trait RandomSampler[T, U] extends Pseudorandom with

[GitHub] spark pull request: [SPARK-13436][SPARKR] Added parameter drop to ...

2016-03-08 Thread olarayej
Github user olarayej commented on the pull request: https://github.com/apache/spark/pull/11318#issuecomment-194128322 @felixcheung @sun-rui @shivaram Can you folks please take a look at this one? Thank you! --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11596#issuecomment-194126735 **[Test build #52725 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52725/consoleFull)** for PR 11596 at commit

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11599#issuecomment-194126377 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11599#issuecomment-194126378 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11599#issuecomment-194126222 **[Test build #52721 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52721/consoleFull)** for PR 11599 at commit

[GitHub] spark pull request: [SPARK-12792][SPARKR] Refactor RRDD to support...

2016-03-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10947#issuecomment-194125116 For bulk movement of code, can you comment in the pr yourself which part was actually changed, and which part was simply moving code from one place to another? --- If

[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

2016-03-08 Thread felixcheung
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/11486#issuecomment-194125311 From SparkR test failure: ``` 1. Error: naiveBayes --- there is no package called 'mlbench'

[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

2016-03-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11486#discussion_r55474419 --- Diff: R/pkg/R/mllib.R --- @@ -192,3 +210,37 @@ setMethod("fitted", signature(object = "PipelineModel"), stop(paste("Unsupported

[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

2016-03-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11486#discussion_r55474361 --- Diff: R/pkg/R/generics.R --- @@ -1168,3 +1168,7 @@ setGeneric("kmeans") #' @rdname fitted #' @export setGeneric("fitted") +

[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

2016-03-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11486#discussion_r55474302 --- Diff: R/pkg/R/mllib.R --- @@ -192,3 +210,37 @@ setMethod("fitted", signature(object = "PipelineModel"), stop(paste("Unsupported

[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

2016-03-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11486#discussion_r55474272 --- Diff: R/pkg/R/mllib.R --- @@ -192,3 +210,37 @@ setMethod("fitted", signature(object = "PipelineModel"), stop(paste("Unsupported

[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

2016-03-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11486#discussion_r55474264 --- Diff: R/pkg/R/mllib.R --- @@ -192,3 +210,37 @@ setMethod("fitted", signature(object = "PipelineModel"), stop(paste("Unsupported

[GitHub] spark pull request: [SPARK-13327][SPARKR] Added parameter validati...

2016-03-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11220#discussion_r55474213 --- Diff: R/pkg/R/DataFrame.R --- @@ -303,8 +303,28 @@ setMethod("colnames", #' @rdname columns #' @name colnames<-

[GitHub] spark pull request: [SPARK-12718][SPARK-13720][SQL] SQL generation...

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11555#issuecomment-194122022 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12718][SPARK-13720][SQL] SQL generation...

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11555#issuecomment-194122020 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12718][SPARK-13720][SQL] SQL generation...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11555#issuecomment-194121910 **[Test build #52720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52720/consoleFull)** for PR 11555 at commit

[GitHub] spark pull request: [SPARK-13696] Remove BlockStore class & simpli...

2016-03-08 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/11534#discussion_r55473498 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -789,117 +752,193 @@ private[spark] class BlockManager( //

[GitHub] spark pull request: [SPARK-13696] Remove BlockStore class & simpli...

2016-03-08 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/11534#discussion_r55473363 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -789,117 +752,193 @@ private[spark] class BlockManager( //

[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11486#issuecomment-194117566 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11486#issuecomment-194117569 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11486#issuecomment-194117451 **[Test build #52723 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52723/consoleFull)** for PR 11486 at commit

[GitHub] spark pull request: [SPARK-13696] Remove BlockStore class & simpli...

2016-03-08 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/11534#discussion_r55472870 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskStore.scala --- @@ -17,112 +17,100 @@ package org.apache.spark.storage -import

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11599#issuecomment-194115203 **[Test build #52724 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52724/consoleFull)** for PR 11599 at commit

[GitHub] spark pull request: [SPARK-12718][SPARK-13720][SQL] SQL generation...

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11555#issuecomment-194114531 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12718][SPARK-13720][SQL] SQL generation...

2016-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11555#issuecomment-194114532 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12718][SPARK-13720][SQL] SQL generation...

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11555#issuecomment-194113999 **[Test build #52718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52718/consoleFull)** for PR 11555 at commit

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55472267 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ColumnPruningSuite.scala --- @@ -157,6 +157,14 @@ class ColumnPruningSuite

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55472012 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ColumnPruningSuite.scala --- @@ -157,6 +157,14 @@ class ColumnPruningSuite

[GitHub] spark pull request: [SPARK-13656][SQL] Remove spark.sql.parquet.ca...

2016-03-08 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/11576#issuecomment-194112721 Yes, ... Scanning all files are pretty expensive though, we can add a metadata file for detecting it; when `DataFrameWriter` writes something in the files, it

[GitHub] spark pull request: [SPARK-13763] [SQL] Remove Project when its pr...

2016-03-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/11599#discussion_r55471810 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ColumnPruningSuite.scala --- @@ -157,6 +157,14 @@ class ColumnPruningSuite

[GitHub] spark pull request: [SPARK-13242] [SQL] codegen fallback in case-w...

2016-03-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11592#issuecomment-194112069 This looks pretty good. Just some minor comments on naming. It'd be great to be more consistent among "branch", "case", and "switch". --- If your project is set up for

[GitHub] spark pull request: [SPARK-13327][SPARKR] Added parameter validati...

2016-03-08 Thread sun-rui
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/11220#issuecomment-194111890 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-13696] Remove BlockStore class & simpli...

2016-03-08 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/11534#discussion_r55471546 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -34,13 +35,15 @@ private case class MemoryEntry(value: Any, size:

[GitHub] spark pull request: [SPARK-13242] [SQL] codegen fallback in case-w...

2016-03-08 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/11592#discussion_r55471524 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CodeGenerationSuite.scala --- @@ -58,6 +58,27 @@ class CodeGenerationSuite

[GitHub] spark pull request: [SPARK-13242] [SQL] codegen fallback in case-w...

2016-03-08 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/11592#discussion_r55471486 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -136,7 +136,16 @@ case class

[GitHub] spark pull request: [SPARK-13242] [SQL] codegen fallback in case-w...

2016-03-08 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/11592#discussion_r55471461 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -136,7 +136,16 @@ case class

[GitHub] spark pull request: [SPARK-13242] [SQL] codegen fallback in case-w...

2016-03-08 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/11592#discussion_r55471445 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -136,7 +136,16 @@ case class

[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

2016-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11486#issuecomment-194111388 **[Test build #52723 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52723/consoleFull)** for PR 11486 at commit

[GitHub] spark pull request: [SPARK-13242] [SQL] codegen fallback in case-w...

2016-03-08 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/11592#discussion_r55471424 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -205,6 +214,9 @@ case class

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/11596#discussion_r55471264 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/LogicalPlanToSQLSuite.scala --- @@ -445,4 +461,98 @@ class LogicalPlanToSQLSuite extends

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/11596#discussion_r55471184 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/LogicalPlanToSQLSuite.scala --- @@ -445,4 +461,98 @@ class LogicalPlanToSQLSuite extends

[GitHub] spark pull request: [SPARK-12719][SQL] [WIP] SQL generation suppor...

2016-03-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/11596#discussion_r55471043 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/SQLBuilder.scala --- @@ -297,6 +303,60 @@ class SQLBuilder(logicalPlan: LogicalPlan,

  1   2   3   4   5   6   7   8   >