[GitHub] spark pull request #17430: [SPARK-20096][Spark Submit][Minor]Expose the righ...

2017-03-26 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/17430#discussion_r108094996 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -190,6 +190,7 @@ private[deploy] class SparkSubmitArguments(args:

[GitHub] spark pull request #17407: [SPARK-20043][ML] DecisionTreeModel: ImpurityCalc...

2017-03-26 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17407#discussion_r108094236 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/DecisionTreeClassifierSuite.scala --- @@ -385,6 +385,22 @@ class

[GitHub] spark pull request #17407: [SPARK-20043][ML] DecisionTreeModel: ImpurityCalc...

2017-03-26 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17407#discussion_r108094126 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/DecisionTreeClassifierSuite.scala --- @@ -385,6 +385,22 @@ class

[GitHub] spark pull request #17407: [SPARK-20043][ML] DecisionTreeModel: ImpurityCalc...

2017-03-26 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17407#discussion_r108094114 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/DecisionTreeRegressorSuite.scala --- @@ -178,6 +178,22 @@ class DecisionTreeRegressorSuite

[GitHub] spark issue #17438: [SPARK-20104][SQL] Don't estimate IsNull or IsNotNull pr...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17438 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17438: [SPARK-20104][SQL] Don't estimate IsNull or IsNotNull pr...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17438 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75250/ Test FAILed. ---

[GitHub] spark issue #17438: [SPARK-20104][SQL] Don't estimate IsNull or IsNotNull pr...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17438 **[Test build #75250 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75250/testReport)** for PR 17438 at commit

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108093020 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -179,88 +132,295 @@ private[sql] class SessionState( }

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2017-03-26 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14617 Thanks @squito , I will change some event logs files to test non zero the off heap memory. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #17430: [SPARK-20096][Spark Submit][Minor]Expose the righ...

2017-03-26 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/17430#discussion_r108092473 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -190,6 +190,7 @@ private[deploy] class SparkSubmitArguments(args:

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108091767 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -179,88 +132,295 @@ private[sql] class SessionState( }

[GitHub] spark issue #17388: [SPARK-20059][YARN] Use the correct classloader for HBas...

2017-03-26 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17388 Thanks @vanzin for your comments. Yes, remote jars will also not be added to client's classpath currently. Any further comments? --- If your project is set up for it, you can reply to

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108091528 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -179,88 +132,295 @@ private[sql] class SessionState( }

[GitHub] spark issue #17375: [SPARK-19019][PYTHON][BRANCH-1.6] Fix hijacked `collecti...

2017-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17375 gentle ping @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108090394 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -179,88 +132,295 @@ private[sql] class SessionState( }

[GitHub] spark issue #17440: [MINOR][SPARKR] Move 'Data type mapping between R and Sp...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17440 **[Test build #75253 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75253/testReport)** for PR 17440 at commit

[GitHub] spark issue #17440: [MINOR][SPARKR] Move 'Data type mapping between R and Sp...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17440 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17440: [MINOR][SPARKR] Move 'Data type mapping between R and Sp...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17440 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75253/ Test PASSed. ---

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108089769 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlanner.scala --- @@ -42,6 +43,8 @@ class SparkPlanner( InMemoryScans

[GitHub] spark issue #17440: [MINOR][SPARKR] Move 'Data type mapping between R and Sp...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17440 **[Test build #75253 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75253/testReport)** for PR 17440 at commit

[GitHub] spark issue #17440: [MINOR][SPARKR] Move 'Data type mapping between R and Sp...

2017-03-26 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/17440 cc @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #17440: [MINOR][SPARKR] Move 'Data type mapping between R...

2017-03-26 Thread yanboliang
GitHub user yanboliang opened a pull request: https://github.com/apache/spark/pull/17440 [MINOR][SPARKR] Move 'Data type mapping between R and Spark' to right place in SparkR doc. Section ```Data type mapping between R and Spark``` was put in the wrong place in SparkR doc

[GitHub] spark issue #17439: [SPARK-20105][TESTS][R] Add tests for checkType and type...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17439 **[Test build #75251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75251/testReport)** for PR 17439 at commit

[GitHub] spark issue #17439: [SPARK-20105][TESTS][R] Add tests for checkType and type...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17439 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75251/ Test PASSed. ---

[GitHub] spark issue #17439: [SPARK-20105][TESTS][R] Add tests for checkType and type...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17439 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Commands Usi...

2017-03-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17394 after this PR, what's the difference between `DESC EXTENDED` and `DESC FORMATTED`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #17439: [SPARK-20105][TESTS][R] Add tests for checkType and type...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17439 **[Test build #75249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75249/testReport)** for PR 17439 at commit

[GitHub] spark issue #17439: [SPARK-20105][TESTS][R] Add tests for checkType and type...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17439 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75249/ Test PASSed. ---

[GitHub] spark issue #17439: [SPARK-20105][TESTS][R] Add tests for checkType and type...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17439 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17406: [SPARK-20009][SQL] Use DDL strings for defining schema i...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17406 **[Test build #75252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75252/testReport)** for PR 17406 at commit

[GitHub] spark issue #17295: [SPARK-19556][core] Do not encrypt block manager data in...

2017-03-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17295 LGTM, cc @mallman to check the `unmap` part --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17406: [SPARK-20009][SQL] Use DDL strings for defining s...

2017-03-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17406#discussion_r108087790 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DataType.scala --- @@ -103,6 +104,12 @@ object DataType { def fromJson(json:

[GitHub] spark pull request #17295: [SPARK-19556][core] Do not encrypt block manager ...

2017-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17295#discussion_r108087419 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerManagedBuffer.scala --- @@ -17,31 +17,52 @@ package

[GitHub] spark issue #17242: [SPARK-19902][SQL] Add optimization rule to simplify exp...

2017-03-26 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17242 ping @cloud-fan @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17439: [SPARK-20105][R] Add tests for checkType and type string...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17439 **[Test build #75251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75251/testReport)** for PR 17439 at commit

[GitHub] spark pull request #17428: [SPARK-20094][SQL] Don't put predicate with IN su...

2017-03-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17428#discussion_r108086458 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -90,6 +90,7 @@ trait PredicateHelper { *

[GitHub] spark pull request #17432: [SPARK-20086][SQL] CollapseWindow should not coll...

2017-03-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17432#discussion_r108086367 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -597,12 +597,14 @@ object CollapseRepartition

[GitHub] spark issue #17439: [SPARK-20105][R] Add tests for checkType and type string...

2017-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17439 cc @felixcheung, this adds the tests for the current status. Probably, let me try to investigate and propose the changes about the type definition thing later. I guess the branch is going to be

[GitHub] spark issue #17438: [SPARK-20104][SQL] Don't estimate IsNull or IsNotNull pr...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17438 **[Test build #75250 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75250/testReport)** for PR 17438 at commit

[GitHub] spark issue #17439: [SPARK-20105][R] Add tests for checkType and type string...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17439 **[Test build #75249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75249/testReport)** for PR 17439 at commit

[GitHub] spark issue #17432: [SPARK-20086][SQL] CollapseWindow should not collapse de...

2017-03-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17432 Thank you! uh, I have another case with different data types. That is why I did not correctly reproduce it. The project is not removed due to data type mismatches. : ) ```Scala val

[GitHub] spark pull request #17439: [SPARK-20105][R] Add tests for checkType and type...

2017-03-26 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/17439 [SPARK-20105][R] Add tests for checkType and type string in structField in R ## What changes were proposed in this pull request? It seems `checkType` and the type string in

[GitHub] spark issue #17438: [SPARK-20104][SQL] Don't estimate IsNull or IsNotNull pr...

2017-03-26 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/17438 cc @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #17438: [SPARK-20104][SQL] Don't estimate IsNull or IsNot...

2017-03-26 Thread wzhfy
GitHub user wzhfy opened a pull request: https://github.com/apache/spark/pull/17438 [SPARK-20104][SQL] Don't estimate IsNull or IsNotNull predicates for non-leaf node ## What changes were proposed in this pull request? In current stage, we don't have advanced statistics

[GitHub] spark pull request #17406: [SPARK-20009][SQL] Use DDL strings for defining s...

2017-03-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17406#discussion_r108085204 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DataType.scala --- @@ -103,6 +104,12 @@ object DataType { def fromJson(json:

[GitHub] spark pull request #17406: [SPARK-20009][SQL] Use DDL strings for defining s...

2017-03-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17406#discussion_r108083206 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DataType.scala --- @@ -103,6 +104,12 @@ object DataType { def fromJson(json:

[GitHub] spark issue #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Commands Usi...

2017-03-26 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17394 Please update the PR description too since you update the title. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-03-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17394#discussion_r108082510 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -21,6 +21,7 @@ import java.net.URI import

[GitHub] spark pull request #17435: [SPARK-20098][PYSPARK] dataType's typeName fix

2017-03-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17435#discussion_r108082197 --- Diff: python/pyspark/sql/types.py --- @@ -57,7 +57,25 @@ def __ne__(self, other): @classmethod def typeName(cls): -

[GitHub] spark issue #17429: [MINOR][DOCS] Match several documentation changes in Sca...

2017-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17429 Thank you @felixcheung, IMO, I think it is fine. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r108080284 --- Diff: R/pkg/inst/tests/testthat/test_mllib_fpm.R --- @@ -0,0 +1,76 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r108080520 --- Diff: mllib/src/main/scala/org/apache/spark/ml/r/FPGrowthWrapper.scala --- @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r108080492 --- Diff: mllib/src/main/scala/org/apache/spark/ml/r/FPGrowthWrapper.scala --- @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r108079970 --- Diff: R/pkg/R/mllib_fpm.R --- @@ -0,0 +1,148 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request #17429: [MINOR][DOCS] Match several documentation changes...

2017-03-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17429 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17429: [MINOR][DOCS] Match several documentation changes in Sca...

2017-03-26 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17429 thanks, merged to master, since part of the fix in scala was in master only. if you think it should also be in branch-2.1, let me know. --- If your project is set up for it, you can reply to

[GitHub] spark pull request #17435: [SPARK-20098][PYSPARK] dataType's typeName fix

2017-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17435#discussion_r108079001 --- Diff: python/pyspark/sql/types.py --- @@ -57,7 +57,25 @@ def __ne__(self, other): @classmethod def typeName(cls): -

[GitHub] spark pull request #17437: [SPARK-20102] Fix nightly packaging and RC packag...

2017-03-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17437#discussion_r108078826 --- Diff: python/setup.py --- @@ -167,7 +167,6 @@ def _supports_symlinks(): 'pyspark.ml', 'pyspark.ml.linalg',

[GitHub] spark pull request #17435: [SPARK-20098][PYSPARK] dataType's typeName fix

2017-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17435#discussion_r108078597 --- Diff: python/pyspark/sql/types.py --- @@ -57,7 +57,25 @@ def __ne__(self, other): @classmethod def typeName(cls): -

[GitHub] spark issue #17428: [SPARK-20094][SQL] Don't put predicate with IN subquery ...

2017-03-26 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/17428 also cc @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #17429: [MINOR][DOCS] Match several documentation changes in Sca...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17429 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17164: [SPARK-16844][SQL] Support codegen for sort-based aggrea...

2017-03-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17164 okay, it'd be better to close this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17429: [MINOR][DOCS] Match several documentation changes in Sca...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17429 **[Test build #75248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75248/testReport)** for PR 17429 at commit

[GitHub] spark issue #17429: [MINOR][DOCS] Match several documentation changes in Sca...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17429 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75248/ Test PASSed. ---

[GitHub] spark issue #17437: [SPARK-20102] Fix nightly packaging and RC packaging scr...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17437 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75246/ Test PASSed. ---

[GitHub] spark issue #17437: [SPARK-20102] Fix nightly packaging and RC packaging scr...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17437 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17429: [MINOR][DOCS] Match several documentation changes in Sca...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17429 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75247/ Test PASSed. ---

[GitHub] spark issue #17437: [SPARK-20102] Fix nightly packaging and RC packaging scr...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17437 **[Test build #75246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75246/testReport)** for PR 17437 at commit

[GitHub] spark issue #17429: [MINOR][DOCS] Match several documentation changes in Sca...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17429 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17429: [MINOR][DOCS] Match several documentation changes in Sca...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17429 **[Test build #75247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75247/testReport)** for PR 17429 at commit

[GitHub] spark issue #17218: [SPARK-19281][PYTHON][ML] spark.ml Python API for FPGrow...

2017-03-26 Thread indyragandy
Github user indyragandy commented on the issue: https://github.com/apache/spark/pull/17218 Get updated --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #17218: [SPARK-19281][PYTHON][ML] spark.ml Python API for...

2017-03-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17218 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17218: [SPARK-19281][PYTHON][ML] spark.ml Python API for FPGrow...

2017-03-26 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/17218 @indyragandy What do you mean by "get directly"? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17218: [SPARK-19281][PYTHON][ML] spark.ml Python API for FPGrow...

2017-03-26 Thread indyragandy
Github user indyragandy commented on the issue: https://github.com/apache/spark/pull/17218 Get directly --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17218: [SPARK-19281][PYTHON][ML] spark.ml Python API for FPGrow...

2017-03-26 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/17218 LGTM Merging with master Thanks a lot! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17429: [MINOR][DOCS] Match several documentation changes in Sca...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17429 **[Test build #75248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75248/testReport)** for PR 17429 at commit

[GitHub] spark issue #12823: [SPARK-14985][ML] Update LinearRegression, LogisticRegre...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12823 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12823: [SPARK-14985][ML] Update LinearRegression, LogisticRegre...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12823 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75245/ Test PASSed. ---

[GitHub] spark issue #12823: [SPARK-14985][ML] Update LinearRegression, LogisticRegre...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12823 **[Test build #75245 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75245/testReport)** for PR 12823 at commit

[GitHub] spark issue #17429: [MINOR][DOCS] Match several documentation changes in Sca...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17429 **[Test build #75247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75247/testReport)** for PR 17429 at commit

[GitHub] spark pull request #17432: [SPARK-20086][SQL] CollapseWindow should not coll...

2017-03-26 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17432#discussion_r108074254 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -597,12 +597,14 @@ object CollapseRepartition

[GitHub] spark issue #17432: [SPARK-20086][SQL] CollapseWindow should not collapse de...

2017-03-26 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17432 The project will get eliminated by the optimizer. From the JIRA ticket: ```scala import org.apache.spark.sql.expressions.Window val win_spec_max =

[GitHub] spark issue #17432: [SPARK-20086][SQL] CollapseWindow should not collapse de...

2017-03-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17432 @hvanhovell Could you share a Scala end-to-end test case for reproducing the error? If using `withColumn`, we always insert a `Project` between two `Window`s. Not sure why the users hit this

[GitHub] spark pull request #17432: [SPARK-20086][SQL] CollapseWindow should not coll...

2017-03-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17432#discussion_r108072266 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -597,12 +597,14 @@ object CollapseRepartition

[GitHub] spark pull request #17437: [SPARK-20102] Fix nightly snapshot and RC packagi...

2017-03-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/17437#discussion_r108071973 --- Diff: python/setup.py --- @@ -167,7 +167,6 @@ def _supports_symlinks(): 'pyspark.ml',

[GitHub] spark issue #17437: [SPARK-20102] Fix nightly snapshot and RC packaging scri...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17437 **[Test build #75246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75246/testReport)** for PR 17437 at commit

[GitHub] spark issue #17437: [SPARK-20102] Fix nightly snapshot and RC packaging scri...

2017-03-26 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/17437 /cc @marmbrus @holdenk, this patch should fix the build issues blocking Spark 2.1.1 RC1 builds. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #17437: [SPARK-20102] Fix nightly snapshot and RC packagi...

2017-03-26 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/17437 [SPARK-20102] Fix nightly snapshot and RC packaging scripts w/ two minor build fixes ## What changes were proposed in this pull request? The master snapshot publisher builds are

[GitHub] spark issue #17423: [SPARK-20088] Do not create new SparkContext in SparkR c...

2017-03-26 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/17423 got it. Thanks :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17432: [SPARK-20086][SQL] CollapseWindow should not collapse de...

2017-03-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17432 Thank you for fixing this! It seems that this should be included in RC2. When will be cut for RC2? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #17432: [SPARK-20086][SQL] CollapseWindow should not coll...

2017-03-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17432 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17432: [SPARK-20086][SQL] CollapseWindow should not collapse de...

2017-03-26 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17432 Thanks for the review. Merging this to master/2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #12823: [SPARK-14985][ML] Update LinearRegression, LogisticRegre...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12823 **[Test build #75245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75245/testReport)** for PR 12823 at commit

[GitHub] spark issue #12823: [SPARK-14985][ML] Update LinearRegression, LogisticRegre...

2017-03-26 Thread BenFradet
Github user BenFradet commented on the issue: https://github.com/apache/spark/pull/12823 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #12823: [SPARK-14985][ML] Update LinearRegression, LogisticRegre...

2017-03-26 Thread BenFradet
Github user BenFradet commented on the issue: https://github.com/apache/spark/pull/12823 `org.apache.spark.storage.BlockManagerProactiveReplicationSuite.proactive block replication - 3 replicas - 2 block manager deletions` doesn't seem to be linked, relaunching the tests. --- If

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17436 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17436 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75244/ Test FAILed. ---

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17436 **[Test build #75244 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75244/testReport)** for PR 17436 at commit

[GitHub] spark issue #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Commands Usi...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17394 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75243/ Test PASSed. ---

[GitHub] spark issue #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Commands Usi...

2017-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17394 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

  1   2   3   >