[GitHub] spark issue #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) should ...

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18852 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80292/ Test PASSed. ---

[GitHub] spark issue #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) should ...

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18852 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18769: [SPARK-21574][SQL] Point out user to set hive con...

2017-08-05 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/18769#discussion_r131534966 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/SetCommand.scala --- @@ -87,6 +88,13 @@ case class SetCommand(kv: Option[(String,

[GitHub] spark issue #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) should ...

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18852 **[Test build #80292 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80292/testReport)** for PR 18852 at commit

[GitHub] spark pull request #18769: [SPARK-21574][SQL] Point out user to set hive con...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18769#discussion_r131534846 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/SetCommand.scala --- @@ -87,6 +88,13 @@ case class SetCommand(kv:

[GitHub] spark issue #18856: [SPARKR][BUILD] AppVeyor change to latest R version

2017-08-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18856 To cut this short, it was ... - 3.3.2 release for Linux only - Gives the latest as 3.3.2 and the download link for 3.3.1 becomes `windows/base/old` - 3.3.2 release for WIndows

[GitHub] spark issue #18856: [SPARKR][BUILD] AppVeyor change to latest R version

2017-08-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18856 Ah, I meant, the PR I linked actually describes a case when we were using 3.3.1 but it was broken after 3.3.2 release. The reason was, 3.3.2 was released but 3.3.2 for Windows was not synced

[GitHub] spark issue #18856: [SPARKR][BUILD] AppVeyor change to latest R version

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18856 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80291/ Test PASSed. ---

[GitHub] spark issue #18856: [SPARKR][BUILD] AppVeyor change to latest R version

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18856 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18856: [SPARKR][BUILD] AppVeyor change to latest R version

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18856 **[Test build #80291 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80291/testReport)** for PR 18856 at commit

[GitHub] spark issue #18856: [SPARKR][BUILD] AppVeyor change to latest R version

2017-08-05 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18856 It's the latest but it's not new - 3.4.1 was release a month ago. I think there shouldn't be the sync problem. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18576 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80293/ Test FAILed. ---

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18576 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18576 **[Test build #80293 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80293/testReport)** for PR 18576 at commit

[GitHub] spark issue #18856: [SPARKR][BUILD] AppVeyor change to latest R version

2017-08-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18856 @felixcheung, BTW, I think optionally we could also consider setting it 3.4.0 a bit more conservatively for now. There was a rather minor problem for using the latest version (see

[GitHub] spark issue #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) should ...

2017-08-05 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18852 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) ...

2017-08-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18852#discussion_r131534323 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -270,4 +270,15 @@ class SQLConfSuite extends QueryTest with

[GitHub] spark pull request #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) ...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18852#discussion_r131534296 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -270,4 +270,15 @@ class SQLConfSuite extends QueryTest with

[GitHub] spark pull request #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) ...

2017-08-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18852#discussion_r131534275 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -270,4 +270,15 @@ class SQLConfSuite extends QueryTest with

[GitHub] spark issue #18724: [SPARK-21519][SQL] Add an option to the JDBC data source...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18724 ping @LucaCanali --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) ...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18852#discussion_r131534246 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -270,4 +270,15 @@ class SQLConfSuite extends QueryTest with

[GitHub] spark pull request #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) ...

2017-08-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18852#discussion_r131534235 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -270,4 +270,15 @@ class SQLConfSuite extends QueryTest with

[GitHub] spark pull request #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) ...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18852#discussion_r131534222 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -270,4 +270,15 @@ class SQLConfSuite extends QueryTest with

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r131534214 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -502,69 +503,311 @@ case class

[GitHub] spark pull request #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) ...

2017-08-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18852#discussion_r131534217 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -270,4 +270,15 @@ class SQLConfSuite extends QueryTest with

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r131534204 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1121,6 +1125,30 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r131534200 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1108,7 +1108,11 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r131534165 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2658,4 +2659,23 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r131534145 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1121,6 +1125,30 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) ...

2017-08-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18852#discussion_r131534140 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -270,4 +270,15 @@ class SQLConfSuite extends QueryTest with

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r131534107 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1121,6 +1125,30 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r131534098 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1121,6 +1125,30 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #18831: [SPARK-21622][ML][SparkR] Support offset in SparkR GLM

2017-08-05 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/18831 Thanks both for the comments. Yes, I think it's be to keep this PR on offset and we can address the other improvements later. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #18857: [SPARK-20963][SQL][FOLLOW-UP] Use UnresolvedSubqueryColu...

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18857 **[Test build #80294 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80294/testReport)** for PR 18857 at commit

[GitHub] spark pull request #18857: [SPARK-20963][SQL][FOLLOW-UP] Use UnresolvedSubqu...

2017-08-05 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/18857 [SPARK-20963][SQL][FOLLOW-UP] Use UnresolvedSubqueryColumnAliases for visitTableName ## What changes were proposed in this pull request? This pr (follow-up of #18772) used

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r131533855 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2304,7 +2304,15 @@ object functions { * @group string_funcs

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18576 **[Test build #80293 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80293/testReport)** for PR 18576 at commit

[GitHub] spark pull request #18772: [SPARK-20963][SQL] Support column aliases for joi...

2017-08-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18772 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18772: [SPARK-20963][SQL] Support column aliases for join relat...

2017-08-05 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18772 Thanks! I'll make a pr later to fix [this](https://github.com/apache/spark/pull/18772#issuecomment-318967315) as follow-up. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #18772: [SPARK-20963][SQL] Support column aliases for join relat...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18772 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) should ...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18852 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18790: [SPARK-21587][SS] Added pushdown through watermar...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18790#discussion_r131533778 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -872,6 +886,25 @@ object PushDownPredicate extends

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-08-05 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18576 Yea, if we can do so, I feel it might be the best. I'll check if we can remove nullability update in `FilterExec`. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) should ...

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18852 **[Test build #80292 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80292/testReport)** for PR 18852 at commit

[GitHub] spark issue #18772: [SPARK-20963][SQL] Support column aliases for join relat...

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18772 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18772: [SPARK-20963][SQL] Support column aliases for join relat...

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18772 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80290/ Test PASSed. ---

[GitHub] spark issue #18772: [SPARK-20963][SQL] Support column aliases for join relat...

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18772 **[Test build #80290 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80290/testReport)** for PR 18772 at commit

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18576 If we can update the nullability in Optimizer rules, do we still need to do it in `FilterExec`? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #18856: [SPARKR][BUILD] AppVeyor change to latest R version

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18856 **[Test build #80291 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80291/testReport)** for PR 18856 at commit

[GitHub] spark pull request #18856: [SPARKR][BUILD] AppVeyor change to latest R versi...

2017-08-05 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/18856 [SPARKR][BUILD] AppVeyor change to latest R version ## What changes were proposed in this pull request? R version update ## How was this patch tested? AppVeyor You

[GitHub] spark pull request #18854: [SPARK-21629][SQL][WIP] Fix Or nullability

2017-08-05 Thread goldmedal
Github user goldmedal closed the pull request at: https://github.com/apache/spark/pull/18854 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18854: [SPARK-21629][SQL][WIP] Fix Or nullability

2017-08-05 Thread goldmedal
Github user goldmedal commented on the issue: https://github.com/apache/spark/pull/18854 @gatorsmile @viirya Thanks a lot. You are right. I close it for now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-08-05 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18576 @gatorsmile If you get time, could you also check this? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #18772: [SPARK-20963][SQL] Support column aliases for join relat...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18772 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18841: [SPARK-21635][SQL] ACOS(2) and ASIN(2) should be ...

2017-08-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18841#discussion_r131532373 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -170,29 +193,29 @@ case class Pi() extends

[GitHub] spark pull request #18841: [SPARK-21635][SQL] ACOS(2) and ASIN(2) should be ...

2017-08-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18841#discussion_r131532351 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -170,29 +193,29 @@ case class Pi() extends

[GitHub] spark issue #18854: [SPARK-21629][SQL][WIP] Fix Or nullability

2017-08-05 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18854 Ok. I think this should not be a problem. We should close this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #18854: [SPARK-21629][SQL][WIP] Fix Or nullability

2017-08-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18854#discussion_r131532208 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -396,6 +396,8 @@ case class Or(left: Expression,

[GitHub] spark issue #18772: [SPARK-20963][SQL] Support column aliases for join relat...

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18772 **[Test build #80290 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80290/testReport)** for PR 18772 at commit

[GitHub] spark issue #18772: [SPARK-20963][SQL] Support column aliases for join relat...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18772 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Tim...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18664#discussion_r131532115 --- Diff: python/pyspark/sql/tests.py --- @@ -3036,6 +3052,9 @@ def test_toPandas_arrow_toggle(self): pdf = df.toPandas()

[GitHub] spark pull request #18668: [SPARK-21637][SPARK-21451][SQL]get `spark.hadoop....

2017-08-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18668 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18668: [SPARK-21637][SPARK-21451][SQL]get `spark.hadoop.*` prop...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18668 Thanks everyone! Merging it to master. If any other comment, we can address it in the follow-up PRs. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #18854: [SPARK-21629][SQL][WIP] Fix Or nullability

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18854#discussion_r131532024 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -396,6 +396,8 @@ case class Or(left:

[GitHub] spark pull request #18854: [SPARK-21629][SQL][WIP] Fix Or nullability

2017-08-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18854#discussion_r131531879 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -396,6 +396,8 @@ case class Or(left: Expression,

[GitHub] spark pull request #18854: [SPARK-21629][SQL][WIP] Fix Or nullability

2017-08-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18854#discussion_r131531690 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -396,6 +396,8 @@ case class Or(left: Expression,

[GitHub] spark pull request #18815: [SPARK-21609][WEB-UI]In the Master ui add "log di...

2017-08-05 Thread guoxiaolongzte
Github user guoxiaolongzte closed the pull request at: https://github.com/apache/spark/pull/18815 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #18829: [SPARK-21620][WEB-UI][CORE]Add metrics url in spa...

2017-08-05 Thread guoxiaolongzte
Github user guoxiaolongzte closed the pull request at: https://github.com/apache/spark/pull/18829 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #18854: [SPARK-21629][SQL][WIP] Fix Or nullability

2017-08-05 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18854#discussion_r131530967 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -396,6 +396,8 @@ case class Or(left:

[GitHub] spark pull request #18851: [SPARK-21644][SQL] LocalLimit.maxRows is defined ...

2017-08-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18851#discussion_r131530034 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -643,6 +657,27 @@ case class

[GitHub] spark issue #18855: [Spark 3151][Block Manager] DiskStore.getBytes fails for...

2017-08-05 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/18855 @rxin, @JoshRosen , @cloud-fan , you seem to be the last guys to touch this class, can you please review? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #18855: [Spark 3151][Block Manager] DiskStore.getBytes fails for...

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18855 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18763: [SPARK-21306][ML] For branch 2.1, OneVsRest shoul...

2017-08-05 Thread facaiy
Github user facaiy commented on a diff in the pull request: https://github.com/apache/spark/pull/18763#discussion_r131529768 --- Diff: python/pyspark/ml/classification.py --- @@ -1423,7 +1425,18 @@ def _fit(self, dataset): numClasses = int(dataset.agg({labelCol:

[GitHub] spark pull request #18855: [Spark 3151][Block Manager] DiskStore.getBytes fa...

2017-08-05 Thread eyalfa
GitHub user eyalfa opened a pull request: https://github.com/apache/spark/pull/18855 [Spark 3151][Block Manager] DiskStore.getBytes fails for files larger than 2GB ## What changes were proposed in this pull request? introduced `DiskBlockData`, a new implementation of

[GitHub] spark pull request #18764: [SPARK-21306][ML] For branch 2.0, OneVsRest shoul...

2017-08-05 Thread facaiy
Github user facaiy commented on a diff in the pull request: https://github.com/apache/spark/pull/18764#discussion_r131529693 --- Diff: python/pyspark/ml/classification.py --- @@ -1344,7 +1346,19 @@ def _fit(self, dataset): numClasses = int(dataset.agg({labelCol:

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80289/ Test PASSed. ---

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18790 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18790 **[Test build #80289 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80289/testReport)** for PR 18790 at commit

[GitHub] spark pull request #18828: [SPARK-21619][SQL] Fail the execution of canonica...

2017-08-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18828#discussion_r131529489 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SparkPlanSuite.scala --- @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #18668: [SPARK-21637][SPARK-21451][SQL]get `spark.hadoop.*` prop...

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18668 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18668: [SPARK-21637][SPARK-21451][SQL]get `spark.hadoop.*` prop...

2017-08-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18668 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80288/ Test PASSed. ---

[GitHub] spark issue #18668: [SPARK-21637][SPARK-21451][SQL]get `spark.hadoop.*` prop...

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18668 **[Test build #80288 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80288/testReport)** for PR 18668 at commit

[GitHub] spark issue #18460: [SPARK-21247][SQL] Type comparision should respect case-...

2017-08-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18460 Hi, @cloud-fan and @gatorsmile . Could you review this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18790 **[Test build #80289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80289/testReport)** for PR 18790 at commit

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18790 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18668: [SPARK-21637][SPARK-21451][SQL]get `spark.hadoop.*` prop...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18668 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18844: [SPARK-21640] Add errorifexists as a valid string...

2017-08-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18844 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18844: [SPARK-21640] Add errorifexists as a valid string for Er...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18844 Thanks! Merging to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) should ...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18852 LGTM except a comment in test cases. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18852: [SPARK-21588][SQL] SQLContext.getConf(key, null) ...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18852#discussion_r131527017 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -270,4 +270,10 @@ class SQLConfSuite extends QueryTest with

[GitHub] spark pull request #18854: [SPARK-21629][SQL][WIP] Fix Or nullability

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18854#discussion_r131526168 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -396,6 +396,8 @@ case class Or(left:

[GitHub] spark issue #18828: [SPARK-21619][SQL] Fail the execution of canonicalized p...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18828 There is another zero argument [ResetCommand](https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/command/SetCommand.scala#L155) --- If your

[GitHub] spark pull request #18749: [SPARK-21485][FOLLOWUP][SQL][DOCS] Describes exam...

2017-08-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18749 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18749: [SPARK-21485][FOLLOWUP][SQL][DOCS] Describes examples an...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18749 LGTM. Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18668: [SPARK-21637][SPARK-21451][SQL]get `spark.hadoop.*` prop...

2017-08-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18668 **[Test build #80288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80288/testReport)** for PR 18668 at commit

[GitHub] spark issue #18668: [SPARK-21637][SPARK-21451][SQL]get `spark.hadoop.*` prop...

2017-08-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18668 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18831: [SPARK-21622][ML][SparkR] Support offset in SparkR GLM

2017-08-05 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18831 @felixcheung Sorry for misunderstand, I agree we can support ```df$myoffset``` as well, the requirement make sense for R users. Let's create a separate JIRA to track it and do this change for

[GitHub] spark pull request #18841: [SPARK-21635][SQL] ACOS(2) and ASIN(2) should be ...

2017-08-05 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/18841#discussion_r131525404 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -170,29 +193,29 @@ case class Pi() extends

[GitHub] spark issue #18831: [SPARK-21622][ML][SparkR] Support offset in SparkR GLM

2017-08-05 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18831 To be clear, I'm not suggesting to rename the parameter. I'm suggest we should support the type being passed in as column like df$myoffset in addition to it being a string. This will be more R

[GitHub] spark pull request #18797: [SPARK-21523][ML] update breeze to 0.13.2 for an ...

2017-08-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/18797#discussion_r131524755 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/optimization/LBFGSSuite.scala --- @@ -191,8 +191,8 @@ class LBFGSSuite extends SparkFunSuite with

  1   2   3   >