[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2021-02-22 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-783179700 **[Test build #135327 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135327/testReport)** for PR 29695 at commit [`e9e984e`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2021-02-22 Thread GitBox
SparkQA removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-783054252 **[Test build #135327 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135327/testReport)** for PR 29695 at commit [`e9e984e`](https://gi

[GitHub] [spark] wangyum commented on pull request #31609: [SPARK-34496][BUILD] Upgrade ZSTD-JNI to 1.4.8-5 for better API compatibility

2021-02-22 Thread GitBox
wangyum commented on pull request #31609: URL: https://github.com/apache/spark/pull/31609#issuecomment-783181373 Thank you @dongjoon-hyun It still has compatibility issue: ``` Job aborted. org.apache.spark.SparkException: Job aborted. at org.apache.spark.sql.execution.datas

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31607: [SPARK-34495][TESTS] Add `DedicatedJVMTest` test tag

2021-02-22 Thread GitBox
AmplabJenkins removed a comment on pull request #31607: URL: https://github.com/apache/spark/pull/31607#issuecomment-783182556 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135332/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31607: [SPARK-34495][TESTS] Add `DedicatedJVMTest` test tag

2021-02-22 Thread GitBox
AmplabJenkins commented on pull request #31607: URL: https://github.com/apache/spark/pull/31607#issuecomment-783182556 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135332/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31575: [SPARK-34450][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. RENAME tests

2021-02-22 Thread GitBox
AmplabJenkins removed a comment on pull request #31575: URL: https://github.com/apache/spark/pull/31575#issuecomment-783182558 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39915/

[GitHub] [spark] AmplabJenkins commented on pull request #31596: [SPARK-34475][SQL] Rename logical nodes of v2 `ALTER` commands

2021-02-22 Thread GitBox
AmplabJenkins commented on pull request #31596: URL: https://github.com/apache/spark/pull/31596#issuecomment-783182555 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39914/ -

[GitHub] [spark] AmplabJenkins commented on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2021-02-22 Thread GitBox
AmplabJenkins commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-783182559 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #31575: [SPARK-34450][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. RENAME tests

2021-02-22 Thread GitBox
AmplabJenkins commented on pull request #31575: URL: https://github.com/apache/spark/pull/31575#issuecomment-783182558 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39915/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31596: [SPARK-34475][SQL] Rename logical nodes of v2 `ALTER` commands

2021-02-22 Thread GitBox
AmplabJenkins removed a comment on pull request #31596: URL: https://github.com/apache/spark/pull/31596#issuecomment-783182555 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39914/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2021-02-22 Thread GitBox
AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-783182557 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31608: [MINOR][DOCS] Add table_identifier in sql-migration-guide for SHOW CREATE TABLE

2021-02-22 Thread GitBox
AmplabJenkins removed a comment on pull request #31608: URL: https://github.com/apache/spark/pull/31608#issuecomment-783156378 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135336/ -

[GitHub] [spark] SparkQA commented on pull request #31608: [MINOR][DOCS] Add table_identifier in sql-migration-guide for SHOW CREATE TABLE

2021-02-22 Thread GitBox
SparkQA commented on pull request #31608: URL: https://github.com/apache/spark/pull/31608#issuecomment-783183556 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39916/ -

[GitHub] [spark] viirya commented on pull request #31476: [SPARK-34366][SQL] Add interface for DS v2 metrics

2021-02-22 Thread GitBox
viirya commented on pull request #31476: URL: https://github.com/apache/spark/pull/31476#issuecomment-783184108 @cloud-fan @rdblue @Ngone51 @sunchao @zsxwing If you have more comments, please let me know. Thanks. This is an

[GitHub] [spark] SparkQA commented on pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

2021-02-22 Thread GitBox
SparkQA commented on pull request #31349: URL: https://github.com/apache/spark/pull/31349#issuecomment-783189497 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39917/ -

[GitHub] [spark] SparkQA commented on pull request #31607: [SPARK-34495][TESTS] Add `DedicatedJVMTest` test tag

2021-02-22 Thread GitBox
SparkQA commented on pull request #31607: URL: https://github.com/apache/spark/pull/31607#issuecomment-783193229 **[Test build #135333 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135333/testReport)** for PR 31607 at commit [`1127bcc`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31607: [SPARK-34495][TESTS] Add `DedicatedJVMTest` test tag

2021-02-22 Thread GitBox
SparkQA removed a comment on pull request #31607: URL: https://github.com/apache/spark/pull/31607#issuecomment-783127167 **[Test build #135333 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135333/testReport)** for PR 31607 at commit [`1127bcc`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31596: [SPARK-34475][SQL] Rename logical nodes of v2 `ALTER` commands

2021-02-22 Thread GitBox
SparkQA commented on pull request #31596: URL: https://github.com/apache/spark/pull/31596#issuecomment-783195866 **[Test build #135339 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135339/testReport)** for PR 31596 at commit [`f6ab9ab`](https://github.com

[GitHub] [spark] cloud-fan commented on pull request #31575: [SPARK-34450][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. RENAME tests

2021-02-22 Thread GitBox
cloud-fan commented on pull request #31575: URL: https://github.com/apache/spark/pull/31575#issuecomment-783195964 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] SparkQA commented on pull request #31609: [SPARK-34496][BUILD] Upgrade ZSTD-JNI to 1.4.8-5 for better API compatibility

2021-02-22 Thread GitBox
SparkQA commented on pull request #31609: URL: https://github.com/apache/spark/pull/31609#issuecomment-783195965 **[Test build #135338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135338/testReport)** for PR 31609 at commit [`31d2389`](https://github.com

[GitHub] [spark] cloud-fan closed pull request #31575: [SPARK-34450][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. RENAME tests

2021-02-22 Thread GitBox
cloud-fan closed pull request #31575: URL: https://github.com/apache/spark/pull/31575 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] cloud-fan commented on pull request #31596: [SPARK-34475][SQL] Rename logical nodes of v2 `ALTER` commands

2021-02-22 Thread GitBox
cloud-fan commented on pull request #31596: URL: https://github.com/apache/spark/pull/31596#issuecomment-783197241 RC3 is cut and I don't think we should do this refactor in 3.1.2 (assuming RC3 passes). This is an automated

[GitHub] [spark] SparkQA commented on pull request #31593: [SPARK-34473][SQL] Avoid NPE in DataFrameReader.schema(StructType)

2021-02-22 Thread GitBox
SparkQA commented on pull request #31593: URL: https://github.com/apache/spark/pull/31593#issuecomment-783200209 **[Test build #135330 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135330/testReport)** for PR 31593 at commit [`8a38f52`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31593: [SPARK-34473][SQL] Avoid NPE in DataFrameReader.schema(StructType)

2021-02-22 Thread GitBox
SparkQA removed a comment on pull request #31593: URL: https://github.com/apache/spark/pull/31593#issuecomment-783072520 **[Test build #135330 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135330/testReport)** for PR 31593 at commit [`8a38f52`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

2021-02-22 Thread GitBox
SparkQA commented on pull request #31349: URL: https://github.com/apache/spark/pull/31349#issuecomment-783202135 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39917/ ---

[GitHub] [spark] yaooqinn commented on a change in pull request #31593: [SPARK-34473][SQL] Avoid NPE in DataFrameReader.schema(StructType)

2021-02-22 Thread GitBox
yaooqinn commented on a change in pull request #31593: URL: https://github.com/apache/spark/pull/31593#discussion_r580071141 ## File path: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala ## @@ -64,8 +64,10 @@ final class DataStreamReader private[s

[GitHub] [spark] SparkQA commented on pull request #31608: [MINOR][DOCS] Add table_identifier in sql-migration-guide for SHOW CREATE TABLE

2021-02-22 Thread GitBox
SparkQA commented on pull request #31608: URL: https://github.com/apache/spark/pull/31608#issuecomment-783209378 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39916/ ---

[GitHub] [spark] gengliangwang commented on pull request #31390: [SPARK-28123][SQL] String Functions: support btrim

2021-02-22 Thread GitBox
gengliangwang commented on pull request #31390: URL: https://github.com/apache/spark/pull/31390#issuecomment-783212402 > I created #31080 before this PR to implement the behavior that is returns Binary. But that was a mistake. I don't think it's a total mistake. Is there any existing

[GitHub] [spark] SparkQA commented on pull request #31596: [SPARK-34475][SQL] Rename logical nodes of v2 `ALTER` commands

2021-02-22 Thread GitBox
SparkQA commented on pull request #31596: URL: https://github.com/apache/spark/pull/31596#issuecomment-783213224 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39919/ -

[GitHub] [spark] SparkQA commented on pull request #31607: [SPARK-34495][TESTS] Add `DedicatedJVMTest` test tag

2021-02-22 Thread GitBox
SparkQA commented on pull request #31607: URL: https://github.com/apache/spark/pull/31607#issuecomment-783215663 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39913/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

2021-02-22 Thread GitBox
AmplabJenkins commented on pull request #31349: URL: https://github.com/apache/spark/pull/31349#issuecomment-783223052 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39917/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31593: [SPARK-34473][SQL] Avoid NPE in DataFrameReader.schema(StructType)

2021-02-22 Thread GitBox
AmplabJenkins commented on pull request #31593: URL: https://github.com/apache/spark/pull/31593#issuecomment-783223056 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135330/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31607: [SPARK-34495][TESTS] Add `DedicatedJVMTest` test tag

2021-02-22 Thread GitBox
AmplabJenkins commented on pull request #31607: URL: https://github.com/apache/spark/pull/31607#issuecomment-783223048 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #31608: [MINOR][DOCS] Add table_identifier in sql-migration-guide for SHOW CREATE TABLE

2021-02-22 Thread GitBox
AmplabJenkins commented on pull request #31608: URL: https://github.com/apache/spark/pull/31608#issuecomment-783223050 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39916/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31607: [SPARK-34495][TESTS] Add `DedicatedJVMTest` test tag

2021-02-22 Thread GitBox
AmplabJenkins removed a comment on pull request #31607: URL: https://github.com/apache/spark/pull/31607#issuecomment-783223046 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31593: [SPARK-34473][SQL] Avoid NPE in DataFrameReader.schema(StructType)

2021-02-22 Thread GitBox
AmplabJenkins removed a comment on pull request #31593: URL: https://github.com/apache/spark/pull/31593#issuecomment-783223056 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135330/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31608: [MINOR][DOCS] Add table_identifier in sql-migration-guide for SHOW CREATE TABLE

2021-02-22 Thread GitBox
AmplabJenkins removed a comment on pull request #31608: URL: https://github.com/apache/spark/pull/31608#issuecomment-783223050 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39916/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

2021-02-22 Thread GitBox
AmplabJenkins removed a comment on pull request #31349: URL: https://github.com/apache/spark/pull/31349#issuecomment-783223052 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39917/

[GitHub] [spark] beliefer commented on pull request #31390: [SPARK-28123][SQL] String Functions: support btrim

2021-02-22 Thread GitBox
beliefer commented on pull request #31390: URL: https://github.com/apache/spark/pull/31390#issuecomment-783224753 @gengliangwang Regardless of the data type, the databases listed above have consistent behavior trim from leading and trailing. Although `btrim(Binary) => Binary` in PostgreS

[GitHub] [spark] cloud-fan commented on pull request #31390: [SPARK-28123][SQL] String Functions: support btrim

2021-02-22 Thread GitBox
cloud-fan commented on pull request #31390: URL: https://github.com/apache/spark/pull/31390#issuecomment-783226778 does any database do `trim(binary) => string` except Spark? This is an automated message from the Apache Git S

[GitHub] [spark] SparkQA commented on pull request #31593: [SPARK-34473][SQL] Avoid NPE in DataFrameReader.schema(StructType)

2021-02-22 Thread GitBox
SparkQA commented on pull request #31593: URL: https://github.com/apache/spark/pull/31593#issuecomment-783226947 **[Test build #135341 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135341/testReport)** for PR 31593 at commit [`643f002`](https://github.com

[GitHub] [spark] cloud-fan commented on a change in pull request #31606: [SPARK-34490][SQL] Analysis should fail if the view refers a dropped table

2021-02-22 Thread GitBox
cloud-fan commented on a change in pull request #31606: URL: https://github.com/apache/spark/pull/31606#discussion_r580093243 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1190,6 +1193,7 @@ class Analyzer(override val ca

[GitHub] [spark] cloud-fan commented on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-22 Thread GitBox
cloud-fan commented on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-783231459 > Replace ' syntax with $"" or col() in examples and documents. +1. This can be done first. > Deprecate APIs which take Symbol type parameters. The API does

[GitHub] [spark] beliefer commented on pull request #31390: [SPARK-28123][SQL] String Functions: support btrim

2021-02-22 Thread GitBox
beliefer commented on pull request #31390: URL: https://github.com/apache/spark/pull/31390#issuecomment-783233306 @cloud-fan @gengliangwang There not any database do `trim(binary) => string`, so I think should remove the `ImplicitCastInputTypes` from `String2TrimExpression`. -

[GitHub] [spark] SparkQA commented on pull request #31596: [SPARK-34475][SQL] Rename logical nodes of v2 `ALTER` commands

2021-02-22 Thread GitBox
SparkQA commented on pull request #31596: URL: https://github.com/apache/spark/pull/31596#issuecomment-783233410 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39919/ ---

[GitHub] [spark] beliefer edited a comment on pull request #31390: [SPARK-28123][SQL] String Functions: support btrim

2021-02-22 Thread GitBox
beliefer edited a comment on pull request #31390: URL: https://github.com/apache/spark/pull/31390#issuecomment-783233306 @cloud-fan @gengliangwang There not any database do `trim(binary) => string`, so I think should remove the `ImplicitCastInputTypes` from `String2TrimExpression`. I

[GitHub] [spark] cloud-fan commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-22 Thread GitBox
cloud-fan commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r580100233 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,371 @@ +/* + * Licensed to the

[GitHub] [spark] SparkQA commented on pull request #31448: [SPARK-28137][SQL] Data Type Formatting Functions: `to_number`.

2021-02-22 Thread GitBox
SparkQA commented on pull request #31448: URL: https://github.com/apache/spark/pull/31448#issuecomment-783234291 **[Test build #135331 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135331/testReport)** for PR 31448 at commit [`c4b2ce6`](https://github.co

[GitHub] [spark] beliefer edited a comment on pull request #31390: [SPARK-28123][SQL] String Functions: support btrim

2021-02-22 Thread GitBox
beliefer edited a comment on pull request #31390: URL: https://github.com/apache/spark/pull/31390#issuecomment-783233306 @cloud-fan @gengliangwang There not any database do `trim(binary) => string`, so I think should remove the `ImplicitCastInputTypes` from `String2TrimExpression`. I

[GitHub] [spark] SparkQA removed a comment on pull request #31448: [SPARK-28137][SQL] Data Type Formatting Functions: `to_number`.

2021-02-22 Thread GitBox
SparkQA removed a comment on pull request #31448: URL: https://github.com/apache/spark/pull/31448#issuecomment-783075049 **[Test build #135331 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135331/testReport)** for PR 31448 at commit [`c4b2ce6`](https://gi

[GitHub] [spark] beliefer commented on pull request #31448: [SPARK-28137][SQL] Data Type Formatting Functions: `to_number`.

2021-02-22 Thread GitBox
beliefer commented on pull request #31448: URL: https://github.com/apache/spark/pull/31448#issuecomment-783234675 ping @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [spark] beliefer edited a comment on pull request #31390: [SPARK-28123][SQL] String Functions: support btrim

2021-02-22 Thread GitBox
beliefer edited a comment on pull request #31390: URL: https://github.com/apache/spark/pull/31390#issuecomment-783233306 @cloud-fan @gengliangwang There not any database do `trim(binary) => string`, so I think should remove the `ImplicitCastInputTypes` from `String2TrimExpression`. I

[GitHub] [spark] cloud-fan commented on a change in pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-22 Thread GitBox
cloud-fan commented on a change in pull request #31560: URL: https://github.com/apache/spark/pull/31560#discussion_r580101743 ## File path: sql/core/src/test/java/test/org/apache/spark/sql/connector/JavaSimpleWritableDataSource.java ## @@ -0,0 +1,371 @@ +/* + * Licensed to the

[GitHub] [spark] beliefer edited a comment on pull request #31390: [SPARK-28123][SQL] String Functions: support btrim

2021-02-22 Thread GitBox
beliefer edited a comment on pull request #31390: URL: https://github.com/apache/spark/pull/31390#issuecomment-783233306 @cloud-fan @gengliangwang There not any database do `trim(binary) => string`, so I think should remove the `ImplicitCastInputTypes` from `String2TrimExpression`. I

[GitHub] [spark] cloud-fan commented on pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-22 Thread GitBox
cloud-fan commented on pull request #31560: URL: https://github.com/apache/spark/pull/31560#issuecomment-783238244 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] c21 commented on a change in pull request #31318: [SPARK-34222][SQL] Enhance boolean simplification rule

2021-02-22 Thread GitBox
c21 commented on a change in pull request #31318: URL: https://github.com/apache/spark/pull/31318#discussion_r580103906 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala ## @@ -366,22 +367,34 @@ object BooleanSimplification ext

[GitHub] [spark] cloud-fan closed pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-22 Thread GitBox
cloud-fan closed pull request #31560: URL: https://github.com/apache/spark/pull/31560 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] cloud-fan commented on pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-22 Thread GitBox
cloud-fan commented on pull request #31560: URL: https://github.com/apache/spark/pull/31560#issuecomment-783239162 @kevincmchen can you open a new PR to add `throws` clause to `planInputPartitions`, `commit`, etc., and trigger the discussion there?

[GitHub] [spark] SparkQA commented on pull request #31606: [SPARK-34490][SQL] Analysis should fail if the view refers a dropped table

2021-02-22 Thread GitBox
SparkQA commented on pull request #31606: URL: https://github.com/apache/spark/pull/31606#issuecomment-783239507 **[Test build #135340 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135340/testReport)** for PR 31606 at commit [`2642877`](https://github.com

[GitHub] [spark] cloud-fan commented on a change in pull request #31273: [SPARK-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

2021-02-22 Thread GitBox
cloud-fan commented on a change in pull request #31273: URL: https://github.com/apache/spark/pull/31273#discussion_r580106368 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -862,7 +862,13 @@ class Analyzer(override val cat

[GitHub] [spark] cloud-fan commented on a change in pull request #31273: [SPARK-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

2021-02-22 Thread GitBox
cloud-fan commented on a change in pull request #31273: URL: https://github.com/apache/spark/pull/31273#discussion_r580107291 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -862,7 +862,17 @@ class Analyzer(override val cat

[GitHub] [spark] cloud-fan commented on a change in pull request #31273: [SPARK-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

2021-02-22 Thread GitBox
cloud-fan commented on a change in pull request #31273: URL: https://github.com/apache/spark/pull/31273#discussion_r580108792 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala ## @@ -622,8 +622,7 @@ class SessionCatalog( }

[GitHub] [spark] cloud-fan commented on a change in pull request #31273: [SPARK-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

2021-02-22 Thread GitBox
cloud-fan commented on a change in pull request #31273: URL: https://github.com/apache/spark/pull/31273#discussion_r580109801 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala ## @@ -541,40 +539,33 @@ object ViewHelper { } /** -

[GitHub] [spark] cloud-fan commented on a change in pull request #31273: [SPARK-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

2021-02-22 Thread GitBox
cloud-fan commented on a change in pull request #31273: URL: https://github.com/apache/spark/pull/31273#discussion_r580110717 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala ## @@ -541,40 +539,33 @@ object ViewHelper { } /** -

[GitHub] [spark] cloud-fan commented on a change in pull request #31273: [SPARK-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

2021-02-22 Thread GitBox
cloud-fan commented on a change in pull request #31273: URL: https://github.com/apache/spark/pull/31273#discussion_r580111666 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala ## @@ -111,7 +111,7 @@ case class CreateViewCommand( //

[GitHub] [spark] cloud-fan commented on pull request #31273: [SPARK-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

2021-02-22 Thread GitBox
cloud-fan commented on pull request #31273: URL: https://github.com/apache/spark/pull/31273#issuecomment-783246992 Looks pretty good. My last concern is changing the `View` signature to hold temp views, as it leads to many other changes. I agree it's a bit overkill to add another wrapper n

[GitHub] [spark] linhongliu-db commented on a change in pull request #31606: [SPARK-34490][SQL] Analysis should fail if the view refers a dropped table

2021-02-22 Thread GitBox
linhongliu-db commented on a change in pull request #31606: URL: https://github.com/apache/spark/pull/31606#discussion_r580114213 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TableLookupCacheSuite.scala ## @@ -46,7 +48,12 @@ class TableLooku

[GitHub] [spark] MaxGekk opened a new pull request #31610: [MINOR][SQL] Fix the comment for CalendarIntervalType about comparability.

2021-02-22 Thread GitBox
MaxGekk opened a new pull request #31610: URL: https://github.com/apache/spark/pull/31610 ### What changes were proposed in this pull request? In the PR, I propose to revert https://github.com/apache/spark/pull/26659 partially regarding to comparability of interval values. The comment be

[GitHub] [spark] MaxGekk commented on pull request #31610: [MINOR][SQL] Fix the comment for CalendarIntervalType about comparability.

2021-02-22 Thread GitBox
MaxGekk commented on pull request #31610: URL: https://github.com/apache/spark/pull/31610#issuecomment-783251094 @yaooqinn @cloud-fan Could you take a look at this minor fix. This is an automated message from the Apache Git S

[GitHub] [spark] sarutak commented on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-22 Thread GitBox
sarutak commented on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-783251606 > The API doesn't take Symbol Actually some `subquery` and `as` in `dsl/package.scala` take Symbol parameter. We should deprecate them right? > Does it really wo

[GitHub] [spark] sarutak edited a comment on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-22 Thread GitBox
sarutak edited a comment on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-783251606 > The API doesn't take Symbol Actually some `subquery` and `as` in `dsl/package.scala` take Symbol parameter. We should deprecate them right? > Does it re

[GitHub] [spark] sarutak edited a comment on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-22 Thread GitBox
sarutak edited a comment on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-783251606 > The API doesn't take Symbol Actually `subquery` and `as` in `dsl/package.scala` take Symbol parameter. We should deprecate them right? > Does it really w

[GitHub] [spark] AmplabJenkins commented on pull request #31609: [SPARK-34496][BUILD] Upgrade ZSTD-JNI to 1.4.8-5 for better API compatibility

2021-02-22 Thread GitBox
AmplabJenkins commented on pull request #31609: URL: https://github.com/apache/spark/pull/31609#issuecomment-783261786 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39918/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31448: [SPARK-28137][SQL] Data Type Formatting Functions: `to_number`.

2021-02-22 Thread GitBox
AmplabJenkins commented on pull request #31448: URL: https://github.com/apache/spark/pull/31448#issuecomment-783261787 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135331/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31596: [SPARK-34475][SQL] Rename logical nodes of v2 `ALTER` commands

2021-02-22 Thread GitBox
AmplabJenkins commented on pull request #31596: URL: https://github.com/apache/spark/pull/31596#issuecomment-783261785 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39919/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31596: [SPARK-34475][SQL] Rename logical nodes of v2 `ALTER` commands

2021-02-22 Thread GitBox
AmplabJenkins removed a comment on pull request #31596: URL: https://github.com/apache/spark/pull/31596#issuecomment-783261785 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39919/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31609: [SPARK-34496][BUILD] Upgrade ZSTD-JNI to 1.4.8-5 for better API compatibility

2021-02-22 Thread GitBox
AmplabJenkins removed a comment on pull request #31609: URL: https://github.com/apache/spark/pull/31609#issuecomment-783261786 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39918/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31448: [SPARK-28137][SQL] Data Type Formatting Functions: `to_number`.

2021-02-22 Thread GitBox
AmplabJenkins removed a comment on pull request #31448: URL: https://github.com/apache/spark/pull/31448#issuecomment-783261787 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135331/ -

[GitHub] [spark] sarutak edited a comment on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-22 Thread GitBox
sarutak edited a comment on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-783251606 > The API doesn't take Symbol Actually `subquery` and `as` in `dsl/package.scala` and `as` and `alias` in `Dataset.scala` take Symbol parameter. We should deprec

[GitHub] [spark] sarutak edited a comment on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-22 Thread GitBox
sarutak edited a comment on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-783251606 > The API doesn't take Symbol Actually `subquery` and `as` in `dsl/package.scala`, `as` and `alias` in `Dataset.scala`, `as` in `Columnlscala` can take Symbol param

[GitHub] [spark] SparkQA commented on pull request #31593: [SPARK-34473][SQL] Avoid NPE in DataFrameReader.schema(StructType)

2021-02-22 Thread GitBox
SparkQA commented on pull request #31593: URL: https://github.com/apache/spark/pull/31593#issuecomment-783267335 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39921/ -

[GitHub] [spark] yaooqinn commented on pull request #31610: [MINOR][SQL] Fix the comment for CalendarIntervalType about comparability.

2021-02-22 Thread GitBox
yaooqinn commented on pull request #31610: URL: https://github.com/apache/spark/pull/31610#issuecomment-783270065 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] gengliangwang commented on pull request #31390: [SPARK-28123][SQL] String Functions: support btrim

2021-02-22 Thread GitBox
gengliangwang commented on pull request #31390: URL: https://github.com/apache/spark/pull/31390#issuecomment-783270727 > But it is a break change. Hmm, then I prefer keeping the current behavior: Trimming binary input is allowed in non-ansi mode, while disallowed in ANSI mode.

[GitHub] [spark] SparkQA commented on pull request #31606: [SPARK-34490][SQL] Analysis should fail if the view refers a dropped table

2021-02-22 Thread GitBox
SparkQA commented on pull request #31606: URL: https://github.com/apache/spark/pull/31606#issuecomment-783270822 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39920/ -

[GitHub] [spark] gengliangwang commented on pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

2021-02-22 Thread GitBox
gengliangwang commented on pull request #31349: URL: https://github.com/apache/spark/pull/31349#issuecomment-783271372 @maropu @cloud-fan all test passes and comments addressed. Could you take another look? This is an auto

[GitHub] [spark] sarutak edited a comment on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-22 Thread GitBox
sarutak edited a comment on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-783251606 > The API doesn't take Symbol Actually the following APIs can take Symbol parameter. * `subquery` and `as` in `dsl/package.scala` * `as` and `alias` in `Da

[GitHub] [spark] sarutak edited a comment on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-22 Thread GitBox
sarutak edited a comment on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-783251606 > The API doesn't take Symbol Actually the following APIs can take Symbol parameter. * `subquery` and `as` in `dsl/package.scala` * `as` and `alias` in `Da

[GitHub] [spark] sarutak edited a comment on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

2021-02-22 Thread GitBox
sarutak edited a comment on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-783251606 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] SparkQA commented on pull request #31610: [MINOR][SQL] Fix the comment for CalendarIntervalType about comparability.

2021-02-22 Thread GitBox
SparkQA commented on pull request #31610: URL: https://github.com/apache/spark/pull/31610#issuecomment-783273413 **[Test build #135342 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135342/testReport)** for PR 31610 at commit [`67ce1f5`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31606: [SPARK-34490][SQL] Analysis should fail if the view refers a dropped table

2021-02-22 Thread GitBox
SparkQA commented on pull request #31606: URL: https://github.com/apache/spark/pull/31606#issuecomment-78321 **[Test build #135343 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135343/testReport)** for PR 31606 at commit [`df0d8be`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31593: [SPARK-34473][SQL] Avoid NPE in DataFrameReader.schema(StructType)

2021-02-22 Thread GitBox
SparkQA commented on pull request #31593: URL: https://github.com/apache/spark/pull/31593#issuecomment-783278945 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39921/ ---

[GitHub] [spark] beliefer commented on pull request #31390: [SPARK-28123][SQL] String Functions: support btrim

2021-02-22 Thread GitBox
beliefer commented on pull request #31390: URL: https://github.com/apache/spark/pull/31390#issuecomment-783281908 > > But it is a break change. > > Hmm, then I prefer keeping the current behavior: Trimming binary input is allowed in non-ansi mode, while disallowed in ANSI mode.

[GitHub] [spark] beliefer commented on a change in pull request #31316: [SPARK-33599][SQL][FOLLOWUP] Group exception messages in catalyst/analysis

2021-02-22 Thread GitBox
beliefer commented on a change in pull request #31316: URL: https://github.com/apache/spark/pull/31316#discussion_r580152159 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala ## @@ -187,10 +187,9 @@ object ResolveHints {

[GitHub] [spark] SparkQA commented on pull request #31575: [SPARK-34450][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. RENAME tests

2021-02-22 Thread GitBox
SparkQA commented on pull request #31575: URL: https://github.com/apache/spark/pull/31575#issuecomment-783282419 **[Test build #135335 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135335/testReport)** for PR 31575 at commit [`ee548c4`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31575: [SPARK-34450][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. RENAME tests

2021-02-22 Thread GitBox
SparkQA removed a comment on pull request #31575: URL: https://github.com/apache/spark/pull/31575#issuecomment-783120465 **[Test build #135335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135335/testReport)** for PR 31575 at commit [`ee548c4`](https://gi

[GitHub] [spark] beliefer commented on pull request #31548: [SPARK-34127][SQL] Support table valued command

2021-02-22 Thread GitBox
beliefer commented on pull request #31548: URL: https://github.com/apache/spark/pull/31548#issuecomment-783283377 cc @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] AngersZhuuuu opened a new pull request #31611: [SPARK-34488][CORE] Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-02-22 Thread GitBox
AngersZh opened a new pull request #31611: URL: https://github.com/apache/spark/pull/31611 ### What changes were proposed in this pull request? For a specific stage, it is useful to show the task metrics in percentile distribution.  This information can help users know whether or not

[GitHub] [spark] SparkQA commented on pull request #31606: [SPARK-34490][SQL] Analysis should fail if the view refers a dropped table

2021-02-22 Thread GitBox
SparkQA commented on pull request #31606: URL: https://github.com/apache/spark/pull/31606#issuecomment-783289461 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39923/ -

[GitHub] [spark] SparkQA commented on pull request #31609: [SPARK-34496][BUILD] Upgrade ZSTD-JNI to 1.4.8-5 for better API compatibility

2021-02-22 Thread GitBox
SparkQA commented on pull request #31609: URL: https://github.com/apache/spark/pull/31609#issuecomment-783291016 **[Test build #135338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135338/testReport)** for PR 31609 at commit [`31d2389`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31609: [SPARK-34496][BUILD] Upgrade ZSTD-JNI to 1.4.8-5 for better API compatibility

2021-02-22 Thread GitBox
SparkQA removed a comment on pull request #31609: URL: https://github.com/apache/spark/pull/31609#issuecomment-783195965 **[Test build #135338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135338/testReport)** for PR 31609 at commit [`31d2389`](https://gi

  1   2   3   4   5   >