[GitHub] [spark] AmplabJenkins removed a comment on pull request #30908: [SPARK-33892][SQL] Display char/varchar in DESC and SHOW CREATE TABLE

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30908:
URL: https://github.com/apache/spark/pull/30908#issuecomment-750790543


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/12/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750790541


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37935/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30898: [SPARK-33884][SQL] Simplify CaseWhen when one clause is null and another is boolean

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30898:
URL: https://github.com/apache/spark/pull/30898#issuecomment-750790542


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37933/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750794254


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37934/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30865:
URL: https://github.com/apache/spark/pull/30865#issuecomment-750790540


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37932/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30905: [SPARK-33890][SQL] Improve the implement of trim/trimleft/trimright

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30905:
URL: https://github.com/apache/spark/pull/30905#issuecomment-750778086


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/13/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

2020-12-23 Thread GitBox


SparkQA commented on pull request #30869:
URL: https://github.com/apache/spark/pull/30869#issuecomment-750795522


   **[Test build #133348 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133348/testReport)**
 for PR 30869 at commit 
[`f42275f`](https://github.com/apache/spark/commit/f42275f890aee250ed93df35b02a3ad636a78435).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

2020-12-23 Thread GitBox


AngersZh commented on a change in pull request #30869:
URL: https://github.com/apache/spark/pull/30869#discussion_r548436302



##
File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala
##
@@ -920,9 +921,12 @@ object DDLUtils {
 serde == Some("parquet.hive.serde.ParquetHiveSerDe") ||
 serde == 
Some("org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe")) {
 ParquetSchemaConverter.checkFieldNames(colNames)
+  } else if (serde == HiveSerDe.sourceToSerDe("avro").get.serde) {
+AvroFileFormat.checkFieldNames(colNames)
   }
 case "parquet" => ParquetSchemaConverter.checkFieldNames(colNames)
 case "orc" => OrcFileFormat.checkFieldNames(colNames)
+case "avro" => AvroFileFormat.checkFieldNames(colNames)

Review comment:
   Updated as your comment, but many test failed since 
   ```
org.apache.spark.sql.AnalysisException: Failed to find data source: avro. 
Avro is built-in but external data source module since Spark 2.4. Please deploy 
the application as per the deployment section of "Apache Avro Data Source 
Guide".
   ```
   Then move all these test case to Avro module?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750794254


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37934/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


SparkQA commented on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750794245


   Kubernetes integration test status success
   URL: 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37934/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] beliefer commented on pull request #30870: [SPARK-33542][SQL] Group exception messages in catalyst/catalog

2020-12-23 Thread GitBox


beliefer commented on pull request #30870:
URL: https://github.com/apache/spark/pull/30870#issuecomment-750792243


   cc @HyukjinKwon 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] beliefer commented on pull request #30905: [SPARK-33890][SQL] Improve the implement of trim/trimleft/trimright

2020-12-23 Thread GitBox


beliefer commented on pull request #30905:
URL: https://github.com/apache/spark/pull/30905#issuecomment-750792203


   cc @cloud-fan  @wangyum 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30908: [SPARK-33892][SQL] Display char/varchar in DESC and SHOW CREATE TABLE

2020-12-23 Thread GitBox


SparkQA commented on pull request #30908:
URL: https://github.com/apache/spark/pull/30908#issuecomment-750791799


   **[Test build #133347 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133347/testReport)**
 for PR 30908 at commit 
[`057f1ff`](https://github.com/apache/spark/commit/057f1ffe9002df782555b6d7d7eebc5b50521c86).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan closed pull request #30914: [SPARK-33895][SQL] Char and Varchar fail in MetaOperation of ThriftServer

2020-12-23 Thread GitBox


cloud-fan closed pull request #30914:
URL: https://github.com/apache/spark/pull/30914


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on pull request #30914: [SPARK-33895][SQL] Char and Varchar fail in MetaOperation of ThriftServer

2020-12-23 Thread GitBox


cloud-fan commented on pull request #30914:
URL: https://github.com/apache/spark/pull/30914#issuecomment-750790913


   thanks, merging to master/3.1!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750790541


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37935/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30898: [SPARK-33884][SQL] Simplify CaseWhen when one clause is null and another is boolean

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30898:
URL: https://github.com/apache/spark/pull/30898#issuecomment-750790542


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37933/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30865:
URL: https://github.com/apache/spark/pull/30865#issuecomment-750790540


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37932/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30908: [SPARK-33892][SQL] Display char/varchar in DESC and SHOW CREATE TABLE

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30908:
URL: https://github.com/apache/spark/pull/30908#issuecomment-750790543


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/12/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30893: [SPARK-33881][SQL][TESTS] Check null and empty string as partition values in DS v1 and v2 tests

2020-12-23 Thread GitBox


SparkQA commented on pull request #30893:
URL: https://github.com/apache/spark/pull/30893#issuecomment-750789727


   Kubernetes integration test starting
   URL: 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37937/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #30908: [SPARK-33892][SQL] Display char/varchar in DESC and SHOW CREATE TABLE

2020-12-23 Thread GitBox


SparkQA removed a comment on pull request #30908:
URL: https://github.com/apache/spark/pull/30908#issuecomment-750713271


   **[Test build #12 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12/testReport)**
 for PR 30908 at commit 
[`2adf7f9`](https://github.com/apache/spark/commit/2adf7f94a1b0b2f64525f0062c4916e916176afb).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30908: [SPARK-33892][SQL] Display char/varchar in DESC and SHOW CREATE TABLE

2020-12-23 Thread GitBox


SparkQA commented on pull request #30908:
URL: https://github.com/apache/spark/pull/30908#issuecomment-750788108


   **[Test build #12 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12/testReport)**
 for PR 30908 at commit 
[`2adf7f9`](https://github.com/apache/spark/commit/2adf7f94a1b0b2f64525f0062c4916e916176afb).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30898: [SPARK-33884][SQL] Simplify CaseWhen when one clause is null and another is boolean

2020-12-23 Thread GitBox


SparkQA commented on pull request #30898:
URL: https://github.com/apache/spark/pull/30898#issuecomment-750786814


   Kubernetes integration test status failure
   URL: 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37933/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


SparkQA commented on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750785864


   Kubernetes integration test starting
   URL: 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37934/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] viirya commented on pull request #30812: [SPARK-33814][SS] Provide preferred locations for stateful operations without reported state store locations

2020-12-23 Thread GitBox


viirya commented on pull request #30812:
URL: https://github.com/apache/spark/pull/30812#issuecomment-750781778


   > I'm wondering whether this would solve the issue. There are so many 
factors involved. I feel the initial even task distribution (assuming the 
executors are free so the task scheduler will respect the preferred locations) 
would become uneven quickly after some micro batches, caused by, such as, 
uneven partitions, executor lost, concurrent queries, etc... Did you verify 
that this would make a real workload that didn't work before become working?
   
   During running stateful streaming queries recently, it caused some troubles 
by bad initial locations of stores. I agree that this is not ideal, but it 
solves the problem I saw during I tested SS recently. It is simple and 
shouldn't have bad impact/regression to SS queries. Combined with task locality 
configuration, it makes SS queries more stable in my local test.
   
   > If the issue is about memory, a low memory state store implementation such 
as https://github.com/qubole/spark-state-store is a better solution.
   
   The problem is, we have a built-in in memory store. And, I don't think 
Rocksdb-based state store is the answer to all cases. Even with Rocksdb-based 
store, is it good to have skew stores on few executors? Then local disk space 
might be the next issue.

   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


SparkQA commented on pull request #30865:
URL: https://github.com/apache/spark/pull/30865#issuecomment-750779938


   Kubernetes integration test status success
   URL: 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37932/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30905: [SPARK-33890][SQL] Improve the implement of trim/trimleft/trimright

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30905:
URL: https://github.com/apache/spark/pull/30905#issuecomment-750778086


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/13/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30885: [SPARK-33659][SS] Document the current behavior for DataStreamWriter.toTable API

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30885:
URL: https://github.com/apache/spark/pull/30885#issuecomment-750775594


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/14/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750775294


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37931/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #30885: [SPARK-33659][SS] Document the current behavior for DataStreamWriter.toTable API

2020-12-23 Thread GitBox


SparkQA removed a comment on pull request #30885:
URL: https://github.com/apache/spark/pull/30885#issuecomment-750712512


   **[Test build #14 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14/testReport)**
 for PR 30885 at commit 
[`eff4b9d`](https://github.com/apache/spark/commit/eff4b9d9acfece6c2e7fae4111ce106fb282d13e).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #30905: [SPARK-33890][SQL] Improve the implement of trim/trimleft/trimright

2020-12-23 Thread GitBox


SparkQA removed a comment on pull request #30905:
URL: https://github.com/apache/spark/pull/30905#issuecomment-750712497


   **[Test build #13 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13/testReport)**
 for PR 30905 at commit 
[`21fb9b9`](https://github.com/apache/spark/commit/21fb9b9f1d3ed725b6700421c45c2a2d14d26612).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30905: [SPARK-33890][SQL] Improve the implement of trim/trimleft/trimright

2020-12-23 Thread GitBox


SparkQA commented on pull request #30905:
URL: https://github.com/apache/spark/pull/30905#issuecomment-750777478


   **[Test build #13 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13/testReport)**
 for PR 30905 at commit 
[`21fb9b9`](https://github.com/apache/spark/commit/21fb9b9f1d3ed725b6700421c45c2a2d14d26612).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30893: [SPARK-33881][SQL][TESTS] Check null and empty string as partition values in DS v1 and v2 tests

2020-12-23 Thread GitBox


SparkQA commented on pull request #30893:
URL: https://github.com/apache/spark/pull/30893#issuecomment-750776425


   **[Test build #133346 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133346/testReport)**
 for PR 30893 at commit 
[`4e0ca6c`](https://github.com/apache/spark/commit/4e0ca6c203c7624cf2f5b95c0ed88b66fa6ccd41).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30898: [SPARK-33884][SQL] Simplify CaseWhen when one clause is null and another is boolean

2020-12-23 Thread GitBox


SparkQA commented on pull request #30898:
URL: https://github.com/apache/spark/pull/30898#issuecomment-750776236


   Kubernetes integration test starting
   URL: 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37933/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30885: [SPARK-33659][SS] Document the current behavior for DataStreamWriter.toTable API

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30885:
URL: https://github.com/apache/spark/pull/30885#issuecomment-750775594


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/14/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750775294


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37931/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


SparkQA commented on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750775283


   Kubernetes integration test status success
   URL: 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37931/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30885: [SPARK-33659][SS] Document the current behavior for DataStreamWriter.toTable API

2020-12-23 Thread GitBox


SparkQA commented on pull request #30885:
URL: https://github.com/apache/spark/pull/30885#issuecomment-750774931


   **[Test build #14 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14/testReport)**
 for PR 30885 at commit 
[`eff4b9d`](https://github.com/apache/spark/commit/eff4b9d9acfece6c2e7fae4111ce106fb282d13e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


SparkQA commented on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750773611


   **[Test build #133343 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133343/testReport)**
 for PR 30912 at commit 
[`0461b8d`](https://github.com/apache/spark/commit/0461b8da388931891c504308c2e404abcff7195f).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


SparkQA commented on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750772861


   **[Test build #133345 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133345/testReport)**
 for PR 30387 at commit 
[`4f18319`](https://github.com/apache/spark/commit/4f183190df6a324364b5f895acaacbfebc16b7ca).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30893: [SPARK-33881][SQL][TESTS] Check null and empty string as partition values in DS v1 and v2 tests

2020-12-23 Thread GitBox


SparkQA commented on pull request #30893:
URL: https://github.com/apache/spark/pull/30893#issuecomment-750772639


   **[Test build #133344 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133344/testReport)**
 for PR 30893 at commit 
[`fe2a4f8`](https://github.com/apache/spark/commit/fe2a4f86e3401a80340fc3dbb1549ff50af9d471).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750771518







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750771519







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] zsxwing commented on pull request #30812: [SPARK-33814][SS] Provide preferred locations for stateful operations without reported state store locations

2020-12-23 Thread GitBox


zsxwing commented on pull request #30812:
URL: https://github.com/apache/spark/pull/30812#issuecomment-750770232


   I'm wondering whether this would solve the issue. There are so many factors 
involved. I feel the initial even task distribution (assuming the executors are 
free so the task scheduler will respect the preferred locations) would become 
uneven quickly after some micro batches, caused by, such as, uneven partitions, 
executor lost, concurrent queries, etc... Did you verify that this would make a 
real workload that didn't work before become working?
   
   If the issue is about memory, a low memory state store implementation such 
as https://github.com/qubole/spark-state-store is a better solution.
   
   > If we want to draw the ideal picture here, IMO my ideal picture is to pin 
executors and force these executors to serve these stateful tasks on the 
lifetime of the query. It's ideal to guarantee these stateful tasks never have 
to reload the state unless crash. This would not be ideal if the application 
runs multiple queries where batch and streaming are mixed and streaming queries 
have longer trigger interval hence the chance to be idle. Either the query 
should wait to be assigned to the executor, or executor should be allowed to be 
idle for the query.
   
   I think the answer to your ideal picture is continuous processing. Although 
we have not invested resources on continuous processing for a while, it's 
unlikely we would pin executors in micro batch since micro batch trades latency 
for throughput.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


SparkQA commented on pull request #30865:
URL: https://github.com/apache/spark/pull/30865#issuecomment-750767838


   Kubernetes integration test starting
   URL: 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37932/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] saikocat commented on pull request #30902: [SPARK-33888][SQL] Add support for TimeMillis logicalType to TimestampType

2020-12-23 Thread GitBox


saikocat commented on pull request #30902:
URL: https://github.com/apache/spark/pull/30902#issuecomment-750766426


   Noted with thanks. Should I just make a change/fix to `JdbcUtils.scala` 
then? Should be an easy fix if we choose to go with `IntegerType`.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


SparkQA commented on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750765191


   Kubernetes integration test starting
   URL: 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37931/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on pull request #30902: [SPARK-33888][SQL] Add support for TimeMillis logicalType to TimestampType

2020-12-23 Thread GitBox


cloud-fan commented on pull request #30902:
URL: https://github.com/apache/spark/pull/30902#issuecomment-750764030


   TIME is a standard SQL data type, which is not TIMESTAMP or INTERVAL. 
Unfortunately, Spark doesn't support this data type yet, and can only fail (or 
read as the physical type int).
   
   The JDBC mapping is wrong and we should fix it.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] viirya commented on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


viirya commented on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750761612


   For Json's e2e tests, I will work in other PR. Thanks.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


SparkQA removed a comment on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750486073


   **[Test build #133324 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133324/testReport)**
 for PR 30912 at commit 
[`b500344`](https://github.com/apache/spark/commit/b500344c7a5b0bf72b9b868d3f7a6ff211a328bf).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


SparkQA commented on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750760830


   **[Test build #133324 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133324/testReport)**
 for PR 30912 at commit 
[`b500344`](https://github.com/apache/spark/commit/b500344c7a5b0bf72b9b868d3f7a6ff211a328bf).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] viirya commented on a change in pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


viirya commented on a change in pull request #30912:
URL: https://github.com/apache/spark/pull/30912#discussion_r548401034



##
File path: 
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/OptimizeCsvExprsSuite.scala
##
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.optimizer
+
+import org.apache.spark.sql.catalyst.dsl.expressions._
+import org.apache.spark.sql.catalyst.dsl.plans._
+import org.apache.spark.sql.catalyst.expressions._
+import org.apache.spark.sql.catalyst.plans.PlanTest
+import org.apache.spark.sql.catalyst.plans.logical.{LocalRelation, LogicalPlan}
+import org.apache.spark.sql.catalyst.rules.RuleExecutor
+import org.apache.spark.sql.internal.SQLConf
+import org.apache.spark.sql.types._
+
+class OptimizeCsvExprsSuite extends PlanTest with ExpressionEvalHelper {

Review comment:
   Added e2e tests now. Thanks.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750750934







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750750932


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37928/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30865:
URL: https://github.com/apache/spark/pull/30865#issuecomment-750750931


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/16/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #29966: [SPARK-33084][CORE][SQL] Add jar support ivy path

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #29966:
URL: https://github.com/apache/spark/pull/29966#issuecomment-750750939


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/15/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] beliefer commented on a change in pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


beliefer commented on a change in pull request #30387:
URL: https://github.com/apache/spark/pull/30387#discussion_r548399696



##
File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowFunctionFrame.scala
##
@@ -153,26 +153,24 @@ class FrameLessOffsetWindowFunctionFrame(
   extends OffsetWindowFunctionFrameBase(
 target, ordinal, expressions, inputSchema, newMutableProjection, offset) {
 
-  assert(expressions.toSeq.filterNot(_.input.isInstanceOf[Attribute]).isEmpty)
-
-  /** The input expression of Lead/Lag. */
-  private lazy val inputExpression = expressions.toSeq.map(_.input).head
-
-  /** The index of input expression in the row. */
-  private lazy val idx = inputAttrs.zipWithIndex.find(_._1 == 
inputExpression).map(_._2).head
-
   /** Holder the UnsafeRow where the input operator by function is not null. */
   private var nextSelectedRow = EmptyRow
 
   // The number of rows skipped to get the next UnsafeRow where the input 
operator by function
   // is not null.
   private var skippedNonNullCount = 0
 
+  /** Create the projection to determine whether input is null. */
+  private val project = 
UnsafeProjection.create(Seq(IsNull(expressions.head.input)), inputSchema)
+
+  /** Check if the output value of the first index is null. */
+  private val nullCheck: InternalRow => Boolean = row => 
project(row).getBoolean(0)

Review comment:
   OK





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30898: [SPARK-33884][SQL] Simplify CaseWhen when one clause is null and another is boolean

2020-12-23 Thread GitBox


SparkQA commented on pull request #30898:
URL: https://github.com/apache/spark/pull/30898#issuecomment-750756902


   **[Test build #133342 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133342/testReport)**
 for PR 30898 at commit 
[`15c74ca`](https://github.com/apache/spark/commit/15c74ca9c63e80fb70e0d71ae89914c4b8f35272).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on a change in pull request #30881: [SPARK-33875][SQL] Implement DESCRIBE COLUMN for v2 tables

2020-12-23 Thread GitBox


cloud-fan commented on a change in pull request #30881:
URL: https://github.com/apache/spark/pull/30881#discussion_r548399246



##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##
@@ -344,10 +344,11 @@ case class DescribeRelation(
  */
 case class DescribeColumn(
 relation: LogicalPlan,
-colNameParts: Seq[String],
+column: NamedExpression,
 isExtended: Boolean) extends Command {
   override def children: Seq[LogicalPlan] = Seq(relation)
   override def output: Seq[Attribute] = 
DescribeCommandSchema.describeColumnAttributes()
+  override lazy val references: AttributeSet = AttributeSet.empty

Review comment:
   I agree. About `ResolvedView`, I think eventually it will have output as 
well, after we have v2 view APII. For now, `ResolvedView` always go to v1 
command, so we don't need its output.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] LuciferYang commented on a change in pull request #30484: [SPARK-33532][SQL] Remove unreachable branch in SpecificParquetRecordReaderBase.initialize method

2020-12-23 Thread GitBox


LuciferYang commented on a change in pull request #30484:
URL: https://github.com/apache/spark/pull/30484#discussion_r548398249



##
File path: 
sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java
##
@@ -93,48 +93,17 @@ public void initialize(InputSplit inputSplit, 
TaskAttemptContext taskAttemptCont
   throws IOException, InterruptedException {
 Configuration configuration = taskAttemptContext.getConfiguration();
 ParquetInputSplit split = (ParquetInputSplit)inputSplit;
+Preconditions.checkState(split.getRowGroupOffsets() == null,
+"rowGroupOffsets in ParquetInputSplit should be null.");
 this.file = split.getPath();
-long[] rowGroupOffsets = split.getRowGroupOffsets();
-
-ParquetMetadata footer;
-List blocks;
-
-// if task.side.metadata is set, rowGroupOffsets is null
-if (rowGroupOffsets == null) {

Review comment:
   Add a comment and revert the code change?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


SparkQA commented on pull request #30865:
URL: https://github.com/apache/spark/pull/30865#issuecomment-750754445


   **[Test build #133341 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133341/testReport)**
 for PR 30865 at commit 
[`878beb0`](https://github.com/apache/spark/commit/878beb0be0ee1f76186a59ec9db900445a52ddfa).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


SparkQA commented on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750754315


   **[Test build #133340 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133340/testReport)**
 for PR 30387 at commit 
[`0cd9bcc`](https://github.com/apache/spark/commit/0cd9bcce7475c2b189be32c0209c0ff017a0736e).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on a change in pull request #30908: [SPARK-33892][SQL] Display char/varchar in DESC and SHOW CREATE TABLE

2020-12-23 Thread GitBox


cloud-fan commented on a change in pull request #30908:
URL: https://github.com/apache/spark/pull/30908#discussion_r548395417



##
File path: 
sql/hive/src/test/scala/org/apache/spark/sql/HiveCharVarcharTestSuite.scala
##
@@ -41,6 +41,15 @@ class HiveCharVarcharTestSuite extends CharVarcharTestSuite 
with TestHiveSinglet
 }
 super.afterAll()
   }
+
+  test("SHOW CREATE TABLE AS SERDE w/ char/varchar") {

Review comment:
   ditto





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on a change in pull request #30908: [SPARK-33892][SQL] Display char/varchar in DESC and SHOW CREATE TABLE

2020-12-23 Thread GitBox


cloud-fan commented on a change in pull request #30908:
URL: https://github.com/apache/spark/pull/30908#discussion_r548395346



##
File path: 
sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala
##
@@ -443,6 +443,14 @@ trait CharVarcharTestSuite extends QueryTest with 
SQLTestUtils {
 ("c1 IN (c2)", true)))
 }
   }
+
+  test("DESCRIBE TABLE w/ char/varchar") {

Review comment:
   +1





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #29966: [SPARK-33084][CORE][SQL] Add jar support ivy path

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #29966:
URL: https://github.com/apache/spark/pull/29966#issuecomment-750750939


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/15/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750750935







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750750932


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37928/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30865:
URL: https://github.com/apache/spark/pull/30865#issuecomment-750750931


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/16/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on a change in pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


cloud-fan commented on a change in pull request #30387:
URL: https://github.com/apache/spark/pull/30387#discussion_r548391607



##
File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowFunctionFrame.scala
##
@@ -153,26 +153,24 @@ class FrameLessOffsetWindowFunctionFrame(
   extends OffsetWindowFunctionFrameBase(
 target, ordinal, expressions, inputSchema, newMutableProjection, offset) {
 
-  assert(expressions.toSeq.filterNot(_.input.isInstanceOf[Attribute]).isEmpty)
-
-  /** The input expression of Lead/Lag. */
-  private lazy val inputExpression = expressions.toSeq.map(_.input).head
-
-  /** The index of input expression in the row. */
-  private lazy val idx = inputAttrs.zipWithIndex.find(_._1 == 
inputExpression).map(_._2).head
-
   /** Holder the UnsafeRow where the input operator by function is not null. */
   private var nextSelectedRow = EmptyRow
 
   // The number of rows skipped to get the next UnsafeRow where the input 
operator by function
   // is not null.
   private var skippedNonNullCount = 0
 
+  /** Create the projection to determine whether input is null. */
+  private val project = 
UnsafeProjection.create(Seq(IsNull(expressions.head.input)), inputSchema)
+
+  /** Check if the output value of the first index is null. */
+  private val nullCheck: InternalRow => Boolean = row => 
project(row).getBoolean(0)

Review comment:
   this can be a def
   ```
   private def nullCheck(row: InternalRow): Boolean = project(row).getBoolean(0)
   ```





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on a change in pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


cloud-fan commented on a change in pull request #30387:
URL: https://github.com/apache/spark/pull/30387#discussion_r548391607



##
File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowFunctionFrame.scala
##
@@ -153,26 +153,24 @@ class FrameLessOffsetWindowFunctionFrame(
   extends OffsetWindowFunctionFrameBase(
 target, ordinal, expressions, inputSchema, newMutableProjection, offset) {
 
-  assert(expressions.toSeq.filterNot(_.input.isInstanceOf[Attribute]).isEmpty)
-
-  /** The input expression of Lead/Lag. */
-  private lazy val inputExpression = expressions.toSeq.map(_.input).head
-
-  /** The index of input expression in the row. */
-  private lazy val idx = inputAttrs.zipWithIndex.find(_._1 == 
inputExpression).map(_._2).head
-
   /** Holder the UnsafeRow where the input operator by function is not null. */
   private var nextSelectedRow = EmptyRow
 
   // The number of rows skipped to get the next UnsafeRow where the input 
operator by function
   // is not null.
   private var skippedNonNullCount = 0
 
+  /** Create the projection to determine whether input is null. */
+  private val project = 
UnsafeProjection.create(Seq(IsNull(expressions.head.input)), inputSchema)
+
+  /** Check if the output value of the first index is null. */
+  private val nullCheck: InternalRow => Boolean = row => 
project(row).getBoolean(0)

Review comment:
   this can be a def
   ```
   def nullCheck(row: InternalRow): Boolean = project(row).getBoolean(0)
   ```





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] kozakana commented on pull request #30803: [SPARK-33897][SQL]Can't set option 'cross' in join method.

2020-12-23 Thread GitBox


kozakana commented on pull request #30803:
URL: https://github.com/apache/spark/pull/30803#issuecomment-750749214


   @HyukjinKwon 
   I created [SPARK-33897](https://issues.apache.org/jira/browse/SPARK-33897) 
in JIRA and modify title of github issue.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

2020-12-23 Thread GitBox


AngersZh commented on a change in pull request #30869:
URL: https://github.com/apache/spark/pull/30869#discussion_r54830



##
File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala
##
@@ -920,9 +921,12 @@ object DDLUtils {
 serde == Some("parquet.hive.serde.ParquetHiveSerDe") ||
 serde == 
Some("org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe")) {
 ParquetSchemaConverter.checkFieldNames(colNames)
+  } else if (serde == HiveSerDe.sourceToSerDe("avro").get.serde) {
+AvroFileFormat.checkFieldNames(colNames)
   }
 case "parquet" => ParquetSchemaConverter.checkFieldNames(colNames)
 case "orc" => OrcFileFormat.checkFieldNames(colNames)
+case "avro" => AvroFileFormat.checkFieldNames(colNames)

Review comment:
   then for hive provider part,  we use similar way?
   ```
   } else if (serde == HiveSerDe.sourceToSerDe("avro").get.serde) {
   DataSource.lookupdataSource("avro", 
conf).checkFieldNames(colNames)
   }
   ```
   
   





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on a change in pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


cloud-fan commented on a change in pull request #30865:
URL: https://github.com/apache/spark/pull/30865#discussion_r548388560



##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/SimplifyConditionalsInPredicate.scala
##
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.optimizer
+
+import org.apache.spark.sql.catalyst.expressions.{And, CaseWhen, Expression, 
If, Literal, Not, Or}
+import org.apache.spark.sql.catalyst.expressions.Literal.{FalseLiteral, 
TrueLiteral}
+import org.apache.spark.sql.catalyst.plans.logical._
+import org.apache.spark.sql.catalyst.rules.Rule
+import org.apache.spark.sql.types.BooleanType
+import org.apache.spark.util.Utils
+
+
+object SimplifyConditionalsInPredicate extends Rule[LogicalPlan] {
+
+  def apply(plan: LogicalPlan): LogicalPlan = plan transform {
+case f @ Filter(cond, _) => f.copy(condition = simplifyConditional(cond))
+case j @ Join(_, _, _, Some(cond), _) => j.copy(condition = 
Some(simplifyConditional(cond)))
+case d @ DeleteFromTable(_, Some(cond)) => d.copy(condition = 
Some(simplifyConditional(cond)))
+case u @ UpdateTable(_, _, Some(cond)) => u.copy(condition = 
Some(simplifyConditional(cond)))
+  }
+
+  private def simplifyConditional(e: Expression): Expression = e match {
+case Literal(null, BooleanType) => FalseLiteral
+case And(left, right) => And(simplifyConditional(left), 
simplifyConditional(right))
+case Or(left, right) => Or(simplifyConditional(left), 
simplifyConditional(right))
+case If(cond, t, FalseLiteral) => And(cond, t)
+case If(cond, t, TrueLiteral) => Or(Not(cond), t)
+case If(cond, FalseLiteral, f) => And(Not(cond), f)
+case If(cond, TrueLiteral, f) => Or(cond, f)
+case CaseWhen(Seq((cond, trueValue)),
+Some(FalseLiteral) | Some(Literal(null, BooleanType)) | None) =>
+  And(cond, trueValue)
+case CaseWhen(Seq((cond, trueValue)), Some(TrueLiteral)) =>
+  Or(Not(cond), trueValue)
+case CaseWhen(Seq((cond, FalseLiteral)), elseValue) =>
+  And(Not(cond), elseValue.getOrElse(Literal(null, BooleanType)))
+case CaseWhen(Seq((cond, TrueLiteral)), elseValue) =>
+  Or(cond, elseValue.getOrElse(Literal(null, BooleanType)))
+case e if e.dataType == BooleanType => e

Review comment:
   I mean something like
   ```
   case e =>
 assert(e.dataType != BooleanType, ...)
 e
   ```





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on a change in pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


cloud-fan commented on a change in pull request #30865:
URL: https://github.com/apache/spark/pull/30865#discussion_r548388326



##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/SimplifyConditionalsInPredicate.scala
##
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.optimizer
+
+import org.apache.spark.sql.catalyst.expressions.{And, CaseWhen, Expression, 
If, Literal, Not, Or}
+import org.apache.spark.sql.catalyst.expressions.Literal.{FalseLiteral, 
TrueLiteral}
+import org.apache.spark.sql.catalyst.plans.logical._
+import org.apache.spark.sql.catalyst.rules.Rule
+import org.apache.spark.sql.types.BooleanType
+
+/**
+ * A rule that converting conditional expressions to predicate expressions, if 
possible, in the

Review comment:
   `that converting` -> `that converts`





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #29966: [SPARK-33084][CORE][SQL] Add jar support ivy path

2020-12-23 Thread GitBox


SparkQA removed a comment on pull request #29966:
URL: https://github.com/apache/spark/pull/29966#issuecomment-750712775


   **[Test build #15 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15/testReport)**
 for PR 29966 at commit 
[`4c44dae`](https://github.com/apache/spark/commit/4c44daecc7c77527e150d30051170f7ee8667f70).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on a change in pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

2020-12-23 Thread GitBox


cloud-fan commented on a change in pull request #30869:
URL: https://github.com/apache/spark/pull/30869#discussion_r548388043



##
File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala
##
@@ -920,9 +921,12 @@ object DDLUtils {
 serde == Some("parquet.hive.serde.ParquetHiveSerDe") ||
 serde == 
Some("org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe")) {
 ParquetSchemaConverter.checkFieldNames(colNames)
+  } else if (serde == HiveSerDe.sourceToSerDe("avro").get.serde) {
+AvroFileFormat.checkFieldNames(colNames)
   }
 case "parquet" => ParquetSchemaConverter.checkFieldNames(colNames)
 case "orc" => OrcFileFormat.checkFieldNames(colNames)
+case "avro" => AvroFileFormat.checkFieldNames(colNames)

Review comment:
   What we can do here is:
   ```
   DataSource.lookupDataSource(provider, conf) match  {
 case f: FileFormat => f.checkFieldNames... // a new API
 case _ =>
   }
   ```





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #29966: [SPARK-33084][CORE][SQL] Add jar support ivy path

2020-12-23 Thread GitBox


SparkQA commented on pull request #29966:
URL: https://github.com/apache/spark/pull/29966#issuecomment-750747800


   **[Test build #15 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15/testReport)**
 for PR 29966 at commit 
[`4c44dae`](https://github.com/apache/spark/commit/4c44daecc7c77527e150d30051170f7ee8667f70).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan closed pull request #30900: [SPARK-33886][SQL] UnresolvedTable should retain SQL text position for DDL commands

2020-12-23 Thread GitBox


cloud-fan closed pull request #30900:
URL: https://github.com/apache/spark/pull/30900


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on pull request #30900: [SPARK-33886][SQL] UnresolvedTable should retain SQL text position for DDL commands

2020-12-23 Thread GitBox


cloud-fan commented on pull request #30900:
URL: https://github.com/apache/spark/pull/30900#issuecomment-750746667


   thanks, merging to master!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on a change in pull request #30864: [SPARK-33857][SQL] Unify the default seed of random functions

2020-12-23 Thread GitBox


cloud-fan commented on a change in pull request #30864:
URL: https://github.com/apache/spark/pull/30864#discussion_r548386088



##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/randomExpressions.scala
##
@@ -83,14 +87,13 @@ trait ExpressionWithRandomSeed {
   """,
   since = "1.5.0")
 // scalastyle:on line.size.limit
-case class Rand(child: Expression, hideSeed: Boolean = false)
-  extends RDG with ExpressionWithRandomSeed {
+case class Rand(child: Expression, hideSeed: Boolean = false) extends RDG {
 
-  def this() = this(Literal(Utils.random.nextLong(), LongType), true)
+  def this() = this(UnresolvedSeed, true)
 
   def this(child: Expression) = this(child, false)
 
-  override def withNewSeed(seed: Long): Rand = Rand(Literal(seed, LongType))
+  override def withNewSeed(seed: Long): Rand = Rand(Literal(seed, LongType), 
hideSeed)

Review comment:
   This is an orthogonal bug fix, but it only affects the `sql` method. I 
don't have a strong opinion about backporting the fix or not.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


SparkQA commented on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750745148


   Kubernetes integration test status success
   URL: 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37929/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on pull request #30776: [SPARK-33787][SQL] Add the `purge` parameter to `dropPartition()` of `SupportsPartitionManagement`

2020-12-23 Thread GitBox


cloud-fan commented on pull request #30776:
URL: https://github.com/apache/spark/pull/30776#issuecomment-750744615


   This was closed in favor of https://github.com/apache/spark/pull/30886



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] beliefer commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


beliefer commented on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750744600


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


SparkQA removed a comment on pull request #30865:
URL: https://github.com/apache/spark/pull/30865#issuecomment-750726287


   **[Test build #16 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16/testReport)**
 for PR 30865 at commit 
[`8bd9ef9`](https://github.com/apache/spark/commit/8bd9ef9e3136e67a49dbbfe8574725dfe9f38c94).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


SparkQA commented on pull request #30865:
URL: https://github.com/apache/spark/pull/30865#issuecomment-750743768


   **[Test build #16 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16/testReport)**
 for PR 30865 at commit 
[`8bd9ef9`](https://github.com/apache/spark/commit/8bd9ef9e3136e67a49dbbfe8574725dfe9f38c94).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] kozakana commented on pull request #30803: [SPARK-30803][SQL]Can't set option 'cross' in join method.

2020-12-23 Thread GitBox


kozakana commented on pull request #30803:
URL: https://github.com/apache/spark/pull/30803#issuecomment-750742951


   ok! I understand the number and I can post the JIRA.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


SparkQA commented on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750742064


   Kubernetes integration test status success
   URL: 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37928/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


SparkQA removed a comment on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750727016


   **[Test build #18 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18/testReport)**
 for PR 30912 at commit 
[`58cc2ba`](https://github.com/apache/spark/commit/58cc2ba4fc5a032cefb285a8c35f4bfb8dff9a07).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750741601


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/18/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750741601


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/18/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


SparkQA commented on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750741485


   **[Test build #18 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18/testReport)**
 for PR 30912 at commit 
[`58cc2ba`](https://github.com/apache/spark/commit/58cc2ba4fc5a032cefb285a8c35f4bfb8dff9a07).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30865:
URL: https://github.com/apache/spark/pull/30865#issuecomment-750741136


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37927/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


AmplabJenkins commented on pull request #30865:
URL: https://github.com/apache/spark/pull/30865#issuecomment-750741136


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37927/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30865: [SPARK-33861][SQL] Simplify conditional in predicate

2020-12-23 Thread GitBox


SparkQA commented on pull request #30865:
URL: https://github.com/apache/spark/pull/30865#issuecomment-750741123


   Kubernetes integration test status success
   URL: 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37927/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #30912: [SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs

2020-12-23 Thread GitBox


SparkQA commented on pull request #30912:
URL: https://github.com/apache/spark/pull/30912#issuecomment-750738156


   **[Test build #19 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19/testReport)**
 for PR 30912 at commit 
[`fc5dd38`](https://github.com/apache/spark/commit/fc5dd38c1c0f8c90fe083f0e7f603ccdcc6e7b19).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30908: [SPARK-33892][SQL] Display char/varchar in DESC and SHOW CREATE TABLE

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30908:
URL: https://github.com/apache/spark/pull/30908#issuecomment-750737030


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37923/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30885: [SPARK-33659][SS] Document the current behavior for DataStreamWriter.toTable API

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30885:
URL: https://github.com/apache/spark/pull/30885#issuecomment-750737026


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37925/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30914: [SPARK-33895][SQL] Char and Varchar fail in MetaOperation of ThriftServer

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30914:
URL: https://github.com/apache/spark/pull/30914#issuecomment-750737025


   
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37921/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #29966: [SPARK-33084][CORE][SQL] Add jar support ivy path

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #29966:
URL: https://github.com/apache/spark/pull/29966#issuecomment-750737028


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133329/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-23 Thread GitBox


AmplabJenkins removed a comment on pull request #30387:
URL: https://github.com/apache/spark/pull/30387#issuecomment-750737029


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/17/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



  1   2   3   4   5   6   7   8   9   >