[GitHub] [spark] beliefer commented on a change in pull request #34644: [SPARK-36357][SQL] Support pushdown Timestamp without time zone for orc

2021-11-19 Thread GitBox
beliefer commented on a change in pull request #34644: URL: https://github.com/apache/spark/pull/34644#discussion_r752933470 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilters.scala ## @@ -168,6 +168,8 @@ private[sql] object OrcFilt

[GitHub] [spark] SparkQA commented on pull request #34648: [SPARK-37282][TESTS][FOLLOWUP] Extract `Utils.isMacOnAppleSilicon` for reuse in UTs

2021-11-19 Thread GitBox
SparkQA commented on pull request #34648: URL: https://github.com/apache/spark/pull/34648#issuecomment-973847248 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49910/ -- This is an automated message from the A

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752936812 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala ## @@ -438,3 +438,22 @@ object VirtualColumn

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752940105 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -194,10 +195,22 @@ case class FileSourceScanExec(

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752940728 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -194,10 +195,22 @@ case class FileSourceScanExec(

[GitHub] [spark] AngersZhuuuu commented on pull request #34623: [SPARK-37347][SQL] Spark Thrift Server (STS) driver fullFC becourse of timeoutExecutor not shutdown correctly

2021-11-19 Thread GitBox
AngersZh commented on pull request #34623: URL: https://github.com/apache/spark/pull/34623#issuecomment-973851143 Title `because`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] SparkQA commented on pull request #33893: [SPARK-36638][SQL] Generalize OptimizeSkewedJoin

2021-11-19 Thread GitBox
SparkQA commented on pull request #33893: URL: https://github.com/apache/spark/pull/33893#issuecomment-973851609 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49914/ -- This is an automated message from the A

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752943643 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -355,7 +377,14 @@ case class FileSourceScanExec(

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752940105 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -194,10 +195,22 @@ case class FileSourceScanExec(

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752946851 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormat.scala ## @@ -171,6 +171,27 @@ trait FileFormat { def sup

[GitHub] [spark] SparkQA commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
SparkQA commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973859761 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49912/ -- This is an automated message from the A

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752953419 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala ## @@ -103,6 +116,123 @@ class FileScanRDD(

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752960194 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala ## @@ -103,6 +116,123 @@ class FileScanRDD(

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752960655 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala ## @@ -103,6 +116,123 @@ class FileScanRDD(

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-19 Thread GitBox
SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973865174 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49913/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34651: [SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-19 Thread GitBox
SparkQA commented on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973866215 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49911/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
SparkQA commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973866416 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49916/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752964031 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala ## @@ -103,6 +116,123 @@ class FileScanRDD(

[GitHub] [spark] LuciferYang commented on pull request #34454: [SPARK-37013][CORE][SQL][FOLLOWUP] Use the new error framework to throw error in `FormatString`

2021-11-19 Thread GitBox
LuciferYang commented on pull request #34454: URL: https://github.com/apache/spark/pull/34454#issuecomment-973867853 Gentle ping @karenfeng once more -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] SparkQA commented on pull request #34664: [SPARK-35672][FOLLOWUP][TESTS] Add more exclusion rules to MimaExcludes.scala for Scala 2.13

2021-11-19 Thread GitBox
SparkQA commented on pull request #34664: URL: https://github.com/apache/spark/pull/34664#issuecomment-973868028 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49915/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #34651: [SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973868203 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49911/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973868205 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49912/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973868211 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49913/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34648: [SPARK-37282][TESTS][FOLLOWUP] Extract `Utils.isMacOnAppleSilicon` for reuse in UTs

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #34648: URL: https://github.com/apache/spark/pull/34648#issuecomment-973868207 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49910/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33893: [SPARK-36638][SQL] Generalize OptimizeSkewedJoin

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #33893: URL: https://github.com/apache/spark/pull/33893#issuecomment-973868208 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49914/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973868205 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49912/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33893: [SPARK-36638][SQL] Generalize OptimizeSkewedJoin

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #33893: URL: https://github.com/apache/spark/pull/33893#issuecomment-973868208 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49914/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34651: [SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973868203 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49911/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973868211 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49913/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34648: [SPARK-37282][TESTS][FOLLOWUP] Extract `Utils.isMacOnAppleSilicon` for reuse in UTs

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #34648: URL: https://github.com/apache/spark/pull/34648#issuecomment-973868207 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49910/

[GitHub] [spark] SparkQA commented on pull request #34644: [SPARK-36357][SQL] Support pushdown Timestamp without time zone for orc

2021-11-19 Thread GitBox
SparkQA commented on pull request #34644: URL: https://github.com/apache/spark/pull/34644#issuecomment-973868742 **[Test build #145446 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145446/testReport)** for PR 34644 at commit [`3b765f1`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
SparkQA commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973868780 **[Test build #145447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145447/testReport)** for PR 34642 at commit [`eb85130`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-19 Thread GitBox
SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973869088 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49917/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752967539 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala ## @@ -103,6 +116,123 @@ class FileScanRDD(

[GitHub] [spark] SparkQA commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
SparkQA commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973869399 **[Test build #145440 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145440/testReport)** for PR 34642 at commit [`7dc5b51`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973869648 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145440/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973869648 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145440/ -

[GitHub] [spark] zhengruifeng closed pull request #33298: [SPARK-36087][SQL][WIP] An Impl of skew key detection and data inflation optimization

2021-11-19 Thread GitBox
zhengruifeng closed pull request #33298: URL: https://github.com/apache/spark/pull/33298 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-u

[GitHub] [spark] SparkQA removed a comment on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
SparkQA removed a comment on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973795437 **[Test build #145440 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145440/testReport)** for PR 34642 at commit [`7dc5b51`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33893: [SPARK-36638][SQL] Generalize OptimizeSkewedJoin

2021-11-19 Thread GitBox
SparkQA commented on pull request #33893: URL: https://github.com/apache/spark/pull/33893#issuecomment-973871931 **[Test build #145448 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145448/testReport)** for PR 33893 at commit [`d7c6678`](https://github.com

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752973640 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileMetadataColumnsSuite.scala ## @@ -0,0 +1,481 @@ +/* + * Licensed t

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752975918 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileMetadataColumnsSuite.scala ## @@ -0,0 +1,481 @@ +/* + * Licensed t

[GitHub] [spark] advancedxy commented on pull request #34623: [SPARK-37347][SQL] Spark Thrift Server (STS) driver fullFC becourse of timeoutExecutor not shutdown correctly

2021-11-19 Thread GitBox
advancedxy commented on pull request #34623: URL: https://github.com/apache/spark/pull/34623#issuecomment-973875309 > Also, is it possible for you to add a unit test for this case. > When we set spark.sql.thriftServer.queryTimeout to 10800s, and STS running for a long time and finish som

[GitHub] [spark] advancedxy commented on pull request #34623: [SPARK-37347][SQL] Spark Thrift Server (STS) driver fullFC becourse of timeoutExecutor not shutdown correctly

2021-11-19 Thread GitBox
advancedxy commented on pull request #34623: URL: https://github.com/apache/spark/pull/34623#issuecomment-973876125 > > Also, is it possible for you to add a unit test for this case. > > When we set spark.sql.thriftServer.queryTimeout to 10800s, and STS running for a long time and finish

[GitHub] [spark] cloud-fan commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-19 Thread GitBox
cloud-fan commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r752977298 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileMetadataColumnsSuite.scala ## @@ -0,0 +1,481 @@ +/* + * Licensed t

[GitHub] [spark] SparkQA commented on pull request #34648: [SPARK-37282][TESTS][FOLLOWUP] Extract `Utils.isMacOnAppleSilicon` for reuse in UTs

2021-11-19 Thread GitBox
SparkQA commented on pull request #34648: URL: https://github.com/apache/spark/pull/34648#issuecomment-973877177 **[Test build #145438 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145438/testReport)** for PR 34648 at commit [`d98e2cb`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34648: [SPARK-37282][TESTS][FOLLOWUP] Extract `Utils.isMacOnAppleSilicon` for reuse in UTs

2021-11-19 Thread GitBox
SparkQA removed a comment on pull request #34648: URL: https://github.com/apache/spark/pull/34648#issuecomment-973774298 **[Test build #145438 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145438/testReport)** for PR 34648 at commit [`d98e2cb`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34648: [SPARK-37282][TESTS][FOLLOWUP] Extract `Utils.isMacOnAppleSilicon` for reuse in UTs

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #34648: URL: https://github.com/apache/spark/pull/34648#issuecomment-973878755 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145438/ -

[GitHub] [spark] AmplabJenkins commented on pull request #34648: [SPARK-37282][TESTS][FOLLOWUP] Extract `Utils.isMacOnAppleSilicon` for reuse in UTs

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #34648: URL: https://github.com/apache/spark/pull/34648#issuecomment-973878755 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145438/ -- This

[GitHub] [spark] Peng-Lei opened a new pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

2021-11-19 Thread GitBox
Peng-Lei opened a new pull request #34666: URL: https://github.com/apache/spark/pull/34666 ### What changes were proposed in this pull request? 1. Migrate `SHOW TBLPROPERTIES` to use V2 command by default 2. Sort the result of V1 command of `SHOW TBLPROPERTIES` 3. handle the `output

[GitHub] [spark] SparkQA commented on pull request #34663: [SPARK-37385][SQL][TESTS] Add tests for TimestampNTZ and TimestampLTZ for Parquet data source

2021-11-19 Thread GitBox
SparkQA commented on pull request #34663: URL: https://github.com/apache/spark/pull/34663#issuecomment-973881154 **[Test build #145450 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145450/testReport)** for PR 34663 at commit [`cd444bf`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

2021-11-19 Thread GitBox
SparkQA commented on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-973881098 **[Test build #145449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145449/testReport)** for PR 34666 at commit [`c4da96f`](https://github.com

[GitHub] [spark] beliefer commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

2021-11-19 Thread GitBox
beliefer commented on pull request #34647: URL: https://github.com/apache/spark/pull/34647#issuecomment-973884852 ping @gengliangwang @MaxGekk cc @cloud-fan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] SparkQA commented on pull request #34651: [SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-19 Thread GitBox
SparkQA commented on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973887182 **[Test build #145439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145439/testReport)** for PR 34651 at commit [`f15f9da`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34651: [SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-19 Thread GitBox
SparkQA removed a comment on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973795464 **[Test build #145439 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145439/testReport)** for PR 34651 at commit [`f15f9da`](https://gi

[GitHub] [spark] wangyum commented on pull request #34070: [SPARK-36840][SQL] Support DPP if there is no selective predicate on the filtering side

2021-11-19 Thread GitBox
wangyum commented on pull request #34070: URL: https://github.com/apache/spark/pull/34070#issuecomment-973893038 Another case from cluster: ![image](https://user-images.githubusercontent.com/5399861/142597105-722ad9dc-4e0c-4cd5-af74-6eac6099a302.png) -- This is an automated messag

[GitHub] [spark] Peng-Lei commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

2021-11-19 Thread GitBox
Peng-Lei commented on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-973895291 @imback82 @cloud-fan If you have time, Could you take a look? Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #34664: [SPARK-35672][FOLLOWUP][TESTS] Add more exclusion rules to MimaExcludes.scala for Scala 2.13

2021-11-19 Thread GitBox
SparkQA commented on pull request #34664: URL: https://github.com/apache/spark/pull/34664#issuecomment-973895916 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49915/ -- This is an automated message from the A

[GitHub] [spark] LuciferYang commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scala-

2021-11-19 Thread GitBox
LuciferYang commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-973898635 The problem still exists, but the method I provided before can not reproduce stably. I need to find a way to reproduce stably -- This is an automated message from

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-19 Thread GitBox
SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973899541 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49917/ -- This is an automated message from the A

[GitHub] [spark] LuciferYang commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scala-

2021-11-19 Thread GitBox
LuciferYang commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-973899774 ``` mvn clean package -DskipTests -Phadoop-3.2 -Pyarn rm -rf assembly/target/scala-2.12 mvn clean test -pl resource-managers/yarn -Phadoop-3.2 -Pyarn -Dtest=none -Dw

[GitHub] [spark] LuciferYang edited a comment on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target

2021-11-19 Thread GitBox
LuciferYang edited a comment on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-973899774 ``` mvn clean package -DskipTests -Phadoop-3.2 -Pyarn rm -rf assembly/target/scala-2.12 mvn clean test -pl resource-managers/yarn -Phadoop-3.2 -Pyarn -Dtest=no

[GitHub] [spark] dchvn opened a new pull request #34667: [WIP][SPARK-36902][SQL] Migrate CreateTableAsSelectStatement to v2 command

2021-11-19 Thread GitBox
dchvn opened a new pull request #34667: URL: https://github.com/apache/spark/pull/34667 ### What changes were proposed in this pull request? Migrate CreateTableAsSelectStatement to v2 command ### Why are the changes needed? Migrate CreateTableAsSelectStatement to v2 comm

[GitHub] [spark] dchvn commented on pull request #34667: [WIP][SPARK-36902][SQL] Migrate CreateTableAsSelectStatement to v2 command

2021-11-19 Thread GitBox
dchvn commented on pull request #34667: URL: https://github.com/apache/spark/pull/34667#issuecomment-973902647 CC @huaxingao FYI, Please take a look when you find some time, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [spark] SparkQA commented on pull request #34644: [SPARK-36357][SQL] Support pushdown Timestamp without time zone for orc

2021-11-19 Thread GitBox
SparkQA commented on pull request #34644: URL: https://github.com/apache/spark/pull/34644#issuecomment-973902795 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49918/ -- This is an automated message from the Apache

[GitHub] [spark] LuciferYang commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scala-

2021-11-19 Thread GitBox
LuciferYang commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-973904992 But use similar way to test with `-Phadoop-2.7` will trigger another problem ``` mvn clean package -DskipTests -Phadoop-2.7 -Pyarn rm -rf assembly/target/scala-2.

[GitHub] [spark] SparkQA commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
SparkQA commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973904990 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49919/ -- This is an automated message from the Apache

[GitHub] [spark] dchvn commented on a change in pull request #34667: [WIP][SPARK-36902][SQL] Migrate CreateTableAsSelectStatement to v2 command

2021-11-19 Thread GitBox
dchvn commented on a change in pull request #34667: URL: https://github.com/apache/spark/pull/34667#discussion_r753016110 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala ## @@ -190,6 +190,19 @@ trait V2CreateTablePlan exte

[GitHub] [spark] SparkQA commented on pull request #33893: [SPARK-36638][SQL] Generalize OptimizeSkewedJoin

2021-11-19 Thread GitBox
SparkQA commented on pull request #33893: URL: https://github.com/apache/spark/pull/33893#issuecomment-973905495 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49920/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
SparkQA commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973906248 **[Test build #145444 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145444/testReport)** for PR 34642 at commit [`9c057b1`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
SparkQA removed a comment on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973831608 **[Test build #145444 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145444/testReport)** for PR 34642 at commit [`9c057b1`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
SparkQA commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973909518 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49916/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #34664: [SPARK-35672][FOLLOWUP][TESTS] Add more exclusion rules to MimaExcludes.scala for Scala 2.13

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #34664: URL: https://github.com/apache/spark/pull/34664#issuecomment-973909639 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49915/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34651: [SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973909645 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145439/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973909556 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973909644 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49917/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973909556 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34651: [SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973909645 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145439/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34664: [SPARK-35672][FOLLOWUP][TESTS] Add more exclusion rules to MimaExcludes.scala for Scala 2.13

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #34664: URL: https://github.com/apache/spark/pull/34664#issuecomment-973909639 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49915/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973909644 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49917/

[GitHub] [spark] SparkQA commented on pull request #34667: [WIP][SPARK-36902][SQL] Migrate CreateTableAsSelectStatement to v2 command

2021-11-19 Thread GitBox
SparkQA commented on pull request #34667: URL: https://github.com/apache/spark/pull/34667#issuecomment-973911887 **[Test build #145451 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145451/testReport)** for PR 34667 at commit [`145f22b`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34644: [SPARK-36357][SQL] Support pushdown Timestamp without time zone for orc

2021-11-19 Thread GitBox
SparkQA commented on pull request #34644: URL: https://github.com/apache/spark/pull/34644#issuecomment-973912016 **[Test build #145452 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145452/testReport)** for PR 34644 at commit [`24848b6`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34662: [SPARK-37386][SQL] Simplify OptimizeSkewedJoin to not run the cost evaluator

2021-11-19 Thread GitBox
SparkQA commented on pull request #34662: URL: https://github.com/apache/spark/pull/34662#issuecomment-973914278 **[Test build #145437 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145437/testReport)** for PR 34662 at commit [`62948ec`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34662: [SPARK-37386][SQL] Simplify OptimizeSkewedJoin to not run the cost evaluator

2021-11-19 Thread GitBox
SparkQA removed a comment on pull request #34662: URL: https://github.com/apache/spark/pull/34662#issuecomment-973770064 **[Test build #145437 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145437/testReport)** for PR 34662 at commit [`62948ec`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34663: [SPARK-37385][SQL][TESTS] Add tests for TimestampNTZ and TimestampLTZ for Parquet data source

2021-11-19 Thread GitBox
SparkQA commented on pull request #34663: URL: https://github.com/apache/spark/pull/34663#issuecomment-973914772 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49921/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #34662: [SPARK-37386][SQL] Simplify OptimizeSkewedJoin to not run the cost evaluator

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #34662: URL: https://github.com/apache/spark/pull/34662#issuecomment-973914949 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145437/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34662: [SPARK-37386][SQL] Simplify OptimizeSkewedJoin to not run the cost evaluator

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #34662: URL: https://github.com/apache/spark/pull/34662#issuecomment-973914949 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145437/ -

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-19 Thread GitBox
SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973916233 **[Test build #145445 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145445/testReport)** for PR 34611 at commit [`8708ff1`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-19 Thread GitBox
AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973916470 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145445/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-19 Thread GitBox
SparkQA removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973839046 **[Test build #145445 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145445/testReport)** for PR 34611 at commit [`8708ff1`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-19 Thread GitBox
AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973916470 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145445/ -

[GitHub] [spark] SparkQA commented on pull request #34644: [SPARK-36357][SQL] Support pushdown Timestamp without time zone for orc

2021-11-19 Thread GitBox
SparkQA commented on pull request #34644: URL: https://github.com/apache/spark/pull/34644#issuecomment-973936088 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49918/ -- This is an automated message from the A

[GitHub] [spark] ulysses-you commented on pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

2021-11-19 Thread GitBox
ulysses-you commented on pull request #34568: URL: https://github.com/apache/spark/pull/34568#issuecomment-973936481 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [spark] SparkQA commented on pull request #33893: [SPARK-36638][SQL] Generalize OptimizeSkewedJoin

2021-11-19 Thread GitBox
SparkQA commented on pull request #33893: URL: https://github.com/apache/spark/pull/33893#issuecomment-973936937 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49920/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34663: [SPARK-37385][SQL][TESTS] Add tests for TimestampNTZ and TimestampLTZ for Parquet data source

2021-11-19 Thread GitBox
SparkQA commented on pull request #34663: URL: https://github.com/apache/spark/pull/34663#issuecomment-973937616 **[Test build #145436 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145436/testReport)** for PR 34663 at commit [`fb59f00`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
SparkQA commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973937736 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49919/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #34663: [SPARK-37385][SQL][TESTS] Add tests for TimestampNTZ and TimestampLTZ for Parquet data source

2021-11-19 Thread GitBox
SparkQA removed a comment on pull request #34663: URL: https://github.com/apache/spark/pull/34663#issuecomment-973769988 **[Test build #145436 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145436/testReport)** for PR 34663 at commit [`fb59f00`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34667: [WIP][SPARK-36902][SQL] Migrate CreateTableAsSelectStatement to v2 command

2021-11-19 Thread GitBox
SparkQA commented on pull request #34667: URL: https://github.com/apache/spark/pull/34667#issuecomment-973947420 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49922/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34644: [SPARK-36357][SQL] Support pushdown Timestamp without time zone for orc

2021-11-19 Thread GitBox
SparkQA commented on pull request #34644: URL: https://github.com/apache/spark/pull/34644#issuecomment-973951646 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49924/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

2021-11-19 Thread GitBox
SparkQA commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-973952114 **[Test build #145447 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145447/testReport)** for PR 34642 at commit [`eb85130`](https://github.co

  1   2   3   4   5   >