[jira] [Assigned] (SPARK-35159) extract doc of hive format

[ https://issues.apache.org/jira/browse/SPARK-35159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35159: --- Assignee: angerszhu > extract doc of hive format > -- > >

[jira] [Resolved] (SPARK-35159) extract doc of hive format

[ https://issues.apache.org/jira/browse/SPARK-35159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35159. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32264

[jira] [Assigned] (SPARK-35143) Add default log config for spark-sql

[ https://issues.apache.org/jira/browse/SPARK-35143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35143: Assignee: hong dongdong > Add default log config for spark-sql >

[jira] [Resolved] (SPARK-35143) Add default log config for spark-sql

[ https://issues.apache.org/jira/browse/SPARK-35143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35143. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32248

[jira] [Assigned] (SPARK-35141) Support two level map for final hash aggregation

[ https://issues.apache.org/jira/browse/SPARK-35141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35141: --- Assignee: Cheng Su > Support two level map for final hash aggregation >

[jira] [Resolved] (SPARK-35141) Support two level map for final hash aggregation

[ https://issues.apache.org/jira/browse/SPARK-35141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35141. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32242

[jira] [Assigned] (SPARK-35195) Move InMemoryTable etc to org.apache.spark.sql.connector.catalog

[ https://issues.apache.org/jira/browse/SPARK-35195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35195: --- Assignee: Chao Sun > Move InMemoryTable etc to org.apache.spark.sql.connector.catalog >

[jira] [Resolved] (SPARK-35195) Move InMemoryTable etc to org.apache.spark.sql.connector.catalog

[ https://issues.apache.org/jira/browse/SPARK-35195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35195. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32302

[jira] [Updated] (SPARK-35200) Avoid to recompute the pending speculative tasks in the ExecutorAllocationManager and remove unnecessary code

[ https://issues.apache.org/jira/browse/SPARK-35200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-35200: - Fix Version/s: 3.0.1 3.0.2 3.1.0 3.1.1 > Avoid

[jira] [Updated] (SPARK-35200) Avoid to recompute the pending speculative tasks in the ExecutorAllocationManager and remove unnecessary code

[ https://issues.apache.org/jira/browse/SPARK-35200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-35200: - Description: The number of the pending speculative tasks is recomputed in the

[jira] [Assigned] (SPARK-35075) Migrate to transformWithPruning or resolveWithPruning for subquery related rules

[ https://issues.apache.org/jira/browse/SPARK-35075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-35075: -- Assignee: Yingyi Bu > Migrate to transformWithPruning or resolveWithPruning for

[jira] [Resolved] (SPARK-35075) Migrate to transformWithPruning or resolveWithPruning for subquery related rules

[ https://issues.apache.org/jira/browse/SPARK-35075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35075. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32247

[jira] [Commented] (SPARK-35176) Raise TypeError in inappropriate type case rather than ValueError

[ https://issues.apache.org/jira/browse/SPARK-35176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329971#comment-17329971 ] Yikun Jiang commented on SPARK-35176: - I write up a POC in

[jira] [Commented] (SPARK-35200) Avoid to recompute the pending speculative tasks in the ExecutorAllocationManager and remove unnecessary code

[ https://issues.apache.org/jira/browse/SPARK-35200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329949#comment-17329949 ] Apache Spark commented on SPARK-35200: -- User 'weixiuli' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35200) Avoid to recompute the pending speculative tasks in the ExecutorAllocationManager and remove unnecessary code

[ https://issues.apache.org/jira/browse/SPARK-35200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35200: Assignee: Apache Spark > Avoid to recompute the pending speculative tasks in the >

[jira] [Assigned] (SPARK-35200) Avoid to recompute the pending speculative tasks in the ExecutorAllocationManager and remove unnecessary code

[ https://issues.apache.org/jira/browse/SPARK-35200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35200: Assignee: (was: Apache Spark) > Avoid to recompute the pending speculative tasks in

[jira] [Updated] (SPARK-35200) Avoid to recompute the pending speculative tasks in the ExecutorAllocationManager and remove unnecessary code

[ https://issues.apache.org/jira/browse/SPARK-35200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-35200: - Summary: Avoid to recompute the pending speculative tasks in the ExecutorAllocationManager and remove

[jira] [Created] (SPARK-35200) Avoid to recompute the pending tasks in the ExecutorAllocationManager and remove unnecessary code

weixiuli created SPARK-35200: Summary: Avoid to recompute the pending tasks in the ExecutorAllocationManager and remove unnecessary code Key: SPARK-35200 URL: https://issues.apache.org/jira/browse/SPARK-35200

[jira] [Commented] (SPARK-21499) Support creating persistent function for Spark UDAF(UserDefinedAggregateFunction)

[ https://issues.apache.org/jira/browse/SPARK-21499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329937#comment-17329937 ] Apache Spark commented on SPARK-21499: -- User 'turboFei' has created a pull request for this issue:

[jira] [Commented] (SPARK-21499) Support creating persistent function for Spark UDAF(UserDefinedAggregateFunction)

[ https://issues.apache.org/jira/browse/SPARK-21499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329936#comment-17329936 ] Apache Spark commented on SPARK-21499: -- User 'turboFei' has created a pull request for this issue:

[jira] [Created] (SPARK-35199) Tasks are failing with zstd default of spark.shuffle.mapStatus.compression.codec

Leonard Lausen created SPARK-35199: -- Summary: Tasks are failing with zstd default of spark.shuffle.mapStatus.compression.codec Key: SPARK-35199 URL: https://issues.apache.org/jira/browse/SPARK-35199

[jira] [Updated] (SPARK-35184) Filtering a dataframe after groupBy and user-define-aggregate-function in Pyspark will cause java.lang.UnsupportedOperationException

[ https://issues.apache.org/jira/browse/SPARK-35184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Jin updated SPARK-35184: - Description: I found some strange error when I'm coding Pyspark UDAF. After I call groupBy function

[jira] [Commented] (SPARK-35191) all columns are read even if column pruning applies when spark3.0 read table written by spark2.2

[ https://issues.apache.org/jira/browse/SPARK-35191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329932#comment-17329932 ] Yuming Wang commented on SPARK-35191: - Could you check if it works after

[jira] [Resolved] (SPARK-35040) Remove Spark-version related codes from test codes.

[ https://issues.apache.org/jira/browse/SPARK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35040. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue

[jira] [Assigned] (SPARK-35182) Support driver-owned on-demand PVC

[ https://issues.apache.org/jira/browse/SPARK-35182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35182: - Assignee: Dongjoon Hyun > Support driver-owned on-demand PVC >

[jira] [Resolved] (SPARK-35182) Support driver-owned on-demand PVC

[ https://issues.apache.org/jira/browse/SPARK-35182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35182. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32288

[jira] [Updated] (SPARK-35198) Add support for calling debugCodegen from Python & Java

[ https://issues.apache.org/jira/browse/SPARK-35198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-35198: - Labels: starter (was: ) > Add support for calling debugCodegen from Python & Java >

[jira] [Created] (SPARK-35198) Add support for calling debugCodegen from Python & Java

Holden Karau created SPARK-35198: Summary: Add support for calling debugCodegen from Python & Java Key: SPARK-35198 URL: https://issues.apache.org/jira/browse/SPARK-35198 Project: Spark

[jira] [Assigned] (SPARK-35197) Accumulators Explore Page on Spark UI in History Server

[ https://issues.apache.org/jira/browse/SPARK-35197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35197: Assignee: (was: Apache Spark) > Accumulators Explore Page on Spark UI in History

[jira] [Commented] (SPARK-35197) Accumulators Explore Page on Spark UI in History Server

[ https://issues.apache.org/jira/browse/SPARK-35197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329867#comment-17329867 ] Apache Spark commented on SPARK-35197: -- User 'FridaPolished' has created a pull request for this

[jira] [Assigned] (SPARK-35197) Accumulators Explore Page on Spark UI in History Server

[ https://issues.apache.org/jira/browse/SPARK-35197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35197: Assignee: Apache Spark > Accumulators Explore Page on Spark UI in History Server >

[jira] [Created] (SPARK-35197) Accumulators Explore Page on Spark UI in History Server

Frida Montserrat Pulido Padilla created SPARK-35197: --- Summary: Accumulators Explore Page on Spark UI in History Server Key: SPARK-35197 URL: https://issues.apache.org/jira/browse/SPARK-35197

[jira] [Commented] (SPARK-35169) Wrong result of min ANSI interval division by -1

[ https://issues.apache.org/jira/browse/SPARK-35169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329405#comment-17329405 ] Max Gekk commented on SPARK-35169: -- Since https://github.com/apache/spark/pull/32260 has been merged,

[jira] [Resolved] (SPARK-34999) Consolidate PySpark testing utils

[ https://issues.apache.org/jira/browse/SPARK-34999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-34999. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue

[jira] [Assigned] (SPARK-34382) ANSI SQL: LATERAL derived table(T491)

[ https://issues.apache.org/jira/browse/SPARK-34382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34382: Assignee: (was: Apache Spark) > ANSI SQL: LATERAL derived table(T491) >

[jira] [Commented] (SPARK-34382) ANSI SQL: LATERAL derived table(T491)

[ https://issues.apache.org/jira/browse/SPARK-34382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329395#comment-17329395 ] Apache Spark commented on SPARK-34382: -- User 'allisonwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-34382) ANSI SQL: LATERAL derived table(T491)

[ https://issues.apache.org/jira/browse/SPARK-34382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34382: Assignee: Apache Spark > ANSI SQL: LATERAL derived table(T491) >

[jira] [Commented] (SPARK-34382) ANSI SQL: LATERAL derived table(T491)

[ https://issues.apache.org/jira/browse/SPARK-34382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329394#comment-17329394 ] Apache Spark commented on SPARK-34382: -- User 'allisonwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-35195) Move InMemoryTable etc to org.apache.spark.sql.connector.catalog

[ https://issues.apache.org/jira/browse/SPARK-35195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35195: Assignee: Apache Spark > Move InMemoryTable etc to

[jira] [Assigned] (SPARK-35195) Move InMemoryTable etc to org.apache.spark.sql.connector.catalog

[ https://issues.apache.org/jira/browse/SPARK-35195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35195: Assignee: (was: Apache Spark) > Move InMemoryTable etc to

[jira] [Commented] (SPARK-35195) Move InMemoryTable etc to org.apache.spark.sql.connector.catalog

[ https://issues.apache.org/jira/browse/SPARK-35195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329393#comment-17329393 ] Apache Spark commented on SPARK-35195: -- User 'sunchao' has created a pull request for this issue:

[jira] [Created] (SPARK-35196) DataFrameWriter.text support zstd compression

Leonard Lausen created SPARK-35196: -- Summary: DataFrameWriter.text support zstd compression Key: SPARK-35196 URL: https://issues.apache.org/jira/browse/SPARK-35196 Project: Spark Issue

[jira] [Created] (SPARK-35195) Move InMemoryTable etc to org.apache.spark.sql.connector.catalog

Chao Sun created SPARK-35195: Summary: Move InMemoryTable etc to org.apache.spark.sql.connector.catalog Key: SPARK-35195 URL: https://issues.apache.org/jira/browse/SPARK-35195 Project: Spark

[jira] [Assigned] (SPARK-35187) Failure on minimal interval literal

[ https://issues.apache.org/jira/browse/SPARK-35187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35187: Assignee: angerszhu > Failure on minimal interval literal > ---

[jira] [Resolved] (SPARK-35187) Failure on minimal interval literal

[ https://issues.apache.org/jira/browse/SPARK-35187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35187. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32296

[jira] [Commented] (SPARK-35194) Improve readability of NestingColumnAliasing

[ https://issues.apache.org/jira/browse/SPARK-35194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329309#comment-17329309 ] Apache Spark commented on SPARK-35194: -- User 'karenfeng' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35194) Improve readability of NestingColumnAliasing

[ https://issues.apache.org/jira/browse/SPARK-35194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35194: Assignee: Apache Spark > Improve readability of NestingColumnAliasing >

[jira] [Assigned] (SPARK-35194) Improve readability of NestingColumnAliasing

[ https://issues.apache.org/jira/browse/SPARK-35194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35194: Assignee: (was: Apache Spark) > Improve readability of NestingColumnAliasing >

[jira] [Commented] (SPARK-35194) Improve readability of NestingColumnAliasing

[ https://issues.apache.org/jira/browse/SPARK-35194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329308#comment-17329308 ] Apache Spark commented on SPARK-35194: -- User 'karenfeng' has created a pull request for this issue:

[jira] [Created] (SPARK-35194) Improve readability of NestingColumnAliasing

Karen Feng created SPARK-35194: -- Summary: Improve readability of NestingColumnAliasing Key: SPARK-35194 URL: https://issues.apache.org/jira/browse/SPARK-35194 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-35040) Remove Spark-version related codes from test codes.

[ https://issues.apache.org/jira/browse/SPARK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35040: Assignee: (was: Apache Spark) > Remove Spark-version related codes from test codes.

[jira] [Commented] (SPARK-35040) Remove Spark-version related codes from test codes.

[ https://issues.apache.org/jira/browse/SPARK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329293#comment-17329293 ] Apache Spark commented on SPARK-35040: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-35040) Remove Spark-version related codes from test codes.

[ https://issues.apache.org/jira/browse/SPARK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35040: Assignee: Apache Spark > Remove Spark-version related codes from test codes. >

[jira] [Commented] (SPARK-34198) Add RocksDB StateStore as external module

[ https://issues.apache.org/jira/browse/SPARK-34198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329276#comment-17329276 ] Yuanjian Li commented on SPARK-34198: - [~viirya] [~kabhwan] Make sense, I should bring it to the

[jira] [Commented] (SPARK-35040) Remove Spark-version related codes from test codes.

[ https://issues.apache.org/jira/browse/SPARK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329268#comment-17329268 ] Xinrong Meng commented on SPARK-35040: -- May I work on this ticket? > Remove Spark-version related

[jira] [Created] (SPARK-35193) Scala/Java compatibility issue Re: how to use externalResource in java transformer from Scala Transformer?

Arthur created SPARK-35193: -- Summary: Scala/Java compatibility issue Re: how to use externalResource in java transformer from Scala Transformer? Key: SPARK-35193 URL: https://issues.apache.org/jira/browse/SPARK-35193

[jira] [Commented] (SPARK-34483) Make check point compress codec configurable

[ https://issues.apache.org/jira/browse/SPARK-34483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329247#comment-17329247 ] Holden Karau commented on SPARK-34483: -- Are you still working on this? > Make check point compress

[jira] [Updated] (SPARK-34953) inferSchema for type date

[ https://issues.apache.org/jira/browse/SPARK-34953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-34953: - Component/s: SQL > inferSchema for type date > -- > >

[jira] [Commented] (SPARK-34997) Make graceful shutdown controllable by property

[ https://issues.apache.org/jira/browse/SPARK-34997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329244#comment-17329244 ] Holden Karau commented on SPARK-34997: -- We could add a configuration for which signals spark should

[jira] [Resolved] (SPARK-34363) Allow users to configure a maximum amount of remote shuffle block storage

[ https://issues.apache.org/jira/browse/SPARK-34363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-34363. -- Fix Version/s: 3.2.0 Assignee: Holden Karau Resolution: Fixed > Allow users

[jira] [Assigned] (SPARK-34079) Improvement CTE table scan

[ https://issues.apache.org/jira/browse/SPARK-34079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34079: Assignee: (was: Apache Spark) > Improvement CTE table scan >

[jira] [Comment Edited] (SPARK-34883) Setting CSV reader option "multiLine" to "true" causes URISyntaxException when colon is in file path

[ https://issues.apache.org/jira/browse/SPARK-34883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329132#comment-17329132 ] Brady Tello edited comment on SPARK-34883 at 4/22/21, 1:52 PM: ---

[jira] [Updated] (SPARK-33121) Spark Streaming 3.1.1 hangs on shutdown

[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Priority: Critical (was: Major) > Spark Streaming 3.1.1 hangs on shutdown >

[jira] [Assigned] (SPARK-34079) Improvement CTE table scan

[ https://issues.apache.org/jira/browse/SPARK-34079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34079: Assignee: Apache Spark > Improvement CTE table scan > -- > >

[jira] [Assigned] (SPARK-35110) Handle ANSI intervals in WindowExecBase

[ https://issues.apache.org/jira/browse/SPARK-35110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35110: Assignee: jiaan.geng > Handle ANSI intervals in WindowExecBase >

[jira] [Updated] (SPARK-33121) Spark does not shutdown gracefully

[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Description: Hi. I am trying to migrate from spark 2.4.5 to 3.1.1 and there is a

[jira] [Commented] (SPARK-28098) Native ORC reader doesn't support subdirectories with Hive tables

[ https://issues.apache.org/jira/browse/SPARK-28098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329120#comment-17329120 ] Yu-Tang Lin commented on SPARK-28098: - Hi [~ddrinka], I already made a PR to resolve the issue,

[jira] [Resolved] (SPARK-35110) Handle ANSI intervals in WindowExecBase

[ https://issues.apache.org/jira/browse/SPARK-35110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35110. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32294

[jira] [Updated] (SPARK-35184) Filtering a dataframe after groupBy and user-define-aggregate-function in Pyspark will cause java.lang.UnsupportedOperationException

[ https://issues.apache.org/jira/browse/SPARK-35184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Jin updated SPARK-35184: - Description: I found some strange error when I'm coding Pyspark UDAF. After I call groupBy function

[jira] [Commented] (SPARK-35187) Failure on minimal interval literal

[ https://issues.apache.org/jira/browse/SPARK-35187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329118#comment-17329118 ] Apache Spark commented on SPARK-35187: -- User 'AngersZh' has created a pull request for this

[jira] [Updated] (SPARK-33121) Spark Streaming 3.1.1 hangs on shutdown

[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Summary: Spark Streaming 3.1.1 hangs on shutdown (was: Spark does not shutdown

[jira] [Commented] (SPARK-34079) Improvement CTE table scan

[ https://issues.apache.org/jira/browse/SPARK-34079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329138#comment-17329138 ] Apache Spark commented on SPARK-34079: -- User 'peter-toth' has created a pull request for this

[jira] [Assigned] (SPARK-35187) Failure on minimal interval literal

[ https://issues.apache.org/jira/browse/SPARK-35187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35187: Assignee: (was: Apache Spark) > Failure on minimal interval literal >

[jira] [Commented] (SPARK-35192) Port minimal TPC-DS datagen code from databricks/spark-sql-perf

[ https://issues.apache.org/jira/browse/SPARK-35192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329108#comment-17329108 ] Apache Spark commented on SPARK-35192: -- User 'maropu' has created a pull request for this issue:

[jira] [Updated] (SPARK-33121) Spark does not shutdown gracefully

[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Description: Hi. I am trying to migrate from spark 2.4.5 to 3.0.1 and there is a

[jira] [Commented] (SPARK-33121) Spark does not shutdown gracefully

[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329129#comment-17329129 ] Dmitry Tverdokhleb commented on SPARK-33121: Tested on Spark Streaming 3.1.1 with a simple

[jira] [Updated] (SPARK-33121) Spark does not shutdown gracefully

[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Description: Hi. I am trying to migrate from spark 2.4.5 to 3.1.1 and there is a

[jira] [Updated] (SPARK-33121) Spark Streaming 3.1.1 hangs on shutdown

[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Affects Version/s: (was: 3.0.1) 3.1.1 > Spark Streaming

[jira] [Commented] (SPARK-35192) Port minimal TPC-DS datagen code from databricks/spark-sql-perf

[ https://issues.apache.org/jira/browse/SPARK-35192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329107#comment-17329107 ] Apache Spark commented on SPARK-35192: -- User 'maropu' has created a pull request for this issue:

[jira] [Commented] (SPARK-34883) Setting CSV reader option "multiLine" to "true" causes URISyntaxException when colon is in file path

[ https://issues.apache.org/jira/browse/SPARK-34883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329132#comment-17329132 ] Brady Tello commented on SPARK-34883: - [~Vikas_Yadav] You don't have a colon in your `inputFile`

[jira] [Updated] (SPARK-33121) Spark Streaming 3.1.1 hangs on shutdown

[ https://issues.apache.org/jira/browse/SPARK-33121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Tverdokhleb updated SPARK-33121: --- Labels: 3.1.1 Streaming hang shutdown (was: ) > Spark Streaming 3.1.1 hangs on

[jira] [Commented] (SPARK-35187) Failure on minimal interval literal

[ https://issues.apache.org/jira/browse/SPARK-35187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329116#comment-17329116 ] Apache Spark commented on SPARK-35187: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-32380) sparksql cannot access hive table while data in hbase

[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17329119#comment-17329119 ] Yennam ShowryPremSagar Reddy commented on SPARK-32380: -- [~meimile] Will there be

[jira] [Assigned] (SPARK-35192) Port minimal TPC-DS datagen code from databricks/spark-sql-perf

[ https://issues.apache.org/jira/browse/SPARK-35192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35192: Assignee: Apache Spark > Port minimal TPC-DS datagen code from databricks/spark-sql-perf

[jira] [Assigned] (SPARK-35026) Support use CUBE/ROLLUP in GROUPING SETS

[ https://issues.apache.org/jira/browse/SPARK-35026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35026: --- Assignee: angerszhu > Support use CUBE/ROLLUP in GROUPING SETS >

[jira] [Resolved] (SPARK-35026) Support use CUBE/ROLLUP in GROUPING SETS

[ https://issues.apache.org/jira/browse/SPARK-35026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35026. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32201

[jira] [Created] (SPARK-35192) Port minimal TPC-DS datagen code from databricks/spark-sql-perf

Takeshi Yamamuro created SPARK-35192: Summary: Port minimal TPC-DS datagen code from databricks/spark-sql-perf Key: SPARK-35192 URL: https://issues.apache.org/jira/browse/SPARK-35192 Project:

[jira] [Assigned] (SPARK-35183) CombineConcats should call transformAllExpressions

[ https://issues.apache.org/jira/browse/SPARK-35183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35183: --- Assignee: Yingyi Bu > CombineConcats should call transformAllExpressions >

[jira] [Assigned] (SPARK-35187) Failure on minimal interval literal

[ https://issues.apache.org/jira/browse/SPARK-35187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35187: Assignee: Apache Spark > Failure on minimal interval literal >

[jira] [Assigned] (SPARK-35192) Port minimal TPC-DS datagen code from databricks/spark-sql-perf

[ https://issues.apache.org/jira/browse/SPARK-35192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35192: Assignee: (was: Apache Spark) > Port minimal TPC-DS datagen code from

[jira] [Resolved] (SPARK-35183) CombineConcats should call transformAllExpressions

[ https://issues.apache.org/jira/browse/SPARK-35183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35183. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32290

[jira] [Resolved] (SPARK-35180) Allow to build SparkR with SBT

[ https://issues.apache.org/jira/browse/SPARK-35180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35180. -- Fix Version/s: 3.2.0 Resolution: Fixed Fixed in 

[jira] [Resolved] (SPARK-35127) When we switch between different stage-detail pages, the entry item in the newly-opened page may be blank.

[ https://issues.apache.org/jira/browse/SPARK-35127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35127. Fix Version/s: 3.2.0 3.1.2 3.0.3 Assignee:

[jira] [Commented] (SPARK-35090) Extract a field from ANSI interval

[ https://issues.apache.org/jira/browse/SPARK-35090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17328043#comment-17328043 ] Kent Yao commented on SPARK-35090: -- yes. thanks > Extract a field from ANSI interval >

[jira] [Commented] (SPARK-35091) Support ANSI intervals by date_part()

[ https://issues.apache.org/jira/browse/SPARK-35091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17328019#comment-17328019 ] Kent Yao commented on SPARK-35091: -- thanks [~maxgekk] for pinging me. I will take it. > Support ANSI

[jira] [Commented] (SPARK-35090) Extract a field from ANSI interval

[ https://issues.apache.org/jira/browse/SPARK-35090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17327637#comment-17327637 ] Max Gekk commented on SPARK-35090: -- [~Qin Yao] Would you like to take this? > Extract a field from

[jira] [Created] (SPARK-35191) all columns are read even if column pruning applies when spark3.0 read table written by spark2.2

xiaoli created SPARK-35191: -- Summary: all columns are read even if column pruning applies when spark3.0 read table written by spark2.2 Key: SPARK-35191 URL: https://issues.apache.org/jira/browse/SPARK-35191

[jira] [Commented] (SPARK-35091) Support ANSI intervals by date_part()

[ https://issues.apache.org/jira/browse/SPARK-35091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17327644#comment-17327644 ] Max Gekk commented on SPARK-35091: -- [~Qin Yao] Would you like to take this? > Support ANSI intervals

[jira] [Created] (SPARK-35190) all columns are read even if column pruning applies when spark3.0 read table written by spark2.2

xiaoli created SPARK-35190: -- Summary: all columns are read even if column pruning applies when spark3.0 read table written by spark2.2 Key: SPARK-35190 URL: https://issues.apache.org/jira/browse/SPARK-35190

[jira] [Assigned] (SPARK-35189) Disallow implicit conversion for `String_Column - Date_Column`

[ https://issues.apache.org/jira/browse/SPARK-35189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35189: Assignee: Apache Spark (was: Gengliang Wang) > Disallow implicit conversion for

  1   2   >