[jira] [Resolved] (SPARK-52699) Support aggregating TIME type in interpreted mode

[ https://issues.apache.org/jira/browse/SPARK-52699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-52699. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51395 [https://github.com

[jira] [Assigned] (SPARK-52699) Support aggregating TIME type in interpreted mode

[ https://issues.apache.org/jira/browse/SPARK-52699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-52699: Assignee: Bruce Robbins > Support aggregating TIME type in interpreted mode > ---

[jira] [Resolved] (SPARK-52693) Add day-time interval to time

[ https://issues.apache.org/jira/browse/SPARK-52693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-52693. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51383 [https://github.com

[jira] [Resolved] (SPARK-52576) In Declarative Pipelines, drop/recreate on full refresh and MV update

[ https://issues.apache.org/jira/browse/SPARK-52576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-52576. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51280 [https://gith

[jira] [Assigned] (SPARK-52576) In Declarative Pipelines, drop/recreate on full refresh and MV update

[ https://issues.apache.org/jira/browse/SPARK-52576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-52576: --- Assignee: Sandy Ryza > In Declarative Pipelines, drop/recreate on full refresh and MV updat

[jira] [Created] (SPARK-52706) Fix inconsistencies and refactor primitive types in parser

Mihailo Milosevic created SPARK-52706: - Summary: Fix inconsistencies and refactor primitive types in parser Key: SPARK-52706 URL: https://issues.apache.org/jira/browse/SPARK-52706 Project: Spark

[jira] [Created] (SPARK-52705) Refactor deterministic check for grouping expressions

Mihailo Timotic created SPARK-52705: --- Summary: Refactor deterministic check for grouping expressions Key: SPARK-52705 URL: https://issues.apache.org/jira/browse/SPARK-52705 Project: Spark

[jira] [Commented] (SPARK-52495) Include partition columns in the single variant column

[ https://issues.apache.org/jira/browse/SPARK-52495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003655#comment-18003655 ] Anastasia Filippova commented on SPARK-52495: - User 'xiaonanyang-db' has cre

[jira] [Commented] (SPARK-52703) Upgrade minimum python version of pandas api to 3.10

[ https://issues.apache.org/jira/browse/SPARK-52703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003656#comment-18003656 ] Anastasia Filippova commented on SPARK-52703: - User 'zhengruifeng' has creat

[jira] [Commented] (SPARK-52703) Upgrade minimum python version of pandas api to 3.10

[ https://issues.apache.org/jira/browse/SPARK-52703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003657#comment-18003657 ] Anastasia Filippova commented on SPARK-52703: - User 'zhengruifeng' has creat

[jira] [Created] (SPARK-52704) Simplify options in TextBasedFileFormats

Kent Yao created SPARK-52704: Summary: Simplify options in TextBasedFileFormats Key: SPARK-52704 URL: https://issues.apache.org/jira/browse/SPARK-52704 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-52703) Upgrade minimum python version of pandas api to 3.10

[ https://issues.apache.org/jira/browse/SPARK-52703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-52703: -- Summary: Upgrade minimum python version of pandas api to 3.10 (was: Upgrade minimum python ve

[jira] [Created] (SPARK-52703) Upgrade minimum python version of pandas api to 3.11

Ruifeng Zheng created SPARK-52703: - Summary: Upgrade minimum python version of pandas api to 3.11 Key: SPARK-52703 URL: https://issues.apache.org/jira/browse/SPARK-52703 Project: Spark Issue

[jira] [Commented] (SPARK-51224) Upgrade Maven to 4.0.0

[ https://issues.apache.org/jira/browse/SPARK-51224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003651#comment-18003651 ] Cheng Pan commented on SPARK-51224: --- [~dongjoon] Ack, will keep eyes on the Maven 4.0

[jira] [Resolved] (SPARK-52695) User Defined Type write support for xml file format

[ https://issues.apache.org/jira/browse/SPARK-52695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-52695. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51388 [https://github.com

[jira] [Assigned] (SPARK-52695) User Defined Type write support for xml file format

[ https://issues.apache.org/jira/browse/SPARK-52695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-52695: Assignee: Kent Yao > User Defined Type write support for xml file format > --

[jira] [Reopened] (SPARK-52698) Improve type hints in datasource module

[ https://issues.apache.org/jira/browse/SPARK-52698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-52698: -- Assignee: (was: Allison Wang) Reverted at https://github.com/apache/spark/commit/00cf5d

[jira] [Updated] (SPARK-52698) Improve type hints in datasource module

[ https://issues.apache.org/jira/browse/SPARK-52698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-52698: - Fix Version/s: (was: 4.1.0) > Improve type hints in datasource module >

[jira] [Commented] (SPARK-31561) Add QUALIFY Clause

[ https://issues.apache.org/jira/browse/SPARK-31561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003641#comment-18003641 ] Hoang Minh Quang commented on SPARK-31561: -- I mean Spark Databrick have the Qua

[jira] [Created] (SPARK-52702) Support using latest generated CRD yaml in Helm charts

Zhou JIANG created SPARK-52702: -- Summary: Support using latest generated CRD yaml in Helm charts Key: SPARK-52702 URL: https://issues.apache.org/jira/browse/SPARK-52702 Project: Spark Issue Type

[jira] [Resolved] (SPARK-52698) Improve type hints in datasource module

[ https://issues.apache.org/jira/browse/SPARK-52698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-52698. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51392 [https://gi

[jira] [Created] (SPARK-52701) Fix float32 type widening in `mod` with bool under ANSI

Xinrong Meng created SPARK-52701: Summary: Fix float32 type widening in `mod` with bool under ANSI Key: SPARK-52701 URL: https://issues.apache.org/jira/browse/SPARK-52701 Project: Spark Issue

[jira] [Updated] (SPARK-52700) Dtype difference for Pandas on Spark

[ https://issues.apache.org/jira/browse/SPARK-52700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-52700: - Summary: Dtype difference for Pandas on Spark (was: Dtype difference) > Dtype difference for P

[jira] [Updated] (SPARK-52700) Dtype difference for Pandas on Spark in ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-52700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-52700: - Summary: Dtype difference for Pandas on Spark in ANSI mode (was: Dtype difference for Pandas on

[jira] [Created] (SPARK-52700) Dtype difference

Xinrong Meng created SPARK-52700: Summary: Dtype difference Key: SPARK-52700 URL: https://issues.apache.org/jira/browse/SPARK-52700 Project: Spark Issue Type: Umbrella Components: P

[jira] [Commented] (SPARK-52681) V2SessionCatalog alter table add column warns on HiveCatalog

[ https://issues.apache.org/jira/browse/SPARK-52681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003631#comment-18003631 ] Szehon Ho commented on SPARK-52681: --- This can be resolved, its fixed as part of https

[jira] [Created] (SPARK-52699) Support aggregating TIME type in interpreted mode

Bruce Robbins created SPARK-52699: - Summary: Support aggregating TIME type in interpreted mode Key: SPARK-52699 URL: https://issues.apache.org/jira/browse/SPARK-52699 Project: Spark Issue Typ

[jira] [Comment Edited] (SPARK-52619) Cast TimeType to IntegralType

[ https://issues.apache.org/jira/browse/SPARK-52619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003624#comment-18003624 ] Mike Artz edited comment on SPARK-52619 at 7/7/25 9:40 PM: --- I

[jira] [Comment Edited] (SPARK-52619) Cast TimeType to IntegralType

[ https://issues.apache.org/jira/browse/SPARK-52619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003624#comment-18003624 ] Mike Artz edited comment on SPARK-52619 at 7/7/25 9:40 PM: --- I

[jira] [Comment Edited] (SPARK-52619) Cast TimeType to IntegralType

[ https://issues.apache.org/jira/browse/SPARK-52619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003624#comment-18003624 ] Mike Artz edited comment on SPARK-52619 at 7/7/25 9:39 PM: --- I

[jira] [Created] (SPARK-52698) Improve type hints in datasource module

Allison Wang created SPARK-52698: Summary: Improve type hints in datasource module Key: SPARK-52698 URL: https://issues.apache.org/jira/browse/SPARK-52698 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-52619) Cast TimeType to IntegralType

[ https://issues.apache.org/jira/browse/SPARK-52619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003624#comment-18003624 ] Mike Artz commented on SPARK-52619: --- I can work on this if no one else is > Cast Time

[jira] [Commented] (SPARK-52505) Connection timeout on Kubernetes to decommissioned executor

[ https://issues.apache.org/jira/browse/SPARK-52505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003623#comment-18003623 ] Enrico Minack commented on SPARK-52505: --- This is fixed in [https://github.com/apac

[jira] [Commented] (SPARK-52507) Quick fallback to fallback storage on fetch failure

[ https://issues.apache.org/jira/browse/SPARK-52507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003621#comment-18003621 ] Enrico Minack commented on SPARK-52507: --- This is fixed in [https://github.com/apac

[jira] [Commented] (SPARK-52506) Migrate to fallback storage only

[ https://issues.apache.org/jira/browse/SPARK-52506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003622#comment-18003622 ] Enrico Minack commented on SPARK-52506: --- This is fixed in [https://github.com/apac

[jira] [Commented] (SPARK-52508) Read from fallback storage should consider replication delay

[ https://issues.apache.org/jira/browse/SPARK-52508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003620#comment-18003620 ] Enrico Minack commented on SPARK-52508: --- This is fixed in https://github.com/apach

[jira] [Commented] (SPARK-52509) Fallback storage accumulates removed shuffle data

[ https://issues.apache.org/jira/browse/SPARK-52509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003619#comment-18003619 ] Enrico Minack commented on SPARK-52509: --- This is fixed in [https://github.com/apac

[jira] [Resolved] (SPARK-51557) Add tests for TIME data type in Java API

[ https://issues.apache.org/jira/browse/SPARK-51557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-51557. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51387 [https://github.com

[jira] [Assigned] (SPARK-51557) Add tests for TIME data type in Java API

[ https://issues.apache.org/jira/browse/SPARK-51557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-51557: Assignee: Mike Artz > Add tests for TIME data type in Java API >

[jira] [Assigned] (SPARK-52679) Revise `appender.console.layout.pattern` to include MDC keys and thread name

[ https://issues.apache.org/jira/browse/SPARK-52679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52679: - Assignee: Zhou JIANG > Revise `appender.console.layout.pattern` to include MDC keys and

[jira] [Resolved] (SPARK-52679) Revise `appender.console.layout.pattern` to include MDC keys and thread name

[ https://issues.apache.org/jira/browse/SPARK-52679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52679. --- Fix Version/s: kubernetes-operator-0.5.0 Resolution: Fixed Issue resolved by pull req

[jira] [Updated] (SPARK-52679) Revise `appender.console.layout.pattern` to include MDC keys and thread name

[ https://issues.apache.org/jira/browse/SPARK-52679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52679: -- Summary: Revise `appender.console.layout.pattern` to include MDC keys and thread name (was: U

[jira] [Commented] (SPARK-51224) Upgrade Maven to 4.0.0

[ https://issues.apache.org/jira/browse/SPARK-51224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003607#comment-18003607 ] Dongjoon Hyun commented on SPARK-51224: --- Since you are working on this actively, I

[jira] [Assigned] (SPARK-51224) Upgrade Maven to 4.0.0

[ https://issues.apache.org/jira/browse/SPARK-51224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51224: - Assignee: Cheng Pan > Upgrade Maven to 4.0.0 > -- > >

[jira] [Resolved] (SPARK-52697) Make Maven plugins up-to-date

[ https://issues.apache.org/jira/browse/SPARK-52697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52697. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51390 [https://

[jira] [Assigned] (SPARK-52697) Make Maven plugins up-to-date

[ https://issues.apache.org/jira/browse/SPARK-52697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52697: - Assignee: Dongjoon Hyun > Make Maven plugins up-to-date > -

[jira] [Updated] (SPARK-52679) Update default log4j2 properties in Helm chart to include MDC keys and thread name

[ https://issues.apache.org/jira/browse/SPARK-52679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou JIANG updated SPARK-52679: --- Summary: Update default log4j2 properties in Helm chart to include MDC keys and thread name (was: B

[jira] [Created] (SPARK-52697) Make Maven plugins up-to-date

Dongjoon Hyun created SPARK-52697: - Summary: Make Maven plugins up-to-date Key: SPARK-52697 URL: https://issues.apache.org/jira/browse/SPARK-52697 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-52544) Allow configuring Json datasource string length limit through SQLConf

[ https://issues.apache.org/jira/browse/SPARK-52544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003576#comment-18003576 ] Anastasia Filippova commented on SPARK-52544: - User 'tianhanhu' has created

[jira] [Commented] (SPARK-52401) Unexpected behavior when using .collect() after modifying an external Spark table

[ https://issues.apache.org/jira/browse/SPARK-52401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18003577#comment-18003577 ] Anastasia Filippova commented on SPARK-52401: - User 'ikshitiz' has created a

[jira] [Updated] (SPARK-52684) Make CACHE TABLE Commands atomic while encounting execution errors

[ https://issues.apache.org/jira/browse/SPARK-52684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52684: -- Fix Version/s: 3.5.7 > Make CACHE TABLE Commands atomic while encounting execution errors > --

[jira] [Created] (SPARK-52696) Strip `__is_duplicate` metadata after analysis

Mihailo Timotic created SPARK-52696: --- Summary: Strip `__is_duplicate` metadata after analysis Key: SPARK-52696 URL: https://issues.apache.org/jira/browse/SPARK-52696 Project: Spark Issue Ty

[jira] [Commented] (SPARK-52650) User Defined Type Improvements

[ https://issues.apache.org/jira/browse/SPARK-52650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18002676#comment-18002676 ] Kent Yao commented on SPARK-52650: -- Thank you [~dongjoon] > User Defined Type Improvem

[jira] [Updated] (SPARK-52695) User Defined Type write support for xml file format

[ https://issues.apache.org/jira/browse/SPARK-52695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-52695: - Summary: User Defined Type write support for xml file format (was: handle User Defined Type in StaxXmlG

[jira] [Created] (SPARK-52695) handle User Defined Type in StaxXmlGenerator

Kent Yao created SPARK-52695: Summary: handle User Defined Type in StaxXmlGenerator Key: SPARK-52695 URL: https://issues.apache.org/jira/browse/SPARK-52695 Project: Spark Issue Type: Sub-task

[jira] [Comment Edited] (SPARK-51557) Add tests for TIME data type in Java API

[ https://issues.apache.org/jira/browse/SPARK-51557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18002681#comment-18002681 ] Mike Artz edited comment on SPARK-51557 at 7/7/25 9:27 AM: --- I

[jira] [Commented] (SPARK-52684) Make CACHE TABLE Commands atomic while encounting execution errors

[ https://issues.apache.org/jira/browse/SPARK-52684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18002679#comment-18002679 ] Kent Yao commented on SPARK-52684: -- Thank you [~dongjoon] I seemed to be distracted wh

[jira] [Commented] (SPARK-51557) Add tests for TIME data type in Java API

[ https://issues.apache.org/jira/browse/SPARK-51557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18002681#comment-18002681 ] Mike Artz commented on SPARK-51557: --- I have a PR for this one > Add tests for TIME da

[jira] [Created] (SPARK-52694) Add a DeveloperApi - org.apache.spark.sql.Encoders#udt

Kent Yao created SPARK-52694: Summary: Add a DeveloperApi - org.apache.spark.sql.Encoders#udt Key: SPARK-52694 URL: https://issues.apache.org/jira/browse/SPARK-52694 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-52692) Support caching the TIME type

[ https://issues.apache.org/jira/browse/SPARK-52692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-52692. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51385 [https://github.com

[jira] [Assigned] (SPARK-52692) Support caching the TIME type

[ https://issues.apache.org/jira/browse/SPARK-52692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-52692: Assignee: Bruce Robbins > Support caching the TIME type > - > >