[jira] [Commented] (SPARK-52866) Add support for `try_to_date`

2025-07-17 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007992#comment-18007992 ] Mihailo Milosevic commented on SPARK-52866: --- Working on this. > Add support f

[jira] [Updated] (SPARK-52866) Add support for `try_to_date`

2025-07-17 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-52866: -- Description: As part of [https://github.com/apache/spark/pull/49227] we have changed t

[jira] [Updated] (SPARK-52866) Add support for `try_to_date`

2025-07-17 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-52866: -- Epic Link: SPARK-49577 > Add support for `try_to_date` > -

[jira] [Updated] (SPARK-52866) Add support for `try_to_date`

2025-07-17 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-52866: -- Description: As part of https://github.com/apache/spark/pull/49227 we have changed the

[jira] [Created] (SPARK-52866) Add support for `try_to_date`

2025-07-17 Thread Mihailo Milosevic (Jira)
Mihailo Milosevic created SPARK-52866: - Summary: Add support for `try_to_date` Key: SPARK-52866 URL: https://issues.apache.org/jira/browse/SPARK-52866 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-52865) Remove usage of deprecated FileCommitProtocol.newTaskTempFile method

2025-07-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52865: --- Labels: pull-request-available (was: ) > Remove usage of deprecated FileCommitProtocol.newT

[jira] [Updated] (SPARK-52865) Remove usage of deprecated FileCommitProtocol.newTaskTempFile method

2025-07-17 Thread morvenhuang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] morvenhuang updated SPARK-52865: Description: Method  {code:java} newTaskTempFile(taskContext: TaskAttemptContext, dir: Option[Stri

[jira] [Updated] (SPARK-52865) Remove usage of deprecated FileCommitProtocol.newTaskTempFile method

2025-07-17 Thread morvenhuang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] morvenhuang updated SPARK-52865: Description: Method  {code:java} newTaskTempFile(taskContext: TaskAttemptContext, dir: Option[Stri

[jira] [Created] (SPARK-52865) Remove usage of deprecated FileCommitProtocol.newTaskTempFile method

2025-07-17 Thread morvenhuang (Jira)
morvenhuang created SPARK-52865: --- Summary: Remove usage of deprecated FileCommitProtocol.newTaskTempFile method Key: SPARK-52865 URL: https://issues.apache.org/jira/browse/SPARK-52865 Project: Spark

[jira] [Updated] (SPARK-52864) Let LocalSparkContext clear active context in beforeAll

2025-07-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52864: --- Labels: pull-request-available (was: ) > Let LocalSparkContext clear active context in befo

[jira] [Created] (SPARK-52864) Let LocalSparkContext clear active context in beforeAll

2025-07-17 Thread Price Qian (Jira)
Price Qian created SPARK-52864: -- Summary: Let LocalSparkContext clear active context in beforeAll Key: SPARK-52864 URL: https://issues.apache.org/jira/browse/SPARK-52864 Project: Spark Issue Typ

[jira] [Commented] (SPARK-52621) Cast TIME to/from VARIANT

2025-07-17 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-52621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007975#comment-18007975 ] Uroš Bojanić commented on SPARK-52621: -- Working on this item. WIP PR: [https://git

[jira] [Updated] (SPARK-52621) Cast TIME to/from VARIANT

2025-07-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52621: --- Labels: pull-request-available (was: ) > Cast TIME to/from VARIANT > --

[jira] [Commented] (SPARK-51562) Add the time() function

2025-07-17 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-51562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007972#comment-18007972 ] Uroš Bojanić commented on SPARK-51562: -- Given that there haven't been any updates i

[jira] [Commented] (SPARK-51554) Add the time_trunc() function

2025-07-17 Thread Sakthi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007974#comment-18007974 ] Sakthi commented on SPARK-51554: I'm sorry but the linked PR is active with active revie

[jira] [Updated] (SPARK-52637) Fix VERSIONID_MISMATCH error when RocksDB compaction leads to incorrect file mapping during load

2025-07-17 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar updated SPARK-52637: --- Fix Version/s: 4.1.0 > Fix VERSIONID_MISMATCH error when RocksDB compaction leads to

[jira] [Updated] (SPARK-51806) Upgrade kryo-shaded to 4.0.3

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51806: -- Affects Version/s: (was: 4.1.0) > Upgrade kryo-shaded to 4.0.3 > -

[jira] [Resolved] (SPARK-52842) New functionality and bugfixes for single-pass analyzer

2025-07-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-52842. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51513 [https://gith

[jira] [Updated] (SPARK-52863) Clean up code paths for old pandas versions

2025-07-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52863: --- Labels: pull-request-available (was: ) > Clean up code paths for old pandas versions >

[jira] [Commented] (SPARK-51564) TIME literals in the 12hr clock format

2025-07-17 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-51564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007967#comment-18007967 ] Uroš Bojanić commented on SPARK-51564: -- Given that there haven't been any updates i

[jira] [Created] (SPARK-52863) Clean up code paths for old pandas versions

2025-07-17 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-52863: - Summary: Clean up code paths for old pandas versions Key: SPARK-52863 URL: https://issues.apache.org/jira/browse/SPARK-52863 Project: Spark Issue Type: Imp

[jira] [Updated] (SPARK-51564) TIME literals in the 12hr clock format

2025-07-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51564: --- Labels: pull-request-available (was: ) > TIME literals in the 12hr clock format > -

[jira] [Updated] (SPARK-52862) Fix the nullability check of Decimal('NaN')

2025-07-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52862: --- Labels: pull-request-available (was: ) > Fix the nullability check of Decimal('NaN') > ---

[jira] [Assigned] (SPARK-52842) New functionality and bugfixes for single-pass analyzer

2025-07-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-52842: --- Assignee: Mihailo Timotic > New functionality and bugfixes for single-pass analyzer > -

[jira] [Created] (SPARK-52862) Fix the nullability check of Decimal('NaN')

2025-07-17 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-52862: - Summary: Fix the nullability check of Decimal('NaN') Key: SPARK-52862 URL: https://issues.apache.org/jira/browse/SPARK-52862 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-51554) Add the time_trunc() function

2025-07-17 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-51554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007962#comment-18007962 ] Uroš Bojanić commented on SPARK-51554: -- Given that there haven't been any updates i

[jira] [Updated] (SPARK-52861) Skip Row object creation in Arrow-optimized UDTF execution

2025-07-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52861: --- Labels: pull-request-available (was: ) > Skip Row object creation in Arrow-optimized UDTF e

[jira] [Updated] (SPARK-52861) Skip Row object creation in Arrow-optimized UDTF execution

2025-07-17 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-52861: -- Summary: Skip Row object creation in Arrow-optimized UDTF execution (was: Skip Row object cre

[jira] [Updated] (SPARK-52861) Skip Row object creation if not necessary

2025-07-17 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-52861: -- Summary: Skip Row object creation if not necessary (was: Skip Row object creation) > Skip Ro

[jira] [Created] (SPARK-52861) Skip Row object creation

2025-07-17 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-52861: - Summary: Skip Row object creation Key: SPARK-52861 URL: https://issues.apache.org/jira/browse/SPARK-52861 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-52511) Support "dry-run" in the spark-pipelines CLI

2025-07-17 Thread Sandy Ryza (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved SPARK-52511. Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51489 [https://github

[jira] [Updated] (SPARK-52858) Retry SBT compilation when OOM

2025-07-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52858: --- Labels: pull-request-available (was: ) > Retry SBT compilation when OOM > -

[jira] [Created] (SPARK-52860) Support write schema evolution for InMemoryTable

2025-07-17 Thread Szehon Ho (Jira)
Szehon Ho created SPARK-52860: - Summary: Support write schema evolution for InMemoryTable Key: SPARK-52860 URL: https://issues.apache.org/jira/browse/SPARK-52860 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-52859) Add `SparkSystemUtils` trait

2025-07-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52859: --- Labels: pull-request-available (was: ) > Add `SparkSystemUtils` trait > ---

[jira] [Created] (SPARK-52859) Add `SparkSystemUtils` trait

2025-07-17 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-52859: - Summary: Add `SparkSystemUtils` trait Key: SPARK-52859 URL: https://issues.apache.org/jira/browse/SPARK-52859 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-52858) Retry SBT compilation when OOM

2025-07-17 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-52858: - Summary: Retry SBT compilation when OOM Key: SPARK-52858 URL: https://issues.apache.org/jira/browse/SPARK-52858 Project: Spark Issue Type: Test C

[jira] [Commented] (SPARK-51415) Support the time type by make_timestamp()

2025-07-17 Thread Sakthi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007945#comment-18007945 ] Sakthi commented on SPARK-51415: Sure thanks for letting me know[~uros-db]  > Support t

[jira] [Comment Edited] (SPARK-51415) Support the time type by make_timestamp()

2025-07-17 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-51415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007944#comment-18007944 ] Uroš Bojanić edited comment on SPARK-51415 at 7/18/25 12:08 AM: --

[jira] [Comment Edited] (SPARK-51415) Support the time type by make_timestamp()

2025-07-17 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-51415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007944#comment-18007944 ] Uroš Bojanić edited comment on SPARK-51415 at 7/18/25 12:08 AM: --

[jira] [Comment Edited] (SPARK-51415) Support the time type by make_timestamp()

2025-07-17 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-51415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007944#comment-18007944 ] Uroš Bojanić edited comment on SPARK-51415 at 7/18/25 12:07 AM: --

[jira] [Commented] (SPARK-51415) Support the time type by make_timestamp()

2025-07-17 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-51415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007944#comment-18007944 ] Uroš Bojanić commented on SPARK-51415: -- Given that there haven't been any updates i

[jira] [Updated] (SPARK-51415) Support the time type by make_timestamp()

2025-07-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51415: --- Labels: pull-request-available (was: ) > Support the time type by make_timestamp() > --

[jira] [Updated] (SPARK-52849) Add `stringifyException` to `o.a.s.util.Utils`

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52849: -- Parent: SPARK-51166 Issue Type: Sub-task (was: Improvement) > Add `stringifyException

[jira] [Updated] (SPARK-52850) Skip calling conversions if identity function.

2025-07-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52850: --- Labels: pull-request-available (was: ) > Skip calling conversions if identity function. > -

[jira] [Commented] (SPARK-52637) Fix VERSIONID_MISMATCH error when RocksDB compaction leads to incorrect file mapping during load

2025-07-17 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007926#comment-18007926 ] Jungtaek Lim commented on SPARK-52637: -- [~anishshri-db] Shall we update the fixed v

[jira] [Resolved] (SPARK-52849) Add `stringifyException` to `o.a.s.util.Utils`

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52849. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51540 [https://

[jira] [Assigned] (SPARK-52849) Add `stringifyException` to `o.a.s.util.Utils`

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52849: - Assignee: Dongjoon Hyun > Add `stringifyException` to `o.a.s.util.Utils` >

[jira] [Assigned] (SPARK-52833) Fix VariantBuilder.appendFloat

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52833: - Assignee: Chenhao Li > Fix VariantBuilder.appendFloat > --

[jira] [Resolved] (SPARK-52833) Fix VariantBuilder.appendFloat

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52833. --- Fix Version/s: 4.1.0 4.0.1 Resolution: Fixed Issue resolved by pul

[jira] [Updated] (SPARK-45302) Remove PID communication between Python workers when no demon is used

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45302: -- Parent Issue: SPARK-51166 (was: SPARK-45315) > Remove PID communication between Python worker

[jira] [Updated] (SPARK-45302) Remove PID communication between Python workers when no demon is used

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45302: -- Affects Version/s: 4.1.0 (was: 4.0.0) > Remove PID communication be

[jira] [Closed] (SPARK-45314) Drop Scala 2.12 and make Scala 2.13 by default

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-45314. - > Drop Scala 2.12 and make Scala 2.13 by default > -- >

[jira] [Closed] (SPARK-44111) Prepare Apache Spark 4.0.0

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-44111. - > Prepare Apache Spark 4.0.0 > -- > > Key: SPARK-44111 >

[jira] [Closed] (SPARK-47046) Apache Spark 4.0.0 Dependency Audit and Cleanup

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-47046. - > Apache Spark 4.0.0 Dependency Audit and Cleanup > ---

[jira] [Closed] (SPARK-45923) Spark Kubernetes Operator

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-45923. - > Spark Kubernetes Operator > - > > Key: SPARK-45923 >

[jira] [Closed] (SPARK-43831) Build and Run Spark on Java 21

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-43831. - > Build and Run Spark on Java 21 > -- > > Key: SPARK-438

[jira] [Closed] (SPARK-44112) Drop Java 8 and 11 support

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-44112. - > Drop Java 8 and 11 support > -- > > Key: SPARK-44112 >

[jira] [Closed] (SPARK-45869) Revisit and Improve Spark Standalone Cluster

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-45869. - > Revisit and Improve Spark Standalone Cluster > > >

[jira] [Closed] (SPARK-49524) Improve K8s support

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-49524. - > Improve K8s support > --- > > Key: SPARK-49524 > U

[jira] [Updated] (SPARK-52857) Improve `Variant` data type support

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52857: -- Issue Type: Umbrella (was: Improvement) > Improve `Variant` data type support > -

[jira] [Commented] (SPARK-52857) Improve `Variant` data type support

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007923#comment-18007923 ] Dongjoon Hyun commented on SPARK-52857: --- Thank you for working on `Variant` type,

[jira] [Closed] (SPARK-45891) Support Variant data type

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-45891. - > Support Variant data type > - > > Key: SPARK-45891 >

[jira] [Updated] (SPARK-51298) Support variant in CSV scan

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51298: -- Parent Issue: SPARK-52857 (was: SPARK-45891) > Support variant in CSV scan >

[jira] [Updated] (SPARK-51503) Support variant in XML scan

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51503: -- Parent Issue: SPARK-52857 (was: SPARK-45891) > Support variant in XML scan >

[jira] [Updated] (SPARK-51716) Support serializing Variant as XML

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51716: -- Parent Issue: SPARK-52857 (was: SPARK-45891) > Support serializing Variant as XML > -

[jira] [Updated] (SPARK-51296) Support collecting corrupt data in singleVariantColumn mode

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51296: -- Parent Issue: SPARK-52857 (was: SPARK-45891) > Support collecting corrupt data in singleVaria

[jira] [Updated] (SPARK-52153) Fix from_json and to_json with variant

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52153: -- Parent Issue: SPARK-52857 (was: SPARK-45891) > Fix from_json and to_json with variant > -

[jira] [Updated] (SPARK-51848) XML failed to be parsed with a defined schema of array/struct of Variant fields

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51848: -- Parent Issue: SPARK-52857 (was: SPARK-45891) > XML failed to be parsed with a defined schema

[jira] [Updated] (SPARK-52049) XML attributes can't be parsed as variant type

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52049: -- Parent Issue: SPARK-52857 (was: SPARK-45891) > XML attributes can't be parsed as variant type

[jira] [Updated] (SPARK-52495) Include partition columns in the single variant column

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52495: -- Parent Issue: SPARK-52857 (was: SPARK-45891) > Include partition columns in the single varian

[jira] [Updated] (SPARK-50321) Rebuild variant binary from shredded data.

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50321: -- Parent Issue: SPARK-52857 (was: SPARK-45891) > Rebuild variant binary from shredded data. > -

[jira] [Updated] (SPARK-49723) Add variant metrics to JSON Scan nodes and Project nodes containing variant-constructor expressions

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49723: -- Parent Issue: SPARK-52857 (was: SPARK-45891) > Add variant metrics to JSON Scan nodes and Pro

[jira] [Resolved] (SPARK-48994) Remove support for interval types in Variant

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48994. --- Fix Version/s: 4.0.0 Resolution: Fixed This was resolved via [https://github.com/apac

[jira] [Updated] (SPARK-48994) Remove support for interval types in Variant

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48994: -- Summary: Remove support for interval types in Variant (was: Add support for interval types i

[jira] [Assigned] (SPARK-48994) Remove support for interval types in Variant

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48994: - Assignee: Harsh Motwani > Remove support for interval types in Variant > -

[jira] [Updated] (SPARK-52857) Improve `Variant` data type support

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52857: -- Summary: Improve `Variant` data type support (was: Improve `Variant` support) > Improve `Var

[jira] [Updated] (SPARK-48051) Add Golden Table Tests for Variant from different engines

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48051: -- Parent Issue: SPARK-52857 (was: SPARK-45891) > Add Golden Table Tests for Variant from differ

[jira] [Updated] (SPARK-52833) Fix VariantBuilder.appendFloat

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52833: -- Parent Issue: SPARK-52857 (was: SPARK-45891) > Fix VariantBuilder.appendFloat > -

[jira] [Created] (SPARK-52857) Improve `Variant` support

2025-07-17 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-52857: - Summary: Improve `Variant` support Key: SPARK-52857 URL: https://issues.apache.org/jira/browse/SPARK-52857 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-52843) Support retry timeout tests

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52843. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51537 [https://

[jira] [Assigned] (SPARK-52843) Support retry timeout tests

2025-07-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52843: - Assignee: Ruifeng Zheng > Support retry timeout tests > --- > >

[jira] [Commented] (SPARK-51413) Support TIME in Arrow

2025-07-17 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-51413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007916#comment-18007916 ] Uroš Bojanić commented on SPARK-51413: -- This issue was resolved as part of https:/

[jira] [Assigned] (SPARK-52641) Primary key columns should not be nullable

2025-07-17 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-52641: -- Assignee: Gengliang Wang > Primary key columns should not be nullable > -

[jira] [Resolved] (SPARK-52641) Primary key columns should not be nullable

2025-07-17 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-52641. Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51525 [https:

[jira] [Created] (SPARK-52856) Prevent session state mutation from within Pipelines Python definition files

2025-07-17 Thread Sandy Ryza (Jira)
Sandy Ryza created SPARK-52856: -- Summary: Prevent session state mutation from within Pipelines Python definition files Key: SPARK-52856 URL: https://issues.apache.org/jira/browse/SPARK-52856 Project: Spa

[jira] [Updated] (SPARK-52854) Prevent setting catalog and database on the session within Pipelines Python definition files

2025-07-17 Thread Sandy Ryza (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-52854: --- Parent: SPARK-52856 Issue Type: Sub-task (was: Improvement) > Prevent setting catalog and d

[jira] [Updated] (SPARK-52853) Prevent setting Spark confs on the session within Pipelines Python definition files

2025-07-17 Thread Sandy Ryza (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-52853: --- Parent: SPARK-52856 Issue Type: Sub-task (was: Improvement) > Prevent setting Spark confs o

[jira] [Updated] (SPARK-52855) Prevent creating and dropping temp views on the session within Pipelines Python definition files

2025-07-17 Thread Sandy Ryza (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-52855: --- Parent: SPARK-52856 Issue Type: Sub-task (was: Improvement) > Prevent creating and dropping

[jira] [Created] (SPARK-52855) Prevent creating and dropping temp views on the session within Pipelines Python definition files

2025-07-17 Thread Sandy Ryza (Jira)
Sandy Ryza created SPARK-52855: -- Summary: Prevent creating and dropping temp views on the session within Pipelines Python definition files Key: SPARK-52855 URL: https://issues.apache.org/jira/browse/SPARK-52855

[jira] [Resolved] (SPARK-52637) Fix VERSIONID_MISMATCH error when RocksDB compaction leads to incorrect file mapping during load

2025-07-17 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar resolved SPARK-52637. Resolution: Fixed > Fix VERSIONID_MISMATCH error when RocksDB compaction leads to

[jira] [Closed] (SPARK-52637) Fix VERSIONID_MISMATCH error when RocksDB compaction leads to incorrect file mapping during load

2025-07-17 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar closed SPARK-52637. -- > Fix VERSIONID_MISMATCH error when RocksDB compaction leads to incorrect file > mapping

[jira] [Assigned] (SPARK-52637) Fix VERSIONID_MISMATCH error when RocksDB compaction leads to incorrect file mapping during load

2025-07-17 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar reassigned SPARK-52637: -- Assignee: Livia Zhu > Fix VERSIONID_MISMATCH error when RocksDB compaction le

[jira] [Commented] (SPARK-52637) Fix VERSIONID_MISMATCH error when RocksDB compaction leads to incorrect file mapping during load

2025-07-17 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007912#comment-18007912 ] Anish Shrigondekar commented on SPARK-52637: PR merged here -> https://githu

[jira] [Created] (SPARK-52854) Prevent setting catalog and database on the session within Pipelines Python definition files

2025-07-17 Thread Sandy Ryza (Jira)
Sandy Ryza created SPARK-52854: -- Summary: Prevent setting catalog and database on the session within Pipelines Python definition files Key: SPARK-52854 URL: https://issues.apache.org/jira/browse/SPARK-52854

[jira] [Updated] (SPARK-52853) Prevent setting Spark confs on the session within Pipelines Python definition files

2025-07-17 Thread Sandy Ryza (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-52853: --- Description: Setting spark session confs is an imperative construct that can cause friction and une

[jira] [Created] (SPARK-52853) Prevent setting Spark confs on the session within Pipelines Python definition files

2025-07-17 Thread Sandy Ryza (Jira)
Sandy Ryza created SPARK-52853: -- Summary: Prevent setting Spark confs on the session within Pipelines Python definition files Key: SPARK-52853 URL: https://issues.apache.org/jira/browse/SPARK-52853 Proje

[jira] [Created] (SPARK-52852) Remove spark_conf argument from create_streaming_table

2025-07-17 Thread Sandy Ryza (Jira)
Sandy Ryza created SPARK-52852: -- Summary: Remove spark_conf argument from create_streaming_table Key: SPARK-52852 URL: https://issues.apache.org/jira/browse/SPARK-52852 Project: Spark Issue Type

[jira] [Created] (SPARK-52851) Remove public APIs for append once flows

2025-07-17 Thread Sandy Ryza (Jira)
Sandy Ryza created SPARK-52851: -- Summary: Remove public APIs for append once flows Key: SPARK-52851 URL: https://issues.apache.org/jira/browse/SPARK-52851 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-52142) Display table constraints in SHOW CREATE TABLE COMMAND

2025-07-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52142: --- Labels: pull-request-available (was: ) > Display table constraints in SHOW CREATE TABLE COM

[jira] [Assigned] (SPARK-52640) [SDP] Propagate Python Source Code Location

2025-07-17 Thread Sandy Ryza (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-52640: -- Assignee: Anish Mahto > [SDP] Propagate Python Source Code Location > ---

  1   2   >