[jira] [Created] (SPARK-47080) Fix `HistoryServerSuite` to get `getNumJobs` in `eventually`

Dongjoon Hyun created SPARK-47080: - Summary: Fix `HistoryServerSuite` to get `getNumJobs` in `eventually` Key: SPARK-47080 URL: https://issues.apache.org/jira/browse/SPARK-47080 Project: Spark

[jira] [Resolved] (SPARK-47057) Reenable MyPy data test

[ https://issues.apache.org/jira/browse/SPARK-47057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47057. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45135

[jira] [Assigned] (SPARK-47057) Reenable MyPy data test

[ https://issues.apache.org/jira/browse/SPARK-47057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47057: Assignee: Hyukjin Kwon > Reenable MyPy data test > --- > >

[jira] [Updated] (SPARK-47079) Unable to create PySpark dataframe containing Variant columns

[ https://issues.apache.org/jira/browse/SPARK-47079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Desmond Cheong updated SPARK-47079: --- Description: Trying to create a dataframe containing a variant type results in:

[jira] [Updated] (SPARK-47079) Unable to create PySpark dataframe containing Variant columns

[ https://issues.apache.org/jira/browse/SPARK-47079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Desmond Cheong updated SPARK-47079: --- Description: Trying to create a dataframe containing a variant type results in

[jira] [Updated] (SPARK-47079) Unable to create PySpark dataframe containing Variant columns

[ https://issues.apache.org/jira/browse/SPARK-47079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Desmond Cheong updated SPARK-47079: --- Description: Trying to create a dataframe containing a variant type results in

[jira] [Created] (SPARK-47079) Unable to create PySpark dataframe containing Variant columns

Desmond Cheong created SPARK-47079: -- Summary: Unable to create PySpark dataframe containing Variant columns Key: SPARK-47079 URL: https://issues.apache.org/jira/browse/SPARK-47079 Project: Spark

[jira] [Resolved] (SPARK-47077) sbt build is broken due to selenium change

[ https://issues.apache.org/jira/browse/SPARK-47077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-47077. -- Resolution: Cannot Reproduce After blowing away my maven + ivy cache it works fine – should

[jira] [Updated] (SPARK-42285) Introduce conf spark.sql.parquet.inferTimestampNTZ.enabled for TimestampNTZ inference on Parquet

[ https://issues.apache.org/jira/browse/SPARK-42285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42285: --- Labels: pull-request-available (was: ) > Introduce conf

[jira] [Created] (SPARK-47078) Documentation for SparkSession-based Profilers

Xinrong Meng created SPARK-47078: Summary: Documentation for SparkSession-based Profilers Key: SPARK-47078 URL: https://issues.apache.org/jira/browse/SPARK-47078 Project: Spark Issue Type:

[jira] [Updated] (SPARK-47077) sbt build is broken due to selenium change

[ https://issues.apache.org/jira/browse/SPARK-47077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47077: --- Labels: pull-request-available (was: ) > sbt build is broken due to selenium change >

[jira] [Created] (SPARK-47077) sbt build is broken due to selenium change

Holden Karau created SPARK-47077: Summary: sbt build is broken due to selenium change Key: SPARK-47077 URL: https://issues.apache.org/jira/browse/SPARK-47077 Project: Spark Issue Type:

[jira] [Updated] (SPARK-47001) Pushdown Verification in Optimizer.scala should support changed data types

[ https://issues.apache.org/jira/browse/SPARK-47001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-47001: - Description: When pushing a filter down in a union the data type may not match exactly if the

[jira] [Resolved] (SPARK-47076) Fix HistoryServerSuite.`incomplete apps get refreshed` test to start with empty storeDir

[ https://issues.apache.org/jira/browse/SPARK-47076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47076. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45143

[jira] [Updated] (SPARK-47076) Fix HistoryServerSuite.`incomplete apps get refreshed` test to start with empty storeDir

[ https://issues.apache.org/jira/browse/SPARK-47076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47076: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Bug) > Fix

[jira] [Updated] (SPARK-47076) Fix HistoryServerSuite.`incomplete apps get refreshed` test to start with empty storeDir

[ https://issues.apache.org/jira/browse/SPARK-47076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47076: -- Summary: Fix HistoryServerSuite.`incomplete apps get refreshed` test to start with empty

[jira] [Assigned] (SPARK-47076) Flaky test: HistoryServerSuite - incomplete apps get refreshed

[ https://issues.apache.org/jira/browse/SPARK-47076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47076: - Assignee: Dongjoon Hyun > Flaky test: HistoryServerSuite - incomplete apps get

[jira] [Updated] (SPARK-47076) Flaky test: HistoryServerSuite - incomplete apps get refreshed

[ https://issues.apache.org/jira/browse/SPARK-47076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47076: --- Labels: pull-request-available (was: ) > Flaky test: HistoryServerSuite - incomplete apps

[jira] [Resolved] (SPARK-47075) Add `derby-provided` profile

[ https://issues.apache.org/jira/browse/SPARK-47075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47075. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45138

[jira] [Assigned] (SPARK-47074) Fix outdated comments in GitHub Action scripts

[ https://issues.apache.org/jira/browse/SPARK-47074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47074: - Assignee: Dongjoon Hyun > Fix outdated comments in GitHub Action scripts >

[jira] [Resolved] (SPARK-47074) Fix outdated comments in GitHub Action scripts

[ https://issues.apache.org/jira/browse/SPARK-47074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47074. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45137

[jira] [Updated] (SPARK-47032) Create API for 'analyze' method to send input column(s) to output table unchanged

[ https://issues.apache.org/jira/browse/SPARK-47032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47032: --- Labels: pull-request-available (was: ) > Create API for 'analyze' method to send input

[jira] [Updated] (SPARK-47076) Flaky test: HistoryServerSuite - incomplete apps get refreshed

[ https://issues.apache.org/jira/browse/SPARK-47076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47076: -- Description: This has been observed multiple times. {code:java} [info] - incomplete apps get

[jira] [Created] (SPARK-47076) Flaky test: HistoryServerSuite - incomplete apps get refreshed

Dongjoon Hyun created SPARK-47076: - Summary: Flaky test: HistoryServerSuite - incomplete apps get refreshed Key: SPARK-47076 URL: https://issues.apache.org/jira/browse/SPARK-47076 Project: Spark

[jira] [Updated] (SPARK-47070) Subquery rewrite inside an aggregation makes an aggregation invalid

[ https://issues.apache.org/jira/browse/SPARK-47070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47070: --- Labels: pull-request-available (was: ) > Subquery rewrite inside an aggregation makes an

[jira] [Commented] (SPARK-45357) Maven test `SparkConnectProtoSuite` failed

[ https://issues.apache.org/jira/browse/SPARK-45357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17818057#comment-17818057 ] Dongjoon Hyun commented on SPARK-45357: --- This is backported to branch-3.5 via

[jira] [Updated] (SPARK-45357) Maven test `SparkConnectProtoSuite` failed

[ https://issues.apache.org/jira/browse/SPARK-45357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45357: -- Fix Version/s: 3.5.2 > Maven test `SparkConnectProtoSuite` failed >

[jira] [Closed] (SPARK-47019) AQE dynamic cache partitioning causes SortMergeJoin to result in data loss

[ https://issues.apache.org/jira/browse/SPARK-47019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ridvan Appa Bugis closed SPARK-47019. - closing > AQE dynamic cache partitioning causes SortMergeJoin to result in data loss >

[jira] [Resolved] (SPARK-47073) Upgrade several Maven plugins to the latest versions

[ https://issues.apache.org/jira/browse/SPARK-47073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-47073. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45136

[jira] [Commented] (SPARK-44027) create *permanent* Spark View from DataFrame via PySpark & Scala DataFrame API

[ https://issues.apache.org/jira/browse/SPARK-44027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17817946#comment-17817946 ] Ahmed Sobeh commented on SPARK-44027: - is it ok if I pick this up? Is it actually newbie level? >

[jira] [Resolved] (SPARK-47072) Wrong error message for incorrect ANSI intervals

[ https://issues.apache.org/jira/browse/SPARK-47072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-47072. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45127

[jira] [Assigned] (SPARK-47075) Add `derby-provided` profile

[ https://issues.apache.org/jira/browse/SPARK-47075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47075: - Assignee: Dongjoon Hyun > Add `derby-provided` profile >

[jira] [Updated] (SPARK-47075) Add `derby-provided` profile

[ https://issues.apache.org/jira/browse/SPARK-47075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47075: --- Labels: pull-request-available (was: ) > Add `derby-provided` profile >

[jira] [Created] (SPARK-47075) Add `derby-provided` profile

Dongjoon Hyun created SPARK-47075: - Summary: Add `derby-provided` profile Key: SPARK-47075 URL: https://issues.apache.org/jira/browse/SPARK-47075 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-47074) Fix outdated comments in GitHub Action scripts

[ https://issues.apache.org/jira/browse/SPARK-47074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47074: -- Summary: Fix outdated comments in GitHub Action scripts (was: Update comments in GitHub

[jira] [Updated] (SPARK-47074) Fix outdated comments in GitHub Action scripts

[ https://issues.apache.org/jira/browse/SPARK-47074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47074: --- Labels: pull-request-available (was: ) > Fix outdated comments in GitHub Action scripts >

[jira] [Created] (SPARK-47074) Update comments in GitHub Action scripts

Dongjoon Hyun created SPARK-47074: - Summary: Update comments in GitHub Action scripts Key: SPARK-47074 URL: https://issues.apache.org/jira/browse/SPARK-47074 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-47072) Wrong error message for incorrect ANSI intervals

[ https://issues.apache.org/jira/browse/SPARK-47072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-47072: - Affects Version/s: 3.5.0 > Wrong error message for incorrect ANSI intervals >

[jira] [Updated] (SPARK-47072) Wrong error message for incorrect ANSI intervals

[ https://issues.apache.org/jira/browse/SPARK-47072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-47072: - Affects Version/s: 3.4.2 > Wrong error message for incorrect ANSI intervals >

[jira] [Resolved] (SPARK-47060) Check SparkIllegalArgumentException instead of IllegalArgumentException in catalyst

[ https://issues.apache.org/jira/browse/SPARK-47060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-47060. -- Resolution: Fixed Issue resolved by pull request 45118 [https://github.com/apache/spark/pull/45118]

[jira] [Updated] (SPARK-47073) Upgrade `versions-maven-plugin` to 2.16.2

[ https://issues.apache.org/jira/browse/SPARK-47073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47073: --- Labels: pull-request-available (was: ) > Upgrade `versions-maven-plugin` to 2.16.2 >

[jira] [Resolved] (SPARK-47071) inline With expression if it contains special expression

[ https://issues.apache.org/jira/browse/SPARK-47071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47071. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45134

[jira] [Updated] (SPARK-47057) Reenable MyPy data test

[ https://issues.apache.org/jira/browse/SPARK-47057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47057: --- Labels: pull-request-available (was: ) > Reenable MyPy data test > ---