[jira] [Resolved] (SPARK-46392) In Function DataSourceStrategy.translateFilterWithMapping, we need transfer cast expression to data source for filtering

[ https://issues.apache.org/jira/browse/SPARK-46392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiahong.li resolved SPARK-46392. Resolution: Abandoned > In Function DataSourceStrategy.translateFilterWithMapping, we need

[jira] [Updated] (SPARK-46473) Reuse `getPartitionedFile` method

[ https://issues.apache.org/jira/browse/SPARK-46473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46473: --- Labels: pull-request-available (was: ) > Reuse `getPartitionedFile` method >

[jira] [Created] (SPARK-46473) Reuse `getPartitionedFile` method

xiaoping.huang created SPARK-46473: -- Summary: Reuse `getPartitionedFile` method Key: SPARK-46473 URL: https://issues.apache.org/jira/browse/SPARK-46473 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46472) Refine docstring of `array_prepend/array_append/array_insert`

[ https://issues.apache.org/jira/browse/SPARK-46472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46472: --- Labels: pull-request-available (was: ) > Refine docstring of

[jira] [Created] (SPARK-46472) Refine docstring of `array_prepend/array_append/array_insert`

Yang Jie created SPARK-46472: Summary: Refine docstring of `array_prepend/array_append/array_insert` Key: SPARK-46472 URL: https://issues.apache.org/jira/browse/SPARK-46472 Project: Spark Issue

[jira] [Assigned] (SPARK-46469) Clean up useless local variables in `InsertIntoHiveTable`

[ https://issues.apache.org/jira/browse/SPARK-46469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-46469: Assignee: Yang Jie > Clean up useless local variables in `InsertIntoHiveTable` >

[jira] [Resolved] (SPARK-46469) Clean up useless local variables in `InsertIntoHiveTable`

[ https://issues.apache.org/jira/browse/SPARK-46469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-46469. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44433

[jira] [Updated] (SPARK-46471) Reorganize `OpsOnDiffFramesEnabledTests`

[ https://issues.apache.org/jira/browse/SPARK-46471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46471: --- Labels: pull-request-available (was: ) > Reorganize `OpsOnDiffFramesEnabledTests` >

[jira] [Created] (SPARK-46471) Reorganize `OpsOnDiffFramesEnabledTests`

Ruifeng Zheng created SPARK-46471: - Summary: Reorganize `OpsOnDiffFramesEnabledTests` Key: SPARK-46471 URL: https://issues.apache.org/jira/browse/SPARK-46471 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46470) Move `test_series_datetime` to `pyspark.pandas.tests.connect.series.*`

[ https://issues.apache.org/jira/browse/SPARK-46470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46470: --- Labels: pull-request-available (was: ) > Move `test_series_datetime` to

[jira] [Updated] (SPARK-46469) Clean up useless local variables in `InsertIntoHiveTable`

[ https://issues.apache.org/jira/browse/SPARK-46469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46469: --- Labels: pull-request-available (was: ) > Clean up useless local variables in

[jira] [Created] (SPARK-46469) Clean up useless local variables in `InsertIntoHiveTable`

Yang Jie created SPARK-46469: Summary: Clean up useless local variables in `InsertIntoHiveTable` Key: SPARK-46469 URL: https://issues.apache.org/jira/browse/SPARK-46469 Project: Spark Issue

[jira] [Assigned] (SPARK-46207) Support MergeInto in DataFrameWriterV2

[ https://issues.apache.org/jira/browse/SPARK-46207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng reassigned SPARK-46207: -- Assignee: Huaxin Gao > Support MergeInto in DataFrameWriterV2 >

[jira] [Resolved] (SPARK-46207) Support MergeInto in DataFrameWriterV2

[ https://issues.apache.org/jira/browse/SPARK-46207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-46207. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44119

[jira] [Commented] (SPARK-46392) In Function DataSourceStrategy.translateFilterWithMapping, we need transfer cast expression to data source for filtering

[ https://issues.apache.org/jira/browse/SPARK-46392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799227#comment-17799227 ] jiahong.li commented on SPARK-46392: pr: https://github.com/apache/spark/pull/44431 > In Function

[jira] [Updated] (SPARK-46468) COUNT bug in lateral/exists subqueries

[ https://issues.apache.org/jira/browse/SPARK-46468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gubichev updated SPARK-46468: Description: Some further instances of a COUNT bug.   One example is this test from

[jira] [Created] (SPARK-46468) COUNT bug in lateral/exists subqueries

Andrey Gubichev created SPARK-46468: --- Summary: COUNT bug in lateral/exists subqueries Key: SPARK-46468 URL: https://issues.apache.org/jira/browse/SPARK-46468 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-46465) Implement Column.isNaN

[ https://issues.apache.org/jira/browse/SPARK-46465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46465: Assignee: Ruifeng Zheng > Implement Column.isNaN > -- > >

[jira] [Resolved] (SPARK-46465) Implement Column.isNaN

[ https://issues.apache.org/jira/browse/SPARK-46465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46465. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44422

[jira] [Assigned] (SPARK-46462) Reorganize `OpsOnDiffFramesGroupByRollingTests`

[ https://issues.apache.org/jira/browse/SPARK-46462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46462: Assignee: Ruifeng Zheng > Reorganize `OpsOnDiffFramesGroupByRollingTests` >

[jira] [Resolved] (SPARK-46462) Reorganize `OpsOnDiffFramesGroupByRollingTests`

[ https://issues.apache.org/jira/browse/SPARK-46462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46462. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44420

[jira] [Resolved] (SPARK-46463) Reorganize `OpsOnDiffFramesGroupByExpandingTests`

[ https://issues.apache.org/jira/browse/SPARK-46463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46463. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44421

[jira] [Assigned] (SPARK-46463) Reorganize `OpsOnDiffFramesGroupByExpandingTests`

[ https://issues.apache.org/jira/browse/SPARK-46463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46463: Assignee: Ruifeng Zheng > Reorganize `OpsOnDiffFramesGroupByExpandingTests` >

[jira] [Commented] (SPARK-46466) vectorized parquet reader should never do rebase for timestamp ntz

[ https://issues.apache.org/jira/browse/SPARK-46466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799207#comment-17799207 ] Dongjoon Hyun commented on SPARK-46466: --- According to the PR content, I added the `correctness`

[jira] [Updated] (SPARK-46466) vectorized parquet reader should never do rebase for timestamp ntz

[ https://issues.apache.org/jira/browse/SPARK-46466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46466: -- Target Version/s: 3.5.1, 3.4.3 > vectorized parquet reader should never do rebase for

[jira] [Updated] (SPARK-46466) vectorized parquet reader should never do rebase for timestamp ntz

[ https://issues.apache.org/jira/browse/SPARK-46466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46466: -- Labels: correctness pull-request-available (was: pull-request-available) > vectorized

[jira] [Updated] (SPARK-46466) vectorized parquet reader should never do rebase for timestamp ntz

[ https://issues.apache.org/jira/browse/SPARK-46466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46466: -- Priority: Blocker (was: Major) > vectorized parquet reader should never do rebase for

[jira] [Resolved] (SPARK-46413) Validate returnType of Arrow Python UDF

[ https://issues.apache.org/jira/browse/SPARK-46413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46413. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44362

[jira] [Resolved] (SPARK-46398) Test rangeBetween window function (pyspark.sql.window)

[ https://issues.apache.org/jira/browse/SPARK-46398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46398. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44339

[jira] [Assigned] (SPARK-46398) Test rangeBetween window function (pyspark.sql.window)

[ https://issues.apache.org/jira/browse/SPARK-46398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46398: Assignee: Xinrong Meng > Test rangeBetween window function (pyspark.sql.window) >

[jira] [Resolved] (SPARK-46456) Shutdown hook timeouts during ui stop

[ https://issues.apache.org/jira/browse/SPARK-46456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46456. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44413

[jira] [Commented] (SPARK-46450) session_window doesn't identify sessions with provided gap when used as a window function

[ https://issues.apache.org/jira/browse/SPARK-46450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799140#comment-17799140 ] Juan Pumarino commented on SPARK-46450: --- [~kabhwan] thanks for the explanation; I learned a bit

[jira] [Created] (SPARK-46467) Improve and test exceptions of TimedeltaIndex

Xinrong Meng created SPARK-46467: Summary: Improve and test exceptions of TimedeltaIndex Key: SPARK-46467 URL: https://issues.apache.org/jira/browse/SPARK-46467 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46467) Improve and test exceptions of TimedeltaIndex

[ https://issues.apache.org/jira/browse/SPARK-46467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46467: --- Labels: pull-request-available (was: ) > Improve and test exceptions of TimedeltaIndex >

[jira] [Resolved] (SPARK-46447) Remove the legacy datetime rebasing SQL configs

[ https://issues.apache.org/jira/browse/SPARK-46447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46447. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44402

[jira] [Updated] (SPARK-46466) vectorized parquet reader should never do rebase for timestamp ntz

[ https://issues.apache.org/jira/browse/SPARK-46466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46466: --- Labels: pull-request-available (was: ) > vectorized parquet reader should never do rebase

[jira] [Created] (SPARK-46466) vectorized parquet reader should never do rebase for timestamp ntz

Wenchen Fan created SPARK-46466: --- Summary: vectorized parquet reader should never do rebase for timestamp ntz Key: SPARK-46466 URL: https://issues.apache.org/jira/browse/SPARK-46466 Project: Spark

[jira] [Commented] (SPARK-46461) The `sbt console` command is not available

[ https://issues.apache.org/jira/browse/SPARK-46461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798975#comment-17798975 ] Yang Jie commented on SPARK-46461: -- cc [~srowen] [~dongjoon] [~gurwls223]  I'm not certain if `sbt

[jira] [Resolved] (SPARK-46452) Add a new API in DSv2 DataWriter to write an iterator of records

[ https://issues.apache.org/jira/browse/SPARK-46452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46452. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44410

[jira] [Assigned] (SPARK-46452) Add a new API in DSv2 DataWriter to write an iterator of records

[ https://issues.apache.org/jira/browse/SPARK-46452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46452: --- Assignee: Allison Wang > Add a new API in DSv2 DataWriter to write an iterator of records

[jira] [Updated] (SPARK-46464) Fix the scroll issue of tables when overflow

[ https://issues.apache.org/jira/browse/SPARK-46464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46464: --- Labels: pull-request-available (was: ) > Fix the scroll issue of tables when overflow >

[jira] [Updated] (SPARK-46465) Implement Column.isNaN

[ https://issues.apache.org/jira/browse/SPARK-46465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46465: --- Labels: pull-request-available (was: ) > Implement Column.isNaN > -- >

[jira] [Created] (SPARK-46464) Fix the scroll issue of tables when overflow

Kent Yao created SPARK-46464: Summary: Fix the scroll issue of tables when overflow Key: SPARK-46464 URL: https://issues.apache.org/jira/browse/SPARK-46464 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-46463) Reorganize `OpsOnDiffFramesGroupByExpandingTests`

[ https://issues.apache.org/jira/browse/SPARK-46463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46463: --- Labels: pull-request-available (was: ) > Reorganize `OpsOnDiffFramesGroupByExpandingTests`

[jira] [Created] (SPARK-46463) Reorganize `OpsOnDiffFramesGroupByExpandingTests`

Ruifeng Zheng created SPARK-46463: - Summary: Reorganize `OpsOnDiffFramesGroupByExpandingTests` Key: SPARK-46463 URL: https://issues.apache.org/jira/browse/SPARK-46463 Project: Spark Issue

[jira] [Assigned] (SPARK-46052) Remove unnecessary TaskScheduler.killAllTaskAttempts

[ https://issues.apache.org/jira/browse/SPARK-46052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46052: -- Assignee: Apache Spark > Remove unnecessary TaskScheduler.killAllTaskAttempts >

[jira] [Assigned] (SPARK-46052) Remove unnecessary TaskScheduler.killAllTaskAttempts

[ https://issues.apache.org/jira/browse/SPARK-46052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46052: -- Assignee: (was: Apache Spark) > Remove unnecessary

[jira] [Resolved] (SPARK-46330) Loading of Spark UI blocks for a long time when HybridStore enabled

[ https://issues.apache.org/jira/browse/SPARK-46330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-46330. -- Fix Version/s: 3.4.3 3.5.1 4.0.0 Resolution: Fixed Issue

[jira] [Assigned] (SPARK-46330) Loading of Spark UI blocks for a long time when HybridStore enabled

[ https://issues.apache.org/jira/browse/SPARK-46330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-46330: Assignee: Zhou Yifan > Loading of Spark UI blocks for a long time when HybridStore enabled >

[jira] [Updated] (SPARK-46460) The filter of partition including cast function may lead the partition pruning to disable

[ https://issues.apache.org/jira/browse/SPARK-46460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou Tong updated SPARK-46460: -- Summary: The filter of partition including cast function may lead the partition pruning to disable

[jira] [Updated] (SPARK-28386) Cannot resolve ORDER BY columns with GROUP BY and HAVING

[ https://issues.apache.org/jira/browse/SPARK-28386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-28386: - Fix Version/s: 4.0.0 > Cannot resolve ORDER BY columns with GROUP BY and HAVING >

[jira] [Resolved] (SPARK-28386) Cannot resolve ORDER BY columns with GROUP BY and HAVING

[ https://issues.apache.org/jira/browse/SPARK-28386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-28386. -- Assignee: Cheng Pan Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-46462) Reorganize `OpsOnDiffFramesGroupByRollingTests`

[ https://issues.apache.org/jira/browse/SPARK-46462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46462: --- Labels: pull-request-available (was: ) > Reorganize `OpsOnDiffFramesGroupByRollingTests` >

[jira] [Created] (SPARK-46462) Reorganize `OpsOnDiffFramesGroupByRollingTests`

Ruifeng Zheng created SPARK-46462: - Summary: Reorganize `OpsOnDiffFramesGroupByRollingTests` Key: SPARK-46462 URL: https://issues.apache.org/jira/browse/SPARK-46462 Project: Spark Issue

[jira] [Assigned] (SPARK-46399) Add exit status to the Application End event for the use of Spark Listener

[ https://issues.apache.org/jira/browse/SPARK-46399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-46399: --- Assignee: Reza Safi > Add exit status to the Application End event for the

[jira] [Resolved] (SPARK-46399) Add exit status to the Application End event for the use of Spark Listener

[ https://issues.apache.org/jira/browse/SPARK-46399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-46399. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Resolved] (SPARK-46272) Support CTAS using DSv2 sources

[ https://issues.apache.org/jira/browse/SPARK-46272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46272. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44190

[jira] [Assigned] (SPARK-46272) Support CTAS using DSv2 sources

[ https://issues.apache.org/jira/browse/SPARK-46272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46272: --- Assignee: Allison Wang > Support CTAS using DSv2 sources > ---