[jira] [Updated] (SPARK-46460) The filter of partition includes cast function may lead the partition pruning to disable

[ https://issues.apache.org/jira/browse/SPARK-46460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou Tong updated SPARK-46460: -- Attachment: SPARK-46460.patch > The filter of partition includes cast function may lead the partition

[jira] [Updated] (SPARK-46460) The filter of partition includes cast function may lead the partition pruning to disable

[ https://issues.apache.org/jira/browse/SPARK-46460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46460: --- Labels: pull-request-available (was: ) > The filter of partition includes cast function may

[jira] [Updated] (SPARK-46460) The filter of partition includes cast function may lead the partition pruning to disable

[ https://issues.apache.org/jira/browse/SPARK-46460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou Tong updated SPARK-46460: -- Description: SQL：select * from test_db.test_table where day between date_sub('2023-12-01',1) and  '20

[jira] [Updated] (SPARK-46461) The `sbt console` command is not available

[ https://issues.apache.org/jira/browse/SPARK-46461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-46461: - Issue Type: Bug (was: Improvement) > The `sbt console` command is not available > -

[jira] [Updated] (SPARK-46461) The `sbt console` command is not available

[ https://issues.apache.org/jira/browse/SPARK-46461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-46461: - Priority: Minor (was: Major) > The `sbt console` command is not available > ---

[jira] [Created] (SPARK-46461) The `sbt console` command is not available

Yang Jie created SPARK-46461: Summary: The `sbt console` command is not available Key: SPARK-46461 URL: https://issues.apache.org/jira/browse/SPARK-46461 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-46302) Fix maven daily testing

[ https://issues.apache.org/jira/browse/SPARK-46302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-46302: Assignee: BingKun Pan > Fix maven daily testing > --- > > Key

[jira] [Resolved] (SPARK-46302) Fix maven daily testing

[ https://issues.apache.org/jira/browse/SPARK-46302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-46302. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44208 [https://github.com

[jira] [Updated] (SPARK-46330) Loading of Spark UI blocks for a long time when HybridStore enabled

[ https://issues.apache.org/jira/browse/SPARK-46330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou Yifan updated SPARK-46330: --- Affects Version/s: 3.3.1 > Loading of Spark UI blocks for a long time when HybridStore enabled > ---

[jira] [Updated] (SPARK-46460) The filter of partition includes cast function may lead the partition pruning to disable

[ https://issues.apache.org/jira/browse/SPARK-46460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou Tong updated SPARK-46460: -- Description: SQL：select * from test_db.test_table where day between date_sub('2023-12-01',1) and  '20

[jira] [Updated] (SPARK-46460) The filter of partition includes cast function may lead the partition pruning to disable

[ https://issues.apache.org/jira/browse/SPARK-46460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou Tong updated SPARK-46460: -- Description: SQL：select * from test_db.test_table where day between date_sub('2023-12-01',1) and  '20

[jira] [Updated] (SPARK-46460) The filter of partition includes cast function may lead the partition pruning to disable

[ https://issues.apache.org/jira/browse/SPARK-46460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou Tong updated SPARK-46460: -- Summary: The filter of partition includes cast function may lead the partition pruning to disable (wa

[jira] [Created] (SPARK-46460) When spark access HiveMetastore to abtain the meta data about hive table, if the filter of partition includes cast function, the partition pruning may disable. In this s

Zhou Tong created SPARK-46460: - Summary: When spark access HiveMetastore to abtain the meta data about hive table, if the filter of partition includes cast function, the partition pruning may disable. In this situation, the performance of HMS will d

[jira] [Updated] (SPARK-46457) Refactor Python Test Suites

[ https://issues.apache.org/jira/browse/SPARK-46457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46457: -- Description: Some Python tests are large and slow, so they are not friendly to parallelism. W

[jira] [Updated] (SPARK-46278) Re-organize `GroupByTests`

[ https://issues.apache.org/jira/browse/SPARK-46278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46278: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Re-organize `GroupByTests` > -

[jira] [Created] (SPARK-46459) Fix bundler to 2.4.22 to unclock CI

Xinrong Meng created SPARK-46459: Summary: Fix bundler to 2.4.22 to unclock CI Key: SPARK-46459 URL: https://issues.apache.org/jira/browse/SPARK-46459 Project: Spark Issue Type: Story

[jira] [Commented] (SPARK-46458) Replace all Scala `require` with `SparkException.require`

[ https://issues.apache.org/jira/browse/SPARK-46458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17798739#comment-17798739 ] Hannah Amundson commented on SPARK-46458: - [~maxgekk] I am adding this but wante

[jira] [Created] (SPARK-46458) Replace all Scala `require` with `SparkException.require`

Hannah Amundson created SPARK-46458: --- Summary: Replace all Scala `require` with `SparkException.require` Key: SPARK-46458 URL: https://issues.apache.org/jira/browse/SPARK-46458 Project: Spark

[jira] [Resolved] (SPARK-46246) Support EXECUTE IMMEDIATE sytax in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-46246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46246. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44093 [https://github.com

[jira] [Assigned] (SPARK-46246) Support EXECUTE IMMEDIATE sytax in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-46246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-46246: Assignee: Milan Stefanovic > Support EXECUTE IMMEDIATE sytax in Spark SQL > -

[jira] [Commented] (SPARK-19256) Hive bucketing write support

[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17798619#comment-17798619 ] Eric Xiao commented on SPARK-19256: --- Hi [~tejasp], for the remaining work, would i be

[jira] [Resolved] (SPARK-46454) Remove redundant `headOption`

[ https://issues.apache.org/jira/browse/SPARK-46454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46454. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44411 [https://github.com

[jira] [Assigned] (SPARK-46454) Remove redundant `headOption`

[ https://issues.apache.org/jira/browse/SPARK-46454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-46454: Assignee: Yang Jie > Remove redundant `headOption` > - > >

[jira] [Commented] (SPARK-32014) Support calling stored procedure on JDBC data source

[ https://issues.apache.org/jira/browse/SPARK-32014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17798563#comment-17798563 ] Sravan Kumar Vadaga commented on SPARK-32014: - Can this be prioritized?  >

[jira] [Resolved] (SPARK-46451) Reorganize `GroupbyStatTests`

[ https://issues.apache.org/jira/browse/SPARK-46451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46451. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44409 [https://

[jira] [Assigned] (SPARK-46451) Reorganize `GroupbyStatTests`

[ https://issues.apache.org/jira/browse/SPARK-46451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-46451: - Assignee: Ruifeng Zheng > Reorganize `GroupbyStatTests` > -

[jira] [Updated] (SPARK-46439) Move IO-related tests to `pyspark.pandas.tests.io.*`

[ https://issues.apache.org/jira/browse/SPARK-46439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46439: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Move IO-related tests to `pysp

[jira] [Updated] (SPARK-46451) Reorganize `GroupbyStatTests`

[ https://issues.apache.org/jira/browse/SPARK-46451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46451: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Reorganize `GroupbyStatTests`

[jira] [Updated] (SPARK-46433) Reorganize `OpsOnDiffFramesGroupByTests`

[ https://issues.apache.org/jira/browse/SPARK-46433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46433: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Reorganize `OpsOnDiffFramesGro

[jira] [Updated] (SPARK-46418) Reorganize `ReshapeTests`

[ https://issues.apache.org/jira/browse/SPARK-46418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46418: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Reorganize `ReshapeTests` > --

[jira] [Updated] (SPARK-46419) Reorganize `DatetimeIndexTests`: Factor out 3 slow tests

[ https://issues.apache.org/jira/browse/SPARK-46419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46419: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Reorganize `DatetimeIndexTests

[jira] [Updated] (SPARK-46345) Reorganize `DiffFramesParityCovCorrWithTests `

[ https://issues.apache.org/jira/browse/SPARK-46345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46345: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Reorganize `DiffFramesParityCo

[jira] [Updated] (SPARK-46379) Reorganize `FrameInterpolateTests`

[ https://issues.apache.org/jira/browse/SPARK-46379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46379: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Reorganize `FrameInterpolateTe

[jira] [Updated] (SPARK-46347) Reorganize `RollingTests `

[ https://issues.apache.org/jira/browse/SPARK-46347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46347: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Reorganize `RollingTests ` > -

[jira] [Updated] (SPARK-46391) Reorganize `ExpandingParityTests`

[ https://issues.apache.org/jira/browse/SPARK-46391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46391: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Reorganize `ExpandingParityTes

[jira] [Updated] (SPARK-46407) Reorganize `OpsOnDiffFramesDisabledTests`

[ https://issues.apache.org/jira/browse/SPARK-46407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46407: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Reorganize `OpsOnDiffFramesDis

[jira] [Updated] (SPARK-46327) Reorganize `SeriesStringTests`

[ https://issues.apache.org/jira/browse/SPARK-46327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46327: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Reorganize `SeriesStringTests`

[jira] [Updated] (SPARK-46309) Remove unused code in `pyspark.pandas.tests.indexes.* `

[ https://issues.apache.org/jira/browse/SPARK-46309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46309: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Remove unused code in `pyspark

[jira] [Updated] (SPARK-46281) Remove unused code in `pyspark.pandas.tests.computation.*`

[ https://issues.apache.org/jira/browse/SPARK-46281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46281: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Remove unused code in `pyspark

[jira] [Updated] (SPARK-46288) Remove unused code in `pyspark.pandas.tests.frame.*`

[ https://issues.apache.org/jira/browse/SPARK-46288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46288: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Remove unused code in `pyspark

[jira] [Updated] (SPARK-46268) Re-organize `StatsTests`

[ https://issues.apache.org/jira/browse/SPARK-46268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46268: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Re-organize `StatsTests` > ---

[jira] [Updated] (SPARK-46266) Re-organize `NumOpsTests`

[ https://issues.apache.org/jira/browse/SPARK-46266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46266: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Re-organize `NumOpsTests` > --

[jira] [Updated] (SPARK-46280) Move test_parity_frame_resample and test_parity_series_resample to pyspark.pandas.tests.connect.resample

[ https://issues.apache.org/jira/browse/SPARK-46280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46280: -- Parent: SPARK-46457 Issue Type: Sub-task (was: Test) > Move test_parity_frame_resampl

[jira] [Created] (SPARK-46457) Refactor Python Test Suites

Ruifeng Zheng created SPARK-46457: - Summary: Refactor Python Test Suites Key: SPARK-46457 URL: https://issues.apache.org/jira/browse/SPARK-46457 Project: Spark Issue Type: Test Comp

[jira] [Resolved] (SPARK-46453) SessionHolder doesn't throw exceptions from internalError()

[ https://issues.apache.org/jira/browse/SPARK-46453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46453. -- Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-46246) Support EXECUTE IMMEDIATE sytax in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-46246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46246: -- Assignee: (was: Apache Spark) > Support EXECUTE IMMEDIATE sytax in Spark SQL > --

[jira] [Assigned] (SPARK-46246) Support EXECUTE IMMEDIATE sytax in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-46246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46246: -- Assignee: Apache Spark > Support EXECUTE IMMEDIATE sytax in Spark SQL > -

[jira] [Assigned] (SPARK-46452) Add a new API in DSv2 DataWriter to write an iterator of records

[ https://issues.apache.org/jira/browse/SPARK-46452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46452: -- Assignee: Apache Spark > Add a new API in DSv2 DataWriter to write an iterator of rec

[jira] [Assigned] (SPARK-46452) Add a new API in DSv2 DataWriter to write an iterator of records

[ https://issues.apache.org/jira/browse/SPARK-46452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46452: -- Assignee: (was: Apache Spark) > Add a new API in DSv2 DataWriter to write an iter

[jira] [Assigned] (SPARK-46452) Add a new API in DSv2 DataWriter to write an iterator of records

[ https://issues.apache.org/jira/browse/SPARK-46452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46452: -- Assignee: (was: Apache Spark) > Add a new API in DSv2 DataWriter to write an iter

[jira] [Assigned] (SPARK-46452) Add a new API in DSv2 DataWriter to write an iterator of records

[ https://issues.apache.org/jira/browse/SPARK-46452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46452: -- Assignee: Apache Spark > Add a new API in DSv2 DataWriter to write an iterator of rec

[jira] [Updated] (SPARK-46454) Remove redundant `headOption`

[ https://issues.apache.org/jira/browse/SPARK-46454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-46454: - Priority: Trivial (was: Major) > Remove redundant `headOption` > - > >

[jira] [Updated] (SPARK-46454) Remove redundant `headOption`

[ https://issues.apache.org/jira/browse/SPARK-46454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46454: --- Labels: pull-request-available (was: ) > Remove redundant `headOption` > --

[jira] [Created] (SPARK-46454) Remove redundant `headOption`

Yang Jie created SPARK-46454: Summary: Remove redundant `headOption` Key: SPARK-46454 URL: https://issues.apache.org/jira/browse/SPARK-46454 Project: Spark Issue Type: Improvement Compo