[jira] [Updated] (SPARK-51873) For OneVsRest algorithm, allow using save / load to replace cache

[ https://issues.apache.org/jira/browse/SPARK-51873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51873: --- Labels: pull-request-available (was: ) > For OneVsRest algorithm, allow using save / load t

[jira] [Created] (SPARK-51873) For OneVsRest algorithm, allow using save / load to replace cache

Weichen Xu created SPARK-51873: -- Summary: For OneVsRest algorithm, allow using save / load to replace cache Key: SPARK-51873 URL: https://issues.apache.org/jira/browse/SPARK-51873 Project: Spark

[jira] [Updated] (SPARK-51872) AQE: CoalescePartitions causes BHJ performance regression via under-utilization

[ https://issues.apache.org/jira/browse/SPARK-51872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chungmin Lee updated SPARK-51872: - Description: When AQE converts SMJ to BHJ, the probe side shuffle can have data skew. When {{Co

[jira] [Updated] (SPARK-51872) AQE: CoalescePartitions causes BHJ performance regression via under-utilization

[ https://issues.apache.org/jira/browse/SPARK-51872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chungmin Lee updated SPARK-51872: - Description: When AQE converts SMJ to BHJ, the probe side shuffle can have data skew. When {{Co

[jira] [Updated] (SPARK-51872) AQE: CoalescePartitions causes BHJ performance regression via under-utilization

[ https://issues.apache.org/jira/browse/SPARK-51872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chungmin Lee updated SPARK-51872: - Attachment: (was: image-2025-04-22-20-12-11-449.png) > AQE: CoalescePartitions causes BHJ pe

[jira] [Updated] (SPARK-51872) AQE: CoalescePartitions causes BHJ performance regression via under-utilization

[ https://issues.apache.org/jira/browse/SPARK-51872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chungmin Lee updated SPARK-51872: - Description: When AQE converts SMJ to BHJ, the probe side shuffle can have data skew. When {{Co

[jira] [Updated] (SPARK-51872) AQE: CoalescePartitions causes BHJ performance regression via under-utilization

[ https://issues.apache.org/jira/browse/SPARK-51872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chungmin Lee updated SPARK-51872: - Attachment: image-2025-04-22-20-13-43-569.png > AQE: CoalescePartitions causes BHJ performance r

[jira] [Updated] (SPARK-51872) AQE: CoalescePartitions causes BHJ performance regression via under-utilization

[ https://issues.apache.org/jira/browse/SPARK-51872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chungmin Lee updated SPARK-51872: - Attachment: image-2025-04-22-20-12-14-032.png > AQE: CoalescePartitions causes BHJ performance r

[jira] [Updated] (SPARK-51872) AQE: CoalescePartitions causes BHJ performance regression via under-utilization

[ https://issues.apache.org/jira/browse/SPARK-51872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chungmin Lee updated SPARK-51872: - Attachment: image-2025-04-22-20-12-11-449.png > AQE: CoalescePartitions causes BHJ performance r

[jira] [Updated] (SPARK-51872) AQE: CoalescePartitions causes BHJ performance regression via under-utilization

[ https://issues.apache.org/jira/browse/SPARK-51872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chungmin Lee updated SPARK-51872: - Description: When AQE converts SMJ to BHJ, the probe side shuffle can have data skew. When {{Co

[jira] [Updated] (SPARK-51872) AQE: CoalescePartitions causes BHJ performance regression via under-utilization

[ https://issues.apache.org/jira/browse/SPARK-51872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chungmin Lee updated SPARK-51872: - Description: When AQE converts SMJ to BHJ, the probe side shuffle can have data skew. When {{Co

[jira] [Created] (SPARK-51872) AQE: CoalescePartitions causes BHJ performance regression via under-utilization

Chungmin Lee created SPARK-51872: Summary: AQE: CoalescePartitions causes BHJ performance regression via under-utilization Key: SPARK-51872 URL: https://issues.apache.org/jira/browse/SPARK-51872 Proje

[jira] [Resolved] (SPARK-51866) Fix `Allocator closed with outstanding child allocators` issue in `ArrowEncoderSuite`

[ https://issues.apache.org/jira/browse/SPARK-51866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-51866. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50664 [https://github.com

[jira] [Assigned] (SPARK-51866) Fix `Allocator closed with outstanding child allocators` issue in `ArrowEncoderSuite`

[ https://issues.apache.org/jira/browse/SPARK-51866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-51866: Assignee: Yang Jie > Fix `Allocator closed with outstanding child allocators` issue in > `Arrow

[jira] [Updated] (SPARK-51871) Improve `SQLTests` to check column names

[ https://issues.apache.org/jira/browse/SPARK-51871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51871: --- Labels: pull-request-available (was: ) > Improve `SQLTests` to check column names > ---

[jira] [Assigned] (SPARK-51870) Support `SPARK_GENERATE_GOLDEN_FILES` in `SQLTests`

[ https://issues.apache.org/jira/browse/SPARK-51870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51870: - Assignee: Dongjoon Hyun > Support `SPARK_GENERATE_GOLDEN_FILES` in `SQLTests` > ---

[jira] [Resolved] (SPARK-51870) Support `SPARK_GENERATE_GOLDEN_FILES` in `SQLTests`

[ https://issues.apache.org/jira/browse/SPARK-51870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51870. --- Fix Version/s: connect-swift-0.1.0 Resolution: Fixed Issue resolved by pull request 8

[jira] [Updated] (SPARK-51870) Support `SPARK_GENERATE_GOLDEN_FILES` in `SQLTests`

[ https://issues.apache.org/jira/browse/SPARK-51870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51870: --- Labels: pull-request-available (was: ) > Support `SPARK_GENERATE_GOLDEN_FILES` in `SQLTests

[jira] [Created] (SPARK-51870) Support `SPARK_GENERATE_GOLDEN_FILES` in `SQLTests`

Dongjoon Hyun created SPARK-51870: - Summary: Support `SPARK_GENERATE_GOLDEN_FILES` in `SQLTests` Key: SPARK-51870 URL: https://issues.apache.org/jira/browse/SPARK-51870 Project: Spark Issue T

[jira] [Resolved] (SPARK-51856) Add API for estimate saved model size

[ https://issues.apache.org/jira/browse/SPARK-51856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu resolved SPARK-51856. Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50652 [https://github

[jira] [Updated] (SPARK-51869) Create classification for user errors within handleInputRows for Scala TransformWithState

[ https://issues.apache.org/jira/browse/SPARK-51869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51869: --- Labels: pull-request-available (was: ) > Create classification for user errors within handl

[jira] [Assigned] (SPARK-51822) Throwing classified error when disallowed functions are called during StatefulProcessor.init()

[ https://issues.apache.org/jira/browse/SPARK-51822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-51822: Assignee: Eric Marnadi > Throwing classified error when disallowed functions are called d

[jira] [Resolved] (SPARK-51822) Throwing classified error when disallowed functions are called during StatefulProcessor.init()

[ https://issues.apache.org/jira/browse/SPARK-51822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-51822. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50611 [https://gi

[jira] [Created] (SPARK-51869) Create classification for user errors within handleInputRows for Scala TransformWithState

Eric Marnadi created SPARK-51869: Summary: Create classification for user errors within handleInputRows for Scala TransformWithState Key: SPARK-51869 URL: https://issues.apache.org/jira/browse/SPARK-51869

[jira] [Updated] (SPARK-51868) Move type coercion error path to a separate object

[ https://issues.apache.org/jira/browse/SPARK-51868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51868: --- Labels: pull-request-available (was: ) > Move type coercion error path to a separate object

[jira] [Updated] (SPARK-51868) Move type coercion error path to a separate object

[ https://issues.apache.org/jira/browse/SPARK-51868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Golubev updated SPARK-51868: - Summary: Move type coercion error path to a separate object (was: Move type coercion va

[jira] [Created] (SPARK-51868) Move type coercion validation to a separate object

Vladimir Golubev created SPARK-51868: Summary: Move type coercion validation to a separate object Key: SPARK-51868 URL: https://issues.apache.org/jira/browse/SPARK-51868 Project: Spark Is

[jira] [Resolved] (SPARK-51849) Renaming ResolveDDLCommandStringTypes to ApplyDefaultCollationToStringType

[ https://issues.apache.org/jira/browse/SPARK-51849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-51849. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50609 [https://gith

[jira] [Assigned] (SPARK-51849) Renaming ResolveDDLCommandStringTypes to ApplyDefaultCollationToStringType

[ https://issues.apache.org/jira/browse/SPARK-51849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-51849: --- Assignee: Marko Ilic > Renaming ResolveDDLCommandStringTypes to ApplyDefaultCollationToStri

[jira] [Updated] (SPARK-51867) Make scala model supporting save / load against local filesystem path

[ https://issues.apache.org/jira/browse/SPARK-51867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51867: --- Labels: pull-request-available (was: ) > Make scala model supporting save / load against lo

[jira] [Created] (SPARK-51867) Make scala model supporting save / load against local filesystem path

Weichen Xu created SPARK-51867: -- Summary: Make scala model supporting save / load against local filesystem path Key: SPARK-51867 URL: https://issues.apache.org/jira/browse/SPARK-51867 Project: Spark

[jira] [Updated] (SPARK-51865) Print stacktrace message when RootAllocator fails to close

[ https://issues.apache.org/jira/browse/SPARK-51865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51865: --- Labels: pull-request-available (was: ) > Print stacktrace message when RootAllocator fails

[jira] [Updated] (SPARK-51866) Fix `Allocator closed with outstanding child allocators` issue in `ArrowEncoderSuite`

[ https://issues.apache.org/jira/browse/SPARK-51866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51866: --- Labels: pull-request-available (was: ) > Fix `Allocator closed with outstanding child allo

[jira] [Created] (SPARK-51866) Fix `Allocator closed with outstanding child allocators` issue in `ArrowEncoderSuite`

Yang Jie created SPARK-51866: Summary: Fix `Allocator closed with outstanding child allocators` issue in `ArrowEncoderSuite` Key: SPARK-51866 URL: https://issues.apache.org/jira/browse/SPARK-51866 Projec

[jira] [Created] (SPARK-51865) Print stacktrace message when RootAllocator fails to close

Yang Jie created SPARK-51865: Summary: Print stacktrace message when RootAllocator fails to close Key: SPARK-51865 URL: https://issues.apache.org/jira/browse/SPARK-51865 Project: Spark Issue Type

[jira] [Resolved] (SPARK-51862) Clean up ml cache before ReusedConnectTestCase and ReusedMixedTestCase

[ https://issues.apache.org/jira/browse/SPARK-51862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-51862. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50660 [https://

[jira] [Updated] (SPARK-44155) Adding a dev utility to improve error messages based on LLM

[ https://issues.apache.org/jira/browse/SPARK-44155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44155: --- Labels: pull-request-available (was: ) > Adding a dev utility to improve error messages ba

[jira] [Updated] (SPARK-51821) Possible deadlock in UninterruptibleThread

[ https://issues.apache.org/jira/browse/SPARK-51821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51821: --- Labels: pull-request-available (was: ) > Possible deadlock in UninterruptibleThread > -

[jira] [Assigned] (SPARK-51529) Support TLS connections

[ https://issues.apache.org/jira/browse/SPARK-51529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51529: - Assignee: Dongjoon Hyun (was: Martin Grund) > Support TLS connections > --

[jira] [Assigned] (SPARK-51529) Support TLS connections

[ https://issues.apache.org/jira/browse/SPARK-51529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51529: - Assignee: Martin Grund > Support TLS connections > --- > >

[jira] [Updated] (SPARK-51529) Support TLS connections

[ https://issues.apache.org/jira/browse/SPARK-51529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51529: -- Affects Version/s: connect-swift-0.1.0 (was: 4.0.0) > Support TLS c

[jira] [Resolved] (SPARK-51529) Support TLS connections

[ https://issues.apache.org/jira/browse/SPARK-51529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51529. --- Fix Version/s: connect-swift-0.1.0 Resolution: Fixed Issue resolved by pull request 8

[jira] [Updated] (SPARK-51529) Support TLS connections

[ https://issues.apache.org/jira/browse/SPARK-51529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51529: -- Summary: Support TLS connections (was: Support for TLS connections) > Support TLS connections

[jira] [Updated] (SPARK-51529) Support for TLS connections

[ https://issues.apache.org/jira/browse/SPARK-51529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51529: --- Labels: pull-request-available (was: ) > Support for TLS connections >

[jira] [Assigned] (SPARK-51864) Rename parameters and support case-insensitively

[ https://issues.apache.org/jira/browse/SPARK-51864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51864: - Assignee: Dongjoon Hyun > Rename parameters and support case-insensitively > --

[jira] [Resolved] (SPARK-51864) Rename parameters and support case-insensitively

[ https://issues.apache.org/jira/browse/SPARK-51864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51864. --- Fix Version/s: connect-swift-0.1.0 Resolution: Fixed Issue resolved by pull request 8

[jira] [Resolved] (SPARK-51863) Support `join` and `crossJoin` in `DataFrame`

[ https://issues.apache.org/jira/browse/SPARK-51863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51863. --- Fix Version/s: connect-swift-0.1.0 Resolution: Fixed Issue resolved by pull request 8

[jira] [Assigned] (SPARK-51863) Support `join` and `crossJoin` in `DataFrame`

[ https://issues.apache.org/jira/browse/SPARK-51863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51863: - Assignee: Dongjoon Hyun > Support `join` and `crossJoin` in `DataFrame` > -

[jira] [Updated] (SPARK-51864) Rename parameters and support case-insensitively

[ https://issues.apache.org/jira/browse/SPARK-51864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51864: -- Parent: SPARK-51448 Issue Type: Sub-task (was: Bug) > Rename parameters and support c

[jira] [Updated] (SPARK-51864) Rename parameters and support case-insensitively

[ https://issues.apache.org/jira/browse/SPARK-51864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51864: --- Labels: pull-request-available (was: ) > Rename parameters and support case-insensitively >

[jira] [Created] (SPARK-51864) Rename parameters and support case-insensitively

Dongjoon Hyun created SPARK-51864: - Summary: Rename parameters and support case-insensitively Key: SPARK-51864 URL: https://issues.apache.org/jira/browse/SPARK-51864 Project: Spark Issue Type

[jira] [Updated] (SPARK-51826) Update `YuniKorn` docs with `1.6.2`

[ https://issues.apache.org/jira/browse/SPARK-51826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51826: -- Fix Version/s: 4.0.0 (was: 4.1.0) > Update `YuniKorn` docs with `1.6.2`