[jira] [Updated] (SPARK-45467) Replace `Proxy.getProxyClass()` with `Proxy.newProxyInstance().getClass`

[ https://issues.apache.org/jira/browse/SPARK-45467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45467: --- Labels: pull-request-available (was: ) > Replace `Proxy.getProxyClass()` with

[jira] [Updated] (SPARK-45464) [CORE] Fix yarn distribution build

[ https://issues.apache.org/jira/browse/SPARK-45464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45464: --- Labels: pull-request-available (was: ) > [CORE] Fix yarn distribution build >

[jira] [Updated] (SPARK-45467) Replace `Proxy.getProxyClass()` with `Proxy.newProxyInstance().getClass`

[ https://issues.apache.org/jira/browse/SPARK-45467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45467: - Description: {code:java} * @deprecated Proxy classes generated in a named module are encapsulated *

[jira] [Created] (SPARK-45467) Replace `Proxy.getProxyClass()` with `Proxy.newProxyInstance().getClass`

Yang Jie created SPARK-45467: Summary: Replace `Proxy.getProxyClass()` with `Proxy.newProxyInstance().getClass` Key: SPARK-45467 URL: https://issues.apache.org/jira/browse/SPARK-45467 Project: Spark

[jira] [Updated] (SPARK-45466) VectorAssembler should validate the vector values

[ https://issues.apache.org/jira/browse/SPARK-45466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45466: --- Labels: pull-request-available (was: ) > VectorAssembler should validate the vector values

[jira] [Created] (SPARK-45466) VectorAssembler should validate the vector values

Ruifeng Zheng created SPARK-45466: - Summary: VectorAssembler should validate the vector values Key: SPARK-45466 URL: https://issues.apache.org/jira/browse/SPARK-45466 Project: Spark Issue

[jira] [Resolved] (SPARK-45465) Upgrade kubernetes-client to 6.9.0 for K8s 1.28

[ https://issues.apache.org/jira/browse/SPARK-45465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45465. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43284

[jira] [Commented] (SPARK-45428) Add Matomo analytics to all released docs pages

[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773090#comment-17773090 ] BingKun Pan commented on SPARK-45428: - Okay, let me investigate it. > Add Matomo analytics to all

[jira] [Commented] (SPARK-44729) Add canonical links to the PySpark docs page

[ https://issues.apache.org/jira/browse/SPARK-44729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773088#comment-17773088 ] BingKun Pan commented on SPARK-44729: - The historical versions includes: 3.1.1 3.1.2 3.1.3 3.2.0

[jira] [Commented] (SPARK-45428) Add Matomo analytics to all released docs pages

[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773078#comment-17773078 ] Ruifeng Zheng commented on SPARK-45428: --- [~panbingkun]  would you mind taking a look? thanks! >

[jira] [Commented] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773074#comment-17773074 ] XiDuo You commented on SPARK-45443: --- > Can this increase probability of concurrent IMR materialization

[jira] [Updated] (SPARK-44729) Add canonical links to the PySpark docs page

[ https://issues.apache.org/jira/browse/SPARK-44729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44729: --- Labels: pull-request-available (was: ) > Add canonical links to the PySpark docs page >

[jira] [Updated] (SPARK-42716) DataSourceV2 cannot report KeyGroupedPartitioning with multiple keys per partition

[ https://issues.apache.org/jira/browse/SPARK-42716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42716: --- Labels: pull-request-available (was: ) > DataSourceV2 cannot report KeyGroupedPartitioning

[jira] [Resolved] (SPARK-44527) Simplify BinaryComparison if its children contain ScalarSubquery with empty output

[ https://issues.apache.org/jira/browse/SPARK-44527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44527. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42129

[jira] [Updated] (SPARK-44527) Simplify BinaryComparison if its children contain ScalarSubquery with empty output

[ https://issues.apache.org/jira/browse/SPARK-44527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44527: --- Labels: pull-request-available (was: ) > Simplify BinaryComparison if its children contain

[jira] [Updated] (SPARK-44527) Replace ScalarSubquery with null if its maxRows is 0

[ https://issues.apache.org/jira/browse/SPARK-44527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44527: -- Summary: Replace ScalarSubquery with null if its maxRows is 0 (was: Simplify

[jira] [Assigned] (SPARK-44527) Simplify BinaryComparison if its children contain ScalarSubquery with empty output

[ https://issues.apache.org/jira/browse/SPARK-44527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44527: - Assignee: Yuming Wang > Simplify BinaryComparison if its children contain

[jira] [Assigned] (SPARK-45465) Upgrade kubernetes-client to 6.9.0 for K8s 1.28

[ https://issues.apache.org/jira/browse/SPARK-45465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45465: - Assignee: Dongjoon Hyun > Upgrade kubernetes-client to 6.9.0 for K8s 1.28 >

[jira] [Updated] (SPARK-45465) Upgrade kubernetes-client to 6.9.0 for K8s 1.28

[ https://issues.apache.org/jira/browse/SPARK-45465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45465: --- Labels: pull-request-available (was: ) > Upgrade kubernetes-client to 6.9.0 for K8s 1.28 >

[jira] [Updated] (SPARK-45465) Upgrade kubernetes-client to 6.9.0 for K8s 1.28

[ https://issues.apache.org/jira/browse/SPARK-45465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45465: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Upgrade

[jira] [Created] (SPARK-45465) Upgrade kubernetes-client to 6.9.0 for K8s 1.28

Dongjoon Hyun created SPARK-45465: - Summary: Upgrade kubernetes-client to 6.9.0 for K8s 1.28 Key: SPARK-45465 URL: https://issues.apache.org/jira/browse/SPARK-45465 Project: Spark Issue

[jira] [Resolved] (SPARK-45457) Surface sc.setLocalProperty() value=NULL param meaning

[ https://issues.apache.org/jira/browse/SPARK-45457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45457. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43269

[jira] [Resolved] (SPARK-45460) Replace `scala.collection.convert.ImplicitConversions` to `scala.jdk.CollectionConverters`

[ https://issues.apache.org/jira/browse/SPARK-45460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45460. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43275

[jira] [Comment Edited] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773050#comment-17773050 ] Eren Avsarogullari edited comment on SPARK-45443 at 10/8/23 8:42 PM: -

[jira] [Comment Edited] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773050#comment-17773050 ] Eren Avsarogullari edited comment on SPARK-45443 at 10/8/23 8:40 PM: -

[jira] [Comment Edited] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773050#comment-17773050 ] Eren Avsarogullari edited comment on SPARK-45443 at 10/8/23 8:33 PM: -

[jira] [Comment Edited] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773050#comment-17773050 ] Eren Avsarogullari edited comment on SPARK-45443 at 10/8/23 8:29 PM: -

[jira] [Created] (SPARK-45464) [CORE] Fix yarn distribution build

Hasnain Lakhani created SPARK-45464: --- Summary: [CORE] Fix yarn distribution build Key: SPARK-45464 URL: https://issues.apache.org/jira/browse/SPARK-45464 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773050#comment-17773050 ] Eren Avsarogullari edited comment on SPARK-45443 at 10/8/23 8:27 PM: -

[jira] [Comment Edited] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773050#comment-17773050 ] Eren Avsarogullari edited comment on SPARK-45443 at 10/8/23 8:25 PM: -

[jira] [Comment Edited] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773050#comment-17773050 ] Eren Avsarogullari edited comment on SPARK-45443 at 10/8/23 8:24 PM: -

[jira] [Commented] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773050#comment-17773050 ] Eren Avsarogullari commented on SPARK-45443: Hi [~ulysses],  Firstly, thanks for reply.

[jira] [Resolved] (SPARK-45461) Introduce a mapper for StorageLevel

[ https://issues.apache.org/jira/browse/SPARK-45461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45461. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43278

[jira] [Assigned] (SPARK-43704) Enable IndexesParityTests.test_to_series

[ https://issues.apache.org/jira/browse/SPARK-43704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43704: - Assignee: Haejoon Lee > Enable IndexesParityTests.test_to_series >

[jira] [Resolved] (SPARK-43704) Enable IndexesParityTests.test_to_series

[ https://issues.apache.org/jira/browse/SPARK-43704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43704. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43228

[jira] [Resolved] (SPARK-45462) Show `Duration` in `ApplicationPage`

[ https://issues.apache.org/jira/browse/SPARK-45462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45462. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43279

[jira] [Assigned] (SPARK-45462) Show `Duration` in `ApplicationPage`

[ https://issues.apache.org/jira/browse/SPARK-45462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45462: - Assignee: Dongjoon Hyun > Show `Duration` in `ApplicationPage` >

[jira] [Resolved] (SPARK-45413) Add warning for prepare drop LevelDB support

[ https://issues.apache.org/jira/browse/SPARK-45413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45413. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43217

[jira] [Assigned] (SPARK-45413) Add warning for prepare drop LevelDB support

[ https://issues.apache.org/jira/browse/SPARK-45413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45413: - Assignee: Jia Fan > Add warning for prepare drop LevelDB support >

[jira] [Resolved] (SPARK-45454) Set the table's default owner to current_user

[ https://issues.apache.org/jira/browse/SPARK-45454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45454. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43264

[jira] [Updated] (SPARK-45463) Allow ShuffleDriverComponent to support reliable store with specified executorId

[ https://issues.apache.org/jira/browse/SPARK-45463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45463: --- Labels: pull-request-available (was: ) > Allow ShuffleDriverComponent to support reliable

[jira] [Updated] (SPARK-45463) Allow ShuffleDriverComponent to support reliable store with specified executorId

[ https://issues.apache.org/jira/browse/SPARK-45463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoubin updated SPARK-45463: Summary: Allow ShuffleDriverComponent to support reliable store with specified executorId (was: Allow

[jira] [Updated] (SPARK-45463) Allow ShuffleDriverComponent to decide weather shuffle data is reliably stored when different stage has different policy

[ https://issues.apache.org/jira/browse/SPARK-45463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoubin updated SPARK-45463: Description: After SPARK-42689, ShuffleDriverComponent.supportsReliableStorage is determined globally.

[jira] [Created] (SPARK-45463) Allow ShuffleDriverComponent to decide weather shuffle data is reliably stored when different stage has different policy

zhoubin created SPARK-45463: --- Summary: Allow ShuffleDriverComponent to decide weather shuffle data is reliably stored when different stage has different policy Key: SPARK-45463 URL:

[jira] [Updated] (SPARK-45462) Show `Duration` in `ApplicationPage`

[ https://issues.apache.org/jira/browse/SPARK-45462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45462: -- Summary: Show `Duration` in `ApplicationPage` (was: Show `Duration` in ApplicationPage) >

[jira] [Updated] (SPARK-45462) Show `Duration` in `ApplicationPage`

[ https://issues.apache.org/jira/browse/SPARK-45462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45462: --- Labels: pull-request-available (was: ) > Show `Duration` in `ApplicationPage` >

[jira] [Created] (SPARK-45462) Show `Duration` in ApplicationPage

Dongjoon Hyun created SPARK-45462: - Summary: Show `Duration` in ApplicationPage Key: SPARK-45462 URL: https://issues.apache.org/jira/browse/SPARK-45462 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-45352) Eliminate foldable window partitions

[ https://issues.apache.org/jira/browse/SPARK-45352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuml updated SPARK-45352: -- Summary: Eliminate foldable window partitions (was: Remove window partition if partition expression are

[jira] [Assigned] (SPARK-45461) Introduce a mapper for StorageLevel

[ https://issues.apache.org/jira/browse/SPARK-45461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng reassigned SPARK-45461: -- Assignee: Jiaan Geng > Introduce a mapper for StorageLevel >

[jira] [Updated] (SPARK-45461) Introduce a mapper for StorageLevel

[ https://issues.apache.org/jira/browse/SPARK-45461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45461: --- Labels: pull-request-available (was: ) > Introduce a mapper for StorageLevel >

[jira] [Created] (SPARK-45461) Introduce a mapper for StorageLevel

Jiaan Geng created SPARK-45461: -- Summary: Introduce a mapper for StorageLevel Key: SPARK-45461 URL: https://issues.apache.org/jira/browse/SPARK-45461 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-45459) Remove the last 2 extra spaces in the automatically generated `sql-error-conditions.md` file

[ https://issues.apache.org/jira/browse/SPARK-45459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45459: -- Assignee: Apache Spark > Remove the last 2 extra spaces in the automatically

[jira] [Assigned] (SPARK-45459) Remove the last 2 extra spaces in the automatically generated `sql-error-conditions.md` file

[ https://issues.apache.org/jira/browse/SPARK-45459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45459: -- Assignee: (was: Apache Spark) > Remove the last 2 extra spaces in the

[jira] [Updated] (SPARK-45460) Replace `scala.collection.convert.ImplicitConversions` to `scala.jdk.CollectionConverters`

[ https://issues.apache.org/jira/browse/SPARK-45460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45460: --- Labels: pull-request-available (was: ) > Replace

[jira] [Created] (SPARK-45460) Replace `scala.collection.convert.ImplicitConversions` to `scala.jdk.CollectionConverters`

BingKun Pan created SPARK-45460: --- Summary: Replace `scala.collection.convert.ImplicitConversions` to `scala.jdk.CollectionConverters` Key: SPARK-45460 URL: https://issues.apache.org/jira/browse/SPARK-45460

[jira] [Updated] (SPARK-45459) Remove the last 2 extra spaces in the automatically generated `sql-error-conditions.md` file

[ https://issues.apache.org/jira/browse/SPARK-45459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45459: --- Labels: pull-request-available (was: ) > Remove the last 2 extra spaces in the

[jira] [Updated] (SPARK-39600) Enhance pushdown limit through window

[ https://issues.apache.org/jira/browse/SPARK-39600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-39600: --- Labels: pull-request-available (was: ) > Enhance pushdown limit through window >

[jira] [Created] (SPARK-45459) Remove the last 2 extra spaces in the automatically generated `sql-error-conditions.md` file

BingKun Pan created SPARK-45459: --- Summary: Remove the last 2 extra spaces in the automatically generated `sql-error-conditions.md` file Key: SPARK-45459 URL: https://issues.apache.org/jira/browse/SPARK-45459

[jira] [Updated] (SPARK-45455) [SQL][JDBC] Improve the rename interface of Postgres Dialect

[ https://issues.apache.org/jira/browse/SPARK-45455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 蔡灿材 updated SPARK-45455: Description: Improve the rename interface of pgdialect (was: Improve the rename interface of pgdialect and

[jira] [Updated] (SPARK-45458) Convert IllegalArgumentException to SparkIllegalArgumentException in bitwiseExpressions and add some UT

[ https://issues.apache.org/jira/browse/SPARK-45458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45458: --- Labels: pull-request-available (was: ) > Convert IllegalArgumentException to

[jira] [Created] (SPARK-45458) Convert IllegalArgumentException to SparkIllegalArgumentException in bitwiseExpressions and add some UT

BingKun Pan created SPARK-45458: --- Summary: Convert IllegalArgumentException to SparkIllegalArgumentException in bitwiseExpressions and add some UT Key: SPARK-45458 URL: