[jira] [Updated] (SPARK-45767) Delete `TimeStampedHashMap` and its UT

[ https://issues.apache.org/jira/browse/SPARK-45767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45767: --- Labels: pull-request-available (was: ) > Delete `TimeStampedHashMap` and its UT >

[jira] [Created] (SPARK-45767) Delete `TimeStampedHashMap` and its UT

BingKun Pan created SPARK-45767: --- Summary: Delete `TimeStampedHashMap` and its UT Key: SPARK-45767 URL: https://issues.apache.org/jira/browse/SPARK-45767 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-36786) SPIP: Improving the compile time performance, by improving a couple of rules, from 24 hrs to under 8 minutes

[ https://issues.apache.org/jira/browse/SPARK-36786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781976#comment-17781976 ] Asif commented on SPARK-36786: -- I had put this on back burner as my changes were on 3.2, so I have to do a

[jira] [Commented] (SPARK-36786) SPIP: Improving the compile time performance, by improving a couple of rules, from 24 hrs to under 8 minutes

[ https://issues.apache.org/jira/browse/SPARK-36786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781966#comment-17781966 ] Abhinav Kumar commented on SPARK-36786: --- [~ashahid7] [~adou...@sqli.com] where are we on this one?

[jira] [Commented] (SPARK-33164) SPIP: add SQL support to "SELECT * (EXCEPT someColumn) FROM .." equivalent to DataSet.dropColumn(someColumn)

[ https://issues.apache.org/jira/browse/SPARK-33164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781959#comment-17781959 ] Abhinav Kumar commented on SPARK-33164: --- I see value in some use cases like [~arnaud.nauwynck]

[jira] [Resolved] (SPARK-45761) Upgrade `Volcano` to 1.8.1

[ https://issues.apache.org/jira/browse/SPARK-45761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45761. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43624

[jira] [Updated] (SPARK-44419) Support to extract partial filters of datasource v2 table and push them down

[ https://issues.apache.org/jira/browse/SPARK-44419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44419: --- Labels: pull-request-available (was: ) > Support to extract partial filters of datasource

[jira] [Updated] (SPARK-44426) optimize adaptive skew join for ExistenceJoin

[ https://issues.apache.org/jira/browse/SPARK-44426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44426: --- Labels: pull-request-available (was: ) > optimize adaptive skew join for ExistenceJoin >

[jira] [Assigned] (SPARK-45680) ReleaseSession to close Spark Connect session

[ https://issues.apache.org/jira/browse/SPARK-45680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45680: Assignee: Juliusz Sompolski > ReleaseSession to close Spark Connect session >

[jira] [Resolved] (SPARK-45680) ReleaseSession to close Spark Connect session

[ https://issues.apache.org/jira/browse/SPARK-45680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45680. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43546

[jira] [Updated] (SPARK-45766) ObjectSerializerPruning fails to align null types in custom serializer 'If' expressions.

[ https://issues.apache.org/jira/browse/SPARK-45766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Szul updated SPARK-45766: --- Description: We have a custom encoder for union like objects.  The our custom serializer uses an

[jira] [Created] (SPARK-45766) ObjectSerializerPruning fails to align null types in custom serializer 'If' expressions.

Piotr Szul created SPARK-45766: -- Summary: ObjectSerializerPruning fails to align null types in custom serializer 'If' expressions. Key: SPARK-45766 URL: https://issues.apache.org/jira/browse/SPARK-45766

[jira] [Updated] (SPARK-45766) ObjectSerializerPruning fails to align null types in custom serializer 'If' expressions.

[ https://issues.apache.org/jira/browse/SPARK-45766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Szul updated SPARK-45766: --- Attachment: prunning_bug.scala > ObjectSerializerPruning fails to align null types in custom

[jira] [Updated] (SPARK-45765) Improve error messages when loading multiple paths in PySpark

[ https://issues.apache.org/jira/browse/SPARK-45765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-45765: - Description: Currently, the error message is super confusing when a user tries to load

[jira] [Resolved] (SPARK-45765) Improve error messages when loading multiple paths in PySpark

[ https://issues.apache.org/jira/browse/SPARK-45765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang resolved SPARK-45765. -- Resolution: Invalid > Improve error messages when loading multiple paths in PySpark >

[jira] [Created] (SPARK-45765) Improve error messages when loading multiple paths in PySpark

Allison Wang created SPARK-45765: Summary: Improve error messages when loading multiple paths in PySpark Key: SPARK-45765 URL: https://issues.apache.org/jira/browse/SPARK-45765 Project: Spark

[jira] [Assigned] (SPARK-45756) Revisit and Improve Spark Standalone Cluster

[ https://issues.apache.org/jira/browse/SPARK-45756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45756: - Assignee: Dongjoon Hyun > Revisit and Improve Spark Standalone Cluster >

[jira] [Resolved] (SPARK-45756) Revisit and Improve Spark Standalone Cluster

[ https://issues.apache.org/jira/browse/SPARK-45756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45756. --- Fix Version/s: 4.0.0 Resolution: Fixed > Revisit and Improve Spark Standalone

[jira] [Updated] (SPARK-45639) Support loading Python data sources in DataFrameReader

[ https://issues.apache.org/jira/browse/SPARK-45639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45639: --- Labels: pull-request-available (was: ) > Support loading Python data sources in

[jira] [Commented] (SPARK-43972) Tests never succeed on pyspark 3.4.0 (work OK on pyspark 3.3.2)

[ https://issues.apache.org/jira/browse/SPARK-43972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781917#comment-17781917 ] Jamie commented on SPARK-43972: --- This issue appears to be fixed in pyspark 3.5.0   Here's a run of the

[jira] [Resolved] (SPARK-45763) Improve `MasterPage` to show `Resource` column only when it exists

[ https://issues.apache.org/jira/browse/SPARK-45763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45763. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43628

[jira] [Assigned] (SPARK-45763) Improve `MasterPage` to show `Resource` column only when it exists

[ https://issues.apache.org/jira/browse/SPARK-45763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45763: - Assignee: Dongjoon Hyun > Improve `MasterPage` to show `Resource` column only when it

[jira] [Updated] (SPARK-45764) Make code block copyable

[ https://issues.apache.org/jira/browse/SPARK-45764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-45764: - Description: We should consider adding a copy button next to the pyspark code blocks. For

[jira] [Commented] (SPARK-45764) Make code block copyable

[ https://issues.apache.org/jira/browse/SPARK-45764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781887#comment-17781887 ] Allison Wang commented on SPARK-45764: -- cc [~podongfeng] WDYT? > Make code block copyable >

[jira] [Created] (SPARK-45764) Make code block copyable

Allison Wang created SPARK-45764: Summary: Make code block copyable Key: SPARK-45764 URL: https://issues.apache.org/jira/browse/SPARK-45764 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-45731) Update partition statistics with ANALYZE TABLE command

[ https://issues.apache.org/jira/browse/SPARK-45731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45731: --- Labels: pull-request-available (was: ) > Update partition statistics with ANALYZE TABLE

[jira] [Resolved] (SPARK-45754) Support `spark.deploy.appIdPattern`

[ https://issues.apache.org/jira/browse/SPARK-45754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45754. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43616

[jira] [Updated] (SPARK-45763) Improve `MasterPage` to show `Resource` column only when it exists

[ https://issues.apache.org/jira/browse/SPARK-45763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45763: --- Labels: pull-request-available (was: ) > Improve `MasterPage` to show `Resource` column

[jira] [Created] (SPARK-45763) Improve `MasterPage` to show `Resource` column only when it exists

Dongjoon Hyun created SPARK-45763: - Summary: Improve `MasterPage` to show `Resource` column only when it exists Key: SPARK-45763 URL: https://issues.apache.org/jira/browse/SPARK-45763 Project: Spark

[jira] [Commented] (SPARK-38473) Use error classes in org.apache.spark.scheduler

[ https://issues.apache.org/jira/browse/SPARK-38473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781827#comment-17781827 ] Hannah Amundson commented on SPARK-38473: - I am working on this ticket now! > Use error classes

[jira] [Assigned] (SPARK-45761) Upgrade `Volcano` to 1.8.1

[ https://issues.apache.org/jira/browse/SPARK-45761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45761: - Assignee: Dongjoon Hyun > Upgrade `Volcano` to 1.8.1 > -- > >

[jira] [Updated] (SPARK-45761) Upgrade `Volcano` to 1.8.1

[ https://issues.apache.org/jira/browse/SPARK-45761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45761: -- Description: To bring the latest feature and bug fixes in addition to the test coverage for

[jira] [Updated] (SPARK-45762) Shuffle managers defined in user jars are not available for some launch modes

[ https://issues.apache.org/jira/browse/SPARK-45762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45762: --- Labels: pull-request-available (was: ) > Shuffle managers defined in user jars are not

[jira] [Updated] (SPARK-45761) Upgrade `Volcano` to 1.8.1

[ https://issues.apache.org/jira/browse/SPARK-45761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45761: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Bug) > Upgrade `Volcano` to 1.8.1 >

[jira] [Created] (SPARK-45762) Shuffle managers defined in user jars are not available for some launch modes

Alessandro Bellina created SPARK-45762: -- Summary: Shuffle managers defined in user jars are not available for some launch modes Key: SPARK-45762 URL: https://issues.apache.org/jira/browse/SPARK-45762

[jira] [Commented] (SPARK-38668) Spark on Kubernetes: add separate pod watcher service to reduce pressure on K8s API server

[ https://issues.apache.org/jira/browse/SPARK-38668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781821#comment-17781821 ] Hannah Amundson commented on SPARK-38668: - Hello, I will start working on this now! > Spark on

[jira] [Updated] (SPARK-45761) Upgrade `Volcano` to 1.8.1

[ https://issues.apache.org/jira/browse/SPARK-45761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45761: --- Labels: pull-request-available (was: ) > Upgrade `Volcano` to 1.8.1 >

[jira] [Created] (SPARK-45761) Upgrade `Volcano` to 1.8.1

Dongjoon Hyun created SPARK-45761: - Summary: Upgrade `Volcano` to 1.8.1 Key: SPARK-45761 URL: https://issues.apache.org/jira/browse/SPARK-45761 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-45728) Upgrade `kubernetes-client` to 6.9.1

[ https://issues.apache.org/jira/browse/SPARK-45728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45728: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Bug) > Upgrade `kubernetes-client` to

[jira] [Updated] (SPARK-45760) Add With expression to avoid duplicating expressions

[ https://issues.apache.org/jira/browse/SPARK-45760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45760: --- Labels: pull-request-available (was: ) > Add With expression to avoid duplicating

[jira] [Created] (SPARK-45760) Add With expression to avoid duplicating expressions

Wenchen Fan created SPARK-45760: --- Summary: Add With expression to avoid duplicating expressions Key: SPARK-45760 URL: https://issues.apache.org/jira/browse/SPARK-45760 Project: Spark Issue

[jira] [Resolved] (SPARK-45327) Upgrade zstd-jni to 1.5.5-6

[ https://issues.apache.org/jira/browse/SPARK-45327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-45327. -- Assignee: BingKun Pan Resolution: Fixed https://github.com/apache/spark/pull/43113 > Upgrade

[jira] [Resolved] (SPARK-45753) Support `spark.deploy.driverIdPattern`

[ https://issues.apache.org/jira/browse/SPARK-45753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45753. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43615

[jira] [Commented] (SPARK-45502) Upgrade Kafka to 3.6.1

[ https://issues.apache.org/jira/browse/SPARK-45502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781789#comment-17781789 ] Dongjoon Hyun commented on SPARK-45502: --- KAFKA-7109 was the root cause of revert. > Upgrade Kafka

[jira] [Updated] (SPARK-45502) Upgrade Kafka to 3.6.1

[ https://issues.apache.org/jira/browse/SPARK-45502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45502: -- Summary: Upgrade Kafka to 3.6.1 (was: Upgrade Kafka to 3.6.0) > Upgrade Kafka to 3.6.1 >

[jira] [Assigned] (SPARK-45502) Upgrade Kafka to 3.6.0

[ https://issues.apache.org/jira/browse/SPARK-45502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45502: - Assignee: (was: Deng Ziming) > Upgrade Kafka to 3.6.0 > -- > >

[jira] [Updated] (SPARK-45502) Upgrade Kafka to 3.6.0

[ https://issues.apache.org/jira/browse/SPARK-45502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45502: -- Fix Version/s: (was: 4.0.0) > Upgrade Kafka to 3.6.0 > -- > >

[jira] [Assigned] (SPARK-45743) Upgrade dropwizard metrics 4.2.21

[ https://issues.apache.org/jira/browse/SPARK-45743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45743: - Assignee: Yang Jie > Upgrade dropwizard metrics 4.2.21 >

[jira] [Resolved] (SPARK-45743) Upgrade dropwizard metrics 4.2.21

[ https://issues.apache.org/jira/browse/SPARK-45743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45743. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43608

[jira] [Updated] (SPARK-45743) Upgrade dropwizard metrics 4.2.21

[ https://issues.apache.org/jira/browse/SPARK-45743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45743: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Upgrade dropwizard

[jira] [Updated] (SPARK-45759) Custom metrics should be updated after commit too

[ https://issues.apache.org/jira/browse/SPARK-45759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ali Ince updated SPARK-45759: - Description: We have a DataWriter component, which processes records in configurable batches, which

[jira] [Updated] (SPARK-45759) Custom metrics should be updated after commit too

[ https://issues.apache.org/jira/browse/SPARK-45759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ali Ince updated SPARK-45759: - Description: We have a DataWriter component, which processes records in configurable batches, which

[jira] [Updated] (SPARK-45759) Custom metrics should be updated after commit too

[ https://issues.apache.org/jira/browse/SPARK-45759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ali Ince updated SPARK-45759: - Description: We have a DataWriter component, which processes records in configurable batches, which

[jira] [Updated] (SPARK-45758) Introduce a mapper for hadoop compression codecs

[ https://issues.apache.org/jira/browse/SPARK-45758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45758: --- Labels: pull-request-available (was: ) > Introduce a mapper for hadoop compression codecs

[jira] [Created] (SPARK-45759) Custom metrics should be updated after commit too

Ali Ince created SPARK-45759: Summary: Custom metrics should be updated after commit too Key: SPARK-45759 URL: https://issues.apache.org/jira/browse/SPARK-45759 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-45758) Introduce a mapper for hadoop compression codecs

[ https://issues.apache.org/jira/browse/SPARK-45758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45758: --- Description: Currently, Spark supported partial Hadoop compression codecs, but the Hadoop

[jira] [Created] (SPARK-45758) Introduce a mapper for hadoop compression codecs

Jiaan Geng created SPARK-45758: -- Summary: Introduce a mapper for hadoop compression codecs Key: SPARK-45758 URL: https://issues.apache.org/jira/browse/SPARK-45758 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-45755) Push down limit through Dataset.isEmpty()

[ https://issues.apache.org/jira/browse/SPARK-45755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng reassigned SPARK-45755: -- Assignee: Yuming Wang > Push down limit through Dataset.isEmpty() >

[jira] [Resolved] (SPARK-45755) Push down limit through Dataset.isEmpty()

[ https://issues.apache.org/jira/browse/SPARK-45755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45755. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43617

[jira] [Commented] (SPARK-44896) Consider adding information os_prio, cpu, elapsed, tid, nid, etc., from the jstack tool

[ https://issues.apache.org/jira/browse/SPARK-44896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781694#comment-17781694 ] Kent Yao commented on SPARK-44896: -- Hi [~hannahkamundson],   Sure, feel free to send a PR for this

[jira] [Resolved] (SPARK-45751) The default value of ‘spark.executor.logs.rolling.maxRetainedFiles' on the official website is incorrect

[ https://issues.apache.org/jira/browse/SPARK-45751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-45751. -- Fix Version/s: 3.3.4 3.5.1 4.0.0 3.4.2

[jira] [Assigned] (SPARK-45680) ReleaseSession to close Spark Connect session

[ https://issues.apache.org/jira/browse/SPARK-45680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45680: -- Assignee: (was: Apache Spark) > ReleaseSession to close Spark Connect session >

[jira] [Assigned] (SPARK-45751) The default value of ‘spark.executor.logs.rolling.maxRetainedFiles' on the official website is incorrect

[ https://issues.apache.org/jira/browse/SPARK-45751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-45751: Assignee: chenyu > The default value of ‘spark.executor.logs.rolling.maxRetainedFiles' on the >

[jira] [Assigned] (SPARK-45751) The default value of ‘spark.executor.logs.rolling.maxRetainedFiles' on the official website is incorrect

[ https://issues.apache.org/jira/browse/SPARK-45751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45751: -- Assignee: (was: Apache Spark) > The default value of

[jira] [Assigned] (SPARK-45751) The default value of ‘spark.executor.logs.rolling.maxRetainedFiles' on the official website is incorrect

[ https://issues.apache.org/jira/browse/SPARK-45751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45751: -- Assignee: Apache Spark > The default value of

[jira] [Assigned] (SPARK-45022) Provide context for dataset API errors

[ https://issues.apache.org/jira/browse/SPARK-45022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-45022: Assignee: Max Gekk > Provide context for dataset API errors >

[jira] [Resolved] (SPARK-45022) Provide context for dataset API errors

[ https://issues.apache.org/jira/browse/SPARK-45022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-45022. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43334

[jira] [Updated] (SPARK-45174) Support `spark.deploy.maxDrivers`

[ https://issues.apache.org/jira/browse/SPARK-45174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45174: -- Summary: Support `spark.deploy.maxDrivers` (was: Support spark.deploy.maxDrivers) > Support

[jira] [Updated] (SPARK-45497) Add a symbolic link file `spark-examples.jar` in K8s Docker images

[ https://issues.apache.org/jira/browse/SPARK-45497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45497: -- Parent: (was: SPARK-45756) Issue Type: Improvement (was: Sub-task) > Add a

[jira] [Updated] (SPARK-45497) Add a symbolic link file `spark-examples.jar` in K8s Docker images

[ https://issues.apache.org/jira/browse/SPARK-45497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45497: -- Parent: SPARK-45756 Issue Type: Sub-task (was: Improvement) > Add a symbolic link

[jira] [Updated] (SPARK-44214) Support Spark Driver Live Log UI

[ https://issues.apache.org/jira/browse/SPARK-44214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44214: -- Parent: SPARK-45756 Issue Type: Sub-task (was: Improvement) > Support Spark Driver

[jira] [Updated] (SPARK-45756) Revisit and Improve Spark Standalone Cluster

[ https://issues.apache.org/jira/browse/SPARK-45756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45756: -- Labels: releasenotes (was: ) > Revisit and Improve Spark Standalone Cluster >

[jira] [Assigned] (SPARK-45754) Support `spark.deploy.appIdPattern`

[ https://issues.apache.org/jira/browse/SPARK-45754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45754: - Assignee: Dongjoon Hyun > Support `spark.deploy.appIdPattern` >

[jira] [Assigned] (SPARK-45753) Support `spark.deploy.driverIdPattern`

[ https://issues.apache.org/jira/browse/SPARK-45753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45753: - Assignee: Dongjoon Hyun > Support `spark.deploy.driverIdPattern` >

[jira] [Updated] (SPARK-45757) Avoid re-computation of NNZ in Binarizer

[ https://issues.apache.org/jira/browse/SPARK-45757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45757: --- Labels: pull-request-available (was: ) > Avoid re-computation of NNZ in Binarizer >

[jira] [Updated] (SPARK-45753) Support `spark.deploy.driverIdPattern`

[ https://issues.apache.org/jira/browse/SPARK-45753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45753: -- Parent: SPARK-45756 Issue Type: Sub-task (was: Improvement) > Support

[jira] [Updated] (SPARK-45756) Revisit and Improve Spark Standalone Cluster

[ https://issues.apache.org/jira/browse/SPARK-45756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45756: -- Summary: Revisit and Improve Spark Standalone Cluster (was: Improve Spark Standalone

[jira] [Updated] (SPARK-45754) Support `spark.deploy.appIdPattern`

[ https://issues.apache.org/jira/browse/SPARK-45754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45754: -- Parent: SPARK-45756 Issue Type: Sub-task (was: Improvement) > Support

[jira] [Updated] (SPARK-45749) Fix Spark History Server to sort `Duration` column properly

[ https://issues.apache.org/jira/browse/SPARK-45749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45749: -- Parent: SPARK-45756 Issue Type: Sub-task (was: Bug) > Fix Spark History Server to

[jira] [Created] (SPARK-45757) Avoid re-computation of NNZ in Binarizer

Ruifeng Zheng created SPARK-45757: - Summary: Avoid re-computation of NNZ in Binarizer Key: SPARK-45757 URL: https://issues.apache.org/jira/browse/SPARK-45757 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45500) Show the number of abnormally completed drivers in MasterPage

[ https://issues.apache.org/jira/browse/SPARK-45500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45500: -- Parent: SPARK-45756 Issue Type: Sub-task (was: Improvement) > Show the number of

[jira] [Updated] (SPARK-45474) Support top-level filtering in MasterPage JSON API

[ https://issues.apache.org/jira/browse/SPARK-45474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45474: -- Parent: SPARK-45756 Issue Type: Sub-task (was: Improvement) > Support top-level

[jira] [Updated] (SPARK-45197) Make StandaloneRestServer add JavaModuleOptions to drivers

[ https://issues.apache.org/jira/browse/SPARK-45197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45197: -- Parent: SPARK-45756 Issue Type: Sub-task (was: Bug) > Make StandaloneRestServer add

[jira] [Updated] (SPARK-45187) Fix WorkerPage to use the same pattern for `logPage` urls

[ https://issues.apache.org/jira/browse/SPARK-45187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45187: -- Parent: SPARK-45756 Issue Type: Sub-task (was: Bug) > Fix WorkerPage to use the same

[jira] [Updated] (SPARK-45197) Make StandaloneRestServer add JavaModuleOptions to drivers

[ https://issues.apache.org/jira/browse/SPARK-45197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45197: -- Parent: (was: SPARK-43831) Issue Type: Bug (was: Sub-task) > Make

[jira] [Updated] (SPARK-45174) Support spark.deploy.maxDrivers

[ https://issues.apache.org/jira/browse/SPARK-45174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45174: -- Parent: SPARK-45756 Issue Type: Sub-task (was: Improvement) > Support

[jira] [Updated] (SPARK-44857) Fix getBaseURI error in Spark Worker LogPage UI buttons

[ https://issues.apache.org/jira/browse/SPARK-44857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44857: -- Parent: SPARK-45756 Issue Type: Sub-task (was: Bug) > Fix getBaseURI error in Spark

[jira] [Created] (SPARK-45756) Improve Spark Standalone Cluster

Dongjoon Hyun created SPARK-45756: - Summary: Improve Spark Standalone Cluster Key: SPARK-45756 URL: https://issues.apache.org/jira/browse/SPARK-45756 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-45751) The default value of ‘spark.executor.logs.rolling.maxRetainedFiles' on the official website is incorrect

[ https://issues.apache.org/jira/browse/SPARK-45751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781606#comment-17781606 ] chenyu commented on SPARK-45751: I had submit a pr to resolve this question.

[jira] [Updated] (SPARK-45751) The default value of ‘spark.executor.logs.rolling.maxRetainedFiles' on the official website is incorrect

[ https://issues.apache.org/jira/browse/SPARK-45751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45751: --- Labels: pull-request-available (was: ) > The default value of

[jira] [Updated] (SPARK-45755) Push down limit through Dataset.isEmpty()

[ https://issues.apache.org/jira/browse/SPARK-45755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45755: --- Labels: pull-request-available (was: ) > Push down limit through Dataset.isEmpty() >