[jira] [Created] (SPARK-41558) Disable Coverage in python.pyspark.tests.test_memory_profiler

Hyukjin Kwon created SPARK-41558: Summary: Disable Coverage in python.pyspark.tests.test_memory_profiler Key: SPARK-41558 URL: https://issues.apache.org/jira/browse/SPARK-41558 Project: Spark

[jira] [Assigned] (SPARK-41426) Protobuf serializer for ResourceProfileWrapper

[ https://issues.apache.org/jira/browse/SPARK-41426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41426: Assignee: (was: Apache Spark) > Protobuf serializer for ResourceProfileWrapper >

[jira] [Commented] (SPARK-41426) Protobuf serializer for ResourceProfileWrapper

[ https://issues.apache.org/jira/browse/SPARK-41426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648870#comment-17648870 ] Apache Spark commented on SPARK-41426: -- User 'techaddict' has created a pull request for this

[jira] [Assigned] (SPARK-41426) Protobuf serializer for ResourceProfileWrapper

[ https://issues.apache.org/jira/browse/SPARK-41426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41426: Assignee: Apache Spark > Protobuf serializer for ResourceProfileWrapper >

[jira] [Commented] (SPARK-41426) Protobuf serializer for ResourceProfileWrapper

[ https://issues.apache.org/jira/browse/SPARK-41426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648868#comment-17648868 ] Apache Spark commented on SPARK-41426: -- User 'techaddict' has created a pull request for this

[jira] [Commented] (SPARK-41426) Protobuf serializer for ResourceProfileWrapper

[ https://issues.apache.org/jira/browse/SPARK-41426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648867#comment-17648867 ] Sandeep Singh commented on SPARK-41426: --- underlying serializer already there, working on a PR. >

[jira] [Commented] (SPARK-41557) Union of tables with and without metadata column fails when used in join

[ https://issues.apache.org/jira/browse/SPARK-41557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648866#comment-17648866 ] Shardul Mahadik commented on SPARK-41557: - cc: [~Gengliang.Wang] [~cloud_fan] > Union of

[jira] [Updated] (SPARK-41557) Union of tables with and without metadata column fails when used in join

[ https://issues.apache.org/jira/browse/SPARK-41557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shardul Mahadik updated SPARK-41557: Description: Here is a test case that can be added to {{MetadataColumnSuite}} to

[jira] [Updated] (SPARK-41557) Union of tables with and without metadata column fails when used in join

[ https://issues.apache.org/jira/browse/SPARK-41557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shardul Mahadik updated SPARK-41557: Description: Here is a test case that can be added to {{MetadataColumnSuite}} to

[jira] [Created] (SPARK-41557) Union of tables with and without metadata column fails when used in join

Shardul Mahadik created SPARK-41557: --- Summary: Union of tables with and without metadata column fails when used in join Key: SPARK-41557 URL: https://issues.apache.org/jira/browse/SPARK-41557

[jira] [Assigned] (SPARK-41425) Protobuf serializer for RDDStorageInfoWrapper

[ https://issues.apache.org/jira/browse/SPARK-41425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41425: Assignee: (was: Apache Spark) > Protobuf serializer for RDDStorageInfoWrapper >

[jira] [Commented] (SPARK-41425) Protobuf serializer for RDDStorageInfoWrapper

[ https://issues.apache.org/jira/browse/SPARK-41425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648862#comment-17648862 ] Apache Spark commented on SPARK-41425: -- User 'techaddict' has created a pull request for this

[jira] [Assigned] (SPARK-41425) Protobuf serializer for RDDStorageInfoWrapper

[ https://issues.apache.org/jira/browse/SPARK-41425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41425: Assignee: Apache Spark > Protobuf serializer for RDDStorageInfoWrapper >

[jira] [Commented] (SPARK-41425) Protobuf serializer for RDDStorageInfoWrapper

[ https://issues.apache.org/jira/browse/SPARK-41425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648860#comment-17648860 ] Sandeep Singh commented on SPARK-41425: --- Working on this > Protobuf serializer for

[jira] [Comment Edited] (SPARK-41556) input_file_positon

[ https://issues.apache.org/jira/browse/SPARK-41556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648856#comment-17648856 ] gabrywu edited comment on SPARK-41556 at 12/17/22 5:18 AM: --- [~yumwang]

[jira] [Commented] (SPARK-41556) input_file_positon

[ https://issues.apache.org/jira/browse/SPARK-41556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648856#comment-17648856 ] gabrywu commented on SPARK-41556: - [~yumwang] [~ptoth] What do you think of it? > input_file_positon >

[jira] [Updated] (SPARK-41447) Reduce the number of doMergeApplicationListing invocations

[ https://issues.apache.org/jira/browse/SPARK-41447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shuyouZZ updated SPARK-41447: - Description: When restarting the history server, In history server log, we can see many {{INFO

[jira] [Created] (SPARK-41556) input_file_positon

gabrywu created SPARK-41556: --- Summary: input_file_positon Key: SPARK-41556 URL: https://issues.apache.org/jira/browse/SPARK-41556 Project: Spark Issue Type: New Feature Components: SQL

[jira] [Commented] (SPARK-41546) pyspark_types_to_proto_types should supports StructType.

[ https://issues.apache.org/jira/browse/SPARK-41546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648848#comment-17648848 ] Apache Spark commented on SPARK-41546: -- User 'beliefer' has created a pull request for this issue:

[jira] (SPARK-41546) pyspark_types_to_proto_types should supports StructType.

[ https://issues.apache.org/jira/browse/SPARK-41546 ] jiaan.geng deleted comment on SPARK-41546: was (Author: beliefer): I'm working on. > pyspark_types_to_proto_types should supports StructType. >

[jira] [Commented] (SPARK-41546) pyspark_types_to_proto_types should supports StructType.

[ https://issues.apache.org/jira/browse/SPARK-41546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648847#comment-17648847 ] Apache Spark commented on SPARK-41546: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41546) pyspark_types_to_proto_types should supports StructType.

[ https://issues.apache.org/jira/browse/SPARK-41546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41546: Assignee: Apache Spark > pyspark_types_to_proto_types should supports StructType. >

[jira] [Assigned] (SPARK-41546) pyspark_types_to_proto_types should supports StructType.

[ https://issues.apache.org/jira/browse/SPARK-41546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41546: Assignee: (was: Apache Spark) > pyspark_types_to_proto_types should supports

[jira] [Updated] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-41555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiahong.li updated SPARK-41555: --- Priority: Minor (was: Major) > Multi sparkSession should share single SQLAppStatusStore >

[jira] [Resolved] (SPARK-41421) Protobuf serializer for ApplicationEnvironmentInfoWrapper

[ https://issues.apache.org/jira/browse/SPARK-41421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-41421. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39096

[jira] [Assigned] (SPARK-41421) Protobuf serializer for ApplicationEnvironmentInfoWrapper

[ https://issues.apache.org/jira/browse/SPARK-41421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-41421: -- Assignee: Sandeep Singh > Protobuf serializer for ApplicationEnvironmentInfoWrapper

[jira] [Updated] (SPARK-41162) Anti-join must not be pushed below aggregation with ambiguous predicates

[ https://issues.apache.org/jira/browse/SPARK-41162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shardul Mahadik updated SPARK-41162: Labels: correctness (was: ) > Anti-join must not be pushed below aggregation with

[jira] [Updated] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-41555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiahong.li updated SPARK-41555: --- Description: In spark , if we create multi sparkSession in the program, we will get multi-SQLTab

[jira] [Commented] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-41555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648843#comment-17648843 ] Apache Spark commented on SPARK-41555: -- User 'monkeyboy123' has created a pull request for this

[jira] [Commented] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-41555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648841#comment-17648841 ] Apache Spark commented on SPARK-41555: -- User 'monkeyboy123' has created a pull request for this

[jira] [Assigned] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-41555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41555: Assignee: (was: Apache Spark) > Multi sparkSession should share single

[jira] [Assigned] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-41555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41555: Assignee: Apache Spark > Multi sparkSession should share single SQLAppStatusStore >

[jira] [Commented] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-41555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648840#comment-17648840 ] Apache Spark commented on SPARK-41555: -- User 'monkeyboy123' has created a pull request for this

[jira] [Updated] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-41555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiahong.li updated SPARK-41555: --- Description: In spark , if we create multi sparkSession in the program, we will get multi-SQLTab

[jira] [Updated] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-41555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiahong.li updated SPARK-41555: --- Description: In spark , if we create multi sparkSession in the program, we will get multi-SQLTab

[jira] [Updated] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-41555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiahong.li updated SPARK-41555: --- Description: In spark , if we create multi sparkSession in the program, we will get multi-SQLTab

[jira] [Updated] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-41555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiahong.li updated SPARK-41555: --- Description: In spark , if we create multi sparkSession in the program, we will get multi-SQLTab

[jira] [Updated] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-41555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiahong.li updated SPARK-41555: --- Attachment: muti-sqltab.png muti-SQLStore.png > Multi sparkSession should share

[jira] [Created] (SPARK-41555) Multi sparkSession should share single SQLAppStatusStore

jiahong.li created SPARK-41555: -- Summary: Multi sparkSession should share single SQLAppStatusStore Key: SPARK-41555 URL: https://issues.apache.org/jira/browse/SPARK-41555 Project: Spark Issue

[jira] [Comment Edited] (SPARK-41422) Protobuf serializer for ExecutorSummaryWrapper

[ https://issues.apache.org/jira/browse/SPARK-41422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648835#comment-17648835 ] Gengliang Wang edited comment on SPARK-41422 at 12/17/22 12:33 AM: ---

[jira] [Assigned] (SPARK-41422) Protobuf serializer for ExecutorSummaryWrapper

[ https://issues.apache.org/jira/browse/SPARK-41422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41422: Assignee: Apache Spark > Protobuf serializer for ExecutorSummaryWrapper >

[jira] [Commented] (SPARK-41422) Protobuf serializer for ExecutorSummaryWrapper

[ https://issues.apache.org/jira/browse/SPARK-41422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648837#comment-17648837 ] Apache Spark commented on SPARK-41422: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-41422) Protobuf serializer for ExecutorSummaryWrapper

[ https://issues.apache.org/jira/browse/SPARK-41422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41422: Assignee: (was: Apache Spark) > Protobuf serializer for ExecutorSummaryWrapper >

[jira] [Commented] (SPARK-41162) Anti-join must not be pushed below aggregation with ambiguous predicates

[ https://issues.apache.org/jira/browse/SPARK-41162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648836#comment-17648836 ] SHU WANG commented on SPARK-41162: -- [~shardulm] Yes. Checked with Spark 3.1.2, and it's also an issue.

[jira] [Commented] (SPARK-41422) Protobuf serializer for ExecutorSummaryWrapper

[ https://issues.apache.org/jira/browse/SPARK-41422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648835#comment-17648835 ] Gengliang Wang commented on SPARK-41422: [~techaddict] I have a PR for this one already. Sorry I

[jira] [Commented] (SPARK-41422) Protobuf serializer for ExecutorSummaryWrapper

[ https://issues.apache.org/jira/browse/SPARK-41422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648832#comment-17648832 ] Sandeep Singh commented on SPARK-41422: --- Working on this, will create a PR soon. > Protobuf

[jira] [Updated] (SPARK-41530) Rename MedianHeap to PercentileMap and support percentile

[ https://issues.apache.org/jira/browse/SPARK-41530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41530: -- Summary: Rename MedianHeap to PercentileMap and support percentile (was: extend MedianHeap

[jira] [Assigned] (SPARK-41530) extend MedianHeap to support percentile

[ https://issues.apache.org/jira/browse/SPARK-41530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41530: - Assignee: Wenchen Fan > extend MedianHeap to support percentile >

[jira] [Resolved] (SPARK-41530) extend MedianHeap to support percentile

[ https://issues.apache.org/jira/browse/SPARK-41530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41530. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39076

[jira] [Commented] (SPARK-41554) Decimal.changePrecision produces ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-41554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648830#comment-17648830 ] Apache Spark commented on SPARK-41554: -- User 'fe2s' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41554) Decimal.changePrecision produces ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-41554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41554: Assignee: Apache Spark > Decimal.changePrecision produces ArrayIndexOutOfBoundsException

[jira] [Assigned] (SPARK-41554) Decimal.changePrecision produces ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-41554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41554: Assignee: (was: Apache Spark) > Decimal.changePrecision produces

[jira] [Commented] (SPARK-41554) Decimal.changePrecision produces ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-41554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648829#comment-17648829 ] Apache Spark commented on SPARK-41554: -- User 'fe2s' has created a pull request for this issue:

[jira] [Updated] (SPARK-41447) Reduce the number of doMergeApplicationListing invocations

[ https://issues.apache.org/jira/browse/SPARK-41447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41447: -- Priority: Minor (was: Major) > Reduce the number of doMergeApplicationListing invocations >

[jira] [Updated] (SPARK-41447) Reduce the number of doMergeApplicationListing invocations

[ https://issues.apache.org/jira/browse/SPARK-41447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41447: -- Summary: Reduce the number of doMergeApplicationListing invocations (was: clean up expired

[jira] [Resolved] (SPARK-41447) clean up expired event log files that don't exist in listing db

[ https://issues.apache.org/jira/browse/SPARK-41447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41447. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38983

[jira] [Assigned] (SPARK-41447) clean up expired event log files that don't exist in listing db

[ https://issues.apache.org/jira/browse/SPARK-41447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41447: - Assignee: shuyouZZ > clean up expired event log files that don't exist in listing db >

[jira] [Created] (SPARK-41554) Decimal.changePrecision produces ArrayIndexOutOfBoundsException

Oleksiy Dyagilev created SPARK-41554: Summary: Decimal.changePrecision produces ArrayIndexOutOfBoundsException Key: SPARK-41554 URL: https://issues.apache.org/jira/browse/SPARK-41554 Project:

[jira] [Resolved] (SPARK-41552) Upgrade kubernetes-client to 6.3.1

[ https://issues.apache.org/jira/browse/SPARK-41552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41552. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39094

[jira] [Commented] (SPARK-41553) Change num_files to repartition

[ https://issues.apache.org/jira/browse/SPARK-41553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648795#comment-17648795 ] Apache Spark commented on SPARK-41553: -- User 'bjornjorgensen' has created a pull request for this

[jira] [Assigned] (SPARK-41553) Change num_files to repartition

[ https://issues.apache.org/jira/browse/SPARK-41553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41553: Assignee: Apache Spark > Change num_files to repartition >

[jira] [Commented] (SPARK-41553) Change num_files to repartition

[ https://issues.apache.org/jira/browse/SPARK-41553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648793#comment-17648793 ] Apache Spark commented on SPARK-41553: -- User 'bjornjorgensen' has created a pull request for this

[jira] [Assigned] (SPARK-41553) Change num_files to repartition

[ https://issues.apache.org/jira/browse/SPARK-41553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41553: Assignee: (was: Apache Spark) > Change num_files to repartition >

[jira] [Assigned] (SPARK-41049) Nondeterministic expressions have unstable values if they are children of CodegenFallback expressions

[ https://issues.apache.org/jira/browse/SPARK-41049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41049: Assignee: Apache Spark > Nondeterministic expressions have unstable values if they are

[jira] [Commented] (SPARK-41049) Nondeterministic expressions have unstable values if they are children of CodegenFallback expressions

[ https://issues.apache.org/jira/browse/SPARK-41049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648792#comment-17648792 ] Apache Spark commented on SPARK-41049: -- User 'NarekDW' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41049) Nondeterministic expressions have unstable values if they are children of CodegenFallback expressions

[ https://issues.apache.org/jira/browse/SPARK-41049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41049: Assignee: (was: Apache Spark) > Nondeterministic expressions have unstable values if

[jira] [Created] (SPARK-41553) Change num_files to repartition

Bjørn Jørgensen created SPARK-41553: --- Summary: Change num_files to repartition Key: SPARK-41553 URL: https://issues.apache.org/jira/browse/SPARK-41553 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-41053) Better Spark UI scalability and Driver stability for large applications

[ https://issues.apache.org/jira/browse/SPARK-41053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648786#comment-17648786 ] Gengliang Wang edited comment on SPARK-41053 at 12/16/22 8:30 PM: --

[jira] [Commented] (SPARK-41053) Better Spark UI scalability and Driver stability for large applications

[ https://issues.apache.org/jira/browse/SPARK-41053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648786#comment-17648786 ] Gengliang Wang commented on SPARK-41053: [~techaddict] Thanks! Feel free to take anyone from the

[jira] [Commented] (SPARK-41421) Protobuf serializer for ApplicationEnvironmentInfoWrapper

[ https://issues.apache.org/jira/browse/SPARK-41421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648785#comment-17648785 ] Apache Spark commented on SPARK-41421: -- User 'techaddict' has created a pull request for this

[jira] [Commented] (SPARK-41421) Protobuf serializer for ApplicationEnvironmentInfoWrapper

[ https://issues.apache.org/jira/browse/SPARK-41421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648784#comment-17648784 ] Apache Spark commented on SPARK-41421: -- User 'techaddict' has created a pull request for this

[jira] [Assigned] (SPARK-41421) Protobuf serializer for ApplicationEnvironmentInfoWrapper

[ https://issues.apache.org/jira/browse/SPARK-41421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41421: Assignee: Apache Spark > Protobuf serializer for ApplicationEnvironmentInfoWrapper >

[jira] [Assigned] (SPARK-41421) Protobuf serializer for ApplicationEnvironmentInfoWrapper

[ https://issues.apache.org/jira/browse/SPARK-41421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41421: Assignee: (was: Apache Spark) > Protobuf serializer for

[jira] [Commented] (SPARK-41053) Better Spark UI scalability and Driver stability for large applications

[ https://issues.apache.org/jira/browse/SPARK-41053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648782#comment-17648782 ] Sandeep Singh commented on SPARK-41053: --- [~Gengliang.Wang] I'm willing to take some tasks from

[jira] [Assigned] (SPARK-41365) Stages UI page fails to load for proxy in some yarn versions

[ https://issues.apache.org/jira/browse/SPARK-41365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41365: - Assignee: miracle > Stages UI page fails to load for proxy in some yarn versions >

[jira] [Assigned] (SPARK-41365) Stages UI page fails to load for proxy in some yarn versions

[ https://issues.apache.org/jira/browse/SPARK-41365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41365: - Assignee: Mars (was: miracle) > Stages UI page fails to load for proxy in some yarn

[jira] [Assigned] (SPARK-41365) Stages UI page fails to load for proxy in some yarn versions

[ https://issues.apache.org/jira/browse/SPARK-41365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41365: - Assignee: (was: miracle) > Stages UI page fails to load for proxy in some yarn

[jira] [Assigned] (SPARK-41365) Stages UI page fails to load for proxy in some yarn versions

[ https://issues.apache.org/jira/browse/SPARK-41365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41365: - Assignee: miracle > Stages UI page fails to load for proxy in some yarn versions >

[jira] [Resolved] (SPARK-41365) Stages UI page fails to load for proxy in some yarn versions

[ https://issues.apache.org/jira/browse/SPARK-41365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41365. --- Fix Version/s: 3.3.2 3.4.0 Target Version/s: (was: 3.3.1)

[jira] [Assigned] (SPARK-41552) Upgrade kubernetes-client to 6.3.1

[ https://issues.apache.org/jira/browse/SPARK-41552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41552: - Assignee: Dongjoon Hyun > Upgrade kubernetes-client to 6.3.1 >

[jira] [Updated] (SPARK-38062) FallbackStorage shouldn't attempt to resolve arbitrary "remote" hostname

[ https://issues.apache.org/jira/browse/SPARK-38062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38062: -- Parent: SPARK-41550 Issue Type: Sub-task (was: Improvement) > FallbackStorage

[jira] [Updated] (SPARK-40060) Add numberDecommissioningExecutors metric

[ https://issues.apache.org/jira/browse/SPARK-40060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40060: -- Parent: SPARK-41550 Issue Type: Sub-task (was: Improvement) > Add

[jira] [Commented] (SPARK-40060) Add numberDecommissioningExecutors metric

[ https://issues.apache.org/jira/browse/SPARK-40060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648762#comment-17648762 ] Dongjoon Hyun commented on SPARK-40060: --- I collected this as a subtask of SPARK-41550. > Add

[jira] [Commented] (SPARK-40269) Randomize the orders of peer in BlockManagerDecommissioner

[ https://issues.apache.org/jira/browse/SPARK-40269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648761#comment-17648761 ] Dongjoon Hyun commented on SPARK-40269: --- I collected this as a subtask of SPARK-41550. >

[jira] [Updated] (SPARK-40269) Randomize the orders of peer in BlockManagerDecommissioner

[ https://issues.apache.org/jira/browse/SPARK-40269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40269: -- Parent: SPARK-41550 Issue Type: Sub-task (was: Improvement) > Randomize the orders

[jira] [Commented] (SPARK-40636) Fix wrong remained shuffles log in BlockManagerDecommissioner

[ https://issues.apache.org/jira/browse/SPARK-40636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648760#comment-17648760 ] Dongjoon Hyun commented on SPARK-40636: --- I collected this as a subtask of SPARK-41550. > Fix

[jira] [Commented] (SPARK-40481) Ignore stage fetch failure caused by decommissioned executor

[ https://issues.apache.org/jira/browse/SPARK-40481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648759#comment-17648759 ] Dongjoon Hyun commented on SPARK-40481: --- I collected this as a subtask of SPARK-41550. > Ignore

[jira] [Updated] (SPARK-40481) Ignore stage fetch failure caused by decommissioned executor

[ https://issues.apache.org/jira/browse/SPARK-40481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40481: -- Parent: SPARK-41550 Issue Type: Sub-task (was: Improvement) > Ignore stage fetch

[jira] [Updated] (SPARK-40636) Fix wrong remained shuffles log in BlockManagerDecommissioner

[ https://issues.apache.org/jira/browse/SPARK-40636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40636: -- Parent: SPARK-41550 Issue Type: Sub-task (was: Bug) > Fix wrong remained shuffles

[jira] [Commented] (SPARK-40596) Populate ExecutorDecommission with more informative messages

[ https://issues.apache.org/jira/browse/SPARK-40596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648758#comment-17648758 ] Dongjoon Hyun commented on SPARK-40596: --- I collected this as a subtask of SPARK-41550 > Populate

[jira] [Commented] (SPARK-40979) Keep removed executor info in decommission state

[ https://issues.apache.org/jira/browse/SPARK-40979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648757#comment-17648757 ] Dongjoon Hyun commented on SPARK-40979: --- I collected this as a subtask of SPARK-41550 > Keep

[jira] [Updated] (SPARK-40596) Populate ExecutorDecommission with more informative messages

[ https://issues.apache.org/jira/browse/SPARK-40596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40596: -- Parent: SPARK-41550 Issue Type: Sub-task (was: Improvement) > Populate

[jira] [Updated] (SPARK-40979) Keep removed executor info in decommission state

[ https://issues.apache.org/jira/browse/SPARK-40979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40979: -- Parent: SPARK-41550 Issue Type: Sub-task (was: Improvement) > Keep removed executor

[jira] [Commented] (SPARK-40379) Propagate decommission executor loss reason during onDisconnect in K8s

[ https://issues.apache.org/jira/browse/SPARK-40379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648756#comment-17648756 ] Dongjoon Hyun commented on SPARK-40379: --- Hi, [~holden]. We want to go GA with `Dynamic Allocation

[jira] [Updated] (SPARK-40379) Propagate decommission executor loss reason during onDisconnect in K8s

[ https://issues.apache.org/jira/browse/SPARK-40379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40379: -- Parent: SPARK-41550 Issue Type: Sub-task (was: Improvement) > Propagate decommission

[jira] [Assigned] (SPARK-41552) Upgrade kubernetes-client to 6.3.1

[ https://issues.apache.org/jira/browse/SPARK-41552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41552: Assignee: (was: Apache Spark) > Upgrade kubernetes-client to 6.3.1 >

[jira] [Commented] (SPARK-41552) Upgrade kubernetes-client to 6.3.1

[ https://issues.apache.org/jira/browse/SPARK-41552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648752#comment-17648752 ] Apache Spark commented on SPARK-41552: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-41552) Upgrade kubernetes-client to 6.3.1

[ https://issues.apache.org/jira/browse/SPARK-41552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41552: Assignee: Apache Spark > Upgrade kubernetes-client to 6.3.1 >

[jira] [Created] (SPARK-41552) Upgrade kubernetes-client to 6.3.1

Dongjoon Hyun created SPARK-41552: - Summary: Upgrade kubernetes-client to 6.3.1 Key: SPARK-41552 URL: https://issues.apache.org/jira/browse/SPARK-41552 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-39322) Remove `Experimental` from `spark.dynamicAllocation.shuffleTracking.enabled`

[ https://issues.apache.org/jira/browse/SPARK-39322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39322: -- Parent: SPARK-41550 Issue Type: Sub-task (was: Documentation) > Remove

  1   2   >