[jira] [Resolved] (SPARK-40004) Redundant `LevelDB.get` in `RemoteBlockPushResolver`

[ https://issues.apache.org/jira/browse/SPARK-40004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-40004. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-40004) Redundant `LevelDB.get` in `RemoteBlockPushResolver`

[ https://issues.apache.org/jira/browse/SPARK-40004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-40004: --- Assignee: Yang Jie > Redundant `LevelDB.get` in `RemoteBlockPushResolver`

[jira] [Commented] (SPARK-39696) Uncaught exception in thread executor-heartbeater java.util.ConcurrentModificationException: mutation occurred during iteration

[ https://issues.apache.org/jira/browse/SPARK-39696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571169#comment-17571169 ] Mridul Muralidharan commented on SPARK-39696: - Can you give more details on how this can be

[jira] [Resolved] (SPARK-33236) Enable Push-based shuffle service to store state in NM level DB for work preserving restart

[ https://issues.apache.org/jira/browse/SPARK-33236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-33236. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-33236) Enable Push-based shuffle service to store state in NM level DB for work preserving restart

[ https://issues.apache.org/jira/browse/SPARK-33236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-33236: --- Assignee: Ye Zhou > Enable Push-based shuffle service to store state in NM

[jira] [Resolved] (SPARK-39647) Block push fails with java.lang.IllegalArgumentException: Active local dirs list has not been updated by any executor registration even when the NodeManager hasn't been

[ https://issues.apache.org/jira/browse/SPARK-39647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-39647. - Fix Version/s: 3.3.1 3.4.0 Resolution: Fixed Issue

[jira] [Assigned] (SPARK-39647) Block push fails with java.lang.IllegalArgumentException: Active local dirs list has not been updated by any executor registration even when the NodeManager hasn't been

[ https://issues.apache.org/jira/browse/SPARK-39647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-39647: --- Assignee: Chandni Singh > Block push fails with

[jira] [Resolved] (SPARK-30835) Add support for YARN decommissioning & pre-emption

[ https://issues.apache.org/jira/browse/SPARK-30835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-30835. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-30835) Add support for YARN decommissioning & pre-emption

[ https://issues.apache.org/jira/browse/SPARK-30835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-30835: --- Assignee: Abhishek Dixit > Add support for YARN decommissioning &

[jira] [Commented] (SPARK-38987) Handle fallback when merged shuffle blocks are corrupted and spark.shuffle.detectCorrupt is set to true

[ https://issues.apache.org/jira/browse/SPARK-38987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17550681#comment-17550681 ] Mridul Muralidharan commented on SPARK-38987: - Manually marking as resolved - some env issue

[jira] [Resolved] (SPARK-38987) Handle fallback when merged shuffle blocks are corrupted and spark.shuffle.detectCorrupt is set to true

[ https://issues.apache.org/jira/browse/SPARK-38987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-38987. - Fix Version/s: 3.4.0 Assignee: Aravind Patnam Resolution: Fixed

[jira] [Assigned] (SPARK-38973) When push-based shuffle is enabled, a stage may not complete when retried

[ https://issues.apache.org/jira/browse/SPARK-38973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-38973: --- Assignee: Chandni Singh > When push-based shuffle is enabled, a stage may

[jira] [Resolved] (SPARK-38973) When push-based shuffle is enabled, a stage may not complete when retried

[ https://issues.apache.org/jira/browse/SPARK-38973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-38973. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Resolved] (SPARK-38856) Fix a rejectedExecutionException error when push-based shuffle is enabled

[ https://issues.apache.org/jira/browse/SPARK-38856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-38856. - Target Version/s: 3.4.0 Assignee: weixiuli Resolution:

[jira] [Resolved] (SPARK-38555) Avoid contention and get or create clientPools quickly in the TransportClientFactory

[ https://issues.apache.org/jira/browse/SPARK-38555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-38555. - Assignee: weixiuli Resolution: Fixed > Avoid contention and get or

[jira] [Updated] (SPARK-38555) Avoid contention and get or create clientPools quickly in the TransportClientFactory

[ https://issues.apache.org/jira/browse/SPARK-38555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-38555: Fix Version/s: 3.4.0 > Avoid contention and get or create clientPools quickly in

[jira] [Commented] (SPARK-38388) Repartition + Stage retries could lead to incorrect data

[ https://issues.apache.org/jira/browse/SPARK-38388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507257#comment-17507257 ] Mridul Muralidharan commented on SPARK-38388: - Agree with [~jiangxb1987], either the

[jira] [Resolved] (SPARK-38428) Check the FetchShuffleBlocks message only once to improve iteration in external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-38428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-38428. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Resolved] (SPARK-38309) SHS has incorrect percentiles for shuffle read bytes and shuffle total blocks metrics

[ https://issues.apache.org/jira/browse/SPARK-38309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-38309. - Fix Version/s: 3.3.0 3.2.2 3.1.3

[jira] [Assigned] (SPARK-38309) SHS has incorrect percentiles for shuffle read bytes and shuffle total blocks metrics

[ https://issues.apache.org/jira/browse/SPARK-38309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-38309: --- Assignee: Rob Reeves > SHS has incorrect percentiles for shuffle read

[jira] [Resolved] (SPARK-38344) Avoid to submit task when there are no requests to push up in push-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-38344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-38344. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-38344) Avoid to submit task when there are no requests to push up in push-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-38344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-38344: --- Assignee: weixiuli > Avoid to submit task when there are no requests to

[jira] [Resolved] (SPARK-37688) ExecutorMonitor should ignore SparkListenerBlockUpdated event if executor was not active

[ https://issues.apache.org/jira/browse/SPARK-37688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-37688. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-37688) ExecutorMonitor should ignore SparkListenerBlockUpdated event if executor was not active

[ https://issues.apache.org/jira/browse/SPARK-37688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-37688: --- Assignee: hujiahua > ExecutorMonitor should ignore

[jira] [Resolved] (SPARK-34826) Adaptive fetch of shuffle mergers for Push based shuffle

[ https://issues.apache.org/jira/browse/SPARK-34826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-34826. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-34826) Adaptive fetch of shuffle mergers for Push based shuffle

[ https://issues.apache.org/jira/browse/SPARK-34826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-34826: --- Assignee: Venkata krishnan Sowrirajan > Adaptive fetch of shuffle mergers

[jira] [Assigned] (SPARK-37793) Invalid LocalMergedBlockData cause task hang

[ https://issues.apache.org/jira/browse/SPARK-37793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-37793: --- Assignee: Chandni Singh > Invalid LocalMergedBlockData cause task hang >

[jira] [Resolved] (SPARK-37793) Invalid LocalMergedBlockData cause task hang

[ https://issues.apache.org/jira/browse/SPARK-37793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-37793. - Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed >

[jira] [Resolved] (SPARK-37675) Push-based merge finalization bugs in the RemoteBlockPushResolver

[ https://issues.apache.org/jira/browse/SPARK-37675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-37675. - Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue

[jira] [Assigned] (SPARK-37675) Push-based merge finalization bugs in the RemoteBlockPushResolver

[ https://issues.apache.org/jira/browse/SPARK-37675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-37675: --- Assignee: Chandni Singh > Push-based merge finalization bugs in the

[jira] [Assigned] (SPARK-37712) Spark request yarn cluster metrics slow and cause unnecessary delay

[ https://issues.apache.org/jira/browse/SPARK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-37712: --- Assignee: angerszhu > Spark request yarn cluster metrics slow and cause

[jira] [Resolved] (SPARK-37712) Spark request yarn cluster metrics slow and cause unnecessary delay

[ https://issues.apache.org/jira/browse/SPARK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-37712. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-37847) PushBlockStreamCallback should check isTooLate first to avoid NPE

[ https://issues.apache.org/jira/browse/SPARK-37847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-37847: --- Assignee: Cheng Pan > PushBlockStreamCallback should check isTooLate first

[jira] [Resolved] (SPARK-37847) PushBlockStreamCallback should check isTooLate first to avoid NPE

[ https://issues.apache.org/jira/browse/SPARK-37847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-37847. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-37710) Add detailed log message for java.io.IOException occurring on Kryo flow

[ https://issues.apache.org/jira/browse/SPARK-37710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-37710: --- Assignee: Eren Avsarogullari > Add detailed log message for

[jira] [Resolved] (SPARK-37710) Add detailed log message for java.io.IOException occurring on Kryo flow

[ https://issues.apache.org/jira/browse/SPARK-37710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-37710. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Resolved] (SPARK-33701) Adaptive shuffle merge finalization for push-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-33701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-33701. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-33701) Adaptive shuffle merge finalization for push-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-33701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-33701: --- Assignee: Venkata krishnan Sowrirajan > Adaptive shuffle merge

[jira] [Resolved] (SPARK-37023) Avoid fetching merge status when shuffleMergeEnabled is false for a shuffleDependency during retry

[ https://issues.apache.org/jira/browse/SPARK-37023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-37023. - Fix Version/s: 3.3.0 3.2.1 Resolution: Fixed Issue

[jira] [Assigned] (SPARK-37023) Avoid fetching merge status when shuffleMergeEnabled is false for a shuffleDependency during retry

[ https://issues.apache.org/jira/browse/SPARK-37023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-37023: --- Assignee: Minchu Yang > Avoid fetching merge status when

[jira] [Resolved] (SPARK-36834) Namespace log lines in External Shuffle Service

[ https://issues.apache.org/jira/browse/SPARK-36834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36834. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-36834) Namespace log lines in External Shuffle Service

[ https://issues.apache.org/jira/browse/SPARK-36834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36834: --- Assignee: Thejdeep Gudivada > Namespace log lines in External Shuffle

[jira] [Assigned] (SPARK-36834) Namespace log lines in External Shuffle Service

[ https://issues.apache.org/jira/browse/SPARK-36834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36834: --- Assignee: (was: Thejdeep Gudivada) > Namespace log lines in External

[jira] [Assigned] (SPARK-36834) Namespace log lines in External Shuffle Service

[ https://issues.apache.org/jira/browse/SPARK-36834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36834: --- Assignee: Thejdeep Gudivada > Namespace log lines in External Shuffle

[jira] [Assigned] (SPARK-36798) When SparkContext is stopped, metrics system should be flushed after listeners have finished processing

[ https://issues.apache.org/jira/browse/SPARK-36798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36798: --- Assignee: Harsh Panchal > When SparkContext is stopped, metrics system

[jira] [Resolved] (SPARK-36798) When SparkContext is stopped, metrics system should be flushed after listeners have finished processing

[ https://issues.apache.org/jira/browse/SPARK-36798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36798. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Commented] (SPARK-36892) Disable batch fetch for a shuffle when push based shuffle is enabled

[ https://issues.apache.org/jira/browse/SPARK-36892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425016#comment-17425016 ] Mridul Muralidharan commented on SPARK-36892: - Sounds good [~Gengliang.Wang], I am not aware

[jira] [Commented] (SPARK-36926) Discrepancy in Q22 of TPCH for Spark 3.2

[ https://issues.apache.org/jira/browse/SPARK-36926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424097#comment-17424097 ] Mridul Muralidharan commented on SPARK-36926: - +CC [~gengliang], [~cloud_fan] > Discrepancy

[jira] [Comment Edited] (SPARK-36892) Disable batch fetch for a shuffle when push based shuffle is enabled

[ https://issues.apache.org/jira/browse/SPARK-36892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424052#comment-17424052 ] Mridul Muralidharan edited comment on SPARK-36892 at 10/4/21, 5:26 PM:

[jira] [Commented] (SPARK-36892) Disable batch fetch for a shuffle when push based shuffle is enabled

[ https://issues.apache.org/jira/browse/SPARK-36892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424052#comment-17424052 ] Mridul Muralidharan commented on SPARK-36892: - [~Gengliang.Wang] Thanks for holding the

[jira] [Created] (SPARK-36892) Disable batch fetch for a shuffle when push based shuffle is enabled

Mridul Muralidharan created SPARK-36892: --- Summary: Disable batch fetch for a shuffle when push based shuffle is enabled Key: SPARK-36892 URL: https://issues.apache.org/jira/browse/SPARK-36892

[jira] [Resolved] (SPARK-36615) SparkContext should register shutdown hook earlier

[ https://issues.apache.org/jira/browse/SPARK-36615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36615. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-36615) SparkContext should register shutdown hook earlier

[ https://issues.apache.org/jira/browse/SPARK-36615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36615: --- Assignee: angerszhu > SparkContext should register shutdown hook earlier >

[jira] [Commented] (SPARK-36772) FinalizeShuffleMerge fails with an exception due to attempt id not matching

[ https://issues.apache.org/jira/browse/SPARK-36772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415891#comment-17415891 ] Mridul Muralidharan commented on SPARK-36772: - Thanks [~Gengliang.Wang] ! And thanks for

[jira] [Comment Edited] (SPARK-36772) FinalizeShuffleMerge fails with an exception due to attempt id not matching

[ https://issues.apache.org/jira/browse/SPARK-36772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415803#comment-17415803 ] Mridul Muralidharan edited comment on SPARK-36772 at 9/15/21, 11:55 PM:

[jira] [Commented] (SPARK-36772) FinalizeShuffleMerge fails with an exception due to attempt id not matching

[ https://issues.apache.org/jira/browse/SPARK-36772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415803#comment-17415803 ] Mridul Muralidharan commented on SPARK-36772: - +CC [~Gengliang.Wang] > FinalizeShuffleMerge

[jira] [Created] (SPARK-36772) FinalizeShuffleMerge fails with an exception due to attempt id not matching

Mridul Muralidharan created SPARK-36772: --- Summary: FinalizeShuffleMerge fails with an exception due to attempt id not matching Key: SPARK-36772 URL: https://issues.apache.org/jira/browse/SPARK-36772

[jira] [Assigned] (SPARK-36705) Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

[ https://issues.apache.org/jira/browse/SPARK-36705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36705: --- Assignee: Minchu Yang > Disable push based shuffle when IO encryption is

[jira] [Resolved] (SPARK-36705) Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

[ https://issues.apache.org/jira/browse/SPARK-36705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36705. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Commented] (SPARK-36700) BlockManager re-registration is broken due to deferred removal of BlockManager

[ https://issues.apache.org/jira/browse/SPARK-36700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412732#comment-17412732 ] Mridul Muralidharan commented on SPARK-36700: - Thanks for chasing this down [~Ngone51] !

[jira] [Created] (SPARK-36705) Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

Mridul Muralidharan created SPARK-36705: --- Summary: Disable push based shuffle when IO encryption is enabled or serializer is not relocatable Key: SPARK-36705 URL:

[jira] [Assigned] (SPARK-35548) Handling new attempt has started error message in BlockPushErrorHandler in client

[ https://issues.apache.org/jira/browse/SPARK-35548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-35548: --- Assignee: Qi Zhu (was: zhuqi) > Handling new attempt has started error

[jira] [Assigned] (SPARK-36344) Fix some typos in ShuffleBlockPusher class.

[ https://issues.apache.org/jira/browse/SPARK-36344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36344: --- Assignee: Qi Zhu (was: zhuqi) > Fix some typos in ShuffleBlockPusher

[jira] [Assigned] (SPARK-35548) Handling new attempt has started error message in BlockPushErrorHandler in client

[ https://issues.apache.org/jira/browse/SPARK-35548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-35548: --- Assignee: zhuqi > Handling new attempt has started error message in

[jira] [Resolved] (SPARK-35548) Handling new attempt has started error message in BlockPushErrorHandler in client

[ https://issues.apache.org/jira/browse/SPARK-35548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-35548. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Resolved] (SPARK-36374) Push-based shuffle documentation

[ https://issues.apache.org/jira/browse/SPARK-36374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36374. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-36374) Push-based shuffle documentation

[ https://issues.apache.org/jira/browse/SPARK-36374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36374: --- Assignee: Venkata krishnan Sowrirajan > Push-based shuffle documentation >

[jira] [Resolved] (SPARK-36483) Fix intermittent test failure due to netty dependency version bump

[ https://issues.apache.org/jira/browse/SPARK-36483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36483. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-36483) Fix intermittent test failure due to netty dependency version bump

[ https://issues.apache.org/jira/browse/SPARK-36483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36483: --- Assignee: Min Shen > Fix intermittent test failure due to netty dependency

[jira] [Commented] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17397725#comment-17397725 ] Mridul Muralidharan commented on SPARK-36378: - Thanks for marking this resolved [~Ngone51] !

[jira] [Resolved] (SPARK-36423) Randomize blocks within a push request before pushing to improve block merge ratio

[ https://issues.apache.org/jira/browse/SPARK-36423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36423. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-36423) Randomize blocks within a push request before pushing to improve block merge ratio

[ https://issues.apache.org/jira/browse/SPARK-36423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36423: --- Assignee: Min Shen > Randomize blocks within a push request before pushing

[jira] [Assigned] (SPARK-36173) [CORE] Support getting CPU number in TaskContext

[ https://issues.apache.org/jira/browse/SPARK-36173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36173: --- Assignee: Xiaochang Wu > [CORE] Support getting CPU number in TaskContext

[jira] [Resolved] (SPARK-36173) [CORE] Support getting CPU number in TaskContext

[ https://issues.apache.org/jira/browse/SPARK-36173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36173. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-30602: --- Assignee: Min Shen > SPIP: Support push-based shuffle to improve shuffle

[jira] [Commented] (SPARK-35275) Add checksum for shuffle blocks

[ https://issues.apache.org/jira/browse/SPARK-35275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391640#comment-17391640 ] Mridul Muralidharan commented on SPARK-35275: - Do we want to add a documentation task for

[jira] [Assigned] (SPARK-36206) Diagnose shuffle data corruption by checksum

[ https://issues.apache.org/jira/browse/SPARK-36206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36206: --- Assignee: wuyi > Diagnose shuffle data corruption by checksum >

[jira] [Resolved] (SPARK-36206) Diagnose shuffle data corruption by checksum

[ https://issues.apache.org/jira/browse/SPARK-36206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36206. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Updated] (SPARK-35917) Disable push-based shuffle until the feature is complete

[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-35917: Fix Version/s: (was: 3.2.0) > Disable push-based shuffle until the feature is

[jira] [Updated] (SPARK-35917) Disable push-based shuffle until the feature is complete

[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-35917: Fix Version/s: 3.2.0 > Disable push-based shuffle until the feature is complete >

[jira] [Assigned] (SPARK-35917) Disable push-based shuffle until the feature is complete

[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-35917: --- Assignee: (was: Mridul Muralidharan) > Disable push-based shuffle

[jira] [Assigned] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36378: --- Assignee: (was: Mridul Muralidharan) > Minor changes to address a few

[jira] [Assigned] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-30602: --- Shepherd: Mridul Muralidharan Assignee: (was: Mridul Muralidharan)

[jira] [Updated] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-30602: Fix Version/s: 3.2.0 > SPIP: Support push-based shuffle to improve shuffle

[jira] [Comment Edited] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391318#comment-17391318 ] Mridul Muralidharan edited comment on SPARK-30602 at 8/2/21, 5:30 AM:

[jira] [Commented] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391324#comment-17391324 ] Mridul Muralidharan commented on SPARK-30602: - Thanks for all the work in getting this

[jira] [Assigned] (SPARK-36266) Rename classes in shuffle RPC used for block push operations

[ https://issues.apache.org/jira/browse/SPARK-36266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36266: --- Assignee: Min Shen > Rename classes in shuffle RPC used for block push

[jira] [Resolved] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-30602. - Resolution: Fixed The only pending task here is documentation. [~vsowrirajan]

[jira] [Assigned] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-30602: --- Assignee: Mridul Muralidharan > SPIP: Support push-based shuffle to

[jira] [Updated] (SPARK-32923) Add support to properly handle different type of stage retries

[ https://issues.apache.org/jira/browse/SPARK-32923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-32923: Fix Version/s: 3.2.0 > Add support to properly handle different type of stage

[jira] [Assigned] (SPARK-32923) Add support to properly handle different type of stage retries

[ https://issues.apache.org/jira/browse/SPARK-32923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-32923: --- Assignee: Venkata krishnan Sowrirajan > Add support to properly handle

[jira] [Issue Comment Deleted] (SPARK-32923) Add support to properly handle different type of stage retries

[ https://issues.apache.org/jira/browse/SPARK-32923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-32923: Comment: was deleted (was: This has been handled by SPARK-32923) > Add support

[jira] [Resolved] (SPARK-32923) Add support to properly handle different type of stage retries

[ https://issues.apache.org/jira/browse/SPARK-32923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-32923. - Resolution: Fixed This has been handled by SPARK-32923 > Add support to

[jira] [Assigned] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36378: --- Assignee: Mridul Muralidharan > Minor changes to address a few identified

[jira] [Resolved] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36378. - Resolution: Won't Fix Let us move this outside of the SPIP and into individual

[jira] [Assigned] (SPARK-35917) Disable push-based shuffle until the feature is complete

[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-35917: --- Assignee: Mridul Muralidharan > Disable push-based shuffle until the

[jira] [Resolved] (SPARK-35917) Disable push-based shuffle until the feature is complete

[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-35917. - Resolution: Won't Fix > Disable push-based shuffle until the feature is

[jira] [Commented] (SPARK-35917) Disable push-based shuffle until the feature is complete

[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391313#comment-17391313 ] Mridul Muralidharan commented on SPARK-35917: - Closing this Jira - as push based shuffle has

[jira] [Resolved] (SPARK-36266) Rename classes in shuffle RPC used for block push operations

[ https://issues.apache.org/jira/browse/SPARK-36266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36266. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-35546) Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a better way

[ https://issues.apache.org/jira/browse/SPARK-35546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-35546: --- Assignee: Ye Zhou > Enable push-based shuffle when multiple app attempts

<    1   2   3   4   5   6   >