wuyi created SPARK-48394:
Summary: Cleanup mapIdToMapIndex on mapoutput unregister
Key: SPARK-48394
URL: https://issues.apache.org/jira/browse/SPARK-48394
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-46052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-46052:
-
Description:
Spark has two functions to kill all tasks in a Stage:
* `cancelTasks`: Not only kill all the
wuyi created SPARK-46052:
Summary: Remove unnecessary TaskScheduler.killAllTaskAttempts
Key: SPARK-46052
URL: https://issues.apache.org/jira/browse/SPARK-46052
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-45527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774756#comment-17774756
]
wuyi commented on SPARK-45527:
--
cc [~wbo4958] [~tgraves]
> Task fraction resource request is not
wuyi created SPARK-45527:
Summary: Task fraction resource request is not expected
Key: SPARK-45527
URL: https://issues.apache.org/jira/browse/SPARK-45527
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-45527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-45527:
-
Description:
{code:java}
test("SPARK-XXX") {
import org.apache.spark.resource.{ResourceProfileBuilder,
[
https://issues.apache.org/jira/browse/SPARK-45057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769443#comment-17769443
]
wuyi commented on SPARK-45057:
--
In the case of "Received UploadBlock request from T1 (blocked by T4)",
wuyi created SPARK-45310:
Summary: Mapstatus location type changed from external shuffle
service to executor after decommission migration
Key: SPARK-45310
URL: https://issues.apache.org/jira/browse/SPARK-45310
wuyi created SPARK-42577:
Summary: A large stage could run indefinitely due to executor lost
Key: SPARK-42577
URL: https://issues.apache.org/jira/browse/SPARK-42577
Project: Spark
Issue Type:
wuyi created SPARK-41958:
Summary: Disallow arbitrary custom classpath with proxy user in
cluster mode
Key: SPARK-41958
URL: https://issues.apache.org/jira/browse/SPARK-41958
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653781#comment-17653781
]
wuyi commented on SPARK-41497:
--
> If I am not wrong, SQL makes very heavy use of accumulators, and so most
[
https://issues.apache.org/jira/browse/SPARK-41848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-41848:
-
Priority: Blocker (was: Major)
> Tasks are over-scheduled with TaskResourceProfile
>
[
https://issues.apache.org/jira/browse/SPARK-41848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653739#comment-17653739
]
wuyi commented on SPARK-41848:
--
cc [~ivoson]
> Tasks are over-scheduled with TaskResourceProfile
>
wuyi created SPARK-41848:
Summary: Tasks are over-scheduled with TaskResourceProfile
Key: SPARK-41848
URL: https://issues.apache.org/jira/browse/SPARK-41848
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-39853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-39853:
-
Fix Version/s: 3.4.0
> Support stage level schedule for standalone cluster when dynamic allocation
> is
[
https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646971#comment-17646971
]
wuyi edited comment on SPARK-41497 at 12/14/22 7:31 AM:
I'm thinking if we could
[
https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646971#comment-17646971
]
wuyi commented on SPARK-41497:
--
I'm thinking if we could improve the improved Option 4 by changing the rdd
[
https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646969#comment-17646969
]
wuyi commented on SPARK-41497:
--
> do we have a way to do that ?
[~mridulm80] Currently, we only have
[
https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646919#comment-17646919
]
wuyi commented on SPARK-41497:
--
[~mridulm80] For b) and c), shouldn't we allow T2 to use the result of
[
https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646639#comment-17646639
]
wuyi commented on SPARK-41497:
--
[~mridulm80] Sounds like a better idea than option 4. But I think it still
[
https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-41497:
-
Description:
Accumulator could be undercounted when the retried task has rdd cache. See the
example below and
[
https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646097#comment-17646097
]
wuyi commented on SPARK-41497:
--
[~mridulm80] [~tgraves] [~attilapiros] [~ivoson] any good ideas?
>
[
https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-41497:
-
Description:
Accumulator could be undercounted when the retried task has rdd cache. See the
example below and
wuyi created SPARK-41497:
Summary: Accumulator undercounting in the case of retry task with
rdd cache
Key: SPARK-41497
URL: https://issues.apache.org/jira/browse/SPARK-41497
Project: Spark
Issue
wuyi created SPARK-41469:
Summary: Task rerun on decommissioned executor can be avoided if
shuffle data has migrated
Key: SPARK-41469
URL: https://issues.apache.org/jira/browse/SPARK-41469
Project: Spark
wuyi created SPARK-41460:
Summary: Introduce IsolatedThreadSafeRpcEndpoint to extend
IsolatedRpcEndpoint
Key: SPARK-41460
URL: https://issues.apache.org/jira/browse/SPARK-41460
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-41360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-41360:
-
Summary: Avoid BlockManager re-registration if the executor has been lost
(was: Avoid BlockMananger
wuyi created SPARK-41360:
Summary: Avoid BlockMananger re-registration if the executor has
been lost
Key: SPARK-41360
URL: https://issues.apache.org/jira/browse/SPARK-41360
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-35011:
-
Summary: False active executor in UI that caused by BlockManager
reregistration (was: Avoid Block Manager
[
https://issues.apache.org/jira/browse/SPARK-40596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-40596.
--
Assignee: Bo Zhang
Resolution: Fixed
Issue resolved by https://github.com/apache/spark/pull/38030
>
[
https://issues.apache.org/jira/browse/SPARK-39853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-39853.
--
Resolution: Fixed
Issue resolved by https://github.com/apache/spark/pull/37268
> Support stage level
[
https://issues.apache.org/jira/browse/SPARK-39853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi reassigned SPARK-39853:
Assignee: huangtengfei
> Support stage level schedule for standalone cluster when dynamic allocation
>
[
https://issues.apache.org/jira/browse/SPARK-40320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608268#comment-17608268
]
wuyi commented on SPARK-40320:
--
I see. Thanks for the explaination.
> When the Executor plugin fails to
[
https://issues.apache.org/jira/browse/SPARK-40320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600777#comment-17600777
]
wuyi commented on SPARK-40320:
--
> Actually the `CoarseGrainedExecutorBackend` JVM process is active but
[
https://issues.apache.org/jira/browse/SPARK-39957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi reassigned SPARK-39957:
Assignee: Kai-Hsun Chen
> Delay onDisconnected to enable Driver receives ExecutorExitCode
>
[
https://issues.apache.org/jira/browse/SPARK-39957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-39957.
--
Resolution: Fixed
Issue resolved by https://github.com/apache/spark/pull/37400
> Delay onDisconnected to
[
https://issues.apache.org/jira/browse/SPARK-39957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-39957:
-
Fix Version/s: 3.4.0
> Delay onDisconnected to enable Driver receives ExecutorExitCode
>
[
https://issues.apache.org/jira/browse/SPARK-34788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572506#comment-17572506
]
wuyi commented on SPARK-34788:
--
> Why don't you ensure enough disks from the beginning?
On the write side,
[
https://issues.apache.org/jira/browse/SPARK-34788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572478#comment-17572478
]
wuyi commented on SPARK-34788:
--
[~leo wen] are you able to reproduce in your environment? If yes, we could
[
https://issues.apache.org/jira/browse/SPARK-39062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-39062.
--
Fix Version/s: 3.4.0
Assignee: huangtengfei
Resolution: Fixed
Issue resolved by
Title: Message Title
wuyi updated an
Title: Message Title
wuyi resolved as
[
https://issues.apache.org/jira/browse/SPARK-39152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-39152.
--
Fix Version/s: 3.4.0
Resolution: Fixed
Issue resolved by https://github.com/apache/spark/pull/36512
>
[
https://issues.apache.org/jira/browse/SPARK-38683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-38683.
--
Fix Version/s: 3.4.0
Assignee: weixiuli
Resolution: Fixed
Issue resolved by
[
https://issues.apache.org/jira/browse/SPARK-38683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-38683:
-
Issue Type: Improvement (was: Bug)
> It is unnecessary to release the ShuffleManagedBufferIterator or
>
[
https://issues.apache.org/jira/browse/SPARK-38468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506754#comment-17506754
]
wuyi commented on SPARK-38468:
--
Shall we close this one? Seem it's duplicate with
[
https://issues.apache.org/jira/browse/SPARK-37481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506673#comment-17506673
]
wuyi commented on SPARK-37481:
--
Backport fix to 3.1/3.0 also done.
> Disappearance of skipped stages
[
https://issues.apache.org/jira/browse/SPARK-38266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-38266.
--
Resolution: Fixed
> UnresolvedException: Invalid call to dataType on unresolved object caused by
>
[
https://issues.apache.org/jira/browse/SPARK-38266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi reassigned SPARK-38266:
Assignee: wuyi
> UnresolvedException: Invalid call to dataType on unresolved object caused by
>
[
https://issues.apache.org/jira/browse/SPARK-38266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17495285#comment-17495285
]
wuyi commented on SPARK-38266:
--
Issue resolved by https://github.com/apache/spark/pull/35568
>
wuyi created SPARK-38266:
Summary: UnresolvedException: Invalid call to dataType on
unresolved object caused by GetDateFieldOperations
Key: SPARK-38266
URL: https://issues.apache.org/jira/browse/SPARK-38266
[
https://issues.apache.org/jira/browse/SPARK-37580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-37580.
--
Assignee: wangshengjie
Resolution: Fixed
Issue resolved by https://github.com/apache/spark/pull/34834
[
https://issues.apache.org/jira/browse/SPARK-37695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-37695.
--
Fix Version/s: 3.2.1
3.3.0
Assignee: Cheng Pan
Resolution: Fixed
Issue
[
https://issues.apache.org/jira/browse/SPARK-37695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-37695:
-
Description:
Shuffle
{code:java}
21/12/19 18:46:37 WARN TaskSetManager: Lost task 166.0 in stage 1921.0 (TID
[
https://issues.apache.org/jira/browse/SPARK-37695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-37695:
-
Description:
Shuffle corruption diagnosis for push-based shuffle hasn't been supported yet.
So we should skip
wuyi created SPARK-37695:
Summary: Skip diagnosis ob merged blocks from push-based shuffle
Key: SPARK-37695
URL: https://issues.apache.org/jira/browse/SPARK-37695
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-37060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-37060:
-
Fix Version/s: 3.1.3
3.2.1
> Report driver status does not handle response from backup
[
https://issues.apache.org/jira/browse/SPARK-37060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi reassigned SPARK-37060:
Assignee: Mohamadreza Rostami
> Report driver status does not handle response from backup masters
>
[
https://issues.apache.org/jira/browse/SPARK-37060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-37060.
--
Fix Version/s: 3.3.0
Resolution: Fixed
Issue resolved by https://github.com/apache/spark/pull/34331
>
[
https://issues.apache.org/jira/browse/SPARK-37300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-37300.
--
Fix Version/s: 3.3.0
Assignee: hujiahua
Resolution: Fixed
Issue resolved by
[
https://issues.apache.org/jira/browse/SPARK-37481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17458131#comment-17458131
]
wuyi commented on SPARK-37481:
--
Backport fix to 3.1/3.0 is still in progress.
> Disappearance of skipped
[
https://issues.apache.org/jira/browse/SPARK-37481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-37481.
--
Fix Version/s: 3.3.0
3.2.0
Assignee: Kent Yao
Resolution: Fixed
Issue
[
https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441796#comment-17441796
]
wuyi commented on SPARK-36575:
--
FYI: the fix is reverted due to test issues.
> Executor lost may cause
[
https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi reassigned SPARK-36575:
Assignee: hujiahua
> Executor lost may cause spark stage to hang
>
[
https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi reassigned SPARK-36575:
Assignee: (was: wuyi)
> Executor lost may cause spark stage to hang
>
[
https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441486#comment-17441486
]
wuyi commented on SPARK-36575:
--
Issue resolved by [https://github.com/apache/spark/pull/33872.]
To
[
https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-36575:
-
Issue Type: Improvement (was: Bug)
> Executor lost may cause spark stage to hang
>
[
https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-36575:
-
Fix Version/s: 3.3.0
> Executor lost may cause spark stage to hang
>
[
https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi reassigned SPARK-36575:
Assignee: wuyi
> Executor lost may cause spark stage to hang
>
[
https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-36575.
--
Resolution: Fixed
> Executor lost may cause spark stage to hang
> ---
[
https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421971#comment-17421971
]
wuyi commented on SPARK-18105:
--
[~vladimir.prus] Hi, could you also file a sub-task under
[
https://issues.apache.org/jira/browse/SPARK-36700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-36700.
--
Fix Version/s: 3.3.0
3.0.4
3.1.3
3.2.0
[
https://issues.apache.org/jira/browse/SPARK-36700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17413865#comment-17413865
]
wuyi commented on SPARK-36700:
--
Reverted by [https://github.com/apache/spark/pull/33942] and backported to
[
https://issues.apache.org/jira/browse/SPARK-36700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412290#comment-17412290
]
wuyi commented on SPARK-36700:
--
I'm working on the fix.
> BlockManager re-registration is broken due to
[
https://issues.apache.org/jira/browse/SPARK-36700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412285#comment-17412285
]
wuyi commented on SPARK-36700:
--
cc [~mridulm80] [~sumeet.gajjar] [~attilapiros]
cc [~gengliang] for the
wuyi created SPARK-36700:
Summary: BlockManager re-registration is broken due to deferred
removal of BlockManager
Key: SPARK-36700
URL: https://issues.apache.org/jira/browse/SPARK-36700
Project: Spark
wuyi created SPARK-36614:
Summary: Executor loss reason shows "worker lost" rather "Executor
decommission"
Key: SPARK-36614
URL: https://issues.apache.org/jira/browse/SPARK-36614
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-36614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-36614:
-
Attachment: WeChat13c9f1345a096ff83d193e4e9853b165.png
> Executor loss reason shows "worker lost" rather
[
https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406622#comment-17406622
]
wuyi edited comment on SPARK-18105 at 8/30/21, 8:31 AM:
FYI, for users who hit
[
https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406622#comment-17406622
]
wuyi commented on SPARK-18105:
--
FYI, for users who hit the "Stream is corrupted" error, please try to apply
[
https://issues.apache.org/jira/browse/SPARK-36196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406620#comment-17406620
]
wuyi commented on SPARK-36196:
--
Hi [~arghya18] Did you try to apply the fix of
[
https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403556#comment-17403556
]
wuyi commented on SPARK-36558:
--
Discussed with [~vsowrirajan] offline. The issue won't happen in the
[
https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-36558.
--
Resolution: Won't Fix
> Stage has all tasks finished but with ongoing finalization can cause job hang
>
[
https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-36558:
-
Description:
For a stage that all tasks are finished but with ongoing finalization can lead
to job hang.
[
https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-36558:
-
Description:
For a stage that all tasks are finished but with ongoing finalization can lead
to job hang.
[
https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-36558:
-
Description:
For a stage that all tasks are finished but with ongoing finalization can lead
to job hang.
[
https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403456#comment-17403456
]
wuyi commented on SPARK-36558:
--
[~vsowrirajan] Sorry, I missed one tweaked change in `MyRDD`. We should
[
https://issues.apache.org/jira/browse/SPARK-36564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-36564:
-
Summary: LiveRDDDistribution.toApi throws NullPointerException (was:
LiveRDD.doUpdate throws
wuyi created SPARK-36564:
Summary: LiveRDD.doUpdate throws NullPointerException
Key: SPARK-36564
URL: https://issues.apache.org/jira/browse/SPARK-36564
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-36558:
-
Description:
For a stage that all tasks are finished but with ongoing finalization can lead
to job hang.
[
https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-36558:
-
Description:
For a stage that all tasks are finished but with ongoing finalization can lead
to job hang.
[
https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402942#comment-17402942
]
wuyi commented on SPARK-36558:
--
cc [~mridul] [~mshen] [~csingh]
> Stage has all tasks finished but with
wuyi created SPARK-36558:
Summary: Stage has all tasks finished but with ongoing
finalization can cause job hang
Key: SPARK-36558
URL: https://issues.apache.org/jira/browse/SPARK-36558
Project: Spark
wuyi created SPARK-36543:
Summary: Decommission logs too frequent when waiting migration to
finish
Key: SPARK-36543
URL: https://issues.apache.org/jira/browse/SPARK-36543
Project: Spark
Issue Type:
wuyi created SPARK-36532:
Summary: Deadlock in CoarseGrainedExecutorBackend.onDisconnected
Key: SPARK-36532
URL: https://issues.apache.org/jira/browse/SPARK-36532
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-36530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-36530.
--
Resolution: Won't Fix
> Avoid finalizing when there's no push at all in a shuffle
>
[
https://issues.apache.org/jira/browse/SPARK-36530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400118#comment-17400118
]
wuyi commented on SPARK-36530:
--
SGTM. Could you update SPARK-33701 to include this part? Then, I'll close
[
https://issues.apache.org/jira/browse/SPARK-36530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400089#comment-17400089
]
wuyi commented on SPARK-36530:
--
cc [~mridulm80] [~mshen] any thoughts?
> Avoid finalizing when there's no
wuyi created SPARK-36530:
Summary: Avoid finalizing when there's no push at all in a shuffle
Key: SPARK-36530
URL: https://issues.apache.org/jira/browse/SPARK-36530
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi resolved SPARK-36378.
--
Fix Version/s: 3.3.0
3.2.0
Assignee: Min Shen
Resolution: Fixed
Issue
1 - 100 of 426 matches
Mail list logo