[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142227#comment-16142227
]
Apache Spark commented on SPARK-17321:
--
User 'jerryshao' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142229#comment-16142229
]
Apache Spark commented on SPARK-17321:
--
User 'jerryshao' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139502#comment-16139502
]
Saisai Shao commented on SPARK-17321:
-
1. if NM recovery is enabled, then yarn will provide a
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139488#comment-16139488
]
lishuming commented on SPARK-17321:
---
[~jerryshao] I agree with what you said, however there are some
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138347#comment-16138347
]
Thomas Graves commented on SPARK-17321:
---
Yes that sounds good. It wouldn't hurt to verify the
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138320#comment-16138320
]
Saisai Shao commented on SPARK-17321:
-
We're facing the same issue. I think YARN shuffle service
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105164#comment-16105164
]
Thomas Graves commented on SPARK-17321:
---
Can you clarify? as stated above you should not be using
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104373#comment-16104373
]
roncenzhao commented on SPARK-17321:
Hi, I encounter this problem too. Any process about this bug?
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15487203#comment-15487203
]
Thomas Graves commented on SPARK-17321:
---
yes that makes sense and as I stated I think the fix for
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15487166#comment-15487166
]
Alexander Kasper commented on SPARK-17321:
--
No, we're not using NM recovery. What we observed is
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484748#comment-15484748
]
Thomas Graves commented on SPARK-17321:
---
Not sure I follow this comment. So you are using NM
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483413#comment-15483413
]
Alexander Kasper commented on SPARK-17321:
--
I guess then we encountered the 1% where the NM
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477061#comment-15477061
]
Thomas Graves commented on SPARK-17321:
---
Note that if we want to fix this without yarn NM recovery
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477028#comment-15477028
]
Thomas Graves commented on SPARK-17321:
---
it is possible but the point of the recovery dir is it
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476355#comment-15476355
]
Alexander Kasper commented on SPARK-17321:
--
But NM recovery only kicks in if the NM goes down,
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15474557#comment-15474557
]
Thomas Graves commented on SPARK-17321:
---
so there are 2 possible things here:
1) You are using
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15474342#comment-15474342
]
Alexander Kasper commented on SPARK-17321:
--
We discovered the same issue. It seems the shuffle
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451942#comment-15451942
]
yunjiong zhao commented on SPARK-17321:
---
If yarn.nodemanager.recovery.enabled = false & the first
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451925#comment-15451925
]
yunjiong zhao commented on SPARK-17321:
---
Below logs shows that NodeManager already detect the disk
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451742#comment-15451742
]
Sean Owen commented on SPARK-17321:
---
Duplicate of https://issues.apache.org/jira/browse/SPARK-14963 ?
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450424#comment-15450424
]
Apache Spark commented on SPARK-17321:
--
User 'zhaoyunjiong' has created a pull request for this
21 matches
Mail list logo