[ https://issues.apache.org/jira/browse/SPARK-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Wendell updated SPARK-1272: ----------------------------------- Assignee: (was: OuyangJin) > Don't fail job if some local directories are buggy > -------------------------------------------------- > > Key: SPARK-1272 > URL: https://issues.apache.org/jira/browse/SPARK-1272 > Project: Spark > Issue Type: Improvement > Components: Shuffle > Reporter: Patrick Wendell > Fix For: 1.1.0 > > > If Spark cannot create shuffle directories inside of a local directory it > might make sense to just log an error and continue, provided that at least > one valid shuffle directory exists. Otherwise if a single disk is wonky the > entire job can fail. > The down side is that this might mask failures if the person actually > misconfigures the local directories to point to the wrong disk(s). -- This message was sent by Atlassian JIRA (v6.2#6252)