[ https://issues.apache.org/jira/browse/SPARK-24700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcelo Vanzin resolved SPARK-24700. ------------------------------------ Resolution: Cannot Reproduce This needs more details, like logs if available. AFAIK the default status for applications in cluster mode was changed to failed in 2.0 (SPARK-13642). > AM shutdown terminates streaming application but marks it as successful > ----------------------------------------------------------------------- > > Key: SPARK-24700 > URL: https://issues.apache.org/jira/browse/SPARK-24700 > Project: Spark > Issue Type: Bug > Components: Spark Core, Structured Streaming, YARN > Affects Versions: 2.2.0 > Reporter: Niranjani > Priority: Critical > > While running Spark streaming applications on YARN in cluster mode, > {color:#333333}reboot/shutdown{color} of the node hosting AM causes the > application to terminate SparkContext and mark it as SUCCEEDED. This is > misleading as: > # streaming applications are long-running and not expected to terminate > # {color:#172b4d}setting > {color}{color:#172b4d}{{spark.yarn.maxAppAttempts}}{color}{color:#172b4d} for > application re-attempt does not take effect since this scenario is not > recognized as a failure{color} > {color:#172b4d}Please provide a workaround for resubmitting > streaming/long-running applications in the case of AM shutdown.{color} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org