SaintBacchus created SPARK-8619: ----------------------------------- Summary: Can't find the keytab file when recovering the streaming application. Key: SPARK-8619 URL: https://issues.apache.org/jira/browse/SPARK-8619 Project: Spark Issue Type: Bug Components: Streaming Affects Versions: 1.5.0 Reporter: SaintBacchus
In a streaming application, I use *--keytab /root/spark.keytab* to get the token. But when the streaming application failed and I wanted to recover it from checkpoint file, there was an error: {quote} java.io.IOException: Login failure for spark/hadoop.hadoop....@hadoop.com from keytab spark.keytab-1fd8f7bb-0d3c-4f65-990a-9ae09055cc8d: javax.security.auth.login.LoginException: Unable to obtain password from user {quote} Spark had changed the configuration, so the checkpoint can't find the file: {code:title=Client.java @ Function: setupCredentials |borderStyle=solid} val keytabFileName = f.getName + "-" + UUID.randomUUID().toString UserGroupInformation.loginUserFromKeytab(args.principal, args.keytab) loginFromKeytab = true sparkConf.set("spark.yarn.keytab", keytabFileName) {code} So when recovering the application, we should ignore this configurations. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org