[ https://issues.apache.org/jira/browse/SPARK-24062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Saisai Shao resolved SPARK-24062. --------------------------------- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.4.0 2.3.1 > SASL encryption cannot be worked in ThriftServer > ------------------------------------------------ > > Key: SPARK-24062 > URL: https://issues.apache.org/jira/browse/SPARK-24062 > Project: Spark > Issue Type: Bug > Components: Spark Core, SQL > Affects Versions: 2.3.0 > Reporter: Saisai Shao > Assignee: Saisai Shao > Priority: Major > Fix For: 2.3.1, 2.4.0 > > > Spark thrift server will throw an exception when SASL encryption is used. > > {noformat} > 18/04/16 14:36:46 ERROR TransportRequestHandler: Error while invoking > RpcHandler#receive() on RPC id 8384069538832556183 > java.lang.IllegalArgumentException: A secret key must be specified via the > spark.authenticate.secret config > at > org.apache.spark.SecurityManager$$anonfun$getSecretKey$4.apply(SecurityManager.scala:510) > at > org.apache.spark.SecurityManager$$anonfun$getSecretKey$4.apply(SecurityManager.scala:510) > at scala.Option.getOrElse(Option.scala:121) > at org.apache.spark.SecurityManager.getSecretKey(SecurityManager.scala:509) > at org.apache.spark.SecurityManager.getSecretKey(SecurityManager.scala:551) > at > org.apache.spark.network.sasl.SparkSaslServer$DigestCallbackHandler.handle(SparkSaslServer.java:166) > at > com.sun.security.sasl.digest.DigestMD5Server.validateClientResponse(DigestMD5Server.java:589) > at > com.sun.security.sasl.digest.DigestMD5Server.evaluateResponse(DigestMD5Server.java:244) > at > org.apache.spark.network.sasl.SparkSaslServer.response(SparkSaslServer.java:119) > at > org.apache.spark.network.sasl.SaslRpcHandler.receive(SaslRpcHandler.java:103) > at > org.apache.spark.network.server.TransportRequestHandler.processRpcRequest(TransportRequestHandler.java:187) > at > org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:111){noformat} > To investigate it, the issue is: > Spark on Yarn stores SASL secret in current UGI's credentials, this > credentials will be distributed to AM and executors, so that executors and > drive share the same secret to communicate. But STS/Hive library code will > refresh the current UGI by UGI's loginFromKeytab(), this will create a new > UGI in the current context with empty tokens and secret keys, so secret key > is lost in the current context's UGI, that's why Spark driver throws secret > key not found exception. > In Spark 2.2 code, Spark also stores this secret key in {{SecurityManager}}'s > class variable, so even UGI is refreshed, the secret is still existed in the > object, so STS with SASL can still be worked in Spark 2.2. But in Spark 2.3, > we always search key from current UGI, which makes it fail to work in Spark > 2.3. > To fix this issue, there're two possible solutions: > 1. Fix in STS/Hive library, when a new UGI is refreshed, copy the secret key > from original UGI to the new one. The difficulty is that some codes to > refresh the UGI is existed in Hive library, which makes us hard to change the > code. > 2. Roll back the logics in SecurityManager to match Spark 2.2, so that this > issue can be fixed. > 2nd solution seems a simple one. So I will propose a PR with 2nd solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org