[ https://issues.apache.org/jira/browse/SPARK-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcelo Vanzin reassigned SPARK-28150: -------------------------------------- Assignee: Marcelo Vanzin > Failure to create multiple contexts in same JVM with Kerberos auth > ------------------------------------------------------------------ > > Key: SPARK-28150 > URL: https://issues.apache.org/jira/browse/SPARK-28150 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 3.0.0 > Reporter: Marcelo Vanzin > Assignee: Marcelo Vanzin > Priority: Minor > > Take the following small app that creates multiple contexts (not > concurrently): > {code} > from pyspark.context import SparkContext > import time > for i in range(2): > with SparkContext() as sc: > pass > time.sleep(5) > {code} > This fails when kerberos (without dt renewal) is being used: > {noformat} > 19/06/24 11:33:58 ERROR spark.SparkContext: Error initializing SparkContext. > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.security.HBaseDelegationTokenProvider.obtainDelegationTokens(HBaseDelegationTokenProvider.scala:49) > Caused by: > org.apache.hadoop.hbase.shaded.com.google.protobuf.ServiceException: Error > calling method hbase.pb.AuthenticationService.GetAuthenticationToken > at > org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:71) > Caused by: > org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.security.AccessDeniedException): > org.apache.hadoop.hbase.security.AccessDeniedException: Token generation > only allowed for Kerberos authenticated clients > at > org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:126) > {noformat} > If you enable dt renewal things work since the codes takes a slightly > different path when generating the initial delegation tokens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org