[
https://issues.apache.org/jira/browse/DRILL-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Hammer updated DRILL-7581:
---------------------------------
Description:
OpenJDK received a minor update which causes this kerberos authentication issue
on our setup:
# Apache Drill started as drillbit server with zookeeper
# Plain authentication for users and kerberos authentication to HDFS cluster
# drill-override.conf lists the kerberos realm and the keytab file:
#
## principal: "user@REALM",
keytab: "/mnt/kerberos/user.keytab"
# storage-plugins-override.conf specifies a HDFS data source
This Java version works (with Drill 1.17.0):
* openjdk version "11.0.5" 2019-10-15
These new version show the authentication issue mentioned in my previous
comment work:
* openjdk version "11.0.6" 2020-01-14
* openjdk version "1.8.0_242"
{code:java}
[main] INFO o.a.d.exec.server.BootStrapContext - Process user name: 'root' and
logged in successfully as 'user@REALM'
Exception in thread "main"
org.apache.drill.exec.exception.DrillbitStartupException: Failure during
initial startup of Drillbit.
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:584)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:550)
at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:546)
Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Error
during udf area creation [/user/user/drll/udf/registry] on file system
[hdfs://cluster]
at
org.apache.drill.common.exceptions.DrillRuntimeException.format(DrillRuntimeException.java:49)
at
org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry.createArea(RemoteFunctionRegistry.java:280)
at
org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry.prepareAreas(RemoteFunctionRegistry.java:237)
at
org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry.init(RemoteFunctionRegistry.java:107)
at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:227)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:580)
... 2 more
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
User: USER@REALM is not allowed to impersonate user@REALM
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1545)
at org.apache.hadoop.ipc.Client.call(Client.java:1491)
at org.apache.hadoop.ipc.Client.call(Client.java:1388)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
at com.sun.proxy.$Proxy48.mkdirs(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:660)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
at com.sun.proxy.$Proxy49.mkdirs(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2425)
at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2401)
at
org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1318)
at
org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1315)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1332)
at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1307)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2275)
at
org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry.createArea(RemoteFunctionRegistry.java:257)
... 6 more
Apache Drill terminated
{code}
was:
OpenJDK received a minor update which causes this kerberos authentication issue
on our setup:
# Apache Drill started as drillbit server with zookeeper
# Plain authentication for users and kerberos authentication to HDFS cluster
# drill-override.conf lists the kerberos realm and the keytab file:
## principal: "psx1...@emea.bosch.com",
keytab: "/mnt/kerberos/psx1liz.keytab"
# storage-plugins-override.conf specifies a HDFS data source
This Java version works (with Drill 1.17.0):
* openjdk version "11.0.5" 2019-10-15
These new version show the authentication issue mentioned in my previous
comment work:
* openjdk version "11.0.6" 2020-01-14
* openjdk version "1.8.0_242"
{code:java}
[main] INFO o.a.d.exec.server.BootStrapContext - Process user name: 'root' and
logged in successfully as 'user@REALM'
Exception in thread "main"
org.apache.drill.exec.exception.DrillbitStartupException: Failure during
initial startup of Drillbit.
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:584)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:550)
at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:546)
Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Error
during udf area creation [/user/user/drll/udf/registry] on file system
[hdfs://cluster]
at
org.apache.drill.common.exceptions.DrillRuntimeException.format(DrillRuntimeException.java:49)
at
org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry.createArea(RemoteFunctionRegistry.java:280)
at
org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry.prepareAreas(RemoteFunctionRegistry.java:237)
at
org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry.init(RemoteFunctionRegistry.java:107)
at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:227)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:580)
... 2 more
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
User: USER@REALM is not allowed to impersonate user@REALM
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1545)
at org.apache.hadoop.ipc.Client.call(Client.java:1491)
at org.apache.hadoop.ipc.Client.call(Client.java:1388)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
at com.sun.proxy.$Proxy48.mkdirs(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:660)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
at com.sun.proxy.$Proxy49.mkdirs(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2425)
at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2401)
at
org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1318)
at
org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1315)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1332)
at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1307)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2275)
at
org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry.createArea(RemoteFunctionRegistry.java:257)
... 6 more
Apache Drill terminated
{code}
> Kerberos AuthorizationException on creating UDF folders after Java minor
> releases
> ---------------------------------------------------------------------------------
>
> Key: DRILL-7581
> URL: https://issues.apache.org/jira/browse/DRILL-7581
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.17.0
> Reporter: Stefan Hammer
> Priority: Major
>
> OpenJDK received a minor update which causes this kerberos authentication
> issue on our setup:
> # Apache Drill started as drillbit server with zookeeper
> # Plain authentication for users and kerberos authentication to HDFS cluster
> # drill-override.conf lists the kerberos realm and the keytab file:
> #
> ## principal: "user@REALM",
> keytab: "/mnt/kerberos/user.keytab"
> # storage-plugins-override.conf specifies a HDFS data source
>
> This Java version works (with Drill 1.17.0):
> * openjdk version "11.0.5" 2019-10-15
> These new version show the authentication issue mentioned in my previous
> comment work:
> * openjdk version "11.0.6" 2020-01-14
> * openjdk version "1.8.0_242"
> {code:java}
> [main] INFO o.a.d.exec.server.BootStrapContext - Process user name: 'root'
> and logged in successfully as 'user@REALM'
> Exception in thread "main"
> org.apache.drill.exec.exception.DrillbitStartupException: Failure during
> initial startup of Drillbit.
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:584)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:550)
> at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:546)
> Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Error
> during udf area creation [/user/user/drll/udf/registry] on file system
> [hdfs://cluster]
> at
> org.apache.drill.common.exceptions.DrillRuntimeException.format(DrillRuntimeException.java:49)
> at
> org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry.createArea(RemoteFunctionRegistry.java:280)
> at
> org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry.prepareAreas(RemoteFunctionRegistry.java:237)
> at
> org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry.init(RemoteFunctionRegistry.java:107)
> at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:227)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:580)
> ... 2 more
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> User: USER@REALM is not allowed to impersonate user@REALM
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1545)
> at org.apache.hadoop.ipc.Client.call(Client.java:1491)
> at org.apache.hadoop.ipc.Client.call(Client.java:1388)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
> at com.sun.proxy.$Proxy48.mkdirs(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:660)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
> at com.sun.proxy.$Proxy49.mkdirs(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2425)
> at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2401)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1318)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1315)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1332)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1307)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2275)
> at
> org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry.createArea(RemoteFunctionRegistry.java:257)
> ... 6 more
> Apache Drill terminated
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
