[ https://issues.apache.org/jira/browse/IGNITE-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062138#comment-15062138 ]
Ivan Veselovsky commented on IGNITE-2195: ----------------------------------------- Quick fix idea is to make org.apache.ignite.internal.processors.hadoop.SecondaryFileSystemProvider pluggable: this way user can substitute it with anything he needs just via Spring configuration. > Accessing from IGFS to HDFS that is in kerberised environment > ------------------------------------------------------------- > > Key: IGNITE-2195 > URL: https://issues.apache.org/jira/browse/IGNITE-2195 > Project: Ignite > Issue Type: Bug > Components: hadoop, IGFS > Affects Versions: ignite-1.4 > Reporter: Denis Magda > Assignee: Ivan Veselovsky > Priority: Critical > > There is some issue in the current IGFS implementation that doesn't take into > account some Kerberos user related settings which leads to the exception > below when there is an attempt to work with Kerberised cluster > {noformat} > Connecting to HDFS with the following settings [uri=null, cfg=all-site.xml, > userName=null] > log4j:WARN No appenders could be found for logger > (org.apache.hadoop.metrics2.lib.MutableMetricsFactory). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > org.apache.hadoop.security.AccessControlException: SIMPLE authentication is > not enabled. Available:[TOKEN, KERBEROS] > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106) > at > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73) > at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2096) > at > org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:944) > at > org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:927) > at > org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:872) > at > org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:868) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus(DistributedFileSystem.java:868) > at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1694) > at org.apache.hadoop.fs.FileSystem$6.<init>(FileSystem.java:1786) > at org.apache.hadoop.fs.FileSystem.listFiles(FileSystem.java:1783) > at com.ig.HadoopFsIssue.main(HadoopFsIssue.java:35) > Caused by: > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): > SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS] > at org.apache.hadoop.ipc.Client.call(Client.java:1427) > at org.apache.hadoop.ipc.Client.call(Client.java:1358) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) > at com.sun.proxy.$Proxy7.getListing(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:573) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy8.getListing(Unknown Source) > at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2094) > {noformat} > The issue is fixed in the following way. Need to revisit the fix and check > whether it can lead to some other consequences. > {noformat} > /** > * @return {@link org.apache.hadoop.fs.FileSystem} instance for this > secondary Fs. > * @throws IOException > */ > public FileSystem createFileSystem(String userName) throws IOException { > userName = IgfsUtils.fixUserName(userName); > UserGroupInformation.setConfiguration(cfg); > UserGroupInformation ugi = UserGroupInformation.createProxyUser(userName, > UserGroupInformation.getCurrentUser()); > try { > return ugi.doAs(new PrivilegedExceptionAction<FileSystem>() { > @Override > public FileSystem run() throws Exception { > return FileSystem.get(uri, cfg); > } > }); > } catch (InterruptedException e) { > Thread.currentThread().interrupt(); > throw new IOException("Failed to create file system due to > interrupt.", e); > } > } > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)