This is something you ought to raise with the IBM JVM team, as it does appear to be a bug in their JVM.
On 21 August 2012 10:11, James F Walton <[email protected]> wrote: > Found part of the reason while digging through UserGroupInformation.java > > /* Return the OS login module class name */ > private static String getOSLoginModuleName() { > if (System.getProperty("java.vendor").contains("IBM")) { > return windows ? "com.ibm.security.auth.module.NTLoginModule" > : "com.ibm.security.auth.module.LinuxLoginModule"; > } else { > return windows ? "com.sun.security.auth.module.NTLoginModule" > : "com.sun.security.auth.module.UnixLoginModule"; > } > } > > > So basically, if you use IBM java, then you must be on either Windows or > Linux. IBM's java appears to have platform specific LoginModules, there's > AIXLoginModule for 32-bit java on AIX, and AIX64LoginModule for 64-bit java > on AIX; however, the IBM Linux module appears not to have any 32-bit vs > 64-bit differentiation. > > So, unless anyone has a means to disable this whole security setup (I'm > not using a hadoop cluster or anything), or wants to dive headlong into > making the necessary code changes (which I presume from my cursory scanning > would include a little more than just the above snippet, like the > getOSPrincipalClass as well), I guess I'll need to look at moving our > crawler from AIX to Linux. > > My java coding skills are not top notch, though I could probably fix it if > the necessary updates didn't get too convoluted beyond what I think it > might require. I would hope there are not too many other instances where > AIX incompatibility would rear it's head since prior to the security > features overhaul it all worked fine. > > Thoughts? > > James > > > > > From: James F Walton/Southbury/IBM@IBMUS > To: [email protected] > Date: 08/17/2012 02:47 PM > Subject: Hadoop 1.0.3 (nutch-1.5.1) throwing errors on AIX 6.1 > ------------------------------ > > > > When implementing Nutch 1.0 awhile back, we had to point our scripts to > /opt/freeware/bin to allow Nutch's Hadoop code to utilize the more > Linux-like versions of various system commands (like df) in order for > Hadoop to function properly on AIX. After doing that, we've had no issues > with Nutch's hadoop implementation from 1.0 through 1.4. > > Now I'm attempting up migrate my existing, functional configurations from > my Nutch 1.4 installation to Nutch 1.5.1, which is now using Hadoop 1.0.3. > Now when I attempt to run a crawl, I'm getting these errors that seem to > be coming from Hadoop, and seem to want to use a LinuxLoginModule class > (see below) > > Is there some configuration setting, plugin or jar file, etc that is > missing from the new version to make this all work on AIX again? > > 2012-08-02 20:03:21,271 ERROR crawl.Injector - Injector: > java.lang.RuntimeException: java.io.IOException: failure to login > at > org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:546) > at > org.apache.hadoop.mapred.FileInputFormat.addInputPath(FileInputFormat.java:336) > at org.apache.nutch.crawl.Injector.inject(Injector.java:209) > at org.apache.nutch.crawl.Injector.run(Injector.java:248) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.nutch.crawl.Injector.main(Injector.java:238) > Caused by: java.io.IOException: failure to login > at > org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:490) > at > org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:452) > at > org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1494) > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1395) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123) > at > org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:542) > ... 5 more > Caused by: javax.security.auth.login.LoginException: unable to find > LoginModule class: com.ibm.security.auth.module.LinuxLoginModule > at > javax.security.auth.login.LoginContext.invoke(LoginContext.java:834) > at > javax.security.auth.login.LoginContext.access$000(LoginContext.java:209) > at > javax.security.auth.login.LoginContext$5.run(LoginContext.java:732) > at > java.security.AccessController.doPrivileged(AccessController.java:284) > at > javax.security.auth.login.LoginContext.invokeCreatorPriv(LoginContext.java:729) > at > javax.security.auth.login.LoginContext.login(LoginContext.java:599) > at > org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:471) > ... 11 more > > > > James >
