I GOT IT TO WORK!!! (on windopws) =) =) =) Thank you everyone :)
Sas -----Original Message----- From: Markus Jelsma [mailto:markus.jel...@openindex.io] Sent: Wednesday, July 13, 2016 10:44 AM To: user@nutch.apache.org Subject: RE: Running into an Issue Jamal, i really have no idea but it smells like a Windows related problem again. See: http://stackoverflow.com/questions/27201505/hadoop-exception-in-thread-main-java-lang-nullpointerexception http://stackoverflow.com/questions/30379441/mapreduce-development-inside-eclipse-on-windows Markus -----Original message----- > From:Jamal, Sarfaraz <sarfaraz.ja...@verizonwireless.com.INVALID> > Sent: Wednesday 13th July 2016 16:09 > To: user@nutch.apache.org > Subject: RE: Running into an Issue > > Hi Markus, > > So I am now running this on a machine that I have administrative > access to And I am getting a different error message: > > Do you (or anyone) have any ideas? > > $ ./nutch inject ../TestCrawl/crawldb ../url [I tried from the bin > folder and the root nutch folder[ > > Injector: starting at 2016-07-13 10:02:54 > Injector: crawlDb: ../TestCrawl/crawldb > Injector: urlDir: ../url > Injector: Converting injected urls to crawl db entries. > Injector: java.lang.NullPointerException > at java.lang.ProcessBuilder.start(Unknown Source) > at org.apache.hadoop.util.Shell.runCommand(Shell.java:445) > at org.apache.hadoop.util.Shell.run(Shell.java:418) > at >org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java: > >650) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:739) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:722) > at >org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSys > >tem.java:633) > at >org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem. > >java:467) > at >org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.jav > >a:456) > at >org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.jav > >a:424) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:906) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:887) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:849) > at >org.apache.hadoop.fs.FileSystem.createNewFile(FileSystem.java:1149) > at >org.apache.nutch.util.LockUtil.createLockFile(LockUtil.java:58) > at org.apache.nutch.crawl.Injector.inject(Injector.java:357) > at org.apache.nutch.crawl.Injector.run(Injector.java:467) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.nutch.crawl.Injector.main(Injector.java:441) > > > > -----Original Message----- > From: Markus Jelsma [mailto:markus.jel...@openindex.io] > Sent: Tuesday, July 12, 2016 4:52 AM > To: user@nutch.apache.org > Subject: RE: Running into an Issue > > Hi , there are some Windows API calls in there that i will never understand. > Are there some kinds of symlinks you are working with or whatever they are > called in Windows? There must be something with Nutch/Hadoop getting access > to your disk. Check permissions, disk space and whatever you can think of. > > M. > > -----Original message----- > > From:Jamal, Sarfaraz <sarfaraz.ja...@verizonwireless.com.INVALID> > > Sent: Monday 11th July 2016 22:46 > > To: Nutch help <user@nutch.apache.org> > > Subject: Running into an Issue > > > > So I feel I have made some progress on Nutch > > > > However I am now getting another error which I am having difficulty > > navigating through: > > > > bin/nutch inject TestCrawl/crawldb url > > > > produces this below > > > > Do you have to run Cygwin under Admin for it to work? > > > > Injector: Converting injected urls to crawl db entries. > > Exception in thread "main" java.lang.UnsatisfiedLinkError: > >org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/Str > >ing;I)Z > > at > >org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method) > > at > >org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:5 > >70) > > at org.apache.hadoop.fs.FileUtil.canRead(FileUtil.java:977) > > at > >org.apache.hadoop.util.DiskChecker.checkAccessByFileMethods(DiskCheck > >er.java:173) > > at > >org.apache.hadoop.util.DiskChecker.checkDirAccess(DiskChecker.java:16 > >0) > > at > >org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:94) > > at > >org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChange > >d(LocalDirAllocator.java:285) > > at > >org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPa > >thForWrite(LocalDirAllocator.java:344) > > at > >org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirA > >llocator.java:150) > > at > >org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirA > >llocator.java:131) > > at > >org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirA > >llocator.java:115) > > at > >org.apache.hadoop.mapred.LocalDistributedCacheManager.setup(LocalDist > >ributedCacheManager.java:131) > > at > >org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.jav > >a:163) > > at > >org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java > >:731) > > at > >org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitt > >er.java:432) > > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285) > > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282) > > at java.security.AccessController.doPrivileged(Native > >Method) > > at javax.security.auth.Subject.doAs(Unknown Source) > > at > >org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma > >tion.java:1548) > > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282) > > at > >org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303) > > at org.apache.nutch.crawl.Injector.inject(Injector.java:376) > > at org.apache.nutch.crawl.Injector.run(Injector.java:467) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > at org.apache.nutch.crawl.Injector.main(Injector.java:441) > > >
