Andy Liu wrote:
I'm running Hadoop 0.19.0 on Solaris (SunOS 5.10 on x86) and many jobs are
failing with this exception:
This isnt disk space, this is a RAM/swap problem related to fork()
. Search for that error string on hadoop JIRA; I think it's been seen
before on linux, though there the namenode was at fault
Error initializing attempt_200901281655_0004_m_000025_0:
java.io.IOException: Cannot run program "chmod": error=12, Not enough space
at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
at org.apache.hadoop.util.Shell.run(Shell.java:134)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:338)
at
org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:540)
at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:532)
at
org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:274)
at
org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
at
org.apache.hadoop.mapred.MapTask.localizeConfiguration(MapTask.java:107)
at
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:1803)
at
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:1884)
at
org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:784)
at
org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:778)
at
org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1636)
at
org.apache.hadoop.mapred.TaskTracker.access$1200(TaskTracker.java:102)
at
org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:1602)
Caused by: java.io.IOException: error=12, Not enough space
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.(UNIXProcess.java:53)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
... 20 more
However, all the disks have plenty of disk space left (over 800 gigs). Can
somebody point me in the right direction?
Thanks,
Andy
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
