Hello,
I am trying to debug a Hadoop MapReduce job under Eclipse in Windows and I am
running into a problem when the Hadoop framework tries to set up the staging
directory (see the stack trace below) and change its permissions to 700. It is
doing it in RawLocalFileSystem.setPermissions by calling
java.io.File.setReadable(false, false), but under Windows this always fails,
since you're not allowed to set readable to false. Is there a workaround for
this to enable debugging in a Windows environment, or is such a scenario simply
not possible?
(I am using Windows 7, Hadoop 0.20.2, Java 6. I have Cygwin installed, but in
this case it is not using it, since it is making the system call directly.)
Thanks,
Iwona
java.io.IOException: Failed to set permissions of path:
file:/tmp/hadoop-iwonabb/mapred/staging/iwonabb-1931875024/.staging to 0700
at
org.apache.hadoop.fs.RawLocalFileSystem.checkReturnValue(RawLocalFileSystem.java:526)
at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:500)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:310)
at
org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at
org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:799)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:793)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:793)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
at
org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:495)
at
com.xpatterns.xrelevance.hadoop.training.crn.CRNTraining.run(CRNTraining.java:168)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at
com.xpatterns.xrelevance.hadoop.training.crn.CRNTraining.main(CRNTraining.java:46)
at
com.xpatterns.xrelevance.tests.CRNTrainingMRvsSingle.compare(CRNTrainingMRvsSingle.java:88)
at
com.xpatterns.xrelevance.tests.CRNTrainingMRvsSingle.compare(CRNTrainingMRvsSingle.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at
org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
