I do have Cygwin on the path, but in this case, it is not using chmod to set
permissions, but File.setReadable. Below is what
RawLocalFileSystem.setPermissions looks like. It only uses chmod (hence Cygwin)
if group != other (lines 490-494), but this is not the case for 700
permissions, so it does it using File.setReadable and fails (line 499). I guess
I could edit this class, but I'm just wondering if there is a proper way of
dealing with this without editing the Hadoop library.
481 @Override
482 public void setPermission(Path p, FsPermission permission
483 ) throws IOException {
484 FsAction user = permission.getUserAction();
485 FsAction group = permission.getGroupAction();
486 FsAction other = permission.getOtherAction();
487
488 File f = pathToFile(p);
489
490 // Fork chmod if group and other permissions are different...
491 if (group != other) {
492 execSetPermission(f, permission);
493 return;
494 }
495
496 boolean rv = true;
497
498 // read perms
499 rv = f.setReadable(group.implies(FsAction.READ), false);
Thanks,
Iwona
-----Original Message-----
From: Itzhak Pan [mailto:itzhak....@gmail.com]
Sent: Sunday, May 08, 2011 6:12 PM
To: common-user@hadoop.apache.org
Subject: Re: RE: Re:Problem debugging MapReduce job under Windows
I encountered the same problem. It is because the app fails to call 'chmod'
in windows. My solution is to include cygwin directory in PATH.
在 2011 5 8 11:29，"Iwona Bialynicka-Birula" <iwona...@atigeo.com>写道：
> Thanks for the reply, but the command you suggest refers to the Hadoop
file system and I am having trouble with the local file system - in particular
a subset of hadoop.tmp.dir. Also, even if I grant full permissions to this
folder initially, Hadoop still tries to set them to 700, which is not allowed
under Windows.
>
> Thanks,
> Iwona
>
> -----Original Message-----
> From: hadoopfan [mailto:shhaij...@126.com]
> Sent: Saturday, May 07, 2011 5:16 PM
> To: common-user@hadoop.apache.org
> Subject: Re:Problem debugging MapReduce job under Windows
>
> hadoop dfs -chmod 777 /tmp
>
>
>
> At 2011-05-08 04:56:22，"Iwona Bialynicka-Birula" <iwona...@atigeo.com>
wrote:
>
>>Hello,
>>
>>I am trying to debug a Hadoop MapReduce job under Eclipse in Windows
>>and I
am running into a problem when the Hadoop framework tries to set up the staging
directory (see the stack trace below) and change its permissions to 700. It is
doing it in RawLocalFileSystem.setPermissions by calling
java.io.File.setReadable(false, false), but under Windows this always fails,
since you're not allowed to set readable to false. Is there a workaround for
this to enable debugging in a Windows environment, or is such a scenario simply
not possible?
>>
>>(I am using Windows 7, Hadoop 0.20.2, Java 6. I have Cygwin installed,
>>but
in this case it is not using it, since it is making the system call
directly.)
>>
>>Thanks,
>>Iwona
>>
>>java.io.IOException: Failed to set permissions of path:
file:/tmp/hadoop-iwonabb/mapred/staging/iwonabb-1931875024/.staging to 0700
>> at
org.apache.hadoop.fs.RawLocalFileSystem.checkReturnValue(RawLocalFileSystem.java:526)
>> at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:500)
>> at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:310)
>> at
org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
>> at
org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:799)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:793)
>> at java.security.AccessController.doPrivileged(Native Method) at
>> javax.security.auth.Subject.doAs(Unknown Source) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
>> at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:793)
>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
>> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:495)
>> at
com.xpatterns.xrelevance.hadoop.training.crn.CRNTraining.run(CRNTraining.java:168)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at
com.xpatterns.xrelevance.hadoop.training.crn.CRNTraining.main(CRNTraining.java:46)
>> at
com.xpatterns.xrelevance.tests.CRNTrainingMRvsSingle.compare(CRNTrainingMRvsSingle.java:88)
>> at
com.xpatterns.xrelevance.tests.CRNTrainingMRvsSingle.compare(CRNTrainingMRvsSingle.java:136)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
>> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at
>> java.lang.reflect.Method.invoke(Unknown Source) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>> at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>> at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>> at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>> at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
>> at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
>> at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
>> at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
>> at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
>> at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
>> at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
>>
