So what you ultimately need is a piece of java code listing the rwx permissions for user, group and others that is not using ls internally, is that correct? If "RawLocalFileSystem" is not HDFS, do you really need to use hadoop-util for that? Can you tell us more about your use case?
Regards, LLoyd On 11 December 2015 at 13:05, Samuel <samuelri...@gmail.com> wrote: >> Using ls to figure out permissions is a bad design anyway, so I would >> not be surprised if this hardcode was reported as a bug. > > Of course, I have no idea why it was implemented like this. I assume > it was written at some point in time where Java didn't provide the > needed APIS (?) > > Implementing the permission check without relying in ls at all is also > a solution for the problem I have :) > >> LLoyd >> >> On 11 December 2015 at 09:19, Samuel <samuelri...@gmail.com> wrote: >> > Hi, >> > >> > I am experiencing some crashes when using spark over local files (mainly >> > for >> > testing). Some operations fail with >> > >> > java.lang.RuntimeException: Error while running command to get file >> > permissions : java.io.IOException: Cannot run program "/bin/ls": error=2, >> > No >> > such file or directory >> > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) >> > at org.apache.hadoop.util.Shell.runCommand(Shell.java:206) >> > at org.apache.hadoop.util.Shell.run(Shell.java:188) >> > at >> > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) >> > at org.apache.hadoop.util.Shell.execCommand(Shell.java:467) >> > at org.apache.hadoop.util.Shell.execCommand(Shell.java:450) >> > at >> > org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:593) >> > at >> > org.apache.hadoop.fs.RawLocalFileSystem.access$100(RawLocalFileSystem.java:51) >> > at >> > org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:514) >> > at >> > org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus.getPermission(RawLocalFileSystem.java:489) >> > at >> > org.apache.spark.sql.parquet.ParquetRelation2$$anonfun$buildScan$1$$anon$1$$anonfun$12.apply(newParquet.scala:292) >> > >> > etcetera... >> > >> > Which seems to be related to Shell.java in org.apache.hadoop-util, that >> > uses >> > ls -ld to figure out file permissions (that is in >> > RawLocalFileSystem.loadPermissionsInfo). The problem is that instead of >> > just >> > calling ls, Shell .java calls /bin/ls, which is usually available, but in >> > certain circumstances might not. Regardless of the reasons not to have ls >> > in >> > /bin, hardcoding the directory bans users from using the standard >> > mechanisms >> > to decide which binaries to run in their systems (in this case, $PATH), so >> > I >> > wonder if there is a particular reason why that path has been hardcoded to >> > an absolute path instead to something resolvable using$PATH. >> > >> > Or in other words, is this a bug or a feature? >> > >> > Best >> > >> > -- >> > Samuel > > > > > -- > Samuel --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org For additional commands, e-mail: user-h...@hadoop.apache.org
