In the first example you want it to chop off the /sub/ directory. Why? What
is the indicator as to how many lower directories you want chopped off? In the
second example, how can it turn a relative path into an absolute path
considering that there may be multiple top directories scattered
I want to normalize the paths. I don't actually care about the specific
string representations, just that two paths that point to the same location
in HDFS could be recognized as equivalent. Usually the way you do this in
file systems is get an absolute path, so that's why I'm looking for one
Hi,
Looking up the FileStatus has worked for me before:
fs.getFileStatus(new Path(.)).getPath()
On Tue, Mar 13, 2012 at 2:50 AM, W.P. McNeill bill...@gmail.com wrote:
Is there an API that returns absolute HDFS paths? So something that would
turn /user/myname/top/sub/.. into /user/myname/top
Hmmm, that's interesting. Unless I'm misunderstanding you, the associated
operation in conventional file systems (such as the pwd) command only work
because something already knows the absolute path and can produce it on
demand. If all you have is a relative path (such as top/sub in your
That works. Note that this returns the full HDFS URI. If you want to remove
protocol and server information you must do something like:
Path p = fs.getFileStatus(new Path(.)).getPath();
URI(p.toString()).getPath();
HDFS always translates non-absolute paths like top/sub into absolute
paths of the form /user/USERNAME/top/sub. So in HDFS top/sub is
unambiguous.
Why retranslate?
Even better is to instead do:
p.toUri().getPath()
On Tue, Mar 13, 2012 at 3:52 AM, W.P. McNeill bill...@gmail.com wrote:
That works. Note that this returns the full HDFS URI. If you want to remove
protocol and server information you must do something like:
Path p =
Yep, that's better. I just got turned around in all the format conversions.
So, you're say that hadoop assumes there is a specific directory hierarchy on
HDFS? Namely that there exists a top-level directory /user/ and that there
exist various username directories under /user/?
I only ask because I've never bothered to configure an HDFS system that
specific way? I
HDFS has the notion of a working directory which defaults to
/user/username. Check out:
http://hadoop.apache.org/common/docs/r1.0.1/api/org/apache/hadoop/fs/FileSystem.html#getWorkingDirectory()
and
10 matches
Mail list logo
