...@gmail.com]
Subject: Re: Is FileSystem thread-safe?
FileSystem is an abstract class, what concrete class are you using
(DistributedFileSystem, etc) ?
Good point. I am calling FileSystem.get(URI uri, Configuration conf) with an
URI like hdfs://server:port/... on a remote server, so I assume
: Re: Is FileSystem thread-safe?
Hi John,
DistributedFileSystem is intended to be thread-safe, true to its name.
Metadata operations are handled by the NameNode server which synchronizes
concurrent client requests via locks (you can look at the FSNameSystem class).
Some discussion
...@hortonworks.com]
Sent: Friday, May 17, 2013 11:14 AM
To: user@hadoop.apache.org
Subject: Re: Is FileSystem thread-safe?
As of today, there is no atomic append, so no, what you say isn't possible.
FWIU, it is one appender at a time - achieved through a lease per file, and
multiple concurrent leases
, users opt to output a
folder full of file parts into HDFS and then read them directly or unify them
later.
John
From: Vinod Kumar Vavilapalli [mailto:vino...@hortonworks.com]
Sent: Friday, May 17, 2013 11:14 AM
To: user@hadoop.apache.org
Subject: Re: Is FileSystem thread-safe
...@redpoint.net mailto:john.lil...@redpoint.net wrote:
*From:*Ted Yu [mailto:yuzhih...@gmail.com
mailto:yuzhih...@gmail.com]
*Subject:* Re: Is FileSystem thread-safe?
FileSystem is an abstract class, what concrete class are you
using (DistributedFileSystem
through ClientProtocol.
An hdfs committer would be able to give you affirmative answer.
On Sun, Mar 31, 2013 at 11:27 AM, John Lilley john.lil...@redpoint.netwrote:
*From:* Ted Yu [mailto:yuzhih...@gmail.com]
*Subject:* Re: Is FileSystem thread-safe?
FileSystem is an abstract class
FileSystem is an abstract class, what concrete class are you using
(DistributedFileSystem, etc) ? For FileSystem, I find the following for
create() method:
* but the implementation is thread-safe. The other option is to change
the
* value of umask in configuration to be 0, but it is not
From: Ted Yu [mailto:yuzhih...@gmail.com]
Subject: Re: Is FileSystem thread-safe?
FileSystem is an abstract class, what concrete class are you using
(DistributedFileSystem, etc) ?
Good point. I am calling FileSystem.get(URI uri, Configuration conf) with an
URI like hdfs://server:port