Is DFSInputStream.read(long position,...) designed for multi threaded access?

2011-03-01 Thread Jason Rutherglen
It's unsynchronized however it's creating a new BlockReader on each call, that seems like a problem?

Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?

2011-03-02 Thread Jason Rutherglen
at 5:58 PM, Todd Lipcon wrote: > Hi Jason, > > Yes, this method is currently very inefficient.. HDFS-941 will > hopefully improve this situation, but currently there's no > particularly efficient way to do multithreaded random access. > > -Todd > > On Tue, Mar

Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?

2011-03-02 Thread Jason Rutherglen
gt; something that I won't be supporting myself in production against an > increasingly diverging upstream. > >    - Andy > > > --- On Wed, 3/2/11, Todd Lipcon wrote: > >> From: Todd Lipcon >> Subject: Re: Is DFSInputStream.read(long position,...) designed

Re: HDFS + ZooKeeper

2011-04-23 Thread Jason Rutherglen
This is an interesting question. AvatarNode doesn't use Zookeeper? Is there a drawback to implementing ZK for the NameNode other than the potential bugs from a newly engineered feature? Perhaps this issue is the main one for implementing the NameNode in ZK? HDFS-1623 On Fri, Apr 22, 2011 at 10:

Re: HDFS + ZooKeeper

2011-04-24 Thread Jason Rutherglen
ally it was not using ZooKeeper. That is way I raised that > question. Also, AvatorNode is build on HDFS 0.20 but my intension is to > utilize BackupNode that comes with HDFS 0.21 > > Thanks, > Ozcan. > > -Original Message- From: Jason Rutherglen > Sent: Sat

Local block placement policy, request

2011-05-26 Thread Jason Rutherglen
Is there a way to send a request to the name node to replicate block(s) to a specific DataNode? If not, what would be a way to do this? -Thanks

Re: Local block placement policy, request

2011-05-26 Thread Jason Rutherglen
e nice to expose this functionality as a "limited > private evolving" API. > > In general, though, keep in mind that, whenever you write data, you'll > get a local copy first, if the writer is in the cluster. That's how > HBase gets locality for most of its accesse

Re: Local block placement policy, request

2011-05-26 Thread Jason Rutherglen
y 26, 2011 at 12:30 PM, Todd Lipcon wrote: > On Thu, May 26, 2011 at 12:02 PM, Jason Rutherglen > wrote: >> Todd, thanks! >> >>> In general, though, keep in mind that, whenever you write data, you'll >>> get a local copy first, if the writer is in the clus