Re: Question on opening file info from namenode in DFSClient

2008-11-09 Thread stack
Taeho Kang wrote: Hi, thanks for your reply Dhruba, One of my co-workers is writing a BigTable-like application that could be used for online, near-real-time, services. Can your co-worker be convinced to instead spend his time helping-along the ongoing bigtable-like efforts? I think HBase

Re: Question on opening file info from namenode in DFSClient

2008-11-08 Thread Dhruba Borthakur
Hi Taeho, Thanks for ur explanation. If your application opens a dfs file and does not close it, then the dfsclient will automatcally keep block locations cached. So, you could achieve your desired goal by developing a cache layer (above HDFS) that does not close the hdfs file even if the user

Re: Question on opening file info from namenode in DFSClient

2008-11-07 Thread Taeho Kang
Hi, thanks for your reply Dhruba, One of my co-workers is writing a BigTable-like application that could be used for online, near-real-time, services. So since the application could be hooked into online services, there would times when a large number of users (e.g. 1000 users) request to access

Question on opening file info from namenode in DFSClient

2008-11-02 Thread Taeho Kang
Dear Hadoop Users and Developers, I was wondering if there's a plan to add file info cache in DFSClient? It could eliminate network travelling cost for contacting Namenode and I think it would greatly improve the DFSClient's performance. The code I was looking at was this