Re: Can we modify files in HDFS?

2010-06-29 Thread Steve Loughran
elton sky wrote: thanx Jeff, So...it is a significant drawback. As a matter of fact, there are many cases we need to modify. When people say Hadoop filesystems are not posix, this is what they mean. No locks, no read/write. seeking discouraged. Even append is something that is just

Can we modify files in HDFS?

2010-06-28 Thread elton sky
hello everyone, After some research I found HDFS only support create new file and append to exiting file. What if I want to modify some parts of a, say 2 Petabyte, file. Do I have to remove it and create it again or we have some alternative way?

Re: Can we modify files in HDFS?

2010-06-28 Thread elton sky
thanx Jeff, So...it is a significant drawback. As a matter of fact, there are many cases we need to modify. I dont understand why Yahoo didn't provoid that functionality. And as I know no one else is working on this. Why is that?

Re: Can we modify files in HDFS?

2010-06-28 Thread Todd Lipcon
Hi Elton, Typically, large data sets are of the sort that continuously grow, and are not edited or amended. For example, a common Hadoop use case is the analysis of log data or other instrumentation from web or application servers. In these cases, files are simply added, but there is no need to