This feature is not available yet, and is still under active discussion. (The current version of HDFS will make the previous block available to readers.) Michael Stack gave a good summary on the HBase dev list:
http://mail-archives.apache.org/mod_mbox/hadoop-hbase-dev/200905.mbox/%3c7c962aed0905231601g533088ebj4a7a068505ba3...@mail.gmail.com%3e Tom On Tue, May 26, 2009 at 12:08 PM, Stas Oskin <stas.os...@gmail.com> wrote: > Hi. > > I'm trying to continuously write data to HDFS via OutputStream(), and want > to be able to read it at the same time from another client. > > Problem is, that after the file is created on HDFS with size of 0, it stays > that way, and only fills up when I close the OutputStream(). > > Here is a simple code sample illustrating this issue: > > try { > > FSDataOutputStream out=fileSystem.create(new > Path("/test/test.bin")); // Here the file created with 0 size > for(int i=0;i<1000;i++) > { > out.write(1); // Still stays 0 > out.flush(); // Even when I flush it out??? > } > > Thread.currentThread().sleep(10000); > out.close(); //Only here the file is updated > } catch (Exception e) { > e.printStackTrace(); > } > > So, two questions here: > > 1) How it's possible to write the files directly to HDFS, and have them > update there immedaitely? > 2) Just for information, in this case, where the file content stays all the > time - on server local disk, in memory, etc...? > > Thanks in advance. >
