That's the not so working HDFS append feature showing it's ugly face, small amounts of data can be lost (configurable max of ~62MB).
J-D On Fri, Dec 11, 2009 at 10:55 AM, Seth Ladd <sethl...@gmail.com> wrote: >>> Which confuses me, if the write goes straight to a RegionServer, but >>> then the RegionServer fails before the MemStore is flushed, did I just >>> lose data? >> >> No that's the goal of the write-ahead-log (WAL). > > Here's the scenario I just tested on my EC2 cluster. 3 Zookeeper > instances, 1 master, and 3 slaves. > > I created a table, and inserted a single row. > I performed a read (get) to test the insert, and sure enough the row > was returned. > I then noted which slave held the table, and terminated the slave via > the AWS management console. > I then waited approx 30 seconds. > I used the web interfaces (port 60030 and 60010) to note that the > region was indeed moved to another slave. > I performed a read on the same row, but did *not* find the row. > > So it looks like the region for the table was moved, but no data was moved > over. > > Was that a valid test? I would expect the row to get moved with the region. > > Thanks, > Seth >
