-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/179/
-----------------------------------------------------------
(Updated 2010-06-15 14:20:05.992588)
Review request for hbase.
Changes
-------
Main changes are:
+ Implement all of Todds suggestions including adding a test to '...which makes
a region, writes some data, closes the region, opens the region, writes some
data, then recovers logs?' (see 'testReplayEditsWrittenViaHRegion') and adding
a test that shows that the following will NOT happen: 'I think this will crash
when the region contains stores that were the result of an HFOF bulk load (they
have no sequence id)' (see 'testRegionMadeOfBulkLoadedFilesOnly').
+ Also some more cleanup around replayRecoveredEdits. Should be easier to
follow now.
After this goes in, I'll start in on the work to address the scenario Todd
outlines below.
Summary
-------
+ Moved replay of edits up from Store up into Region. This means we play the
edits once only rather than once per Store.
+ Lots of cleanup in the replay of edits code. Uses the new flag introduced by
cosmin – hbase.skip.errors. If set, and exception processing edits, we'll fail.
If false, we'll move the broke edits file aside and keep going. Renamed the
method from doReconstructionLog to replayRecoveredEdits (reconstruction?).
+ The main change in this patch is that recovering edits, we now go in via the
HRegion main API doing put and delete so that only one code path, so replayed
edits are added to the WAL, and so a flush will be triggered if we fill memory.
+ In HStore, lots of removed code and comments since no longer does log replay.
Cleanup of maximum seqid. Calculate it instead rather than save as a data
member. Its only used once on HRegion startup.
+ Change the HRegion#initialize. It used to take 'initial files' which is a
notion never used (it was a means of putting files in place after a split but
split is done internal to HRegion so can do things in HRegions guts w/o need of
exposing notion of initial files). I removed it and added overload that takes
no args which is the usual way this method is invoked.
+ Rename the product of splits, 'recovered.edits' instead of 'oldlogfile.log'
+ Added small facility to HBaseTestingUtility for creating different user in a
Configuration so can have more than one Filesystem instance the easier.
+ Redid the test TestStoreReconstruction as TestWALReplay.
This addresses bug hbase-1025.
http://issues.apache.org/jira/browse/hbase-1025
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/HConstants.java f5d3e94
src/main/java/org/apache/hadoop/hbase/HMerge.java 62f3561
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 06e022c
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java fe9aa8a
src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 2a0dcee
src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 05826f5
src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 05cf17f
src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java dbb21d4
src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 479c661
src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java 43a8a28
src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreReconstruction.java
4f0417d
src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java
86cf4ea
src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
PRE-CREATION
Diff: http://review.hbase.org/r/179/diff
Testing
-------
Thanks,
stack
