On Tue, May 31, 2011 at 2:04 AM, jiangwen w <[email protected]> wrote: > The following code is from execute method in SplitTransaction class > If the region server shut down when the following code is executing. > Some files may be left in HDFS forever? >
They could. Check the CatalogJanitor and or the server shutdown handler. Here we will finish-off interrupted splits IFF at least the parent edit made it up into .META. There may be a hole here though where we should recognize these abandoned daughter directories and do some cleanup. IIRC, it has been suggested in the past that pre-split we write to the WAL log what we are about to do. If we crash in the middle of the split, then on replay of edits when we trip over the special split record, if we did not complete, we could clean up the abandoned daughter regions written into HDFS? But as Anty said, these abandoned daughters are more a nuisance than harmful (We should clean them up though -- at a minimum hbck will flag them). St.Ack
