[
https://issues.apache.org/jira/browse/HBASE-533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12600682#action_12600682
]
stack commented on HBASE-533:
-----------------------------
Here's a sample from a run internal here...
{code}
Timestamp Event Description
Thu, 29 May 2008 05:07:48 compaction Region compaction
Thu, 29 May 2008 05:07:34 open Region opened on server :
aa0-000-13.u.powerset.com
Thu, 29 May 2008 05:07:31 creation Region creation
Thu, 29 May 2008 05:07:29 split Region split from :
TestTable,0000097920,1212037574001
Thu, 29 May 2008 05:07:29 creation Region creation
Thu, 29 May 2008 05:07:28 compaction
[TestTable,0000097920,1212037574001] Region compaction
Thu, 29 May 2008 05:06:48 compaction
[TestTable,0000097920,1212037574001] Region compaction
Thu, 29 May 2008 05:06:21 compaction
[TestTable,0000097920,1212037574001] Region compaction
Thu, 29 May 2008 05:06:18 open [TestTable,0000097920,1212037574001]
Region opened on server : aa0-000-13.u.powerset.com
Thu, 29 May 2008 05:06:15 creation
[TestTable,0000097920,1212037574001] Region creation
Thu, 29 May 2008 05:06:14 split [TestTable,0000097920,1212037574001]
Region split from : TestTable,,1212037455955
Thu, 29 May 2008 05:06:14 creation
[TestTable,0000097920,1212037574001] Region creation
Thu, 29 May 2008 05:06:13 compaction
[TestTable,0000097920,1212037574001] [TestTable,,1212037455955] Region
compaction
Thu, 29 May 2008 05:05:54 compaction
[TestTable,0000097920,1212037574001] [TestTable,,1212037455955] Region
compaction
Thu, 29 May 2008 05:05:18 compaction
[TestTable,0000097920,1212037574001] [TestTable,,1212037455955] Region
compaction
Thu, 29 May 2008 05:04:55 compaction
[TestTable,0000097920,1212037574001] [TestTable,,1212037455955] Region
compaction
Thu, 29 May 2008 05:04:21 open [TestTable,0000097920,1212037574001]
[TestTable,,1212037455955] Region opened on server : aa0-000-13.u.powerset.com
Thu, 29 May 2008 05:04:18 compaction
[TestTable,0000097920,1212037574001] [TestTable,,1212037455955] Region
compaction
Thu, 29 May 2008 05:04:18 creation
[TestTable,0000097920,1212037574001] [TestTable,,1212037455955] Region creation
Thu, 29 May 2008 05:04:16 creation
[TestTable,0000097920,1212037574001] [TestTable,,1212037455955] Region creation
{code}
It looks great.
Yeah, I'd say drop the compactions.
I also wonder about listing parent events in the daughters. A daughter from a
good family who can trace her roots all the ways back to the pilgrims will have
a long line in the table above. I think listing just the immediate parent is
sufficient, at least for the first version.
What about listing the event source? Most edits will be by the master but the
splits will be done by the splitting regionserver.
Regards double creations, I'd guess its not two creations -- just two
invocations of the HRegion constructor. Opening and creating both call new
HRegion. You might move our call to addRegionCreation out of the HRegion
constructor and instead put it into HRegion.createHRegion (if this is
appropriately called in all places a region is initially created).
Dynamic would be nice -- like log level? Tying it to loglevel would make your
flag easy to implement: e.g. send compaction/flush notice when at DEBUG. Bad
thing about tying it to loglevel doesn't have sufficient granularity. There's
effectively INFO and DEBUG. What you think?
Good on you J-D
> Region Historian
> ----------------
>
> Key: HBASE-533
> URL: https://issues.apache.org/jira/browse/HBASE-533
> Project: Hadoop HBase
> Issue Type: Wish
> Reporter: Bryan Duxbury
> Assignee: Jean-Daniel Cryans
> Priority: Minor
> Attachments: hbase-533-v1-fix.patch, hbase-533-v1.patch
>
>
> Whenever we try to debug region splitting, assignment, compaction, etc.
> issues, we always end up having to look in 1-20 different log files for
> cryptic names of regions and try to piece together the chain of events
> ourselves. This is a challenging at best effort most of the time.
> What would be very useful would be a new utility I've nicknamed the Region
> Historian. You give it the text name of a region, and it will track down the
> log messages relevant to it in the master and regionserver logs. Then, it
> will interleave the messages in such a way that the timestamps correctly list
> the order of events. The result is a log summary that accurately describes
> what happened to a region during it's lifetime, making it much easier to try
> and figure out where something went wrong.
> Other things it could do would be replace cryptic log messages with simple
> events like "the region was split into a and b", "the region was assigned to
> server x", and trace the lineage of a region backwards to its parent before
> it came into existence.
> I'm sure there are other things we would think up that would be useful as
> well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
