[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[
https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105849#comment-13105849
]
ramkrishna.s.vasudevan commented on HBASE-4238:
---
@Stack
TestCatalogJanitor.testCleanParent is failing in 0.90.x. In trunk
{code}
assertTrue(fs.delete(p, true));
// We will fail!!! Because split b is empty, which is right... we should
// not remove parent if daughters do not exist in fs.
assertFalse(janitor.cleanParent(parent, r));
{code}
this part of code has been removed whereas in 0.90 it is not.
CatalogJanitor can clear a daughter that split before processing its parent
---
Key: HBASE-4238
URL: https://issues.apache.org/jira/browse/HBASE-4238
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
Assignee: stack
Priority: Critical
Fix For: 0.92.0, 0.90.5
Attachments: 4238-v2.txt, 4238.txt
I didn't dig a lot into this issue, but by splitting a table twice in a row I
was able to trigger a situation where a daughter of the first split was
deleted by the CatalogJanitor before it processed its parent. Will post log
in a comment.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[
https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105852#comment-13105852
]
ramkrishna.s.vasudevan commented on HBASE-4238:
---
{code}
-// We will fail!!! Because split b is empty, which is right... we should
-// not remove parent if daughters do not exist in fs.
-assertFalse(janitor.cleanParent(parent, r));
-// Put in place daughter dir for b... that should make it so parent gets
-// cleaned up.
-storedir = Store.getStoreHomedir(tabledir, splitb.getEncodedName(),
- htd.getColumnFamilies()[0].getName());
{code}
This part of code is removed. Not the one in the previous comment. :)
CatalogJanitor can clear a daughter that split before processing its parent
---
Key: HBASE-4238
URL: https://issues.apache.org/jira/browse/HBASE-4238
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
Assignee: stack
Priority: Critical
Fix For: 0.92.0, 0.90.5
Attachments: 4238-v2.txt, 4238.txt
I didn't dig a lot into this issue, but by splitting a table twice in a row I
was able to trigger a situation where a daughter of the first split was
deleted by the CatalogJanitor before it processed its parent. Will post log
in a comment.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[ https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105855#comment-13105855 ] stack commented on HBASE-4238: -- Let me fix. Thanks for finding this Ram. CatalogJanitor can clear a daughter that split before processing its parent --- Key: HBASE-4238 URL: https://issues.apache.org/jira/browse/HBASE-4238 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Assignee: stack Priority: Critical Fix For: 0.92.0, 0.90.5 Attachments: 4238-v2.txt, 4238.txt I didn't dig a lot into this issue, but by splitting a table twice in a row I was able to trigger a situation where a daughter of the first split was deleted by the CatalogJanitor before it processed its parent. Will post log in a comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[ https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13103404#comment-13103404 ] jirapos...@reviews.apache.org commented on HBASE-4238: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1819/ --- Review request for hbase. Summary --- Previous, we'd not clean up a parent if its daughter region didn't exist in the fs. This stipulation was added by HBASE-3872. This patch undoes this barrier to parent cleanup (See HBASE-3872 for why its ok to do this). CatalogJanitor + Break out the Comparator used by CatalogJanitor. It was an anonymous class. Instead we make it a static inner class so can add test that its actually sorting properly. + Added method hasNoReferences that will return true if no daughter dir OR no refs in daughter dir Added some TODOs around SplitTransaction -- vaguely related to this patch. Added new Test that checks cleanParent to ensure it works properly. Refactored bits of previous tests so they use common code. This addresses bug hbase-4238. https://issues.apache.org/jira/browse/hbase-4238 Diffs - src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java b53e9a0 src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 742aea4 src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java abafe5e src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 78e7d62 Diff: https://reviews.apache.org/r/1819/diff Testing --- Thanks, Michael CatalogJanitor can clear a daughter that split before processing its parent --- Key: HBASE-4238 URL: https://issues.apache.org/jira/browse/HBASE-4238 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Assignee: stack Priority: Critical Fix For: 0.92.0, 0.90.5 Attachments: 4238-v2.txt, 4238.txt I didn't dig a lot into this issue, but by splitting a table twice in a row I was able to trigger a situation where a daughter of the first split was deleted by the CatalogJanitor before it processed its parent. Will post log in a comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[ https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104020#comment-13104020 ] Hudson commented on HBASE-4238: --- Integrated in HBase-TRUNK #2205 (See [https://builds.apache.org/job/HBase-TRUNK/2205/]) HBASE-4238 CatalogJanitor can clear a daughter that split before processing its parent stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java CatalogJanitor can clear a daughter that split before processing its parent --- Key: HBASE-4238 URL: https://issues.apache.org/jira/browse/HBASE-4238 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Assignee: stack Priority: Critical Fix For: 0.92.0, 0.90.5 Attachments: 4238-v2.txt, 4238.txt I didn't dig a lot into this issue, but by splitting a table twice in a row I was able to trigger a situation where a daughter of the first split was deleted by the CatalogJanitor before it processed its parent. Will post log in a comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[
https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13091419#comment-13091419
]
Jean-Daniel Cryans commented on HBASE-4238:
---
Another way this problem appears is if the 2 daughters are removed from parent
and one of them was split and already cleaned before the CJ went to delete the
parent, this will fail (CJ.cleanParent):
{code}
if ((a.getFirst() !a.getSecond()) (b.getFirst() !b.getSecond())) {
{code}
Here we check if the directory of each daughter exists and if they don't have a
reference. In my case, they don't exist since they were already cleaned, which
means that the parent will _never_ get cleaned.
CatalogJanitor can clear a daughter that split before processing its parent
---
Key: HBASE-4238
URL: https://issues.apache.org/jira/browse/HBASE-4238
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
Priority: Critical
Fix For: 0.90.5
I didn't dig a lot into this issue, but by splitting a table twice in a row I
was able to trigger a situation where a daughter of the first split was
deleted by the CatalogJanitor before it processed its parent. Will post log
in a comment.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[
https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13089099#comment-13089099
]
Jean-Daniel Cryans commented on HBASE-4238:
---
Log of the issue (daughter region being d06b3b8c85af53ccbe139df7f0877e51):
{quote}
2011-08-22 15:06:50,227 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
Deleted daughter reference
TestTable,,1314050587746.7ada9de242de30a4e91f00a8dfcbf693., qualifier=splitA,
from parent TestTable,,1314050584480.d06b3b8c85af53ccbe139df7f0877e51.
2011-08-22 15:06:50,229 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
Deleted daughter reference
TestTable,013824,1314050587746.66c3c9f067203cfb7c0a541e9207766d.,
qualifier=splitB, from parent
TestTable,,1314050584480.d06b3b8c85af53ccbe139df7f0877e51.
2011-08-22 15:06:50,230 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor:
Deleting region TestTable,,1314050584480.d06b3b8c85af53ccbe139df7f0877e51.
because daughter splits no longer hold references
2011-08-22 15:06:50,230 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
DELETING region
file:/tmp/hbase-jdcryans/hbase/TestTable/d06b3b8c85af53ccbe139df7f0877e51
2011-08-22 15:06:50,234 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
Deleted region TestTable,,1314050584480.d06b3b8c85af53ccbe139df7f0877e51. from
META
2011-08-22 15:06:50,235 WARN org.apache.hadoop.hbase.master.CatalogJanitor:
Daughter regiondir does not exist:
file:/tmp/hbase-jdcryans/hbase/TestTable/d06b3b8c85af53ccbe139df7f0877e51
2011-08-22 15:11:50,220 WARN org.apache.hadoop.hbase.master.CatalogJanitor:
Daughter regiondir does not exist:
file:/tmp/hbase-jdcryans/hbase/TestTable/d06b3b8c85af53ccbe139df7f0877e51
2011-08-22 15:16:50,224 WARN org.apache.hadoop.hbase.master.CatalogJanitor:
Daughter regiondir does not exist:
file:/tmp/hbase-jdcryans/hbase/TestTable/d06b3b8c85af53ccbe139df7f0877e51
...
{quote}
Those last lines will happen forever, and .META. will be littered with that row.
CatalogJanitor can clear a daughter that split before processing its parent
---
Key: HBASE-4238
URL: https://issues.apache.org/jira/browse/HBASE-4238
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
Priority: Critical
Fix For: 0.90.5
I didn't dig a lot into this issue, but by splitting a table twice in a row I
was able to trigger a situation where a daughter of the first split was
deleted by the CatalogJanitor before it processed its parent. Will post log
in a comment.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
