[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[ https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105849#comment-13105849 ] ramkrishna.s.vasudevan commented on HBASE-4238: --- @Stack TestCatalogJanitor.testCleanParent is failing in 0.90.x. In trunk {code} assertTrue(fs.delete(p, true)); // We will fail!!! Because split b is empty, which is right... we should // not remove parent if daughters do not exist in fs. assertFalse(janitor.cleanParent(parent, r)); {code} this part of code has been removed whereas in 0.90 it is not. CatalogJanitor can clear a daughter that split before processing its parent --- Key: HBASE-4238 URL: https://issues.apache.org/jira/browse/HBASE-4238 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Assignee: stack Priority: Critical Fix For: 0.92.0, 0.90.5 Attachments: 4238-v2.txt, 4238.txt I didn't dig a lot into this issue, but by splitting a table twice in a row I was able to trigger a situation where a daughter of the first split was deleted by the CatalogJanitor before it processed its parent. Will post log in a comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[ https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105852#comment-13105852 ] ramkrishna.s.vasudevan commented on HBASE-4238: --- {code} -// We will fail!!! Because split b is empty, which is right... we should -// not remove parent if daughters do not exist in fs. -assertFalse(janitor.cleanParent(parent, r)); -// Put in place daughter dir for b... that should make it so parent gets -// cleaned up. -storedir = Store.getStoreHomedir(tabledir, splitb.getEncodedName(), - htd.getColumnFamilies()[0].getName()); {code} This part of code is removed. Not the one in the previous comment. :) CatalogJanitor can clear a daughter that split before processing its parent --- Key: HBASE-4238 URL: https://issues.apache.org/jira/browse/HBASE-4238 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Assignee: stack Priority: Critical Fix For: 0.92.0, 0.90.5 Attachments: 4238-v2.txt, 4238.txt I didn't dig a lot into this issue, but by splitting a table twice in a row I was able to trigger a situation where a daughter of the first split was deleted by the CatalogJanitor before it processed its parent. Will post log in a comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[ https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105855#comment-13105855 ] stack commented on HBASE-4238: -- Let me fix. Thanks for finding this Ram. CatalogJanitor can clear a daughter that split before processing its parent --- Key: HBASE-4238 URL: https://issues.apache.org/jira/browse/HBASE-4238 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Assignee: stack Priority: Critical Fix For: 0.92.0, 0.90.5 Attachments: 4238-v2.txt, 4238.txt I didn't dig a lot into this issue, but by splitting a table twice in a row I was able to trigger a situation where a daughter of the first split was deleted by the CatalogJanitor before it processed its parent. Will post log in a comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[ https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13103404#comment-13103404 ] jirapos...@reviews.apache.org commented on HBASE-4238: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1819/ --- Review request for hbase. Summary --- Previous, we'd not clean up a parent if its daughter region didn't exist in the fs. This stipulation was added by HBASE-3872. This patch undoes this barrier to parent cleanup (See HBASE-3872 for why its ok to do this). CatalogJanitor + Break out the Comparator used by CatalogJanitor. It was an anonymous class. Instead we make it a static inner class so can add test that its actually sorting properly. + Added method hasNoReferences that will return true if no daughter dir OR no refs in daughter dir Added some TODOs around SplitTransaction -- vaguely related to this patch. Added new Test that checks cleanParent to ensure it works properly. Refactored bits of previous tests so they use common code. This addresses bug hbase-4238. https://issues.apache.org/jira/browse/hbase-4238 Diffs - src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java b53e9a0 src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 742aea4 src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java abafe5e src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 78e7d62 Diff: https://reviews.apache.org/r/1819/diff Testing --- Thanks, Michael CatalogJanitor can clear a daughter that split before processing its parent --- Key: HBASE-4238 URL: https://issues.apache.org/jira/browse/HBASE-4238 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Assignee: stack Priority: Critical Fix For: 0.92.0, 0.90.5 Attachments: 4238-v2.txt, 4238.txt I didn't dig a lot into this issue, but by splitting a table twice in a row I was able to trigger a situation where a daughter of the first split was deleted by the CatalogJanitor before it processed its parent. Will post log in a comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[ https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104020#comment-13104020 ] Hudson commented on HBASE-4238: --- Integrated in HBase-TRUNK #2205 (See [https://builds.apache.org/job/HBase-TRUNK/2205/]) HBASE-4238 CatalogJanitor can clear a daughter that split before processing its parent stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java CatalogJanitor can clear a daughter that split before processing its parent --- Key: HBASE-4238 URL: https://issues.apache.org/jira/browse/HBASE-4238 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Assignee: stack Priority: Critical Fix For: 0.92.0, 0.90.5 Attachments: 4238-v2.txt, 4238.txt I didn't dig a lot into this issue, but by splitting a table twice in a row I was able to trigger a situation where a daughter of the first split was deleted by the CatalogJanitor before it processed its parent. Will post log in a comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[ https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13091419#comment-13091419 ] Jean-Daniel Cryans commented on HBASE-4238: --- Another way this problem appears is if the 2 daughters are removed from parent and one of them was split and already cleaned before the CJ went to delete the parent, this will fail (CJ.cleanParent): {code} if ((a.getFirst() !a.getSecond()) (b.getFirst() !b.getSecond())) { {code} Here we check if the directory of each daughter exists and if they don't have a reference. In my case, they don't exist since they were already cleaned, which means that the parent will _never_ get cleaned. CatalogJanitor can clear a daughter that split before processing its parent --- Key: HBASE-4238 URL: https://issues.apache.org/jira/browse/HBASE-4238 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.90.5 I didn't dig a lot into this issue, but by splitting a table twice in a row I was able to trigger a situation where a daughter of the first split was deleted by the CatalogJanitor before it processed its parent. Will post log in a comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4238) CatalogJanitor can clear a daughter that split before processing its parent
[ https://issues.apache.org/jira/browse/HBASE-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13089099#comment-13089099 ] Jean-Daniel Cryans commented on HBASE-4238: --- Log of the issue (daughter region being d06b3b8c85af53ccbe139df7f0877e51): {quote} 2011-08-22 15:06:50,227 INFO org.apache.hadoop.hbase.catalog.MetaEditor: Deleted daughter reference TestTable,,1314050587746.7ada9de242de30a4e91f00a8dfcbf693., qualifier=splitA, from parent TestTable,,1314050584480.d06b3b8c85af53ccbe139df7f0877e51. 2011-08-22 15:06:50,229 INFO org.apache.hadoop.hbase.catalog.MetaEditor: Deleted daughter reference TestTable,013824,1314050587746.66c3c9f067203cfb7c0a541e9207766d., qualifier=splitB, from parent TestTable,,1314050584480.d06b3b8c85af53ccbe139df7f0877e51. 2011-08-22 15:06:50,230 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor: Deleting region TestTable,,1314050584480.d06b3b8c85af53ccbe139df7f0877e51. because daughter splits no longer hold references 2011-08-22 15:06:50,230 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: DELETING region file:/tmp/hbase-jdcryans/hbase/TestTable/d06b3b8c85af53ccbe139df7f0877e51 2011-08-22 15:06:50,234 INFO org.apache.hadoop.hbase.catalog.MetaEditor: Deleted region TestTable,,1314050584480.d06b3b8c85af53ccbe139df7f0877e51. from META 2011-08-22 15:06:50,235 WARN org.apache.hadoop.hbase.master.CatalogJanitor: Daughter regiondir does not exist: file:/tmp/hbase-jdcryans/hbase/TestTable/d06b3b8c85af53ccbe139df7f0877e51 2011-08-22 15:11:50,220 WARN org.apache.hadoop.hbase.master.CatalogJanitor: Daughter regiondir does not exist: file:/tmp/hbase-jdcryans/hbase/TestTable/d06b3b8c85af53ccbe139df7f0877e51 2011-08-22 15:16:50,224 WARN org.apache.hadoop.hbase.master.CatalogJanitor: Daughter regiondir does not exist: file:/tmp/hbase-jdcryans/hbase/TestTable/d06b3b8c85af53ccbe139df7f0877e51 ... {quote} Those last lines will happen forever, and .META. will be littered with that row. CatalogJanitor can clear a daughter that split before processing its parent --- Key: HBASE-4238 URL: https://issues.apache.org/jira/browse/HBASE-4238 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.90.5 I didn't dig a lot into this issue, but by splitting a table twice in a row I was able to trigger a situation where a daughter of the first split was deleted by the CatalogJanitor before it processed its parent. Will post log in a comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira