[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13783542#comment-13783542 ] Hudson commented on HBASE-1212: --- FAILURE: Integrated in HBase-0.94-security #306 (See [https://builds.apache.org/job/HBase-0.94-security/306/]) HBASE-9504 Backport HBASE-1212 to 0.94 - merge tool expects regions all have different sequence ids (Jean-Marc Spaggiari) (larsh: rev 1528280) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jean-Marc Spaggiari Fix For: 0.95.1 Attachments: failure.log, HBASE-1212.patch, HBASE-1212-v2.patch, HBASE-1212-v3-trunk.patch Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13783570#comment-13783570 ] Hudson commented on HBASE-1212: --- SUCCESS: Integrated in HBase-0.94 #1165 (See [https://builds.apache.org/job/HBase-0.94/1165/]) HBASE-9504 Backport HBASE-1212 to 0.94 - merge tool expects regions all have different sequence ids (Jean-Marc Spaggiari) (larsh: rev 1528280) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jean-Marc Spaggiari Fix For: 0.95.1 Attachments: failure.log, HBASE-1212.patch, HBASE-1212-v2.patch, HBASE-1212-v3-trunk.patch Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564368#comment-13564368 ] Jean-Marc Spaggiari commented on HBASE-1212: Should I upload one for 0.94? merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jean-Marc Spaggiari Fix For: 0.96.0 Attachments: failure.log, HBASE-1212.patch, HBASE-1212-v2.patch, HBASE-1212-v3-trunk.patch Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564408#comment-13564408 ] Ted Yu commented on HBASE-1212: --- Integrated to trunk. Thanks for the patch, Jean-Marc. Thanks for the review, Chunhui. merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jean-Marc Spaggiari Fix For: 0.96.0 Attachments: failure.log, HBASE-1212.patch, HBASE-1212-v2.patch, HBASE-1212-v3-trunk.patch Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564461#comment-13564461 ] Hudson commented on HBASE-1212: --- Integrated in HBase-TRUNK #3814 (See [https://builds.apache.org/job/HBase-TRUNK/3814/]) HBASE-1212 merge tool expects regions all have different sequence ids (Jean-Marc) (Revision 1439495) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jean-Marc Spaggiari Fix For: 0.96.0 Attachments: failure.log, HBASE-1212.patch, HBASE-1212-v2.patch, HBASE-1212-v3-trunk.patch Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564893#comment-13564893 ] Hudson commented on HBASE-1212: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #381 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/381/]) HBASE-1212 merge tool expects regions all have different sequence ids (Jean-Marc) (Revision 1439495) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jean-Marc Spaggiari Fix For: 0.96.0 Attachments: failure.log, HBASE-1212.patch, HBASE-1212-v2.patch, HBASE-1212-v3-trunk.patch Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564071#comment-13564071 ] chunhui shen commented on HBASE-1212: - We will use sequenceId to sort HFiles, like in compaction. When merging, there is no overlap between these two files from different regions, I think it's ok even if two files have the same sequence ID since we don't use it to distingush the files in Store any more Patch looks good for me merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jean-Marc Spaggiari Attachments: failure.log, HBASE-1212.patch, HBASE-1212-v2.patch, HBASE-1212-v3-trunk.patch Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[
https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13563332#comment-13563332
]
Hadoop QA commented on HBASE-1212:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12566598/HBASE-1212-v3-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4191//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4191//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4191//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4191//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4191//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4191//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4191//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4191//console
This message is automatically generated.
merge tool expects regions all have different sequence ids
--
Key: HBASE-1212
URL: https://issues.apache.org/jira/browse/HBASE-1212
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: Jean-Marc Spaggiari
Attachments: failure.log, HBASE-1212.patch, HBASE-1212-v2.patch,
HBASE-1212-v3-trunk.patch
Currently merging two regions, the merge tool will compare their sequence
ids. If same, it will decrement one. It needs to do this because on region
open, files are keyed by their sequenceid; if two the same, one will erase
the other.
Well, with the move to the aggregating hfile format, the sequenceid is
written when the file is created and its no longer written into an aside file
but as metadata on to the end of the file. Changing the sequenceid is no
longer an option.
This issue is about figuring a solution for the rare case where two store
files have same sequence id AND we want to merge the two regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[
https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530802#comment-13530802
]
Hadoop QA commented on HBASE-1212:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12556386/HBASE-1212-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
104 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 23 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3526//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3526//console
This message is automatically generated.
merge tool expects regions all have different sequence ids
--
Key: HBASE-1212
URL: https://issues.apache.org/jira/browse/HBASE-1212
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: Jean-Marc Spaggiari
Attachments: failure.log, HBASE-1212.patch, HBASE-1212-v2.patch
Currently merging two regions, the merge tool will compare their sequence
ids. If same, it will decrement one. It needs to do this because on region
open, files are keyed by their sequenceid; if two the same, one will erase
the other.
Well, with the move to the aggregating hfile format, the sequenceid is
written when the file is created and its no longer written into an aside file
but as metadata on to the end of the file. Changing the sequenceid is no
longer an option.
This issue is about figuring a solution for the rare case where two store
files have same sequence id AND we want to merge the two regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[
https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13511343#comment-13511343
]
Jean-Marc Spaggiari commented on HBASE-1212:
Based on Ryan's comment on March 3rd 2011 I did some more testing.
Here is how I proceed:
Creation of a test table.
Truncate the test table if already there and not empty.
Insert 10 records
hbck = ok.
Split into 256 regions using HTML interface
Balancer + major_compact
Try to merge all the regions 2 by 2 until only 1 region remaining or until
error.
If no error, restart from split. (Usually need to do it twice)
Now, at this stage, I have 2 regions with the same sequence ID that I'm not
able to merge with the current un-patched HRegion code.
Displayed:starting merge of regions:
testtable,?\xEC\x80C^\xB2Q\xE8,1354799289826.eb1b2213a6b09a9d989c39abe3d6d18b.
and
testtable,?\xEC\xF9\x88\xBE{Sb,1354799292172.2d81e98f11bdda12dbf06c11eefefb39.
into new region {NAME =
'testtable,?\xEC\x80C^\xB2Q\xE8,1354799434888.e4e403c2bf3b3b2561f26bfb70ef48d1.',
STARTKEY = '?\xEC\x80C^\xB2Q\xE8', ENDKEY = '?\xEDw\xF5\x16\xC4\x81\xB7',
ENCODED = e4e403c2bf3b3b2561f26bfb70ef48d1,} with start key
?\xEC\x80C^\xB2Q\xE8 and end key ?\xEDw\xF5\x16\xC4\x81\xB7
Start-hbase, hbck = One issue on the newly created region (HBASE-7287
e4e403c2bf3b3b2561f26bfb70ef48d)
rowcount = 12/12/06 08:15:14 INFO mapred.JobClient: ROWS=10
Apply HBASE-1212 patch, stop HBase and restart the merge.
Merge went well up to the end (all regions merged 2 by 2 until only 1 region
remaining)
Start-hbase, hbck = One issue on the newly created region (HBASE-7287
e4e403c2bf3b3b2561f26bfb70ef48d) Same as before, so no additionnal issue
because of HBASE-1212.
rowcount = 12/12/06 08:21:54 INFO mapred.JobClient: ROWS=10
The issue created by the merge when not patched (HBASE-7287 or HBASE-1212) is
just an empty directory created which can be manually removed.
If we apply HBASE-1212, then there is no more needs to apply HBASE-7287. If we
want to make more tests for HBASE-1212 then it might be good to apply
HBASE-7287 to avoid system inconsistencies.
Based on my tests I can confirm that we can remove the sequenceID check and the
merge will still be working fine even if 2 files got the same sequenceID.
merge tool expects regions all have different sequence ids
--
Key: HBASE-1212
URL: https://issues.apache.org/jira/browse/HBASE-1212
Project: HBase
Issue Type: Bug
Reporter: stack
Attachments: failure.log, HBASE-1212.patch
Currently merging two regions, the merge tool will compare their sequence
ids. If same, it will decrement one. It needs to do this because on region
open, files are keyed by their sequenceid; if two the same, one will erase
the other.
Well, with the move to the aggregating hfile format, the sequenceid is
written when the file is created and its no longer written into an aside file
but as metadata on to the end of the file. Changing the sequenceid is no
longer an option.
This issue is about figuring a solution for the rare case where two store
files have same sequence id AND we want to merge the two regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13511457#comment-13511457 ] Ted Yu commented on HBASE-1212: --- Can you attach patch for trunk ? Thanks merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jean-Marc Spaggiari Attachments: failure.log, HBASE-1212.patch Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13511488#comment-13511488 ] Jean-Marc Spaggiari commented on HBASE-1212: Hi Ted, I have already attached the patch. I did it on my eclipse which is configured with the trunk. So it should be correct? Or have I done anything wrong? Thanks. merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jean-Marc Spaggiari Attachments: failure.log, HBASE-1212.patch Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[
https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13511529#comment-13511529
]
Ted Yu commented on HBASE-1212:
---
From your patch:
{code}
Index: src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
{code}
In trunk, the path would be
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
Can you use 'svn diff' command to produce next patch under the root of your
workspace ?
merge tool expects regions all have different sequence ids
--
Key: HBASE-1212
URL: https://issues.apache.org/jira/browse/HBASE-1212
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: Jean-Marc Spaggiari
Attachments: failure.log, HBASE-1212.patch
Currently merging two regions, the merge tool will compare their sequence
ids. If same, it will decrement one. It needs to do this because on region
open, files are keyed by their sequenceid; if two the same, one will erase
the other.
Well, with the move to the aggregating hfile format, the sequenceid is
written when the file is created and its no longer written into an aside file
but as metadata on to the end of the file. Changing the sequenceid is no
longer an option.
This issue is about figuring a solution for the rare case where two store
files have same sequence id AND we want to merge the two regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13512008#comment-13512008 ] Lars Hofhansl commented on HBASE-1212: -- Change looks good to me. Taking a cursory look at the code we're indeed not using the sequenceId solely to distinguish HFiles. Can somebody reconfirm that. merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jean-Marc Spaggiari Attachments: failure.log, HBASE-1212.patch, HBASE-1212-v2.patch Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13510670#comment-13510670 ] Jean-Marc Spaggiari commented on HBASE-1212: FYI, I'm facing this issue today while trying to merge all the regions of a table. I have created a 54 regions table, and while trying to merge all of them back into a single one I got this error. I'm able to reproduce it. merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13290022#comment-13290022 ] Lukas commented on HBASE-1212: -- Can someone comment, if this is still an issue? merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002363#comment-13002363 ] stack commented on HBASE-1212: -- This is worse now that bulk upload is popular; all get low seqid so likelihood of clashing more likely. Made it so I had to do manual merge doing fixup of user cluster. merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-1212) merge tool expects regions all have different sequence ids
[ https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002375#comment-13002375 ] ryan rawson commented on HBASE-1212: now that we dont key files by their sequence id in Store.java, this is simpler: just allow duplicate sequence ids. merge tool expects regions all have different sequence ids -- Key: HBASE-1212 URL: https://issues.apache.org/jira/browse/HBASE-1212 Project: HBase Issue Type: Bug Reporter: stack Currently merging two regions, the merge tool will compare their sequence ids. If same, it will decrement one. It needs to do this because on region open, files are keyed by their sequenceid; if two the same, one will erase the other. Well, with the move to the aggregating hfile format, the sequenceid is written when the file is created and its no longer written into an aside file but as metadata on to the end of the file. Changing the sequenceid is no longer an option. This issue is about figuring a solution for the rare case where two store files have same sequence id AND we want to merge the two regions. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-1212) merge tool expects regions all have different sequence ids
[
https://issues.apache.org/jira/browse/HBASE-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001467#comment-13001467
]
Lukas commented on HBASE-1212:
--
Hi there,
When using util.merge on a table (created under hbase 0.20.6, but now running
under 0.90.1) I get the following exception very often (which I think is the
result of this jira):
{code}
FATAL util.Merge: Merge failed
java.io.IOException: Files have same sequenceid: 1283640296746
at org.apache.hadoop.hbase.regionserver.HRegion.merge(HRegion.java:2785)
at org.apache.hadoop.hbase.util.Merge.merge(Merge.java:287)
at org.apache.hadoop.hbase.util.Merge.mergeTwoRegions(Merge.java:238)
at org.apache.hadoop.hbase.util.Merge.run(Merge.java:110)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.hbase.util.Merge.main(Merge.java:379)
{code}
Unfortunately at this point some families of the region were already moved to
the new merged region which leaves the table in an inconsistent state.
Is there any way to detect this failure case before running the merge tool
(except for opening the store files in advance and calling getMaxSequenceId)?
Same issue is also mentioned in the
[comments|https://issues.apache.org/jira/browse/HBASE-1621?focusedCommentId=12993477#comment-12993477]
of HBASE-1621
merge tool expects regions all have different sequence ids
--
Key: HBASE-1212
URL: https://issues.apache.org/jira/browse/HBASE-1212
Project: HBase
Issue Type: Bug
Reporter: stack
Currently merging two regions, the merge tool will compare their sequence
ids. If same, it will decrement one. It needs to do this because on region
open, files are keyed by their sequenceid; if two the same, one will erase
the other.
Well, with the move to the aggregating hfile format, the sequenceid is
written when the file is created and its no longer written into an aside file
but as metadata on to the end of the file. Changing the sequenceid is no
longer an option.
This issue is about figuring a solution for the rare case where two store
files have same sequence id AND we want to merge the two regions.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
