[jira] [Commented] (HBASE-2376) Add special SnapshotScanner which presents view of all data at some time in the past
[ https://issues.apache.org/jira/browse/HBASE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501743#comment-13501743 ] Kannan Muthukkaruppan commented on HBASE-2376: -- Lars wrote: <<>>. This is not true. A simple CF with VERSIONS=1 & no TTL (i.e. TTL of infinity) can also benefit from ability to FlashBack query. Flash back is simply an ability to query the DB as of a previous point in time. Why should we overload that functionality with versions, TTL, etc.? I think it is useful to think of FlashBack as completely independent of other settings like TTL, MAXVERSIONS, MINVERSIONS, etc. The latter should be picked at schema design time based on the application requirements. For example, you may have many tables in your system with different TTL, VERSIONS requirements. Maybe you have different CFs within a table, with differing TTL & VERSION requirements. But on top of all those, suppose across all my tables I want to be able to query the entire DB as of a previous point in time. From a user's point of view, the only setting they need to worry about is the "time period" (back in time) up to which flash back queries are supported. For example, you might have one CF, with VERSIONS=1, where you are keeping hourly rollup data that you want to retain for 1 month (TTL) and, another CF where you keep daily rollup data also with VERSIONS=1 where you want to retain data for 3 years. But separately, I want the ability to be able to do flash back queries up to say 7 days back. This "7 days" should be a completely different setting, and there seems to be no reason to confuse this with TTL & Verions. Now, API wise, we need the ability to say that we are doing a flashback query i.e. "Scan @ T" instead of regular "Scan". In Oracle DB too, for instance, flash back queries have this special syntax: SELECT * FROM employee AS OF TIMESTAMP WHERE name = 'JOHN'; Regarding <<< So the snapshot scanner is special in that only through this specific scanner you can look further back than the TTL.>>>: I think that is by design. Note: Scan @ T (flash back query) is different than doing a Scan with setTimeRange(0, T). A delete done a T+1 of a key is immaterial for Scan @ T query; whereas for a Scan with setTimeRange(0, T), you will still see the effect of the delete done at T+1. In summary, we should not confuse our users by forcing them to change their schema design (i.e. choice of VERSIONS, TTL, etc.) to support flashback queries. Flashback support should be configured using a simple extra knob that can be set a system, table or CF level. We should NOT overload that knob with TTL and Versions. > Add special SnapshotScanner which presents view of all data at some time in > the past > > > Key: HBASE-2376 > URL: https://issues.apache.org/jira/browse/HBASE-2376 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Affects Versions: 0.20.3 >Reporter: Jonathan Gray >Assignee: Pritam Damania > > In order to support a particular kind of database "snapshot" feature which > doesn't require copying data, we came up with the idea for a special > SnapshotScanner that would present a view of your data at some point in the > past. The primary use case for this would be to be able to recover > particular data/rows (but not all data, like a global rollback) should they > have somehow been messed up (application fault, application bug, user error, > etc.). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7009) Port HBaseCluster interface/tests to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-7009: - Release Note: Adds an Integration Test facilty that is run by invoking 'mvn verify'. It can run the same test on a minihbasecluster and also on an already distributed cluster. Adds a ChaosMonkey too. > Port HBaseCluster interface/tests to 0.94 > - > > Key: HBASE-7009 > URL: https://issues.apache.org/jira/browse/HBASE-7009 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.94.3 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.4 > > Attachments: HBASE-7009-p1.patch, HBASE-7009.patch, > HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch, > HBASE-7009-v3-wo-classfinder.patch > > > Need to port. I am porting V5 patch from the original JIRA; I have a > partially ported (V3) patch from Enis with protocol buffers being reverted to > HRegionInterface/HMasterInterface -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7009) Port HBaseCluster interface/tests to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-7009: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed this (v3-wo-classfinder patch). Thanks Sergey for the backport. > Port HBaseCluster interface/tests to 0.94 > - > > Key: HBASE-7009 > URL: https://issues.apache.org/jira/browse/HBASE-7009 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.94.3 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.4 > > Attachments: HBASE-7009-p1.patch, HBASE-7009.patch, > HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch, > HBASE-7009-v3-wo-classfinder.patch > > > Need to port. I am porting V5 patch from the original JIRA; I have a > partially ported (V3) patch from Enis with protocol buffers being reverted to > HRegionInterface/HMasterInterface -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2231) Compaction events should be written to HLog
[ https://issues.apache.org/jira/browse/HBASE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-2231: - Attachment: 2231-testcase_v3.txt This test case fails when files have been removed by a compaction. Not to forward port Todd's fancy WAL writing. > Compaction events should be written to HLog > --- > > Key: HBASE-2231 > URL: https://issues.apache.org/jira/browse/HBASE-2231 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Labels: moved_from_0_20_5 > Fix For: 0.96.0 > > Attachments: 2231-testcase-0.94.txt, 2231-testcase_v2.txt, > 2231-testcase_v3.txt, 2231v2.txt, hbase-2231-testcase.txt, hbase-2231.txt > > > The sequence for a compaction should look like this: > # Compact region to "new" files > # Write a "Compacted Region" entry to the HLog > # Delete "old" files > This deals with a case where the RS has paused between step 1 and 2 and the > regions have since been reassigned. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2376) Add special SnapshotScanner which presents view of all data at some time in the past
[ https://issues.apache.org/jira/browse/HBASE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501710#comment-13501710 ] Lars Hofhansl commented on HBASE-2376: -- MIN_VERSIONS affects expired KVs. It's means: "You can expire KVs after this TTL, but keep at least MIN_VERSIONS versions around". I was going to Jonathan's initial description: bq. TTKAV (Time To Keep All Versions) This setting would define how far back you would be able to perform this snapshot. You'd do that by setting MAX_VERSIONS to MAX_LONG, TTL to the flashback time, and MIN_VERSIONS to the number of version you want keep around. Now within the TTL you'd keep all versions, outside of the TTL MIN_VERSIONS versions are kept. But I see now. So the snapshot scanner is special in that only through this specific scanner you can look further back than the TTL. This does seem a pretty esoteric feature then, though. Flashback only makes sense together with TTL (otherwise you could set the TTL). You have a TTL and a sort of a super TTL for which you can only use a special scanner. > Add special SnapshotScanner which presents view of all data at some time in > the past > > > Key: HBASE-2376 > URL: https://issues.apache.org/jira/browse/HBASE-2376 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Affects Versions: 0.20.3 >Reporter: Jonathan Gray >Assignee: Pritam Damania > > In order to support a particular kind of database "snapshot" feature which > doesn't require copying data, we came up with the idea for a special > SnapshotScanner that would present a view of your data at some point in the > past. The primary use case for this would be to be able to recover > particular data/rows (but not all data, like a global rollback) should they > have somehow been messed up (application fault, application bug, user error, > etc.). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2376) Add special SnapshotScanner which presents view of all data at some time in the past
[ https://issues.apache.org/jira/browse/HBASE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501704#comment-13501704 ] Pritam Damania commented on HBASE-2376: --- I'm not sure how you would achieve it with TTL and MIN_VERSIONS, but lets take an example, suppose the current time in millliseconds is 60 and the table has max versions set to 3. Suppose we want to support FlashBackQueries for upto 10ms in the past. What you want is atmost 3 versions for the time between t=0 to t=50. How does MIN_VERSIONS achieve keeping atmost 3 versions in that time range ? Does MIN_VERSIONS apply only to expired kvs ? Also an issue with TTL and MIN_VERSIONS is that you cannot support something like if I want a TTL of 6 days but a FlashBack upto 8 days. The FlashBack and TTL time have to be the same which some applications might not want. Some applications might want to keep all their other parameters the same and just specify that they want to do a read back in time for 'x' days. Changing the TTL value for an application to provide this functionality would also change what a scan returns since although you are pushing TTL back to retain enough data to do a read in the past, your queries in the current time are also affected since they will surface all kvs which are within TTL. > Add special SnapshotScanner which presents view of all data at some time in > the past > > > Key: HBASE-2376 > URL: https://issues.apache.org/jira/browse/HBASE-2376 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Affects Versions: 0.20.3 >Reporter: Jonathan Gray >Assignee: Pritam Damania > > In order to support a particular kind of database "snapshot" feature which > doesn't require copying data, we came up with the idea for a special > SnapshotScanner that would present a view of your data at some point in the > past. The primary use case for this would be to be able to recover > particular data/rows (but not all data, like a global rollback) should they > have somehow been messed up (application fault, application bug, user error, > etc.). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7162) Prefix Compression - Trie data block encoding; hbase-common and hbase-server changes
[ https://issues.apache.org/jira/browse/HBASE-7162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501701#comment-13501701 ] Lars Hofhansl commented on HBASE-7162: -- In this case it was the ByteBuffer allocation + writing the deserialized KV into it (BufferedDataBlockEncoder.BufferedEncodedSeeker.getKeyValueBuffer()) Some scatter/gather type concept would be good here. What you say makes a lot of sense... And would be a major refactoring it seems. > Prefix Compression - Trie data block encoding; hbase-common and hbase-server > changes > > > Key: HBASE-7162 > URL: https://issues.apache.org/jira/browse/HBASE-7162 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.96.0 >Reporter: stack >Assignee: Matt Corgan > Fix For: 0.96.0 > > Attachments: HBASE-4676-common-and-server-v8.patch, > HBASE-4676-common-and-server-v9.patch, HBASE-7162-common-and-server-v10.patch > > > These are the hbase-common and hbase-server changes for hbase-4676 Prefix > Compression - Trie data block encoding. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7162) Prefix Compression - Trie data block encoding; hbase-common and hbase-server changes
[ https://issues.apache.org/jira/browse/HBASE-7162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501696#comment-13501696 ] Enis Soztutar commented on HBASE-7162: -- bq. Turns out that (in our scenario) 60% of the time was spent is just allocating ByteBuffer to back the decoded KeyValues where it would have been sufficient to just create a KV interface that points to an existing buffer for the key and existing buffer for the value. Does 60% include just the allocation, or allocation + deserialization/copy. I was also thinking along the lines of keeping just the pointers to the block cache for KV's, and only deserializing them to the socket buffers. > Prefix Compression - Trie data block encoding; hbase-common and hbase-server > changes > > > Key: HBASE-7162 > URL: https://issues.apache.org/jira/browse/HBASE-7162 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.96.0 >Reporter: stack >Assignee: Matt Corgan > Fix For: 0.96.0 > > Attachments: HBASE-4676-common-and-server-v8.patch, > HBASE-4676-common-and-server-v9.patch, HBASE-7162-common-and-server-v10.patch > > > These are the hbase-common and hbase-server changes for hbase-4676 Prefix > Compression - Trie data block encoding. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6802) Export Snapshot
[ https://issues.apache.org/jira/browse/HBASE-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501693#comment-13501693 ] terry zhang commented on HBASE-6802: hi Jesse, where is your snapshot branch? Could you please give us the URL to checkout the project? Thanks so much! > Export Snapshot > --- > > Key: HBASE-6802 > URL: https://issues.apache.org/jira/browse/HBASE-6802 > Project: HBase > Issue Type: Sub-task > Components: snapshots >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Fix For: hbase-6055, 0.96.0 > > Attachments: HBASE-6802-v1.patch > > > Export a snapshot to another cluster. > - Copy the .snapshot/name folder with all the references > - Copy the hfiles/hlogs needed by the snapshot > Once the other cluster has the files and the snapshot information it can > restore the snapshot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7187) Create empty hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-7187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark reassigned HBASE-7187: Assignee: Elliott Clark > Create empty hbase-client module > > > Key: HBASE-7187 > URL: https://issues.apache.org/jira/browse/HBASE-7187 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7187-0.patch, HBASE-7187-1.patch, > HBASE-7187-2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7162) Prefix Compression - Trie data block encoding; hbase-common and hbase-server changes
[ https://issues.apache.org/jira/browse/HBASE-7162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501682#comment-13501682 ] Lars Hofhansl commented on HBASE-7162: -- The Cell stuff is a cool step into the right direction. We just came across this today again when profiling scanning when FastDiff is enabled. Turns out that (in our scenario) 60% of the time was spent is just allocating ByteBuffer to back the decoded KeyValues where it would have been sufficient to just create a KV interface that points to an existing buffer for the key and existing buffer for the value. > Prefix Compression - Trie data block encoding; hbase-common and hbase-server > changes > > > Key: HBASE-7162 > URL: https://issues.apache.org/jira/browse/HBASE-7162 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.96.0 >Reporter: stack >Assignee: Matt Corgan > Fix For: 0.96.0 > > Attachments: HBASE-4676-common-and-server-v8.patch, > HBASE-4676-common-and-server-v9.patch, HBASE-7162-common-and-server-v10.patch > > > These are the hbase-common and hbase-server changes for hbase-4676 Prefix > Compression - Trie data block encoding. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2376) Add special SnapshotScanner which presents view of all data at some time in the past
[ https://issues.apache.org/jira/browse/HBASE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501681#comment-13501681 ] Lars Hofhansl commented on HBASE-2376: -- HBASE-4071 is a bit misnamed. It introduces MIN_VERSIONS. So setting TTL and MIN_VERSIONS does give you exactly what you describe above (I think). > Add special SnapshotScanner which presents view of all data at some time in > the past > > > Key: HBASE-2376 > URL: https://issues.apache.org/jira/browse/HBASE-2376 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Affects Versions: 0.20.3 >Reporter: Jonathan Gray >Assignee: Pritam Damania > > In order to support a particular kind of database "snapshot" feature which > doesn't require copying data, we came up with the idea for a special > SnapshotScanner that would present a view of your data at some point in the > past. The primary use case for this would be to be able to recover > particular data/rows (but not all data, like a global rollback) should they > have somehow been messed up (application fault, application bug, user error, > etc.). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501678#comment-13501678 ] Otis Gospodnetic commented on HBASE-4120: - [~liujia_ict] it looks like this got stuck last year. I quickly looked at the comments here in JIRA and see this went through some reviews and was generally welcomes. Would it be possible for you to upload a fresh patch to revive this? TIP: please use the same/single name for the patch. JIRA will overwrite the old one. This way devs reviewing this JIRA won't have to figure out which patch to use (yes, one can look at dates, but when there are >10 patches attached, even that becomes painful) > isolation and allocation > > > Key: HBASE-4120 > URL: https://issues.apache.org/jira/browse/HBASE-4120 > Project: HBase > Issue Type: New Feature > Components: master, regionserver >Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 >Reporter: Liu Jia >Assignee: Liu Jia > Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, > Design_document_for_HBase_isolation_and_allocation_Revised.pdf, > HBase_isolation_and_allocation_user_guide.pdf, > Performance_of_Table_priority.pdf, > Simple_YCSB_Tests_For_TablePriority_Trunk_and_0.90.4.pdf, System > Structure.jpg, TablePriority.patch, TablePriority_v12.patch, > TablePriority_v12.patch, TablePriority_v15_with_coprocessor.patch, > TablePriority_v16_with_coprocessor.patch, TablePriority_v17.patch, > TablePriority_v17.patch, TablePriority_v8_for_trunk.patch, > TablePriority_v8.patch, TablePriority_v8.patch, TablePrioriy_v9.patch > > > The HBase isolation and allocation tool is designed to help users manage > cluster resource among different application and tables. > When we have a large scale of HBase cluster with many applications running on > it, there will be lots of problems. In Taobao there is a cluster for many > departments to test their applications performance, these applications are > based on HBase. With one cluster which has 12 servers, there will be only one > application running exclusively on this server, and many other applications > must wait until the previous test finished. > After we add allocation manage function to the cluster, applications can > share the cluster and run concurrently. Also if the Test Engineer wants to > make sure there is no interference, he/she can move out other tables from > this group. > In groups we use table priority to allocate resource, when system is busy; we > can make sure high-priority tables are not affected lower-priority tables > Different groups can have different region server configurations, some groups > optimized for reading can have large block cache size, and others optimized > for writing can have large memstore size. > Tables and region servers can be moved easily between groups; after changing > the configuration, a group can be restarted alone instead of restarting the > whole cluster. > git entry : https://github.com/ICT-Ope/HBase_allocation . > We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-74) [performance] When a get or scan request spans multiple columns, execute the reads in parallel
[ https://issues.apache.org/jira/browse/HBASE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501677#comment-13501677 ] Otis Gospodnetic commented on HBASE-74: --- Let's dig out this 5+ years old issue with last comment from 4.5+ years ago :) Maybe this was actually implemented by now? > [performance] When a get or scan request spans multiple columns, execute the > reads in parallel > -- > > Key: HBASE-74 > URL: https://issues.apache.org/jira/browse/HBASE-74 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Jim Kellerman > > When a get or scan request spans multiple columns, execute the reads in > parallel and use a CountDownLatch to wait for them to complete before > returning the results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7187) Create empty hbase-client module
[
https://issues.apache.org/jira/browse/HBASE-7187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501675#comment-13501675
]
Hadoop QA commented on HBASE-7187:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554441/HBASE-7187-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
99 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 24 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3383//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3383//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3383//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3383//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3383//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3383//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3383//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3383//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3383//console
This message is automatically generated.
> Create empty hbase-client module
>
>
> Key: HBASE-7187
> URL: https://issues.apache.org/jira/browse/HBASE-7187
> Project: HBase
> Issue Type: Sub-task
> Components: Client
>Reporter: Elliott Clark
> Fix For: 0.96.0
>
> Attachments: HBASE-7187-0.patch, HBASE-7187-1.patch,
> HBASE-7187-2.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2376) Add special SnapshotScanner which presents view of all data at some time in the past
[ https://issues.apache.org/jira/browse/HBASE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501669#comment-13501669 ] Pritam Damania commented on HBASE-2376: --- I don't think those two JIRAs give us that flexibility. For example if TTL is 5 days and max versions is 3. If we want to support a query that gives us a view upto 7 days in the past, then we need to retain versions upto 7 + 5 days (effective TTL for compactions and flushes is 12 days and not 5) and we need to retain a maximum of 3 versions before 7 days so that if we do a query for data on 7 days in the past, we have 3 versions to surface for that query if present. The basic idea is if we support queries in the past upto 'x' days, the compactions and flushes should be behaving as if they were happening 'x days' in the past. Simply retaining deletes or retaining just one version is not sufficient. > Add special SnapshotScanner which presents view of all data at some time in > the past > > > Key: HBASE-2376 > URL: https://issues.apache.org/jira/browse/HBASE-2376 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Affects Versions: 0.20.3 >Reporter: Jonathan Gray >Assignee: Pritam Damania > > In order to support a particular kind of database "snapshot" feature which > doesn't require copying data, we came up with the idea for a special > SnapshotScanner that would present a view of your data at some point in the > past. The primary use case for this would be to be able to recover > particular data/rows (but not all data, like a global rollback) should they > have somehow been messed up (application fault, application bug, user error, > etc.). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7110) refactor the compaction selection and config code similarly to 0.89-fb changes
[ https://issues.apache.org/jira/browse/HBASE-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501667#comment-13501667 ] Enis Soztutar commented on HBASE-7110: -- Overall looks good. Some points: - I think HStore.getTtl() should just returns the configured this.ttl, and should not look at min versions. Looking at min versions only apply at the compaction selection for expiry. - Some lines exceed 100 chars. - Checked whether the current compaction policy is unchanged with this patch. Although there are some refactorings and splitting up the work, it seems so. > refactor the compaction selection and config code similarly to 0.89-fb changes > -- > > Key: HBASE-7110 > URL: https://issues.apache.org/jira/browse/HBASE-7110 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-6371-v5-refactor-only-squashed.patch, > HBASE-7110-v6-squashed.patch > > > Separate JIRA for refactoring changes from HBASE-7055 (and further ones after > code review) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[ https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501662#comment-13501662 ] Oleg Zhurakousky commented on HBASE-7191: - That might be the case. I'll check in the AM. The funny thing is that they all pass in IDE and/or if ran in isolation via maven, but not with other tests. But you may be right, i may have 1.7 as a default. > HBCK - Add offline create/fix hbase.version and hbase.id > - > > Key: HBASE-7191 > URL: https://issues.apache.org/jira/browse/HBASE-7191 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.94.1 >Reporter: Enis Soztutar > Attachments: 7191-2.patch > > > One of our clients run into a problem, in which they have the hbase.root.dir, > and cluster data, but their hbase.id and hbase.version files are corrupted. > HMaster creates those on start, but not if there is already existing data. > We can add smt like --fixIdFile, and ability for HBCK to do some offline > repairs for the version file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7201) Convert HLog / HFile metadata content to PB
Enis Soztutar created HBASE-7201: Summary: Convert HLog / HFile metadata content to PB Key: HBASE-7201 URL: https://issues.apache.org/jira/browse/HBASE-7201 Project: HBase Issue Type: Sub-task Components: HFile, Protobufs, wal Reporter: Enis Soztutar Some of the remaining discussions for PB conversions: - Convert the HFile/HLog metadata to PB. - WALEdit, HLogKey should be converted? We don't want to repeat the PBMagic, and the PB overhead can be high, but this is needed for replication? - We said no to converting KV. These should not block 0.96. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2376) Add special SnapshotScanner which presents view of all data at some time in the past
[ https://issues.apache.org/jira/browse/HBASE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501651#comment-13501651 ] Lars Hofhansl commented on HBASE-2376: -- Is this addressed with HBASE-4536 together with HBASE-4071? > Add special SnapshotScanner which presents view of all data at some time in > the past > > > Key: HBASE-2376 > URL: https://issues.apache.org/jira/browse/HBASE-2376 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Affects Versions: 0.20.3 >Reporter: Jonathan Gray >Assignee: Pritam Damania > > In order to support a particular kind of database "snapshot" feature which > doesn't require copying data, we came up with the idea for a special > SnapshotScanner that would present a view of your data at some point in the > past. The primary use case for this would be to be able to recover > particular data/rows (but not all data, like a global rollback) should they > have somehow been messed up (application fault, application bug, user error, > etc.). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7187) Create empty hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-7187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7187: - Attachment: HBASE-7187-2.patch > Create empty hbase-client module > > > Key: HBASE-7187 > URL: https://issues.apache.org/jira/browse/HBASE-7187 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7187-0.patch, HBASE-7187-1.patch, > HBASE-7187-2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7187) Create empty hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-7187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7187: - Attachment: HBASE-7187-1.patch new patch. I removed some dependencies in hbase-common that were un-used but being dragged into hbase-client. > Create empty hbase-client module > > > Key: HBASE-7187 > URL: https://issues.apache.org/jira/browse/HBASE-7187 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7187-0.patch, HBASE-7187-1.patch, > HBASE-7187-2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501646#comment-13501646 ] Jimmy Xiang commented on HBASE-6423: @Stack, I addressed couple of your comments and put the patch on RB: https://reviews.apache.org/r/8149/. It will make the review easier. This patch will make sure it is possible that we don't wait for the lock forever, so that the IPC handler is released if the region is busy. Therefore, other none-busy regions can be accessed. That's how we address Karthik's original comment in the issue. Sure, we can backport, once we are ready to integrate it into the trunk branch. > Writes should not block reads on blocking updates to memstores > -- > > Key: HBASE-6423 > URL: https://issues.apache.org/jira/browse/HBASE-6423 > Project: HBase > Issue Type: Bug >Reporter: Karthik Ranganathan >Assignee: Jimmy Xiang > Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, > trunk-6423_v2.patch > > > We have a big data use case where we turn off WAL and have a ton of reads and > writes. We found that: > 1. flushing a memstore takes a while (GZIP compression) > 2. incoming writes cause the new memstore to grow in an unbounded fashion > 3. this triggers blocking memstore updates > 4. in turn, this causes all the RPC handler threads to block on writes to > that memstore > 5. we are not able to read during this time as RPC handlers are blocked > At a higher level, we should not hold up the RPC threads while blocking > updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7200) create integration test for balancing regions and killing region servers
[ https://issues.apache.org/jira/browse/HBASE-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7200: Description: See related JIRA (was: See related JIRAs) > create integration test for balancing regions and killing region servers > > > Key: HBASE-7200 > URL: https://issues.apache.org/jira/browse/HBASE-7200 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > > See related JIRA -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7200) create integration test for balancing regions and killing region servers
Sergey Shelukhin created HBASE-7200: --- Summary: create integration test for balancing regions and killing region servers Key: HBASE-7200 URL: https://issues.apache.org/jira/browse/HBASE-7200 Project: HBase Issue Type: Sub-task Affects Versions: 0.96.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin See related JIRAs -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7109) integration tests on cluster are not getting picked up from distribution
[ https://issues.apache.org/jira/browse/HBASE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501634#comment-13501634 ] Sergey Shelukhin commented on HBASE-7109: - The behavior is identical to previous code, I will revise tomorrow. > integration tests on cluster are not getting picked up from distribution > > > Key: HBASE-7109 > URL: https://issues.apache.org/jira/browse/HBASE-7109 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-7109-squashed.patch, HBASE-7109-v2-squashed.patch, > HBASE-7109-v3-squashed.patch, HBASE-7109-v4-squashed.patch, > HBASE-7109-v5.patch, HBASE-7109-v5.patch > > > The method of finding test classes only works on local build (or its full > copy), not if the distribution is used. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7187) Create empty hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-7187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7187: - Attachment: (was: HBASE-7187-0.patch) > Create empty hbase-client module > > > Key: HBASE-7187 > URL: https://issues.apache.org/jira/browse/HBASE-7187 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7187-0.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7187) Create empty hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-7187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7187: - Attachment: HBASE-7187-0.patch > Create empty hbase-client module > > > Key: HBASE-7187 > URL: https://issues.apache.org/jira/browse/HBASE-7187 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7187-0.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7109) integration tests on cluster are not getting picked up from distribution
[
https://issues.apache.org/jira/browse/HBASE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501628#comment-13501628
]
Ted Yu commented on HBASE-7109:
---
{code}
+ * A class is considered as a test class if:
+ * - it's not Abstract AND
+ * - one or more of its methods is annotated with org.junit.Test OR
+ * - the class is annotated with Suite.SuiteClasses
+ * */
+ private static class TestClassFilter implements ClassFilter {
{code}
Do we need to consider tests that extend junit.framework.TestCase ?
A few examples: TestCompare, TestBase64.
There is also:
{code}
public abstract class HBaseTestCase extends TestCase {
{code}
We have:
{code}
public class TestHRegion extends HBaseTestCase {
{code}
{code}
+public boolean isCandidateClass(Class c) {
+ return isTestClass(c) && isCategorizedClass(c);
{code}
Some test classes don't satisfy both conditions above (methods aren't annotated
with @Test in TestBase64).
> integration tests on cluster are not getting picked up from distribution
>
>
> Key: HBASE-7109
> URL: https://issues.apache.org/jira/browse/HBASE-7109
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Affects Versions: 0.96.0
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HBASE-7109-squashed.patch, HBASE-7109-v2-squashed.patch,
> HBASE-7109-v3-squashed.patch, HBASE-7109-v4-squashed.patch,
> HBASE-7109-v5.patch, HBASE-7109-v5.patch
>
>
> The method of finding test classes only works on local build (or its full
> copy), not if the distribution is used.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7187) Create empty hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-7187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7187: - Attachment: HBASE-7187-0.patch Create the empty hbase-client module. > Create empty hbase-client module > > > Key: HBASE-7187 > URL: https://issues.apache.org/jira/browse/HBASE-7187 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7187-0.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7187) Create empty hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-7187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7187: - Status: Patch Available (was: Open) > Create empty hbase-client module > > > Key: HBASE-7187 > URL: https://issues.apache.org/jira/browse/HBASE-7187 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7187-0.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7109) integration tests on cluster are not getting picked up from distribution
[
https://issues.apache.org/jira/browse/HBASE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501622#comment-13501622
]
Hadoop QA commented on HBASE-7109:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554426/HBASE-7109-v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
99 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 24 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3381//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3381//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3381//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3381//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3381//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3381//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3381//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3381//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3381//console
This message is automatically generated.
> integration tests on cluster are not getting picked up from distribution
>
>
> Key: HBASE-7109
> URL: https://issues.apache.org/jira/browse/HBASE-7109
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Affects Versions: 0.96.0
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HBASE-7109-squashed.patch, HBASE-7109-v2-squashed.patch,
> HBASE-7109-v3-squashed.patch, HBASE-7109-v4-squashed.patch,
> HBASE-7109-v5.patch, HBASE-7109-v5.patch
>
>
> The method of finding test classes only works on local build (or its full
> copy), not if the distribution is used.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7199) hbck should check lingering reference hfile and have option to sideline them automatically
Jimmy Xiang created HBASE-7199: -- Summary: hbck should check lingering reference hfile and have option to sideline them automatically Key: HBASE-7199 URL: https://issues.apache.org/jira/browse/HBASE-7199 Project: HBase Issue Type: Improvement Components: hbck Reporter: Jimmy Xiang Sometimes, some lingering reference hfile points to some region doesn't exist any more. This will prevent the region to open. Hbck should be able to find these files and sideline them automatically if requested. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7109) integration tests on cluster are not getting picked up from distribution
[
https://issues.apache.org/jira/browse/HBASE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501616#comment-13501616
]
Hadoop QA commented on HBASE-7109:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554426/HBASE-7109-v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
99 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 24 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//console
This message is automatically generated.
> integration tests on cluster are not getting picked up from distribution
>
>
> Key: HBASE-7109
> URL: https://issues.apache.org/jira/browse/HBASE-7109
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Affects Versions: 0.96.0
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HBASE-7109-squashed.patch, HBASE-7109-v2-squashed.patch,
> HBASE-7109-v3-squashed.patch, HBASE-7109-v4-squashed.patch,
> HBASE-7109-v5.patch, HBASE-7109-v5.patch
>
>
> The method of finding test classes only works on local build (or its full
> copy), not if the distribution is used.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Open (was: Patch Available) > Writes should not block reads on blocking updates to memstores > -- > > Key: HBASE-6423 > URL: https://issues.apache.org/jira/browse/HBASE-6423 > Project: HBase > Issue Type: Bug >Reporter: Karthik Ranganathan >Assignee: Jimmy Xiang > Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, > trunk-6423_v2.patch > > > We have a big data use case where we turn off WAL and have a ton of reads and > writes. We found that: > 1. flushing a memstore takes a while (GZIP compression) > 2. incoming writes cause the new memstore to grow in an unbounded fashion > 3. this triggers blocking memstore updates > 4. in turn, this causes all the RPC handler threads to block on writes to > that memstore > 5. we are not able to read during this time as RPC handlers are blocked > At a higher level, we should not hold up the RPC threads while blocking > updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7185) Create hbase-protocol module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501603#comment-13501603 ] Hudson commented on HBASE-7185: --- Integrated in HBase-TRUNK #3554 (See [https://builds.apache.org/job/HBase-TRUNK/3554/]) HBASE-7185 Create hbase-protocol module (Revision 1411922) Result = FAILURE eclark : Files : * /hbase/trunk/hbase-protocol * /hbase/trunk/hbase-protocol/pom.xml * /hbase/trunk/hbase-protocol/src * /hbase/trunk/hbase-protocol/src/main * /hbase/trunk/hbase-protocol/src/main/java * /hbase/trunk/hbase-protocol/src/main/java/org * /hbase/trunk/hbase-protocol/src/main/java/org/apache * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AccessControlProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AggregateProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterIdProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterStatusProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ComparatorProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FSProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FilterProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/LoadBalancerProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterMonitorProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MultiRowMutation.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionServerStatusProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/Tracing.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf * /hbase/trunk/hbase-protocol/src/main/protobuf/AccessControl.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/Admin.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/Aggregate.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/BulkDelete.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/ClusterId.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/ClusterStatus.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/Comparator.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/Examples.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/FS.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/Filter.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/LoadBalancer.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/Master.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/MasterAdmin.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/MasterMonitor.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/MultiRowMutation.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/README.txt * /hbase/trunk/hbase-protocol/src/main/protobuf/RPC.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/RegionServerStatus.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/Tracing.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/ZooKeeper.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/hbase.proto * /hbase/trunk/hbase-server/pom.xml * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated * /hbase/trunk/hbase-server/src/main/protobuf * /hbase/trunk/pom.xml > Create hbase-protocol module > > > Key: HBASE-7185 > URL: https://issues.apache.org/jira/browse/HBASE-7185 > Project: HBase > Issue Type: Sub-task >Reporter: Ellio
[jira] [Commented] (HBASE-2376) Add special SnapshotScanner which presents view of all data at some time in the past
[ https://issues.apache.org/jira/browse/HBASE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501602#comment-13501602 ] Pritam Damania commented on HBASE-2376: --- Here are the relevant patches for this feature in the 89fb branch : http://svn.apache.org/viewvc?view=revision&revision=r1395032 http://svn.apache.org/viewvc?view=revision&revision=r1406789 http://svn.apache.org/viewvc?view=revision&revision=r1410118 > Add special SnapshotScanner which presents view of all data at some time in > the past > > > Key: HBASE-2376 > URL: https://issues.apache.org/jira/browse/HBASE-2376 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Affects Versions: 0.20.3 >Reporter: Jonathan Gray >Assignee: Pritam Damania > > In order to support a particular kind of database "snapshot" feature which > doesn't require copying data, we came up with the idea for a special > SnapshotScanner that would present a view of your data at some point in the > past. The primary use case for this would be to be able to recover > particular data/rows (but not all data, like a global rollback) should they > have somehow been messed up (application fault, application bug, user error, > etc.). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2376) Add special SnapshotScanner which presents view of all data at some time in the past
[ https://issues.apache.org/jira/browse/HBASE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kannan Muthukkaruppan updated HBASE-2376: - Assignee: Pritam Damania > Add special SnapshotScanner which presents view of all data at some time in > the past > > > Key: HBASE-2376 > URL: https://issues.apache.org/jira/browse/HBASE-2376 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Affects Versions: 0.20.3 >Reporter: Jonathan Gray >Assignee: Pritam Damania > > In order to support a particular kind of database "snapshot" feature which > doesn't require copying data, we came up with the idea for a special > SnapshotScanner that would present a view of your data at some point in the > past. The primary use case for this would be to be able to recover > particular data/rows (but not all data, like a global rollback) should they > have somehow been messed up (application fault, application bug, user error, > etc.). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7198) fix lastSeqId logic in regionserver
Amitanand Aiyer created HBASE-7198: -- Summary: fix lastSeqId logic in regionserver Key: HBASE-7198 URL: https://issues.apache.org/jira/browse/HBASE-7198 Project: HBase Issue Type: Bug Affects Versions: 0.89-fb Reporter: Amitanand Aiyer Priority: Minor Fix For: 0.89-fb -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7109) integration tests on cluster are not getting picked up from distribution
[ https://issues.apache.org/jira/browse/HBASE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7109: Attachment: HBASE-7109-v5.patch > integration tests on cluster are not getting picked up from distribution > > > Key: HBASE-7109 > URL: https://issues.apache.org/jira/browse/HBASE-7109 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-7109-squashed.patch, HBASE-7109-v2-squashed.patch, > HBASE-7109-v3-squashed.patch, HBASE-7109-v4-squashed.patch, > HBASE-7109-v5.patch, HBASE-7109-v5.patch > > > The method of finding test classes only works on local build (or its full > copy), not if the distribution is used. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7109) integration tests on cluster are not getting picked up from distribution
[ https://issues.apache.org/jira/browse/HBASE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7109: Attachment: HBASE-7109-v5.patch Added tests... unfortunately I didn't realize when I started how tedious mucking with jar files and ClassLoader is... But now it's there :) > integration tests on cluster are not getting picked up from distribution > > > Key: HBASE-7109 > URL: https://issues.apache.org/jira/browse/HBASE-7109 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-7109-squashed.patch, HBASE-7109-v2-squashed.patch, > HBASE-7109-v3-squashed.patch, HBASE-7109-v4-squashed.patch, > HBASE-7109-v5.patch > > > The method of finding test classes only works on local build (or its full > copy), not if the distribution is used. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7185) Create hbase-protocol module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7185: - Resolution: Fixed Release Note: Created hbase-protocol module to extract all protobuf files that could be transmitted. Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed revision 1411922. > Create hbase-protocol module > > > Key: HBASE-7185 > URL: https://issues.apache.org/jira/browse/HBASE-7185 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch, > HBASE-7185-2.patch > > > Create an hbase-rpc module where protobuf's that can cross the wire(either to > zk or from a server) will be stored. This should allow others to create > clients without relying on too much of the hbase code base. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[ https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501568#comment-13501568 ] Ted Yu commented on HBASE-7191: --- @Oleg: w.r.t. the test failures in TestHeapSize, did you run the tests using Java 1.7 ? I encountered the same failure on MacBook with 1.7.0_07-b10 > HBCK - Add offline create/fix hbase.version and hbase.id > - > > Key: HBASE-7191 > URL: https://issues.apache.org/jira/browse/HBASE-7191 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.94.1 >Reporter: Enis Soztutar > Attachments: 7191-2.patch > > > One of our clients run into a problem, in which they have the hbase.root.dir, > and cluster data, but their hbase.id and hbase.version files are corrupted. > HMaster creates those on start, but not if there is already existing data. > We can add smt like --fixIdFile, and ability for HBCK to do some offline > repairs for the version file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[ https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501530#comment-13501530 ] Oleg Zhurakousky commented on HBASE-7191: - [~yuzhih...@gmail.com] Sure, will do > HBCK - Add offline create/fix hbase.version and hbase.id > - > > Key: HBASE-7191 > URL: https://issues.apache.org/jira/browse/HBASE-7191 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.94.1 >Reporter: Enis Soztutar > Attachments: 7191-2.patch > > > One of our clients run into a problem, in which they have the hbase.root.dir, > and cluster data, but their hbase.id and hbase.version files are corrupted. > HMaster creates those on start, but not if there is already existing data. > We can add smt like --fixIdFile, and ability for HBCK to do some offline > repairs for the version file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[ https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501529#comment-13501529 ] Oleg Zhurakousky commented on HBASE-7191: - Thanks Deveraj The 3 failing tests are mentioned in the previous comment. Actually they pass in isolation, but not as a full build (unrelated to the patch). I'll check the indentation and resubmit. Let me know if there is anything else you'll see in the meantime. I'll also do another one anyway from the trunk > HBCK - Add offline create/fix hbase.version and hbase.id > - > > Key: HBASE-7191 > URL: https://issues.apache.org/jira/browse/HBASE-7191 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.94.1 >Reporter: Enis Soztutar > Attachments: 7191-2.patch > > > One of our clients run into a problem, in which they have the hbase.root.dir, > and cluster data, but their hbase.id and hbase.version files are corrupted. > HMaster creates those on start, but not if there is already existing data. > We can add smt like --fixIdFile, and ability for HBCK to do some offline > repairs for the version file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501528#comment-13501528
]
Ted Yu commented on HBASE-7191:
---
For OfflineVersionAndClusterIdFixTests.java, usually test class starts with
Test. Name it TestOfflineVersionAndClusterIdFix
It should carry the following annotation:
{code}
@Category(SmallTests.class)
{code}
{code}
- System.exit(success ? 0 : 1);
+ shutDownFsck(fsck);
{code}
Can you log the value of success above ?
For shutDownFsck(), can you add a method in HBaseFsck which shuts down executor
?
> HBCK - Add offline create/fix hbase.version and hbase.id
> -
>
> Key: HBASE-7191
> URL: https://issues.apache.org/jira/browse/HBASE-7191
> Project: HBase
> Issue Type: Improvement
> Components: hbck
>Affects Versions: 0.94.1
>Reporter: Enis Soztutar
> Attachments: 7191-2.patch
>
>
> One of our clients run into a problem, in which they have the hbase.root.dir,
> and cluster data, but their hbase.id and hbase.version files are corrupted.
> HMaster creates those on start, but not if there is already existing data.
> We can add smt like --fixIdFile, and ability for HBCK to do some offline
> repairs for the version file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[ https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501523#comment-13501523 ] Devaraj Das commented on HBASE-7191: Haven't looked at the patch in detail but some things that you should take care of: (1) indentation of code (two spaces, look at the existing code for getting a feel of how it should look like). (2) patch should be submitted for trunk first (ideally) There are flaky tests in the codebase. If you see an unrelated test failing on hudson, and the test passes locally, please make a note of it as a comment on jira. > HBCK - Add offline create/fix hbase.version and hbase.id > - > > Key: HBASE-7191 > URL: https://issues.apache.org/jira/browse/HBASE-7191 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.94.1 >Reporter: Enis Soztutar > Attachments: 7191-2.patch > > > One of our clients run into a problem, in which they have the hbase.root.dir, > and cluster data, but their hbase.id and hbase.version files are corrupted. > HMaster creates those on start, but not if there is already existing data. > We can add smt like --fixIdFile, and ability for HBCK to do some offline > repairs for the version file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7185) Create hbase-protocol module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501510#comment-13501510 ] stack commented on HBASE-7185: -- +1 > Create hbase-protocol module > > > Key: HBASE-7185 > URL: https://issues.apache.org/jira/browse/HBASE-7185 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch, > HBASE-7185-2.patch > > > Create an hbase-rpc module where protobuf's that can cross the wire(either to > zk or from a server) will be stored. This should allow others to create > clients without relying on too much of the hbase code base. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501489#comment-13501489
]
Oleg Zhurakousky commented on HBASE-7191:
-
Not sure what I can do. The test does pass but there are other failing tests:
{code}
Failed tests:
testSplitCalculatorEq(org.apache.hadoop.hbase.util.TestRegionSplitCalculator):
expected:<2> but was:<1>
testNativeSizes(org.apache.hadoop.hbase.io.TestHeapSize): expected:<56> but
was:<64>
testSizes(org.apache.hadoop.hbase.io.TestHeapSize): expected:<192> but
was:<200>
{code}
and looking at the Jenkins report there hasn't been a successful build since
October 30 (as far as I can see).
The irony is that for example TestRegionSplitCalculator test passes locally if
ran in isolation
> HBCK - Add offline create/fix hbase.version and hbase.id
> -
>
> Key: HBASE-7191
> URL: https://issues.apache.org/jira/browse/HBASE-7191
> Project: HBase
> Issue Type: Improvement
> Components: hbck
>Affects Versions: 0.94.1
>Reporter: Enis Soztutar
> Attachments: 7191-2.patch
>
>
> One of our clients run into a problem, in which they have the hbase.root.dir,
> and cluster data, but their hbase.id and hbase.version files are corrupted.
> HMaster creates those on start, but not if there is already existing data.
> We can add smt like --fixIdFile, and ability for HBCK to do some offline
> repairs for the version file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6802) Export Snapshot
[ https://issues.apache.org/jira/browse/HBASE-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-6802: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to snapshot branch > Export Snapshot > --- > > Key: HBASE-6802 > URL: https://issues.apache.org/jira/browse/HBASE-6802 > Project: HBase > Issue Type: Sub-task > Components: snapshots >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Fix For: hbase-6055, 0.96.0 > > Attachments: HBASE-6802-v1.patch > > > Export a snapshot to another cluster. > - Copy the .snapshot/name folder with all the references > - Copy the hfiles/hlogs needed by the snapshot > Once the other cluster has the files and the snapshot information it can > restore the snapshot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501462#comment-13501462
]
Hadoop QA commented on HBASE-7191:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554407/7191-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3379//console
This message is automatically generated.
> HBCK - Add offline create/fix hbase.version and hbase.id
> -
>
> Key: HBASE-7191
> URL: https://issues.apache.org/jira/browse/HBASE-7191
> Project: HBase
> Issue Type: Improvement
> Components: hbck
>Affects Versions: 0.94.1
>Reporter: Enis Soztutar
> Attachments: 7191-2.patch
>
>
> One of our clients run into a problem, in which they have the hbase.root.dir,
> and cluster data, but their hbase.id and hbase.version files are corrupted.
> HMaster creates those on start, but not if there is already existing data.
> We can add smt like --fixIdFile, and ability for HBCK to do some offline
> repairs for the version file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[ https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated HBASE-7191: Attachment: 7191-2.patch The new patch includes modified hbase-site.xml file which has hbase.root set which is required by the test > HBCK - Add offline create/fix hbase.version and hbase.id > - > > Key: HBASE-7191 > URL: https://issues.apache.org/jira/browse/HBASE-7191 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.94.1 >Reporter: Enis Soztutar > Attachments: 7191-2.patch > > > One of our clients run into a problem, in which they have the hbase.root.dir, > and cluster data, but their hbase.id and hbase.version files are corrupted. > HMaster creates those on start, but not if there is already existing data. > We can add smt like --fixIdFile, and ability for HBCK to do some offline > repairs for the version file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[ https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated HBASE-7191: Attachment: (was: 7191.patch) > HBCK - Add offline create/fix hbase.version and hbase.id > - > > Key: HBASE-7191 > URL: https://issues.apache.org/jira/browse/HBASE-7191 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.94.1 >Reporter: Enis Soztutar > > One of our clients run into a problem, in which they have the hbase.root.dir, > and cluster data, but their hbase.id and hbase.version files are corrupted. > HMaster creates those on start, but not if there is already existing data. > We can add smt like --fixIdFile, and ability for HBCK to do some offline > repairs for the version file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6230) [brainstorm] "Restore" snapshots for HBase 0.96
[ https://issues.apache.org/jira/browse/HBASE-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates resolved HBASE-6230. Resolution: Fixed Committed to https://github.com/jyates/hbase/tree/snapshots. > [brainstorm] "Restore" snapshots for HBase 0.96 > --- > > Key: HBASE-6230 > URL: https://issues.apache.org/jira/browse/HBASE-6230 > Project: HBase > Issue Type: Brainstorming >Reporter: Jesse Yates >Assignee: Matteo Bertozzi > Attachments: HBASE-6230-v1.patch, HBASE-6230-v2.patch, > HBASE-6230-v3.patch, SnapshotRestore-v0.pdf > > > Discussion ticket around the definitions/expectations of different parts of > snapshot restoration. This is complementary, but separate from the _how_ of > taking a snapshot of a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501442#comment-13501442
]
Hadoop QA commented on HBASE-7191:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554402/7191.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3378//console
This message is automatically generated.
> HBCK - Add offline create/fix hbase.version and hbase.id
> -
>
> Key: HBASE-7191
> URL: https://issues.apache.org/jira/browse/HBASE-7191
> Project: HBase
> Issue Type: Improvement
> Components: hbck
>Affects Versions: 0.94.1
>Reporter: Enis Soztutar
> Attachments: 7191.patch
>
>
> One of our clients run into a problem, in which they have the hbase.root.dir,
> and cluster data, but their hbase.id and hbase.version files are corrupted.
> HMaster creates those on start, but not if there is already existing data.
> We can add smt like --fixIdFile, and ability for HBCK to do some offline
> repairs for the version file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zhurakousky updated HBASE-7191:
Attachment: 7191.patch
Attaching a patch which basically adds two options to OfflineMetaRepair
{code}
$ hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair -fixVersionFile
$ hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair -fixClusterIdFile
{code}
Also fixed a few other minor things such as System.exit(..) which prevented
proper testing and quite frankly not needed as access to the executor can still
be obtain via Reflection.
> HBCK - Add offline create/fix hbase.version and hbase.id
> -
>
> Key: HBASE-7191
> URL: https://issues.apache.org/jira/browse/HBASE-7191
> Project: HBase
> Issue Type: Improvement
> Components: hbck
>Affects Versions: 0.94.1
>Reporter: Enis Soztutar
> Attachments: 7191.patch
>
>
> One of our clients run into a problem, in which they have the hbase.root.dir,
> and cluster data, but their hbase.id and hbase.version files are corrupted.
> HMaster creates those on start, but not if there is already existing data.
> We can add smt like --fixIdFile, and ability for HBCK to do some offline
> repairs for the version file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7009) Port HBaseCluster interface/tests to 0.94
[
https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501437#comment-13501437
]
Hadoop QA commented on HBASE-7009:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554396/HBASE-7009-v3-wo-classfinder.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 71 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3377//console
This message is automatically generated.
> Port HBaseCluster interface/tests to 0.94
> -
>
> Key: HBASE-7009
> URL: https://issues.apache.org/jira/browse/HBASE-7009
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Affects Versions: 0.94.3
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 0.94.4
>
> Attachments: HBASE-7009-p1.patch, HBASE-7009.patch,
> HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch,
> HBASE-7009-v3-wo-classfinder.patch
>
>
> Need to port. I am porting V5 patch from the original JIRA; I have a
> partially ported (V3) patch from Enis with protocol buffers being reverted to
> HRegionInterface/HMasterInterface
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[ https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated HBASE-7191: Affects Version/s: (was: 0.96.0) 0.94.1 Status: Patch Available (was: Open) > HBCK - Add offline create/fix hbase.version and hbase.id > - > > Key: HBASE-7191 > URL: https://issues.apache.org/jira/browse/HBASE-7191 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.94.1 >Reporter: Enis Soztutar > > One of our clients run into a problem, in which they have the hbase.root.dir, > and cluster data, but their hbase.id and hbase.version files are corrupted. > HMaster creates those on start, but not if there is already existing data. > We can add smt like --fixIdFile, and ability for HBCK to do some offline > repairs for the version file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6230) [brainstorm] "Restore" snapshots for HBase 0.96
[ https://issues.apache.org/jira/browse/HBASE-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-6230: --- Attachment: HBASE-6230-v3.patch > [brainstorm] "Restore" snapshots for HBase 0.96 > --- > > Key: HBASE-6230 > URL: https://issues.apache.org/jira/browse/HBASE-6230 > Project: HBase > Issue Type: Brainstorming >Reporter: Jesse Yates >Assignee: Matteo Bertozzi > Attachments: HBASE-6230-v1.patch, HBASE-6230-v2.patch, > HBASE-6230-v3.patch, SnapshotRestore-v0.pdf > > > Discussion ticket around the definitions/expectations of different parts of > snapshot restoration. This is complementary, but separate from the _how_ of > taking a snapshot of a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6777) Snapshot Restore interface
[ https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-6777: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to snapshot branch > Snapshot Restore interface > -- > > Key: HBASE-6777 > URL: https://issues.apache.org/jira/browse/HBASE-6777 > Project: HBase > Issue Type: Sub-task > Components: Client, master, snapshots >Affects Versions: 0.96.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Fix For: hbase-6055 > > Attachments: HBASE-6777-v10.patch, hbase-6777-v9.patch, > HBASE--v1.patch > > > Add interfaces for restoring a snapshot -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6777) Snapshot Restore interface
[ https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501431#comment-13501431 ] Jesse Yates commented on HBASE-6777: [~mbertozzi] hmm, odd. When I tried applying the first time it was getting mad at the HMaster. Oh well, thanks for the rebase! > Snapshot Restore interface > -- > > Key: HBASE-6777 > URL: https://issues.apache.org/jira/browse/HBASE-6777 > Project: HBase > Issue Type: Sub-task > Components: Client, master, snapshots >Affects Versions: 0.96.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Fix For: hbase-6055 > > Attachments: HBASE-6777-v10.patch, hbase-6777-v9.patch, > HBASE--v1.patch > > > Add interfaces for restoring a snapshot -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6777) Snapshot Restore interface
[
https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501430#comment-13501430
]
Hadoop QA commented on HBASE-6777:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554397/HBASE-6777-v10.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3375//console
This message is automatically generated.
> Snapshot Restore interface
> --
>
> Key: HBASE-6777
> URL: https://issues.apache.org/jira/browse/HBASE-6777
> Project: HBase
> Issue Type: Sub-task
> Components: Client, master, snapshots
>Affects Versions: 0.96.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: hbase-6055
>
> Attachments: HBASE-6777-v10.patch, hbase-6777-v9.patch,
> HBASE--v1.patch
>
>
> Add interfaces for restoring a snapshot
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7009) Port HBaseCluster interface/tests to 0.94
[
https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501429#comment-13501429
]
Hadoop QA commented on HBASE-7009:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554396/HBASE-7009-v3-wo-classfinder.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 71 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3376//console
This message is automatically generated.
> Port HBaseCluster interface/tests to 0.94
> -
>
> Key: HBASE-7009
> URL: https://issues.apache.org/jira/browse/HBASE-7009
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Affects Versions: 0.94.3
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 0.94.4
>
> Attachments: HBASE-7009-p1.patch, HBASE-7009.patch,
> HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch,
> HBASE-7009-v3-wo-classfinder.patch
>
>
> Need to port. I am porting V5 patch from the original JIRA; I have a
> partially ported (V3) patch from Enis with protocol buffers being reverted to
> HRegionInterface/HMasterInterface
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7185) Create hbase-protocol module
[
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501424#comment-13501424
]
Hadoop QA commented on HBASE-7185:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554391/HBASE-7185-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
99 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 24 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestSplitTransaction
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//console
This message is automatically generated.
> Create hbase-protocol module
>
>
> Key: HBASE-7185
> URL: https://issues.apache.org/jira/browse/HBASE-7185
> Project: HBase
> Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.96.0
>
> Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch,
> HBASE-7185-2.patch
>
>
> Create an hbase-rpc module where protobuf's that can cross the wire(either to
> zk or from a server) will be stored. This should allow others to create
> clients without relying on too much of the hbase code base.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6777) Snapshot Restore interface
[ https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-6777: --- Attachment: HBASE-6777-v10.patch Attached new patch, no changes just regenerated the MasterAdminProtos also the branch with the patch applied is here: https://github.com/matteobertozzi/hbase/commits/offline-snapshot-review-v4 > Snapshot Restore interface > -- > > Key: HBASE-6777 > URL: https://issues.apache.org/jira/browse/HBASE-6777 > Project: HBase > Issue Type: Sub-task > Components: Client, master, snapshots >Affects Versions: 0.96.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Fix For: hbase-6055 > > Attachments: HBASE-6777-v10.patch, hbase-6777-v9.patch, > HBASE--v1.patch > > > Add interfaces for restoring a snapshot -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7009) Port HBaseCluster interface/tests to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7009: Attachment: HBASE-7009-v3-wo-classfinder.patch > Port HBaseCluster interface/tests to 0.94 > - > > Key: HBASE-7009 > URL: https://issues.apache.org/jira/browse/HBASE-7009 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.94.3 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.4 > > Attachments: HBASE-7009-p1.patch, HBASE-7009.patch, > HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch, > HBASE-7009-v3-wo-classfinder.patch > > > Need to port. I am porting V5 patch from the original JIRA; I have a > partially ported (V3) patch from Enis with protocol buffers being reverted to > HRegionInterface/HMasterInterface -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7185) Create hbase-protocol module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7185: - Attachment: HBASE-7185-2.patch Moved to hbase-protocol > Create hbase-protocol module > > > Key: HBASE-7185 > URL: https://issues.apache.org/jira/browse/HBASE-7185 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch, > HBASE-7185-2.patch > > > Create an hbase-rpc module where protobuf's that can cross the wire(either to > zk or from a server) will be stored. This should allow others to create > clients without relying on too much of the hbase code base. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7185) Create hbase-protocol module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7185: - Summary: Create hbase-protocol module (was: Create hbase-rpc module) > Create hbase-protocol module > > > Key: HBASE-7185 > URL: https://issues.apache.org/jira/browse/HBASE-7185 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch, > HBASE-7185-2.patch > > > Create an hbase-rpc module where protobuf's that can cross the wire(either to > zk or from a server) will be stored. This should allow others to create > clients without relying on too much of the hbase code base. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6055) Snapshots in HBase 0.96
[ https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-6055: -- Comment: was deleted (was: I will be out of the office on November 19 - 25, 2012 with limited access to email. If this is a support issue please use the appropriate on-call procedures.[1] If this is an emergency please contact Greg Whitsitt. [1] - http://wiki.cloud.cerner.corp/index.php/Big_Data/On_Call_Support CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024. ) > Snapshots in HBase 0.96 > --- > > Key: HBASE-6055 > URL: https://issues.apache.org/jira/browse/HBASE-6055 > Project: HBase > Issue Type: New Feature > Components: Client, master, regionserver, snapshots, Zookeeper >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: hbase-6055, 0.96.0 > > Attachments: Snapshots in HBase.docx > > > Continuation of HBASE-50 for the current trunk. Since the implementation has > drastically changed, opening as a new ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96
[ https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501391#comment-13501391 ] Micah Whitacre commented on HBASE-6055: --- I will be out of the office on November 19 - 25, 2012 with limited access to email. If this is a support issue please use the appropriate on-call procedures.[1] If this is an emergency please contact Greg Whitsitt. [1] - http://wiki.cloud.cerner.corp/index.php/Big_Data/On_Call_Support CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024. > Snapshots in HBase 0.96 > --- > > Key: HBASE-6055 > URL: https://issues.apache.org/jira/browse/HBASE-6055 > Project: HBase > Issue Type: New Feature > Components: Client, master, regionserver, snapshots, Zookeeper >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: hbase-6055, 0.96.0 > > Attachments: Snapshots in HBase.docx > > > Continuation of HBASE-50 for the current trunk. Since the implementation has > drastically changed, opening as a new ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96
[ https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501387#comment-13501387 ] Jonathan Hsieh commented on HBASE-6055: --- Here's a repo with the online branch merged to the snapshot branch with snapshot unit tests passing now. https://github.com/jmhsieh/hbase/tree/snapshots-online-merge hash d1299347c0c1afcc0264b14ee12beee170efc4c2 mvn clean mvn test -Dtest=errorhandling/* -PlocalTests mvn test -Dtest=Test*Procedure* -PlocalTests mvn test -Dtest=Test*Snapshot*,snapshot/*,TestFSUtils -PlocalTests It needs some cleanup (cleanup duplicate/commented code from merge) but patches of pieces will be coming out later today / tommorrow. Likely pieces: ( might break a few more of these down if they are excessive) * External Exceptions + snapshot manager refactor * Barrier Procedure * online Timestamp snapshots tasks * online timestampe snapshots + snapshot manager > Snapshots in HBase 0.96 > --- > > Key: HBASE-6055 > URL: https://issues.apache.org/jira/browse/HBASE-6055 > Project: HBase > Issue Type: New Feature > Components: Client, master, regionserver, snapshots, Zookeeper >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: hbase-6055, 0.96.0 > > Attachments: Snapshots in HBase.docx > > > Continuation of HBASE-50 for the current trunk. Since the implementation has > drastically changed, opening as a new ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7180) RegionScannerImpl.next() is inefficient.
[ https://issues.apache.org/jira/browse/HBASE-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501375#comment-13501375 ] Lars Hofhansl commented on HBASE-7180: -- Yeah I don't like it either. We somehow need to expose more internals to coprocessors in a clean way. * KeyValue already is needed for RegionScanner, since it extends internal scanner. * start/closeRegionOperation should be available to coprocessors anyway (I think). Otherwise it is hard to implement these types of things in coprocessors. * I mainly do not like nextInternal on the interface. Is there a better way to expose the inner workings of RegionScannerImpl to avoid expensive setup at each iteration? Another option is to keep the RegionScanner interface as it, and just make these methods public in RegionScannerImpl. A coprocessor can then cast the RegionScanner to RegionScannerImpl and access the stuff it needs. > RegionScannerImpl.next() is inefficient. > > > Key: HBASE-7180 > URL: https://issues.apache.org/jira/browse/HBASE-7180 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > Attachments: 7180-0.94-SKETCH.txt, 7180-0.94-v1.txt > > > We just came across a special scenario. > For our Phoenix project (SQL runtime for HBase), we push a lot of work into > HBase via coprocessors. One method is to wrap RegionScanner in coprocessor > hooks and then do processing in the hook to avoid returning a lot of data to > the client unnecessarily. > In this specific case this is pretty bad. Since the wrapped RegionScanner's > next() does not "know" that it is called this way is still does all of this > on each invocation: > # Starts a RegionOperation > # Increments the request count > # set the current read point on a thread local (because generally each call > could come from a different thread) > # Finally does the next on its StoreScanner(s) > # Ends the RegionOperation > When this is done in a tight loop millions of times (as is the case for us) > it starts to become significant. > Not sure what to do about this, really. Opening this issue for discussion. > One way is to extend the RegionScanner with an "internal" next() method of > sorts, so that all this overhead can be avoided. The coprocessor could call > the regular next() methods once and then just call the cheaper internal > version. > Are there better/cleaner ways? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7009) Port HBaseCluster interface/tests to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501373#comment-13501373 ] Sergey Shelukhin commented on HBASE-7009: - I am going to remove the backport of 7109 from the patch (it was started here but review in 7109 resulted in changes, so it's easier to commit them separately, 7109 in 7109 and 6241 here). > Port HBaseCluster interface/tests to 0.94 > - > > Key: HBASE-7009 > URL: https://issues.apache.org/jira/browse/HBASE-7009 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.94.3 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.4 > > Attachments: HBASE-7009-p1.patch, HBASE-7009.patch, > HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch > > > Need to port. I am porting V5 patch from the original JIRA; I have a > partially ported (V3) patch from Enis with protocol buffers being reverted to > HRegionInterface/HMasterInterface -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6777) Snapshot Restore interface
[ https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501371#comment-13501371 ] Jesse Yates commented on HBASE-6777: Looks like this is a little behind the current branch; I guess with the changes to HMaster from offline, this got a little off. I took the opportunity to rebase the snapshot branch onto the current trunk (its pushed up now). Any chance you can rebase this patch Matteo (and maybe HBASE-6230 on this as well)? > Snapshot Restore interface > -- > > Key: HBASE-6777 > URL: https://issues.apache.org/jira/browse/HBASE-6777 > Project: HBase > Issue Type: Sub-task > Components: Client, master, snapshots >Affects Versions: 0.96.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Fix For: hbase-6055 > > Attachments: hbase-6777-v9.patch, HBASE--v1.patch > > > Add interfaces for restoring a snapshot -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7185) Create hbase-rpc module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501365#comment-13501365 ] Elliott Clark commented on HBASE-7185: -- Talked with stack and I think we both agree that rpc is probably a bad name for this module. Thinking hbase-protocol would be better. It would convey more that this module contains only the code of protobuf's that are pushed across the wire to ZK and to/from the servers. That's also why the ipc namespace is not moved into this module. The ipc module it much more tied to our client and server implementations. I wanted to make it possible for an external client to use this module. The skipping stuff is copied from the rest of the modules. -Dskip-server-tests will skip the server tests, -Dskip-common-tests will skip common etc. > Create hbase-rpc module > --- > > Key: HBASE-7185 > URL: https://issues.apache.org/jira/browse/HBASE-7185 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch > > > Create an hbase-rpc module where protobuf's that can cross the wire(either to > zk or from a server) will be stored. This should allow others to create > clients without relying on too much of the hbase code base. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2231) Compaction events should be written to HLog
[ https://issues.apache.org/jira/browse/HBASE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501364#comment-13501364 ] stack commented on HBASE-2231: -- So, test passes because we are able to read from deleted file because dfsclient has block locations cached. I suppose this makes sense it is just a little unexpected (It took me a while to figure it). Not sure how test failed in the past. My guess is that in the past we were strict when you went to compact that all referenced files had to be present. That is no longer the case in that we seem to be loosey-goosey about it to handle the case where a compaction may have run before ours. Let me look into this more. > Compaction events should be written to HLog > --- > > Key: HBASE-2231 > URL: https://issues.apache.org/jira/browse/HBASE-2231 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Labels: moved_from_0_20_5 > Fix For: 0.96.0 > > Attachments: 2231-testcase-0.94.txt, 2231-testcase_v2.txt, > 2231v2.txt, hbase-2231-testcase.txt, hbase-2231.txt > > > The sequence for a compaction should look like this: > # Compact region to "new" files > # Write a "Compacted Region" entry to the HLog > # Delete "old" files > This deals with a case where the RS has paused between step 1 and 2 and the > regions have since been reassigned. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7185) Create hbase-rpc module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501356#comment-13501356 ] stack commented on HBASE-7185: -- Highlevel, this is rpc module so why the pb stuff in here? Its scope is messy when it has stuff like ClusterIdProtos and AggregateProtos and zk protos? Did you have to pull in stuff to make things like AggregateProtos happy? Is it odd having RegionServerProto in here, in this module that hbase-server, another module downstream on the dependency river, depends on? Any particular reason you need it beyond what is listed in issue? It does not move the content of the rpc package? Just the pbs? So the below makes it so I can skip unit tests in rpc? skip-rpc-tests Its a profile? -Pskip-rpc-tests Or I do it w/ -Dskip-rpc-tests=true Should be in release notes how to do above. If above is correct, why doesn't skipTests work for this module? It does but you just want flag for skipping rpcs? > Create hbase-rpc module > --- > > Key: HBASE-7185 > URL: https://issues.apache.org/jira/browse/HBASE-7185 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch > > > Create an hbase-rpc module where protobuf's that can cross the wire(either to > zk or from a server) will be stored. This should allow others to create > clients without relying on too much of the hbase code base. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7180) RegionScannerImpl.next() is inefficient.
[ https://issues.apache.org/jira/browse/HBASE-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501355#comment-13501355 ] stack commented on HBASE-7180: -- getReadPt has to be public? nextInternal has to be public too? closeRegionOperation and startRegionOperation too? So CPs can get at them? You add KeyValue to RegionScanner when it did not need it previous. You have to? And a nextInternal in an Interface like RegionScanner seems wrong? I am for speedup but as you say, this one is ugly > RegionScannerImpl.next() is inefficient. > > > Key: HBASE-7180 > URL: https://issues.apache.org/jira/browse/HBASE-7180 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > Attachments: 7180-0.94-SKETCH.txt, 7180-0.94-v1.txt > > > We just came across a special scenario. > For our Phoenix project (SQL runtime for HBase), we push a lot of work into > HBase via coprocessors. One method is to wrap RegionScanner in coprocessor > hooks and then do processing in the hook to avoid returning a lot of data to > the client unnecessarily. > In this specific case this is pretty bad. Since the wrapped RegionScanner's > next() does not "know" that it is called this way is still does all of this > on each invocation: > # Starts a RegionOperation > # Increments the request count > # set the current read point on a thread local (because generally each call > could come from a different thread) > # Finally does the next on its StoreScanner(s) > # Ends the RegionOperation > When this is done in a tight loop millions of times (as is the case for us) > it starts to become significant. > Not sure what to do about this, really. Opening this issue for discussion. > One way is to extend the RegionScanner with an "internal" next() method of > sorts, so that all this overhead can be avoided. The coprocessor could call > the regular next() methods once and then just call the cheaper internal > version. > Are there better/cleaner ways? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501354#comment-13501354 ] stack commented on HBASE-6423: -- Needs release note of new config. You think Long.MAX_VALUE as default? In trunk why not have it a reasonable value and in 0.94 have your MAX_VALUE? Should default be socket timeout? What is this comment about? +// block waiting for the lock for closing lock.writeLock().lock(); No busy waiting when closing? I see the comment in a few places. Removing getRecentFlushInfo is ok because HRegion is annotated private? This is a little ugly: + int multiplier = Math.min(acquiredLocks.size(), maxBusyWaitMultiplier); + lock(this.updatesLock.readLock(), multiplier); You pass in multiplier. Why not pass in calculated time to busy wait? Have a little function that consults multiplier and does Math.min, etc., and figures how long to wait? + int multiplier = Math.min(acquiredLocks.size(), maxBusyWaitMultiplier); For example, put the above into the lock method? Pass in size? Fix your capialization of the 'N' in + * @throws RegioNTooBusyException if failed to get the lock in time Can we backport? So write locks are unbounded, its just the reads that we'll give up on. How does this address Karthik's original comment in the issue? Reads fail faster releasing handlers so we can read/write against other memstores other than the one that has a write lock on it (and is currently at its limit)? Good patch Jimmy > Writes should not block reads on blocking updates to memstores > -- > > Key: HBASE-6423 > URL: https://issues.apache.org/jira/browse/HBASE-6423 > Project: HBase > Issue Type: Bug >Reporter: Karthik Ranganathan >Assignee: Jimmy Xiang > Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, > trunk-6423_v2.patch > > > We have a big data use case where we turn off WAL and have a ton of reads and > writes. We found that: > 1. flushing a memstore takes a while (GZIP compression) > 2. incoming writes cause the new memstore to grow in an unbounded fashion > 3. this triggers blocking memstore updates > 4. in turn, this causes all the RPC handler threads to block on writes to > that memstore > 5. we are not able to read during this time as RPC handlers are blocked > At a higher level, we should not hold up the RPC threads while blocking > updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7197) Add multi get to RemoteHTable
[ https://issues.apache.org/jira/browse/HBASE-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7197: - Component/s: REST Client Affects Version/s: 0.96.0 0.94.2 > Add multi get to RemoteHTable > - > > Key: HBASE-7197 > URL: https://issues.apache.org/jira/browse/HBASE-7197 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.2, 0.96.0 >Reporter: Elliott Clark >Priority: Minor > > REST server supports multi-get, so the RemoteHTable class should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7197) Add multi get to RemoteHTable
Elliott Clark created HBASE-7197: Summary: Add multi get to RemoteHTable Key: HBASE-7197 URL: https://issues.apache.org/jira/browse/HBASE-7197 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Priority: Minor REST server supports multi-get, so the RemoteHTable class should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6580) New HTable pool, based on HBase(byte[], HConnection, ExecutorService) constructor
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501343#comment-13501343 ] Gary Helmling commented on HBASE-6580: -- bq. Maybe HConnection should just finally get a getTable(...) method. +1. Using HConnection to get an HTable seems more intuitive and more composable than our old monolithic HTable. Lightweight HTables that are easily GC'd would also be very nice. Agree that at that point we don't really need HTablePool. > New HTable pool, based on HBase(byte[], HConnection, ExecutorService) > constructor > - > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.2, 0.94.2 >Reporter: Lars Hofhansl >Priority: Minor > Attachments: HBASE-6580_v1.patch > > > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running
[ https://issues.apache.org/jira/browse/HBASE-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501335#comment-13501335 ] Oleg Zhurakousky commented on HBASE-7196: - It would be great if that was the case, but unfortunately it is not and that is what I've been trying to explain. Based on the current code HBaseAdmin instance can NOT be created unless master is running. > 'hbase shell status' throws exception when HBase is not running > --- > > Key: HBASE-7196 > URL: https://issues.apache.org/jira/browse/HBASE-7196 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.1 >Reporter: Oleg Zhurakousky > > Its kind of a nuisance bug. One would assume that 'status' command should > simply return something along the lines of "HBase is not running" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running
[ https://issues.apache.org/jira/browse/HBASE-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501327#comment-13501327 ] Ted Yu commented on HBASE-7196: --- I think the purpose of isMasterRunning() is for checking whether master is running after HBaseAdmin instance is successfully obtained. > 'hbase shell status' throws exception when HBase is not running > --- > > Key: HBASE-7196 > URL: https://issues.apache.org/jira/browse/HBASE-7196 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.1 >Reporter: Oleg Zhurakousky > > Its kind of a nuisance bug. One would assume that 'status' command should > simply return something along the lines of "HBase is not running" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[ https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501323#comment-13501323 ] Jimmy Xiang commented on HBASE-7191: We can add -fixVersionFile to the offline meta repair tool. > HBCK - Add offline create/fix hbase.version and hbase.id > - > > Key: HBASE-7191 > URL: https://issues.apache.org/jira/browse/HBASE-7191 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.96.0 >Reporter: Enis Soztutar > > One of our clients run into a problem, in which they have the hbase.root.dir, > and cluster data, but their hbase.id and hbase.version files are corrupted. > HMaster creates those on start, but not if there is already existing data. > We can add smt like --fixIdFile, and ability for HBCK to do some offline > repairs for the version file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running
[ https://issues.apache.org/jira/browse/HBASE-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501322#comment-13501322 ] Oleg Zhurakousky commented on HBASE-7196: - [~yuzhih...@gmail.com] That is different. Delete may or may not succeed, while during the deletion the IOException may be thrown. In the above case we have a method that by its name and thus definition suppose to return 'true' if Master is running and 'false' if Master is not running. And it doesn't do that. And as described above it its meaningless since it can never be called unless Master is running which is the prerequisite for creating HBaseAdmin instance. But once the instance is created its already implied that Master is running. Don't you agree? > 'hbase shell status' throws exception when HBase is not running > --- > > Key: HBASE-7196 > URL: https://issues.apache.org/jira/browse/HBASE-7196 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.1 >Reporter: Oleg Zhurakousky > > Its kind of a nuisance bug. One would assume that 'status' command should > simply return something along the lines of "HBase is not running" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running
[
https://issues.apache.org/jira/browse/HBASE-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501313#comment-13501313
]
Oleg Zhurakousky commented on HBASE-7196:
-
Different question.
Is there a reason why HBaseAdmin initializes itself in the constructor? It
makes the above problem even more counter intuitive:
The javadoc for the constructor says:
{code}
@throws MasterNotRunningException if the master is not running
{code}
which means the _isMasterRunning()_ method is meaningless since you can never
call it unless you have a valid instance of HBaseAdmin, but you'll never have
one unless master is running.
Wouldn't it be better to always create HBaseAdmin and perform check for running
master when necessary (of course without exception being thrown by the
isMasterRunning() method)
> 'hbase shell status' throws exception when HBase is not running
> ---
>
> Key: HBASE-7196
> URL: https://issues.apache.org/jira/browse/HBASE-7196
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.1
>Reporter: Oleg Zhurakousky
>
> Its kind of a nuisance bug. One would assume that 'status' command should
> simply return something along the lines of "HBase is not running"
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7146) Fix the wrong reference to getReader survived in theFSHLog javadoc
[
https://issues.apache.org/jira/browse/HBASE-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501308#comment-13501308
]
Gustavo Anatoly commented on HBASE-7146:
Hi, Nicolas.
It's ok, to me. I don't have objections.
Thanks for your review.
> Fix the wrong reference to getReader survived in theFSHLog javadoc
> --
>
> Key: HBASE-7146
> URL: https://issues.apache.org/jira/browse/HBASE-7146
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Affects Versions: 0.96.0
>Reporter: nkeywal
>Assignee: Gustavo Anatoly
>Priority: Trivial
> Attachments: HBASE-7146.patch
>
>
> In FSHLog, a wrong reference to getReader survived in the javadoc:
> {noformat}
> * To read an HLog, call {@link #getReader(org.apache.hadoop.fs.FileSystem,
> * org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration)}.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6580) New HTable pool, based on HBase(byte[], HConnection, ExecutorService) constructor
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501302#comment-13501302 ] Lars Hofhansl commented on HBASE-6580: -- Not really. After HBASE-4805 you'd create a single HConnection (via HConnectionManager.createConnection(...)) and then use this for each HTable created afterwards. HTable creation is very cheap when both HConnection and ExecutorService are passed... Cheaper indeed than then the HTablePool protocol. Maybe HConnection should just finally get a getTable(...) method. > New HTable pool, based on HBase(byte[], HConnection, ExecutorService) > constructor > - > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.2, 0.94.2 >Reporter: Lars Hofhansl >Priority: Minor > Attachments: HBASE-6580_v1.patch > > > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running
[
https://issues.apache.org/jira/browse/HBASE-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501257#comment-13501257
]
Ted Yu commented on HBASE-7196:
---
Take a look at this method in FileSystem from hadoop:
{code}
public abstract boolean delete(Path f) throws IOException;
{code}
> 'hbase shell status' throws exception when HBase is not running
> ---
>
> Key: HBASE-7196
> URL: https://issues.apache.org/jira/browse/HBASE-7196
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.1
>Reporter: Oleg Zhurakousky
>
> Its kind of a nuisance bug. One would assume that 'status' command should
> simply return something along the lines of "HBase is not running"
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running
[
https://issues.apache.org/jira/browse/HBASE-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501230#comment-13501230
]
Oleg Zhurakousky commented on HBASE-7196:
-
Also
{code}
HBaseAdmin.isMasterRunning()
{code}
is throwing an exception if Master is not running. By definition boolean
methods should never throw an exception, don't you agree?
> 'hbase shell status' throws exception when HBase is not running
> ---
>
> Key: HBASE-7196
> URL: https://issues.apache.org/jira/browse/HBASE-7196
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.1
>Reporter: Oleg Zhurakousky
>
> Its kind of a nuisance bug. One would assume that 'status' command should
> simply return something along the lines of "HBase is not running"
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7146) Fix the wrong reference to getReader survived in theFSHLog javadoc
[
https://issues.apache.org/jira/browse/HBASE-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nkeywal updated HBASE-7146:
---
Assignee: Gustavo Anatoly
> Fix the wrong reference to getReader survived in theFSHLog javadoc
> --
>
> Key: HBASE-7146
> URL: https://issues.apache.org/jira/browse/HBASE-7146
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Affects Versions: 0.96.0
>Reporter: nkeywal
>Assignee: Gustavo Anatoly
>Priority: Trivial
> Attachments: HBASE-7146.patch
>
>
> In FSHLog, a wrong reference to getReader survived in the javadoc:
> {noformat}
> * To read an HLog, call {@link #getReader(org.apache.hadoop.fs.FileSystem,
> * org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration)}.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7146) Fix the wrong reference to getReader survived in theFSHLog javadoc
[
https://issues.apache.org/jira/browse/HBASE-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501223#comment-13501223
]
nkeywal commented on HBASE-7146:
Hi Gustavo,
I think the new doc is ok. I will commit this if no ones object.
Thanks;
Nicolas
> Fix the wrong reference to getReader survived in theFSHLog javadoc
> --
>
> Key: HBASE-7146
> URL: https://issues.apache.org/jira/browse/HBASE-7146
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Affects Versions: 0.96.0
>Reporter: nkeywal
>Priority: Trivial
> Attachments: HBASE-7146.patch
>
>
> In FSHLog, a wrong reference to getReader survived in the javadoc:
> {noformat}
> * To read an HLog, call {@link #getReader(org.apache.hadoop.fs.FileSystem,
> * org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration)}.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running
Oleg Zhurakousky created HBASE-7196: --- Summary: 'hbase shell status' throws exception when HBase is not running Key: HBASE-7196 URL: https://issues.apache.org/jira/browse/HBASE-7196 Project: HBase Issue Type: Bug Affects Versions: 0.94.1 Reporter: Oleg Zhurakousky Its kind of a nuisance bug. One would assume that 'status' command should simply return something along the lines of "HBase is not running" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501214#comment-13501214
]
Oleg Zhurakousky commented on HBASE-7191:
-
Actually, this affects other versions as well
{code}
HBase 0.94.1
{code}
Further more; FSUtils spits out a misleading message
{code}
String msg = "File system needs to be upgraded."
+ " You have version " + version
+ " and I want version " + HConstants.FILE_SYSTEM_VERSION
+ ". Run the '${HBASE_HOME}/bin/hbase migrate' script.";
{code}
As far as I can tell 'migrate' script is no longer available
One would think 'hbase hbck -fixVersionFile' would do the trick, but that
requires a running instance of HBase which we can't start unless those two
files are in tact
The current workaround is to temporarily change the value of 'hbase.rootdir' ->
start/stop hbase (this will re-create the two files) -> copy those files to the
original 'hbase.rootdir' and change 'hbase.rootdir' to point back to the
original one. However, this is obviously a hack. May what really needs to
happen is to have and offline option of 'hbase hbck -fixVersionFile'.
> HBCK - Add offline create/fix hbase.version and hbase.id
> -
>
> Key: HBASE-7191
> URL: https://issues.apache.org/jira/browse/HBASE-7191
> Project: HBase
> Issue Type: Improvement
> Components: hbck
>Affects Versions: 0.96.0
>Reporter: Enis Soztutar
>
> One of our clients run into a problem, in which they have the hbase.root.dir,
> and cluster data, but their hbase.id and hbase.version files are corrupted.
> HMaster creates those on start, but not if there is already existing data.
> We can add smt like --fixIdFile, and ability for HBCK to do some offline
> repairs for the version file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7146) Fix the wrong reference to getReader survived in theFSHLog javadoc
[
https://issues.apache.org/jira/browse/HBASE-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gustavo Anatoly updated HBASE-7146:
---
Attachment: HBASE-7146.patch
Hi, Nicolas.
I submitted a patch for review.
Could you verify, please?
> Fix the wrong reference to getReader survived in theFSHLog javadoc
> --
>
> Key: HBASE-7146
> URL: https://issues.apache.org/jira/browse/HBASE-7146
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Affects Versions: 0.96.0
>Reporter: nkeywal
>Priority: Trivial
> Attachments: HBASE-7146.patch
>
>
> In FSHLog, a wrong reference to getReader survived in the javadoc:
> {noformat}
> * To read an HLog, call {@link #getReader(org.apache.hadoop.fs.FileSystem,
> * org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration)}.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6580) New HTable pool, based on HBase(byte[], HConnection, ExecutorService) constructor
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501126#comment-13501126 ] Adrian Muraru commented on HBASE-6580: -- Lars, I agree the HTablePool is overkill here for these lightweight HTable instances but we'd still need a kind of borrow/release pooling mech for them to avoid creating a new instance every time one is needed. > New HTable pool, based on HBase(byte[], HConnection, ExecutorService) > constructor > - > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.2, 0.94.2 >Reporter: Lars Hofhansl >Priority: Minor > Attachments: HBASE-6580_v1.patch > > > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3776) Add Bloom Filter Support to HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-3776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500939#comment-13500939 ] Chip Salzenberg commented on HBASE-3776: Hello? Anyone? Bueller? > Add Bloom Filter Support to HFileOutputFormat > - > > Key: HBASE-3776 > URL: https://issues.apache.org/jira/browse/HBASE-3776 > Project: HBase > Issue Type: Sub-task >Reporter: Nicolas Spiegelberg >Assignee: Jerry Chen >Priority: Critical > Labels: hbase > Fix For: 0.96.0 > > > Add Bloom Filter support for bulk imports. Lacking a bloom filter, even on a > single imported file, can cause perf degradation. Since we now set our > compression type based on the HBase CF configuration, it would be good to > follow this path for the bloom filter addition. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
