[jira] [Created] (HBASE-19292) Numerical comparison support for column value - cell
Kayla Fisher created HBASE-19292:
Summary: Numerical comparison support for column value - cell
Key: HBASE-19292
URL: https://issues.apache.org/jira/browse/HBASE-19292
Project: HBase
Issue Type: Wish
Components: Filters
Affects Versions: 1.3.1
Environment: may not be related
Reporter: Kayla Fisher
I've gotten the following data set:
rowkey1 test:duration =43425
rowkey2 test:duration = 5000
rowkey3 test:duration = 90
rowkey4 test:duration =8882
According to your filter languages, if i want the data in a specific duration
range,e.g. 2000 "SingleColumnValueFilter('test', 'duration', <=, 'binary:1') AND
SingleColumnValueFilter('test', 'duration', >=, 'binary:2000')"}
Finally got nothing, since i foud out that the comparator i used is
BinaryComparator which compares by bits i.e, `90` is greater than `800`, a
little incredible.
Currently i am unable to follow your spectacular project in java, could you
please add support for numerical comparison or give some advice to conform my
needs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19291) Use common header and footer for JSP pages
Appy created HBASE-19291: Summary: Use common header and footer for JSP pages Key: HBASE-19291 URL: https://issues.apache.org/jira/browse/HBASE-19291 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy Use header and footer in our *.jsp pages to avoid unnecessary redundancy (copy-paste of code) Misc edits: - Due to redundancy, new additions make it to some places but not others. For eg there are missing links to "/logLevel", "/processRS.jsp" in few places. - Fix processMaster.jsp wrongly pointing to rs-status instead of master-status (probably due to copy paste from processRS.jsp) - Deleted a bunch of extraneous "" in processMaster.jsp & processRS.jsp - Added missing tag in snapshot.jsp - Deleted fossils of html5shiv.js. It's uses and the js itself were deleted in the commit "819aed4ccd073d818bfef5931ec8d248bfae5f1f" - Fixed wrongly matched heading tags - Deleted some unused variables Tested: Ran standalone cluster and opened each page to make sure it looked right. Sidenote: Looks like HBASE-3835 started the work of converting from jsp to jamon, but the work didn't finish. Now we have a mix of jsp and jamon. Needs reconciling, but later. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19290) Reduce zk request when doing split log
binlijin created HBASE-19290: Summary: Reduce zk request when doing split log Key: HBASE-19290 URL: https://issues.apache.org/jira/browse/HBASE-19290 Project: HBase Issue Type: Improvement Reporter: binlijin Assignee: binlijin We observe once the cluster has 1000+ nodes and when hundreds of nodes abort and doing split log, the split is very very slow, and we find the regionserver and master wait on the zookeeper response, so we need to reduce zookeeper request and pressure for big cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Moving 2.0 forward
On Thu, Nov 16, 2017 at 8:08 PM, ramkrishna vasudevan < ramkrishna.s.vasude...@gmail.com> wrote: > HBASE-18946 Stochastic load balancer assigns replica regions to the same RS > > Am on this today and probably will put up another patch. > > Regards > Ram > > Thanks Ram, St.Ack > On Fri, Nov 17, 2017 at 5:18 AM, Stack wrote: > > > hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last > > remaining features and apply final polish to API. There will be a beta-2 > > but it is about upgrade/rolling-upgrade and bug fixes ONLY). > > > > Myself and Mr. Drob did a pass over the outstanding hbase-2.0.0-beta-1 > list > > this morning. See here [1]. > > > > There are about ~12 issues in progress with most of these about to land. > > There are 37 TODO. Many of these are tests we need to run, some are > related > > to the backup/restore, but a good few are meaty w/o assignees. > > > > The awkward outstanding ones as I see it are the below: > > > > HBASE-18946 Stochastic load balancer assigns replica regions to the same > RS > > HBASE-17204 Make L2 off heap cache default ON > > HBASE-19112 Suspect methods on Cell to be deprecated > > HBASE-19147 All branch-2 unit tests pass > > > > We need to make progress on the above or punt on them (can't punt on the > > last one though). > > > > Any ideas on what configs we should update in hbase2? Dump ideas into: > > HBASE-19148 Edit of default configuration > > > > Still hoping for an early December beta-1 RC. beta-2 hopefully will be > > close behind. > > > > Comments? Thoughts? > > > > Thanks all, > > S > > > > 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861 > > > > On Tue, Oct 31, 2017 at 2:56 PM, Stack wrote: > > > > > > > > > > > On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob wrote: > > > > > >> Hoping to keep momentum going from our Stack working on alpha4, I > tried > > to > > >> take a stab at triaging some of the open beta-1 issues. > > >> > > >> I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done > > sooner > > >> then I'm happy to see it pulled back in. Trying to balance optimism > with > > >> realism here, and knowing that documentation unfortunately often gets > > >> pushed to the back-burner. > > >> > > >> Also, I poked some folks on unassigned issues that they've filed for > > >> beta-1, especially in the last few days. If issues don't have an owner > > >> they > > >> are unlikely to get worked. I chatted with stack and he agreed to take > > on > > >> some of the tasks, but there's a lot of surface area to cover. > > >> > > >> If you you're working on issues that are critical for beta-1, please > > mark > > >> them as such. Then the rest of the community will know to help > > prioritize > > >> feedback and reviews there. > > >> > > >> Do we have a general theme for the betas like we did with the alphas? > > >> > > >> Beta1 is upgrades work from branch1, beta2 is rolling upgrades work as > > >> well? Continue to work on tests throughout? > > >> > > >> > > > Thanks Mike for helping to kick off the beta-1 train. > > > > > > Regards a theme for beta-1, I'd like it to be 'finish'; beta-1 is what > we > > > are going to ship (beta-2 is rolling upgrade and any minor items turned > > up > > > in testing/burn-in). > > > > > > Thanks, > > > S > > > > > > > > > > > >> Mike > > >> > > >> On Tue, Oct 31, 2017 at 10:04 AM, Josh Elser > wrote: > > >> > > >> > +1 go from my POV. > > >> > > > >> > > > >> > On 10/31/17 10:07 AM, Stack wrote: > > >> > > > >> >> I want to push an alpha-4 today. A few items didn't make it > > >> (HBASE-19092). > > >> >> They need more time. We'll pull them in for beta-1. CP API is > > basically > > >> >> done. There may be some changes for beta-1 but hopefully only > changes > > >> >> informed by experience trying to port an existing Coprocessor to > > >> hbase2. > > >> >> > > >> >> Shout if there is anything that needs to make alpha-4. > > >> >> > > >> >> Thanks, > > >> >> St.Ack > > >> >> > > >> >> > > >> >> On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser > > >> wrote: > > >> >> > > >> >> Yup, that was going to be my plan, Mike! > > >> >>> > > >> >>> Making a pass now, and will check back later tonight again. I see > > >> others > > >> >>> have already done some work today on this front. > > >> >>> > > >> >>> > > >> >>> On 10/27/17 11:38 PM, Mike Drob wrote: > > >> >>> > > >> >>> Josh - Do you want to kick off a bunch of QA runs? (Do you know > how > > >> to do > > >> it directly on the jenkins job, so you don't have to bother with > > JIRA > > >> uploads) > > >> > > >> If you're busy, then I can make time tomorrow or Sunday to kick > off > > >> jobs, > > >> but I want to make sure we're not duplicating effort and jenkins > > >> cycles. > > >> > > >> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser > > >> wrote: > > >> > > >> My turn to bump ;) > > >> > > >> > > > >> > By my take: HBASE-18770 and HBASE-19092 are the only issues that > > >> remain
Re: [DISCUSSION] Removing the bypass semantic from the Coprocessor APIs
On Mon, Oct 30, 2017 at 1:56 PM, Stack wrote: > On Mon, Oct 30, 2017 at 1:45 PM, Andrew Purtell > wrote: > >> I think complete and bypass are separate considerations and complete can >> be >> used universally while we've decided to make bypass work only in some >> contexts. >> >> That said, we can consider removing the complete semantic. Let's pose the >> same question we did about bypass. Does anyone use it? Can we live without >> it? As you point out, security interrupts processing by throwing an >> exception, which is meant to propagate all the way back to the user. It >> simplifies the theory of operation for coprocessors if we can assume >> either >> the entire chain will complete or one of the coprocessors in the chain >> will >> throw an exception that not only terminates processing of the rest of the >> chain but also the operation in progress. >> >> > > I'd be game for removing 'complete'. > > I can leave this question hang a while. Purge would be internals and > javadoc changes so it can happen after alpha-4 (HBASE-19123 is the > issue-to-purge). > > Ok. No calls for the preservation of 'complete'. HBASE-19123 "Purge 'complete' support from Coprocesor Observers" has a patch up and I'll commit it in the morning unless objection. It removes the complete facility. Thanks, S > Thanks, > S > > > > > >> >> On Mon, Oct 30, 2017 at 10:46 AM, Stack wrote: >> >> > HBASE-18770 (bypass) is coming along (thanks for the helpful reviews so >> > far!). >> > >> > Of note, I have changed the Coprocessor Observer 'complete' function so >> it >> > is only available on 'bypassable' methods. Is this ok to do (I'm no >> expert >> > on coprocessoring)? I do it in in the name of KISS. Having any >> Coprocessor >> > being able to 'complete' overriding any Coprocessor that comes behind >> it in >> > the processing chain seems obnoxious. I can see the need if a >> Coprocessor >> > is bypasable and has conjured an answer it wants to be back to the >> client >> > without tainting by subsequent Coprocessors -- which seems to be how it >> is >> > used in my survey of Coprocessor implementations -- but perhaps I am >> > missing a use case? (AccessController throws an exception when access is >> > denied). Downside of supporting 'complete' globally is more overrides >> > internally and messaging gets a bit more muddled. >> > >> > Here is more on 'complete' in case you don't know what it is about. If a >> > method's 'pre' hook is wrapped by 10 Coprocessor observers, each >> observer >> > gets called one after the other before we go ahead and do the actual >> > method invocation. If the first Coprocessor in the chain calls >> 'complete' >> > in its context, we will skip calling the remaining 9 coprocessors and >> then >> > go ahead and make the method invocation. >> > >> > Any opinions out there on 'complete'? Any objections to my only allowing >> > 'complete' on bypassable methods? >> > >> > Thanks, >> > St.Ack >> > >> > >> > >> > On Tue, Oct 24, 2017 at 9:53 PM, Stack wrote: >> > >> > > I made a start on HBASE-18770. It has edit of RegionObserver which >> > denotes >> > > methods that support bypass (Unfortunately, because of the varied >> > > signatures, how bypass is signaled varies too). Would appreciate a >> > > once-over. >> > > >> > > of note, a CP cannot bypass flush. Speak up if you think otherwise (or >> > you >> > > can think of a case where this needed). My rationale is CPs won't have >> > > enough insider knowledge to do memory accounting in a world of >> in-memory >> > > compactions, and on/offheap memory in our hosting process. What ye >> > reckon? >> > > >> > > Coprocessors have always been able to adjust what gets compacted in >> any >> > > run and even skirt compaction altogether by returning an empty set of >> > files >> > > to compact. This works as it ever did. >> > > >> > > Thanks, >> > > S >> > > >> > > >> > > >> > > On Tue, Oct 17, 2017 at 9:46 PM, Stack wrote: >> > > >> > >> I was going to pick up on the bypass after HBASE-19007 lands, >> cleaning >> > up >> > >> our exposure of Master/RegionServerServices to Coprocessors >> (HBASE-19007 >> > >> was going bad for a good while but lots of contributors and good >> > discussion >> > >> and now I think we have it). Shouldn't be too much longer. >> > >> >> > >> Its CP API so I was figuring it an alpha-4 item. >> > >> >> > >> St.Ack >> > >> >> > >> On Tue, Oct 17, 2017 at 6:56 PM, 张铎(Duo Zhang) < >> palomino...@gmail.com> >> > >> wrote: >> > >> >> > >>> Fine. Let me change the title of HBASE-18770 and prepare a patch >> there. >> > >>> >> > >>> May still a week or two before alpha4 I think. The scan injection, >> and >> > >>> flush/compaction trigger/track API is still unstable... >> > >>> >> > >>> 2017-10-18 6:12 GMT+08:00 Josh Elser : >> > >>> >> > >>> > (catching up here) >> > >>> > >> > >>> > I'm glad to see you fine folks came to a conclusion around a >> > >>> reduced-scope >> > >>> > solution (correct me if I'm wrong). "Some" bypass mechanism wo
Re: Moving 2.0 forward
HBASE-18946 Stochastic load balancer assigns replica regions to the same RS Am on this today and probably will put up another patch. Regards Ram On Fri, Nov 17, 2017 at 5:18 AM, Stack wrote: > hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last > remaining features and apply final polish to API. There will be a beta-2 > but it is about upgrade/rolling-upgrade and bug fixes ONLY). > > Myself and Mr. Drob did a pass over the outstanding hbase-2.0.0-beta-1 list > this morning. See here [1]. > > There are about ~12 issues in progress with most of these about to land. > There are 37 TODO. Many of these are tests we need to run, some are related > to the backup/restore, but a good few are meaty w/o assignees. > > The awkward outstanding ones as I see it are the below: > > HBASE-18946 Stochastic load balancer assigns replica regions to the same RS > HBASE-17204 Make L2 off heap cache default ON > HBASE-19112 Suspect methods on Cell to be deprecated > HBASE-19147 All branch-2 unit tests pass > > We need to make progress on the above or punt on them (can't punt on the > last one though). > > Any ideas on what configs we should update in hbase2? Dump ideas into: > HBASE-19148 Edit of default configuration > > Still hoping for an early December beta-1 RC. beta-2 hopefully will be > close behind. > > Comments? Thoughts? > > Thanks all, > S > > 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861 > > On Tue, Oct 31, 2017 at 2:56 PM, Stack wrote: > > > > > > > On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob wrote: > > > >> Hoping to keep momentum going from our Stack working on alpha4, I tried > to > >> take a stab at triaging some of the open beta-1 issues. > >> > >> I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done > sooner > >> then I'm happy to see it pulled back in. Trying to balance optimism with > >> realism here, and knowing that documentation unfortunately often gets > >> pushed to the back-burner. > >> > >> Also, I poked some folks on unassigned issues that they've filed for > >> beta-1, especially in the last few days. If issues don't have an owner > >> they > >> are unlikely to get worked. I chatted with stack and he agreed to take > on > >> some of the tasks, but there's a lot of surface area to cover. > >> > >> If you you're working on issues that are critical for beta-1, please > mark > >> them as such. Then the rest of the community will know to help > prioritize > >> feedback and reviews there. > >> > >> Do we have a general theme for the betas like we did with the alphas? > >> > >> Beta1 is upgrades work from branch1, beta2 is rolling upgrades work as > >> well? Continue to work on tests throughout? > >> > >> > > Thanks Mike for helping to kick off the beta-1 train. > > > > Regards a theme for beta-1, I'd like it to be 'finish'; beta-1 is what we > > are going to ship (beta-2 is rolling upgrade and any minor items turned > up > > in testing/burn-in). > > > > Thanks, > > S > > > > > > > >> Mike > >> > >> On Tue, Oct 31, 2017 at 10:04 AM, Josh Elser wrote: > >> > >> > +1 go from my POV. > >> > > >> > > >> > On 10/31/17 10:07 AM, Stack wrote: > >> > > >> >> I want to push an alpha-4 today. A few items didn't make it > >> (HBASE-19092). > >> >> They need more time. We'll pull them in for beta-1. CP API is > basically > >> >> done. There may be some changes for beta-1 but hopefully only changes > >> >> informed by experience trying to port an existing Coprocessor to > >> hbase2. > >> >> > >> >> Shout if there is anything that needs to make alpha-4. > >> >> > >> >> Thanks, > >> >> St.Ack > >> >> > >> >> > >> >> On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser > >> wrote: > >> >> > >> >> Yup, that was going to be my plan, Mike! > >> >>> > >> >>> Making a pass now, and will check back later tonight again. I see > >> others > >> >>> have already done some work today on this front. > >> >>> > >> >>> > >> >>> On 10/27/17 11:38 PM, Mike Drob wrote: > >> >>> > >> >>> Josh - Do you want to kick off a bunch of QA runs? (Do you know how > >> to do > >> it directly on the jenkins job, so you don't have to bother with > JIRA > >> uploads) > >> > >> If you're busy, then I can make time tomorrow or Sunday to kick off > >> jobs, > >> but I want to make sure we're not duplicating effort and jenkins > >> cycles. > >> > >> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser > >> wrote: > >> > >> My turn to bump ;) > >> > >> > > >> > By my take: HBASE-18770 and HBASE-19092 are the only issues that > >> remain > >> > needing some more work. The rest are just awaiting a good QA run. > >> > > >> > Unless I hear otherwise, I'll try to keep an eye on things over > the > >> > weekend, bump them along as necessary, and get them committed. > >> Would be > >> > great to be able get a vote up on Monday. > >> > > >> > > >> > On 10/24/17 6:03 PM, Stack wrote: > >> > > >> > Chatting with my coworker
[jira] [Resolved] (HBASE-18805) Unify Admin and AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-18805. Resolution: Fixed All sub-tasks done. > Unify Admin and AsyncAdmin > -- > > Key: HBASE-18805 > URL: https://issues.apache.org/jira/browse/HBASE-18805 > Project: HBase > Issue Type: Umbrella >Reporter: Balazs Meszaros >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > > Admin and AsyncAdmin differ some places: > - some methods missing from AsyncAdmin (e.g. methods with String regex), > - some methods have different names (listTables vs listTableDescriptors), > - some method parameters are different (e.g. AsyncAdmin has Optional<> > parameters), > - AsyncAdmin returns Lists instead of arrays (e.g. listTableNames), > - unify Javadoc comments, > - ... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18912) Update Admin methods to return Lists instead of arrays
[ https://issues.apache.org/jira/browse/HBASE-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-18912. Resolution: Won't Fix As we need deprecate too many old methods. So if we can't find a better method name or don't have a good reason to deprecate the old methods, I thought we don't need to only change the return type from array to List... Resolve this as won't fix. Thanks > Update Admin methods to return Lists instead of arrays > -- > > Key: HBASE-18912 > URL: https://issues.apache.org/jira/browse/HBASE-18912 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Moving 2.0 forward
hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last remaining features and apply final polish to API. There will be a beta-2 but it is about upgrade/rolling-upgrade and bug fixes ONLY). Myself and Mr. Drob did a pass over the outstanding hbase-2.0.0-beta-1 list this morning. See here [1]. There are about ~12 issues in progress with most of these about to land. There are 37 TODO. Many of these are tests we need to run, some are related to the backup/restore, but a good few are meaty w/o assignees. The awkward outstanding ones as I see it are the below: HBASE-18946 Stochastic load balancer assigns replica regions to the same RS HBASE-17204 Make L2 off heap cache default ON HBASE-19112 Suspect methods on Cell to be deprecated HBASE-19147 All branch-2 unit tests pass We need to make progress on the above or punt on them (can't punt on the last one though). Any ideas on what configs we should update in hbase2? Dump ideas into: HBASE-19148 Edit of default configuration Still hoping for an early December beta-1 RC. beta-2 hopefully will be close behind. Comments? Thoughts? Thanks all, S 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861 On Tue, Oct 31, 2017 at 2:56 PM, Stack wrote: > > > On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob wrote: > >> Hoping to keep momentum going from our Stack working on alpha4, I tried to >> take a stab at triaging some of the open beta-1 issues. >> >> I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done sooner >> then I'm happy to see it pulled back in. Trying to balance optimism with >> realism here, and knowing that documentation unfortunately often gets >> pushed to the back-burner. >> >> Also, I poked some folks on unassigned issues that they've filed for >> beta-1, especially in the last few days. If issues don't have an owner >> they >> are unlikely to get worked. I chatted with stack and he agreed to take on >> some of the tasks, but there's a lot of surface area to cover. >> >> If you you're working on issues that are critical for beta-1, please mark >> them as such. Then the rest of the community will know to help prioritize >> feedback and reviews there. >> >> Do we have a general theme for the betas like we did with the alphas? >> >> Beta1 is upgrades work from branch1, beta2 is rolling upgrades work as >> well? Continue to work on tests throughout? >> >> > Thanks Mike for helping to kick off the beta-1 train. > > Regards a theme for beta-1, I'd like it to be 'finish'; beta-1 is what we > are going to ship (beta-2 is rolling upgrade and any minor items turned up > in testing/burn-in). > > Thanks, > S > > > >> Mike >> >> On Tue, Oct 31, 2017 at 10:04 AM, Josh Elser wrote: >> >> > +1 go from my POV. >> > >> > >> > On 10/31/17 10:07 AM, Stack wrote: >> > >> >> I want to push an alpha-4 today. A few items didn't make it >> (HBASE-19092). >> >> They need more time. We'll pull them in for beta-1. CP API is basically >> >> done. There may be some changes for beta-1 but hopefully only changes >> >> informed by experience trying to port an existing Coprocessor to >> hbase2. >> >> >> >> Shout if there is anything that needs to make alpha-4. >> >> >> >> Thanks, >> >> St.Ack >> >> >> >> >> >> On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser >> wrote: >> >> >> >> Yup, that was going to be my plan, Mike! >> >>> >> >>> Making a pass now, and will check back later tonight again. I see >> others >> >>> have already done some work today on this front. >> >>> >> >>> >> >>> On 10/27/17 11:38 PM, Mike Drob wrote: >> >>> >> >>> Josh - Do you want to kick off a bunch of QA runs? (Do you know how >> to do >> it directly on the jenkins job, so you don't have to bother with JIRA >> uploads) >> >> If you're busy, then I can make time tomorrow or Sunday to kick off >> jobs, >> but I want to make sure we're not duplicating effort and jenkins >> cycles. >> >> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser >> wrote: >> >> My turn to bump ;) >> >> > >> > By my take: HBASE-18770 and HBASE-19092 are the only issues that >> remain >> > needing some more work. The rest are just awaiting a good QA run. >> > >> > Unless I hear otherwise, I'll try to keep an eye on things over the >> > weekend, bump them along as necessary, and get them committed. >> Would be >> > great to be able get a vote up on Monday. >> > >> > >> > On 10/24/17 6:03 PM, Stack wrote: >> > >> > Chatting with my coworker Mr. Mike Drob, we were batting back and >> forth >> > >> >> what remains to be done. Surfacing our thoughts here so you all >> clued >> >> inOr if you think otherwise, please speak up. >> >> >> >> We have ~13 issues to land: >> >> https://issues.apache.org/jira/projects/HBASE/versions/12341594 >> About >> >> two >> >> are meta-issues that are about process which leaves 11. >> >> >> >> Duo and Zheng Hu are to merge the FilterList f
[jira] [Created] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
Ted Yu created HBASE-19289:
--
Summary: CommonFSUtils$StreamLacksCapabilityException: hflush when
running test against hadoop3 beta1
Key: HBASE-19289
URL: https://issues.apache.org/jira/browse/HBASE-19289
Project: HBase
Issue Type: Test
Reporter: Ted Yu
As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
following exception when running unit test against hadoop3 beta1:
{code}
testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
elapsed: 0.061 sec <<< ERROR!
java.io.IOException: cannot get log writer
at
org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
at
org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
Caused by:
org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
hflush
at
org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
at
org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19288) Intermittent test failure in TestHStore.testRunDoubleMemStoreCompactors
Ted Yu created HBASE-19288:
--
Summary: Intermittent test failure in
TestHStore.testRunDoubleMemStoreCompactors
Key: HBASE-19288
URL: https://issues.apache.org/jira/browse/HBASE-19288
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Here was one of the test failures:
https://builds.apache.org/job/PreCommit-HBASE-Build/9812/testReport/junit/org.apache.hadoop.hbase.regionserver/TestHStore/testRunDoubleMemStoreCompactors/
{code}
[ERROR]
org.apache.hadoop.hbase.regionserver.TestHStore.testRunDoubleMemStoreCompactors(org.apache.hadoop.hbase.regionserver.TestHStore)
[ERROR] Run 1: TestHStore.testRunDoubleMemStoreCompactors:1500 expected:<2>
but was:<3>
[ERROR] Run 2: TestHStore.testRunDoubleMemStoreCompactors:1481 expected:<1>
but was:<4>
[ERROR] Run 3: TestHStore.testRunDoubleMemStoreCompactors:1481 expected:<1>
but was:<5>
{code}
>From the counts for second and third runs, we know that RUNNER_COUNT was not
>cleared in between the reruns, leading to failure at the 1st assertion.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Re: Any Eclipse users among devs?
Sure I will try, Josh -Vlad On Thu, Nov 16, 2017 at 10:57 AM, Josh Elser wrote: > I still use Eclipse for most of my development and have been fighting it > being broken for branch-2 (ironically, as a result of some "fixes" I made > previously). > > Anyone with a setup who can test a clean import as a part of > https://issues.apache.org/jira/browse/HBASE-19267 would be greatly > appreciated. > > Thanks! > > - Josh >
[jira] [Created] (HBASE-19287) master hangs forever if RecoverMeta send assign meta region request to target server fail
Yi Liang created HBASE-19287:
Summary: master hangs forever if RecoverMeta send assign meta
region request to target server fail
Key: HBASE-19287
URL: https://issues.apache.org/jira/browse/HBASE-19287
Project: HBase
Issue Type: Bug
Reporter: Yi Liang
2017-11-10 19:26:56,019 INFO [ProcExecWrkr-1] procedure.RecoverMetaProcedure:
pid=138, state=RUNNABLE:RECOVER_META_ASSIGN_REGIONS; RecoverMetaProcedure
failedMetaServer=null, splitWal=true; Retaining meta assignment to
server=hadoop-slave1.hadoop,16020,1510341981454
2017-11-10 19:26:56,029 INFO [ProcExecWrkr-1] procedure2.ProcedureExecutor:
Initialized subprocedures=[{pid=139, ppid=138,
state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure table=hbase:meta,
region=1588230740, target=hadoop-slave1.hadoop,16020,1510341981454}]
2017-11-10 19:26:56,067 INFO [ProcExecWrkr-2]
procedure.MasterProcedureScheduler: pid=139, ppid=138,
state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure table=hbase:meta,
region=1588230740, target=hadoop-slave1.hadoop,16020,1510341981454 hbase:meta
hbase:meta,,1.1588230740
2017-11-10 19:26:56,071 INFO [ProcExecWrkr-2] assignment.AssignProcedure:
Start pid=139, ppid=138, state=RUNNABLE:REGION_TRANSITION_QUEUE;
AssignProcedure table=hbase:meta, region=1588230740,
target=hadoop-slave1.hadoop,16020,1510341981454; rit=OFFLINE,
location=hadoop-slave1.hadoop,16020,1510341981454; forceNewPlan=false,
retain=false
2017-11-10 19:26:56,224 INFO [ProcExecWrkr-4] zookeeper.MetaTableLocator:
Setting hbase:meta (replicaId=0) location in ZooKeeper as
hadoop-slave2.hadoop,16020,1510341988652
2017-11-10 19:26:56,230 INFO [ProcExecWrkr-4]
assignment.RegionTransitionProcedure: Dispatch pid=139, ppid=138,
state=RUNNABLE:REGION_TRANSITION_DISPATCH; AssignProcedure table=hbase:meta,
region=1588230740, target=hadoop-slave1.hadoop,16020,1510341981454;
rit=OPENING, location=hadoop-slave2.hadoop,16020,1510341988652
2017-11-10 19:26:56,382 INFO [ProcedureDispatcherTimeoutThread]
procedure.RSProcedureDispatcher: Using procedure batch rpc execution for
serverName=hadoop-slave2.hadoop,16020,1510341988652 version=2097152
2017-11-10 19:26:57,542 INFO [main-EventThread] zookeeper.RegionServerTracker:
RegionServer ephemeral node deleted, processing expiration
[hadoop-slave2.hadoop,16020,1510341988652]
2017-11-10 19:26:57,543 INFO [main-EventThread] master.ServerManager: Master
doesn't enable ServerShutdownHandler during initialization, delay expiring
server hadoop-slave2.hadoop,16020,1510341988652
2017-11-10 19:26:58,875 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000]
master.ServerManager: Registering
server=hadoop-slave1.hadoop,16020,1510342016106
2017-11-10 19:27:05,832 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000]
master.ServerManager: Registering
server=hadoop-slave2.hadoop,16020,1510342023184
2017-11-10 19:27:05,832 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000]
master.ServerManager: Triggering server recovery; existingServer
hadoop-slave2.hadoop,16020,1510341988652 looks stale, new
server:hadoop-slave2.hadoop,16020,1510342023184
2017-11-10 19:27:05,832 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000]
master.ServerManager: Master doesn't enable ServerShutdownHandler during
initialization, delay expiring server hadoop-slave2.hadoop,16020,1510341988652
2017-11-10 19:27:49,815 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000]
client.RpcRetryingCallerImpl: tarted=38594 ms ago, cancelled=false,
msg=org.apache.hadoop.hbase.NotServingRegionException: hbase:meta,,1 is not
online on hadoop-slave2.hadoop,16020,1510342023184
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3290)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1370)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2401)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41544)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:278)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:258)
row 'hbase:namespace' on table 'hbase:meta' at
region=hbase:meta,,1.1588230740,
hostname=hadoop-slave2.hadoop,16020,1510341988652, seqNum=0
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Any Eclipse users among devs?
I still use Eclipse for most of my development and have been fighting it being broken for branch-2 (ironically, as a result of some "fixes" I made previously). Anyone with a setup who can test a clean import as a part of https://issues.apache.org/jira/browse/HBASE-19267 would be greatly appreciated. Thanks! - Josh
[jira] [Resolved] (HBASE-18703) Inconsistent behavior for preBatchMutate in doMiniBatchMutate and processRowsWithLocks
[ https://issues.apache.org/jira/browse/HBASE-18703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe resolved HBASE-18703. -- Resolution: Fixed > Inconsistent behavior for preBatchMutate in doMiniBatchMutate and > processRowsWithLocks > -- > > Key: HBASE-18703 > URL: https://issues.apache.org/jira/browse/HBASE-18703 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Umesh Agashe >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: hbase-18703.master.001.patch, > hbase-18703.master.002.patch, hbase-18703.master.003.patch, > hbase-18703.master.004.patch, hbase-18703.master.005.patch, > hbase-18703.master.005.patch, hbase-18703.master.005.patch, > hbase-18703.master.006.patch, hbase-18703.master.007.patch > > > In doMiniBatchMutate, the preBatchMutate is called before building WAL, but > in processRowsWithLocks, we suggest the RowProcessor implementation to build > WAL in process method, which is ahead of preBatchMutate. > If a CP modifies the mutations, especially if it removes some cells from the > mutations, then the behavior of processRowsWithLocks is broken. The changes > applied to memstore and WAL will be different. And there is no way to remove > entries from a WALEdit through CP. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19286) ad-hoc test job should take profiles to activate
Sean Busbey created HBASE-19286: --- Summary: ad-hoc test job should take profiles to activate Key: HBASE-19286 URL: https://issues.apache.org/jira/browse/HBASE-19286 Project: HBase Issue Type: Improvement Components: build Reporter: Sean Busbey Priority: Minor we should have the ability to activate profiles in the ad-hoc job. this will let us a) check hadoop 3 b) check with the same profile used in precommit -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19277) hbase sync between 0.9 and 1.2
[ https://issues.apache.org/jira/browse/HBASE-19277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-19277. --- Resolution: Not A Problem > hbase sync between 0.9 and 1.2 > -- > > Key: HBASE-19277 > URL: https://issues.apache.org/jira/browse/HBASE-19277 > Project: HBase > Issue Type: Brainstorming >Affects Versions: 1.2.6 > Environment: hbase 0.9 hbase1.2 >Reporter: SuperbDong > > I expect synchrodata between hbase 0.9 and hbase 1.2. > What's more,I find several ways to do it. > Follow : > 1.replication (need modify) > 2.sync hlog before delete to hdfs .oldlog (need modify) > 3.client writes data to two hbase > 4.client writes data to kafka and consume to two hbase > But, I think the bigest question is one java client how to use two > hbase-cliet jar,It must be conflict,How can I do? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19285) Add per-table latency histograms
Josh Elser created HBASE-19285: -- Summary: Add per-table latency histograms Key: HBASE-19285 URL: https://issues.apache.org/jira/browse/HBASE-19285 Project: HBase Issue Type: Bug Components: metrics Reporter: Josh Elser Assignee: Josh Elser Priority: Critical Fix For: 2.0.0, 1.4.0, 1.3.3 HBASE-17017 removed the per-region latency histograms (e.g. Get, Put, Scan at p75, p85, etc) HBASE-15518 added some per-table metrics, but not the latency histograms. Given the previous conversations, it seems like it these per-table aggregations weren't intentionally omitted, just never re-implemented after the per-region removal. They're some really nice out-of-the-box metrics we can provide to our users/admins as long as it's not detrimental. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19284) Update unsupported methods in Javadoc for RemoteHTable
Rick Kellogg created HBASE-19284: Summary: Update unsupported methods in Javadoc for RemoteHTable Key: HBASE-19284 URL: https://issues.apache.org/jira/browse/HBASE-19284 Project: HBase Issue Type: Task Components: documentation, REST Reporter: Rick Kellogg Priority: Minor The RemoteHTable class implements the Table interface. Many of the methods are not supported and should be documented in the javadoc as such. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19283) InfoServer should allow configuring of cipher suite when TLS is enabled
Sean Busbey created HBASE-19283: --- Summary: InfoServer should allow configuring of cipher suite when TLS is enabled Key: HBASE-19283 URL: https://issues.apache.org/jira/browse/HBASE-19283 Project: HBase Issue Type: Improvement Components: master, regionserver Affects Versions: 2.0.0-alpha-3 Reporter: Sean Busbey Fix For: 2.0.0 When updating to jetty 9, we gained configuration knobs for restricting TLS protocols and cipher suites for the REST service and the Thrift service when running over HTTP. We should provide the same configuration knob for the HttpServer class that's used by the informational web page on other roles. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19282) CellChunkMap Benchmarking and User Interface
Anastasia Braginsky created HBASE-19282: --- Summary: CellChunkMap Benchmarking and User Interface Key: HBASE-19282 URL: https://issues.apache.org/jira/browse/HBASE-19282 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky We have made some experiments how working with CellChunkMap (CCM) influences the performance when running on-heap and off-heap. Based on those results it is suggested to tie the MSLAB usage (off-heap or on-heap) with CCM index usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19281) Snapshot creation failed after splitting table (replica region > 1)
Chandra Sekhar created HBASE-19281:
--
Summary: Snapshot creation failed after splitting table (replica
region > 1)
Key: HBASE-19281
URL: https://issues.apache.org/jira/browse/HBASE-19281
Project: HBase
Issue Type: Bug
Components: snapshots
Affects Versions: 1.3.1
Reporter: Chandra Sekhar
Snapshot creation failed with below error when tried on table with multiple
replica region,
{noformat}
hbase(main):025:0> snapshot 't1','t1_snap'
2017-11-16 18:04:27,930 DEBUG [main] client.HBaseAdmin: Waiting a max of 30
ms for snapshot '{ ss=t1_snap table=t1 type=FLUSH }'' to complete. (max 42857
ms per retry)
2017-11-16 18:04:27,930 DEBUG [main] client.HBaseAdmin: (#1) Sleeping: 100ms
while waiting for snapshot completion.
2017-11-16 18:04:28,030 DEBUG [main] client.HBaseAdmin: Getting current status
of snapshot from master...
2017-11-16 18:04:28,035 DEBUG [main] client.HBaseAdmin: (#2) Sleeping: 200ms
while waiting for snapshot completion.
2017-11-16 18:04:28,236 DEBUG [main] client.HBaseAdmin: Getting current status
of snapshot from master...
2017-11-16 18:04:28,238 DEBUG [main] client.HBaseAdmin: (#3) Sleeping: 300ms
while waiting for snapshot completion.
2017-11-16 18:04:28,538 DEBUG [main] client.HBaseAdmin: Getting current status
of snapshot from master...
ERROR: org.apache.hadoop.hbase.snapshot.HBaseSnapshotException: Snapshot {
ss=t1_snap table=t1 type=FLUSH } had an error. Procedure t1_snap { waiting=[]
done=[] }
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:354)
at
org.apache.hadoop.hbase.master.MasterRpcServices.isSnapshotDone(MasterRpcServices.java:1091)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2418)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:191)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException via
Failed taking snapshot { ss=t1_snap table=t1 type=FLUSH } due to
exception:Manifest region info {ENCODED => 3158abebd655fca73cd87b6e84584197,
NAME => 't1,,1510826577196_0002.3158abebd655fca73cd87b6e84584197.', STARTKEY =>
'', ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID => 2}doesn't match
expected region:{ENCODED => 73aa1a133d3344a67afa46ee135e389a, NAME =>
't1,,1510826577196.73aa1a133d3344a67afa46ee135e389a.', STARTKEY => '', ENDKEY
=> '', OFFLINE => true, SPLIT =>
true}:org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Manifest
region info {ENCODED => 3158abebd655fca73cd87b6e84584197, NAME =>
't1,,1510826577196_0002.3158abebd655fca73cd87b6e84584197.', STARTKEY => '',
ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID => 2}doesn't match
expected region:{ENCODED => 73aa1a133d3344a67afa46ee135e389a, NAME =>
't1,,1510826577196.73aa1a133d3344a67afa46ee135e389a.', STARTKEY => '', ENDKEY
=> '', OFFLINE => true, SPLIT => true}
at
org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:83)
at
org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.rethrowExceptionIfFailed(TakeSnapshotHandler.java:315)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:344)
... 6 more
Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException:
Manifest region info {ENCODED => 3158abebd655fca73cd87b6e84584197, NAME =>
't1,,1510826577196_0002.3158abebd655fca73cd87b6e84584197.', STARTKEY => '',
ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID => 2}doesn't match
expected region:{ENCODED => 73aa1a133d3344a67afa46ee135e389a, NAME =>
't1,,1510826577196.73aa1a133d3344a67afa46ee135e389a.', STARTKEY => '', ENDKEY
=> '', OFFLINE => true, SPLIT => true}
at
org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifyRegionInfo(MasterSnapshotVerifier.java:220)
at
org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifyRegions(MasterSnapshotVerifier.java:198)
at
org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifySnapshot(MasterSnapshotVerifier.java:118)
at
org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.process(TakeSnapshotHandler.java:202)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
Steps to repr
[jira] [Created] (HBASE-19280) Move HFileWriterImpl.compressionByName(String name) to some utility class
Ankit Singhal created HBASE-19280:
-
Summary: Move HFileWriterImpl.compressionByName(String name) to
some utility class
Key: HBASE-19280
URL: https://issues.apache.org/jira/browse/HBASE-19280
Project: HBase
Issue Type: Bug
Reporter: Ankit Singhal
Priority: Trivial
This method can be moved to some utility (related jira PHOENIX-4368).
{code}
public static Compression.Algorithm compressionByName(String algoName) {
if (algoName == null)
return HFile.DEFAULT_COMPRESSION_ALGORITHM;
return Compression.getCompressionAlgorithmByName(algoName);
}
{code}
FYI, [~elserj]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
