[jira] [Created] (HBASE-19146) Hbase3.0 protobuf-maven-plugin do not support Arm64(only for x86)
Yuqi Gu created HBASE-19146:
---
Summary: Hbase3.0 protobuf-maven-plugin do not support Arm64(only
for x86)
Key: HBASE-19146
URL: https://issues.apache.org/jira/browse/HBASE-19146
Project: HBase
Issue Type: Bug
Components: pom
Environment: OS: Ubuntu 16.04.3
OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
Hw platform: AARCH64
Reporter: Yuqi Gu
Priority: Major
We are building the HBase 3.0.0-SNAPSHOT on AARCH64.
It is noted that 'protobuf-maven-plugin' only support x86 shown as follows:
{code:java}
org.xolstice.maven.plugins
protobuf-maven-plugin
${protobuf.plugin.version}
com.google.protobuf:protoc:${external.protobuf.version}:
exe:${os.detected.classifier}
${basedir}/src/main/protobuf/
false
true
{code}
So the build is failed.
{code:java}
[INFO] --- protobuf-maven-plugin:0.5.0:compile (compile-protoc) @
hbase-protocol-shaded ---
[INFO] Compiling 32 proto file(s) to
/root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
Failed to execute goal
org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile (compile-protoc)
on project hbase-protocol-shaded: Missing:
{code}
Then I installed aarch64 protobuf 2.5.0 on the host and modify the pom:
{code:java}
/usr/local/bin/protoc
{code}
The build is also failed:
{code:java}
[INFO] Compiling 32 proto file(s) to
/root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
[ERROR] PROTOC FAILED: google/protobuf/any.proto:31:10: Unrecognized syntax
identifier "proto3". This parser only recognizes "proto2".
{code}
It seems that "internal.protobuf.version" in "hbase-protocol-shaded" is 3.3.0.
How to fix it? Thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-17499) Bound the total heap memory used for the rolling average of RegionLoads
[ https://issues.apache.org/jira/browse/HBASE-17499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-17499. Resolution: Later > Bound the total heap memory used for the rolling average of RegionLoads > --- > > Key: HBASE-17499 > URL: https://issues.apache.org/jira/browse/HBASE-17499 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Priority: Major > Labels: balancer > > Currently "hbase.master.balancer.stochastic.numRegionLoadsToRemember" > controls the number of RegionLoads which are kept by StochasticLoadBalancer > for each region. > The parameter doesn't take into account the number of regions in the cluster. > Meaning, the amount of heap consumed by RegionLoads would be out of norm for > cluster with large number of regions. > This issue is to see if we should bound the total heap memory used for the > rolling average of RegionLoads. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
Thanks. I filed a JIRA to track it. https://issues.apache.org/jira/browse/HBASE-19145 On Mon, Oct 30, 2017 at 9:36 PM, Stack wrote: > On Mon, Oct 30, 2017 at 11:46 AM, Jerry He wrote: > >> Hi, Stack >> >> Coming back to this thread, i have meant to ask this question long ago. >> Do we support hbase-2 client going against hbase-1 server? >> > > It has not been an objective. It might work doing basic Client API on a > later branch-1 but will fail doing Admin functions (and figuring if a Table > is online). I've not tried it Jerry. If it was a small thing to make it > work, lets get it in. > > St.Ack > > > >> We seem to be fine mix-and-match the clients and servers within the >> hbase-1 releases. And IIRC hbase-1 client is ok against 0.98 server. >> Suppose I have a product that depends and bundles HBase client. I >> want to upgrade the dependency to hbase-2 so that it can take >> advantages of and claim support of hbase-2. >> But does it mean that I will need drop the claims that the new version >> of the product support any hbase-1 backend? >> >> Thanks. >> >> >> >> On Tue, Sep 12, 2017 at 10:21 AM, Stack wrote: >> > Let me put this one on this thread, G1GC on by default in hbase2? >> > St.Ack >> > >> > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: >> > >> >> What are our expectations regards compatibility between hbase1 and >> hbase2? >> >> >> >> Lets have a chat about it. Here are some goal posts. >> >> >> >> + You have to upgrade to hbase-1.x before you can migrate to hbase-2. No >> >> migration from < hbase-1 (Is this too onerous? Should we support 0.98 => >> >> 2.0?). >> >> + You do NOT have to upgrade to the latest release of hbase1 to migrate >> to >> >> hbase2; being up on hbase-1.0.0+ will be sufficient. >> >> + You'll have to update your hbase1 coprocessors to deploy them on >> hbase2. >> >> A bunch of CP API has/will change by the time hbase2 comes out; e.g. >> >> watching for region split on RegionServer no longer makes sense given >> >> Master runs all splits now. >> >> + An hbase1 client can run against an hbase2 cluster but it will only be >> >> able to do DML (Get/Put/Scan, etc.). We do not allow being able to do >> admin >> >> ops using an hbase1 Admin client against an hbase2 cluster. We have some >> >> egregious API violations in branch-1; e.g. we have protobuf in our API >> (See >> >> HBASE-15607). The notion is that we can't afford a deprecation cycle >> >> purging this stuff from our Admin API. >> >> >> >> What you all think? >> >> >> >> St.Ack >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>
[jira] [Created] (HBASE-19145) Look into hbase-2 client going to hbase-1 server
Jerry He created HBASE-19145: Summary: Look into hbase-2 client going to hbase-1 server Key: HBASE-19145 URL: https://issues.apache.org/jira/browse/HBASE-19145 Project: HBase Issue Type: Task Affects Versions: 2.0.0-beta-1 Reporter: Jerry He Priority: Major >From the "[DISCUSS] hbase-2.0.0 compatibility expectations" thread. Do we support hbase-2 client going against hbase-1 server? We seem to be fine mix-and-match the clients and servers within the hbase-1 releases. And IIRC hbase-1 client is ok against 0.98 server. Suppose I have a product that depends and bundles HBase client. I want to upgrade the dependency to hbase-2 so that it can take advantages of and claim support of hbase-2. But does it mean that I will need drop the claims that the new version of the product support any hbase-1 backend? It has not been an objective. It might work doing basic Client API on a later branch-1 but will fail doing Admin functions (and figuring if a Table is online). If it was a small thing to make it work, lets get it in. Let's look into it to see what works and what not. Have a statement at least. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: [DISCUSS] Any committers willing to do maintenance releases over the holidays?
Loads of helping hands...woohoo! -Appy On Oct 31, 2017 4:01 PM, "Nick Dimiduk" wrote: > As 1.1 is rounding down, I will be able to apply that time to helping out > on another release line, if that's useful to the community. > > Thanks, > Nick > > On Mon, Oct 30, 2017 at 9:35 PM, Sean Busbey wrote: > > > Hi folks! > > > > I'm in the process of getting us a 1.2.7-RC, hopefully soon[1]. This > > release is badly behind schedule. I'd very much like to get us back on > > a regular cadence for our stable release line, with a goal of once a > > month. Unfortunately, I'm very unlikely to have time to do release > > managing after the 1.2.7 release until spring (luckily it's for > > fortunate-to-me reasons). > > > > Despite the delay in schedule on this release, generally doing a > > maintenance release is relatively low effort. These kinds of > > "substitute RM" situations are a great chance to get familiar with the > > process while having a well-defined end point. If there's anyone who'd > > like to give it a try by handling things while I'm gone, both I and > > I'm sure the project community would be grateful. > > > > To take on this particular task, one must be a committer already (due > > to ASF policy). It's a great way to "act like a PMC" for those > > committers who'd like to make a long term investment in HBase. > > > > Anyone interested? > > > > -busbey > > > > [1]: https://issues.apache.org/jira/browse/HBASE-18276 > > >
[jira] [Resolved] (HBASE-19135) TestWeakObjectPool time out
[
https://issues.apache.org/jira/browse/HBASE-19135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-19135.
---
Resolution: Fixed
Assignee: stack
Hadoop Flags: Reviewed
Fix Version/s: 2.0.0-beta-1
Pushed to master and branch-2. Thanks for reviews. (Tried to backport to
branch-1 but failed apply)
> TestWeakObjectPool time out
> ---
>
> Key: HBASE-19135
> URL: https://issues.apache.org/jira/browse/HBASE-19135
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19135.master.001.patch
>
>
> On last branch-2 build, this small test timed out. Timeout is 1second which
> given how hostile apache infra has been of late is way to short. Change the
> test to do category-base timeout.
> Here is the report:
> {code}
> Regression
> org.apache.hadoop.hbase.util.TestWeakObjectPool.testCongestion
> Failing for the past 1 build (Since Failed#778 )
> Took 1.1 sec.
> add description
> Error Message
> test timed out after 1000 milliseconds
> Stacktrace
> org.junit.runners.model.TestTimedOutException: test timed out after 1000
> milliseconds
> at
> org.apache.hadoop.hbase.util.TestWeakObjectPool.testCongestion(TestWeakObjectPool.java:128)
> Standard Output
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Re: [DISCUSS] Any committers willing to do maintenance releases over the holidays?
As 1.1 is rounding down, I will be able to apply that time to helping out on another release line, if that's useful to the community. Thanks, Nick On Mon, Oct 30, 2017 at 9:35 PM, Sean Busbey wrote: > Hi folks! > > I'm in the process of getting us a 1.2.7-RC, hopefully soon[1]. This > release is badly behind schedule. I'd very much like to get us back on > a regular cadence for our stable release line, with a goal of once a > month. Unfortunately, I'm very unlikely to have time to do release > managing after the 1.2.7 release until spring (luckily it's for > fortunate-to-me reasons). > > Despite the delay in schedule on this release, generally doing a > maintenance release is relatively low effort. These kinds of > "substitute RM" situations are a great chance to get familiar with the > process while having a well-defined end point. If there's anyone who'd > like to give it a try by handling things while I'm gone, both I and > I'm sure the project community would be grateful. > > To take on this particular task, one must be a committer already (due > to ASF policy). It's a great way to "act like a PMC" for those > committers who'd like to make a long term investment in HBase. > > Anyone interested? > > -busbey > > [1]: https://issues.apache.org/jira/browse/HBASE-18276 >
[jira] [Created] (HBASE-19144) [RSgroups] Regions assigned to a RSGroup all go to FAILED_OPEN state when all servers in the group are unavailable
Andrew Purtell created HBASE-19144: -- Summary: [RSgroups] Regions assigned to a RSGroup all go to FAILED_OPEN state when all servers in the group are unavailable Key: HBASE-19144 URL: https://issues.apache.org/jira/browse/HBASE-19144 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 3.0.0, 1.4.0, 1.5.0 Reporter: Andrew Purtell -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Moving 2.0 forward
On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob wrote: > Hoping to keep momentum going from our Stack working on alpha4, I tried to > take a stab at triaging some of the open beta-1 issues. > > I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done sooner > then I'm happy to see it pulled back in. Trying to balance optimism with > realism here, and knowing that documentation unfortunately often gets > pushed to the back-burner. > > Also, I poked some folks on unassigned issues that they've filed for > beta-1, especially in the last few days. If issues don't have an owner they > are unlikely to get worked. I chatted with stack and he agreed to take on > some of the tasks, but there's a lot of surface area to cover. > > If you you're working on issues that are critical for beta-1, please mark > them as such. Then the rest of the community will know to help prioritize > feedback and reviews there. > > Do we have a general theme for the betas like we did with the alphas? > > Beta1 is upgrades work from branch1, beta2 is rolling upgrades work as > well? Continue to work on tests throughout? > > Thanks Mike for helping to kick off the beta-1 train. Regards a theme for beta-1, I'd like it to be 'finish'; beta-1 is what we are going to ship (beta-2 is rolling upgrade and any minor items turned up in testing/burn-in). Thanks, S > Mike > > On Tue, Oct 31, 2017 at 10:04 AM, Josh Elser wrote: > > > +1 go from my POV. > > > > > > On 10/31/17 10:07 AM, Stack wrote: > > > >> I want to push an alpha-4 today. A few items didn't make it > (HBASE-19092). > >> They need more time. We'll pull them in for beta-1. CP API is basically > >> done. There may be some changes for beta-1 but hopefully only changes > >> informed by experience trying to port an existing Coprocessor to hbase2. > >> > >> Shout if there is anything that needs to make alpha-4. > >> > >> Thanks, > >> St.Ack > >> > >> > >> On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser > wrote: > >> > >> Yup, that was going to be my plan, Mike! > >>> > >>> Making a pass now, and will check back later tonight again. I see > others > >>> have already done some work today on this front. > >>> > >>> > >>> On 10/27/17 11:38 PM, Mike Drob wrote: > >>> > >>> Josh - Do you want to kick off a bunch of QA runs? (Do you know how to > do > it directly on the jenkins job, so you don't have to bother with JIRA > uploads) > > If you're busy, then I can make time tomorrow or Sunday to kick off > jobs, > but I want to make sure we're not duplicating effort and jenkins > cycles. > > On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser > wrote: > > My turn to bump ;) > > > > > By my take: HBASE-18770 and HBASE-19092 are the only issues that > remain > > needing some more work. The rest are just awaiting a good QA run. > > > > Unless I hear otherwise, I'll try to keep an eye on things over the > > weekend, bump them along as necessary, and get them committed. Would > be > > great to be able get a vote up on Monday. > > > > > > On 10/24/17 6:03 PM, Stack wrote: > > > > Chatting with my coworker Mr. Mike Drob, we were batting back and > forth > > > >> what remains to be done. Surfacing our thoughts here so you all > clued > >> inOr if you think otherwise, please speak up. > >> > >> We have ~13 issues to land: > >> https://issues.apache.org/jira/projects/HBASE/versions/12341594 > About > >> two > >> are meta-issues that are about process which leaves 11. > >> > >> Duo and Zheng Hu are to merge the FilterList fixes improvements > >> (HBASE-19057, HBASE-18410 et al.). These are blocker because some > >> changed > >> API/semantic that we need to get out earlier rather than later. > >> > >> Once the above is merged, HBASE-13346, change of Filter method names > >> to > >> mention Cell instead of KeyValue can land. > >> > >> HBASE-199048 needs a review (Anoop will probably do it), removing > >> IA.Private objects as params to MasterObserver... Hopefully this > goes > >> in > >> soon. > >> > >> Duo is hard at work on trackers for flush and compaction for CPs > >> (HBASE-18905). How is HBASE-19033 looking Duo (facility for Tephra)? > >> > >> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate after > >> Duo > >> is > >> done w/ his work above. > >> > >> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after > >> all > >> the > >> purges allowing CPs do direct calls against Regions in same Host. > >> > >> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil. > >> > >> Another day or two? > >> > >> St.Ack > >> > >> > >> > >> On Mon, Oct 23, 2017 at 2:52 PM, Stack wrote: > >> > >> > >> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser > >> wrote: > >> > >>> > >>> +1
Re: Moving 2.0 forward
Hoping to keep momentum going from our Stack working on alpha4, I tried to take a stab at triaging some of the open beta-1 issues. I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done sooner then I'm happy to see it pulled back in. Trying to balance optimism with realism here, and knowing that documentation unfortunately often gets pushed to the back-burner. Also, I poked some folks on unassigned issues that they've filed for beta-1, especially in the last few days. If issues don't have an owner they are unlikely to get worked. I chatted with stack and he agreed to take on some of the tasks, but there's a lot of surface area to cover. If you you're working on issues that are critical for beta-1, please mark them as such. Then the rest of the community will know to help prioritize feedback and reviews there. Do we have a general theme for the betas like we did with the alphas? Beta1 is upgrades work from branch1, beta2 is rolling upgrades work as well? Continue to work on tests throughout? Mike On Tue, Oct 31, 2017 at 10:04 AM, Josh Elser wrote: > +1 go from my POV. > > > On 10/31/17 10:07 AM, Stack wrote: > >> I want to push an alpha-4 today. A few items didn't make it (HBASE-19092). >> They need more time. We'll pull them in for beta-1. CP API is basically >> done. There may be some changes for beta-1 but hopefully only changes >> informed by experience trying to port an existing Coprocessor to hbase2. >> >> Shout if there is anything that needs to make alpha-4. >> >> Thanks, >> St.Ack >> >> >> On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser wrote: >> >> Yup, that was going to be my plan, Mike! >>> >>> Making a pass now, and will check back later tonight again. I see others >>> have already done some work today on this front. >>> >>> >>> On 10/27/17 11:38 PM, Mike Drob wrote: >>> >>> Josh - Do you want to kick off a bunch of QA runs? (Do you know how to do it directly on the jenkins job, so you don't have to bother with JIRA uploads) If you're busy, then I can make time tomorrow or Sunday to kick off jobs, but I want to make sure we're not duplicating effort and jenkins cycles. On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser wrote: My turn to bump ;) > > By my take: HBASE-18770 and HBASE-19092 are the only issues that remain > needing some more work. The rest are just awaiting a good QA run. > > Unless I hear otherwise, I'll try to keep an eye on things over the > weekend, bump them along as necessary, and get them committed. Would be > great to be able get a vote up on Monday. > > > On 10/24/17 6:03 PM, Stack wrote: > > Chatting with my coworker Mr. Mike Drob, we were batting back and forth > >> what remains to be done. Surfacing our thoughts here so you all clued >> inOr if you think otherwise, please speak up. >> >> We have ~13 issues to land: >> https://issues.apache.org/jira/projects/HBASE/versions/12341594 About >> two >> are meta-issues that are about process which leaves 11. >> >> Duo and Zheng Hu are to merge the FilterList fixes improvements >> (HBASE-19057, HBASE-18410 et al.). These are blocker because some >> changed >> API/semantic that we need to get out earlier rather than later. >> >> Once the above is merged, HBASE-13346, change of Filter method names >> to >> mention Cell instead of KeyValue can land. >> >> HBASE-199048 needs a review (Anoop will probably do it), removing >> IA.Private objects as params to MasterObserver... Hopefully this goes >> in >> soon. >> >> Duo is hard at work on trackers for flush and compaction for CPs >> (HBASE-18905). How is HBASE-19033 looking Duo (facility for Tephra)? >> >> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate after >> Duo >> is >> done w/ his work above. >> >> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after >> all >> the >> purges allowing CPs do direct calls against Regions in same Host. >> >> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil. >> >> Another day or two? >> >> St.Ack >> >> >> >> On Mon, Oct 23, 2017 at 2:52 PM, Stack wrote: >> >> >> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser >> wrote: >> >>> >>> +1 >>> >>> I was trying to work on helping out on the outstanding alpha-4 stuff last week -- will be continuing to try to do the same this week. If you need any help, Stack, or if others need reviews where I haven't noticed on my own: feel free to @mention me. Thanks for the offer Josh. All items seem assigned and are being actively >>> worked on. If you get a moment, reviews by you (or anyone else) helps >>> move >>> the process along
Re: [ANNOUNCE] New HBase committer Lars Francke
Thank you very much everyone! On Fri, Oct 27, 2017 at 4:48 PM, Abhishek Singh Chouhan < abhishekchouhan...@gmail.com> wrote: > Congrats Lars!! > > On Thu, Oct 26, 2017 at 1:57 AM Andrew Purtell > wrote: > > > Congratulations and welcome! > > > > > > On Wed, Oct 25, 2017 at 12:56 PM, Lars George > > wrote: > > > > > On behalf of the Apache HBase PMC, I am pleased to announce that Lars > > > Francke has accepted the PMC's invitation to become a committer on the > > > project. > > > > > > We appreciate all of Lars' great work thus far and look forward to > > > continued involvement. > > > > > > Please join me in congratulating LarsF! (Opting to use last name > > > initials as we now have three Lars' as committers) > > > > > > -- > > > Best regards, > > > LarsG > > > > > > > > > > > -- > > Best regards, > > Andrew > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > decrepit hands > >- A23, Crosstalk > > >
[jira] [Created] (HBASE-19143) Add has(Option) to ClusterStatus
Chia-Ping Tsai created HBASE-19143: -- Summary: Add has(Option) to ClusterStatus Key: HBASE-19143 URL: https://issues.apache.org/jira/browse/HBASE-19143 Project: HBase Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai It helps user to distinguish between nothing and you-do-not-ask. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19142) Expunge our dev-support/smart-apply-patch
Sean Busbey created HBASE-19142:
---
Summary: Expunge our dev-support/smart-apply-patch
Key: HBASE-19142
URL: https://issues.apache.org/jira/browse/HBASE-19142
Project: HBase
Issue Type: Task
Components: build, community
Reporter: Sean Busbey
Priority: Minor
right now we have an old support script for applying patches locally,
{{dev-support/smart-apply-patch.sh}}. This functionality is now maintained in
Apache Yetus.
We should either:
1) just remove the file and update any docs that point at it to point at yetus
2) update it to be a wrapper for the yetus implementation as Hadoop has done:
https://github.com/apache/hadoop/blob/trunk/dev-support/bin/smart-apply-patch
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19141) [compat 1-2] getClusterStatus always return empty ClusterStatus
Chia-Ping Tsai created HBASE-19141:
--
Summary: [compat 1-2] getClusterStatus always return empty
ClusterStatus
Key: HBASE-19141
URL: https://issues.apache.org/jira/browse/HBASE-19141
Project: HBase
Issue Type: Sub-task
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Priority: Critical
Fix For: 2.0.0-beta-1
We are able to limit the scope to get part of {{ClusterStatus}} in 2.0. However
the request sent by 1.x client has no specific scope info to retrieve any
information from {{ClusterStatus}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19140) hbase-cleanup.sh uses deprecated call to remove files in hdfs
Artem Ervits created HBASE-19140:
Summary: hbase-cleanup.sh uses deprecated call to remove files in
hdfs
Key: HBASE-19140
URL: https://issues.apache.org/jira/browse/HBASE-19140
Project: HBase
Issue Type: Improvement
Components: scripts
Affects Versions: 1.1.12, 1.2.6, 1.3.1, 2.0.0, 1.4.1
Environment: MVN: 3.5.2
JDK: 1.8.0.144
Reporter: Artem Ervits
Assignee: Artem Ervits
Priority: Trivial
steps to reproduce:
{code}
bin/hbase-cleanup.sh --cleanAll
{code}
{code}
rmr: DEPRECATED: Please use 'rm -r' instead.
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19139) Create Async Admin methods for Clear Block Cache
Zach York created HBASE-19139: - Summary: Create Async Admin methods for Clear Block Cache Key: HBASE-19139 URL: https://issues.apache.org/jira/browse/HBASE-19139 Project: HBase Issue Type: Improvement Components: Admin Reporter: Zach York Assignee: Zach York As part of the review for HBASE-18624, reviewers suggested to add the clear_block_cache to the AsyncAdmin as well. Since the issue was very large, we decided to split this into a follow-up JIRA. The purpose of this JIRA will be to finish the work on the AsyncAdmin. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19138) Rare failure in TestLruBlockCache
Andrew Purtell created HBASE-19138: -- Summary: Rare failure in TestLruBlockCache Key: HBASE-19138 URL: https://issues.apache.org/jira/browse/HBASE-19138 Project: HBase Issue Type: Bug Components: test Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Trivial Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 [ERROR] testCacheEvictionThreadSafe(org.apache.hadoop.hbase.io.hfile.TestLruBlockCache) Time elapsed: 0.028 s <<< FAILURE! java.lang.AssertionError: expected:<0> but was:<1> at org.apache.hadoop.hbase.io.hfile.TestLruBlockCache.testCacheEvictionThreadSafe(TestLruBlockCache.java:86) Fix for this I think is waiting for the block count to drop to zero after awaiting shutdown of the executor pool. There's a rare race here. Tests look good using a Waiter predicate so far, but I haven't finished all 100 iterations yet. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19030) nightly runs should attempt to log test results after archiving
[ https://issues.apache.org/jira/browse/HBASE-19030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-19030. --- Resolution: Fixed Closing again. Opened HBASE-19137 to fix the issue [~busbey] identified w/ the fix here. Thanks. > nightly runs should attempt to log test results after archiving > --- > > Key: HBASE-19030 > URL: https://issues.apache.org/jira/browse/HBASE-19030 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13, 2.0.0-alpha-4 > > Attachments: HBASE-19030.0.patch > > > right now on the nightly tests the first post-action we do is log junit > results. due to current limitations of Jenkins DSL, if this step fails none > of the other post actions will happen. > Since we might not make junit test results, e.g. in the case of a timeout of > yetus itself, we should log the junit results after we've saved whatever we > can of yetus output. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19137) Fix failures caused by 'nightly runs should attempt to log results after archiving'
stack created HBASE-19137: - Summary: Fix failures caused by 'nightly runs should attempt to log results after archiving' Key: HBASE-19137 URL: https://issues.apache.org/jira/browse/HBASE-19137 Project: HBase Issue Type: Bug Reporter: stack HBASE-19030 "nightly runs should attempt to log tests results after archiving" was reopened because its fix was causing failures (reported by our @busbey). This issue is about fixing the new failures (originally to be done against reopened HBASE-19030 but time passed). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19136) Set branch-2 version to 2.0.0-alpha4 from 2.0.0-alpha4-SNAPSHOT
[ https://issues.apache.org/jira/browse/HBASE-19136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-19136. --- Resolution: Fixed Assignee: stack Done. > Set branch-2 version to 2.0.0-alpha4 from 2.0.0-alpha4-SNAPSHOT > --- > > Key: HBASE-19136 > URL: https://issues.apache.org/jira/browse/HBASE-19136 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-4 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19136) Set branch-2 version to 2.0.0-alpha4 from 2.0.0-alpha4-SNAPSHOT
stack created HBASE-19136: - Summary: Set branch-2 version to 2.0.0-alpha4 from 2.0.0-alpha4-SNAPSHOT Key: HBASE-19136 URL: https://issues.apache.org/jira/browse/HBASE-19136 Project: HBase Issue Type: Sub-task Reporter: stack -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19135) TestWeakObjectPool time out
stack created HBASE-19135:
-
Summary: TestWeakObjectPool time out
Key: HBASE-19135
URL: https://issues.apache.org/jira/browse/HBASE-19135
Project: HBase
Issue Type: Bug
Reporter: stack
On last branch-2 build, this small test timed out. Timeout is 1second which
given how hostile apache infra has been of late is way to short. Change the
test to do category-base timeout.
Here is the report:
{code}
Regression
org.apache.hadoop.hbase.util.TestWeakObjectPool.testCongestion
Failing for the past 1 build (Since Failed#778 )
Took 1.1 sec.
add description
Error Message
test timed out after 1000 milliseconds
Stacktrace
org.junit.runners.model.TestTimedOutException: test timed out after 1000
milliseconds
at
org.apache.hadoop.hbase.util.TestWeakObjectPool.testCongestion(TestWeakObjectPool.java:128)
Standard Output
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Re: [DISCUSS] Any committers willing to do maintenance releases over the holidays?
Can't (realistically) commit to doing one on my own, but I'll be around to help the process along. Appy/others, feel free to ask directly or on list and I'll try to help out! On 10/31/17 11:37 AM, Apekshit Sharma wrote: I can do it. -- Appy On Mon, Oct 30, 2017 at 9:35 PM, Sean Busbey wrote: Hi folks! I'm in the process of getting us a 1.2.7-RC, hopefully soon[1]. This release is badly behind schedule. I'd very much like to get us back on a regular cadence for our stable release line, with a goal of once a month. Unfortunately, I'm very unlikely to have time to do release managing after the 1.2.7 release until spring (luckily it's for fortunate-to-me reasons). Despite the delay in schedule on this release, generally doing a maintenance release is relatively low effort. These kinds of "substitute RM" situations are a great chance to get familiar with the process while having a well-defined end point. If there's anyone who'd like to give it a try by handling things while I'm gone, both I and I'm sure the project community would be grateful. To take on this particular task, one must be a committer already (due to ASF policy). It's a great way to "act like a PMC" for those committers who'd like to make a long term investment in HBase. Anyone interested? -busbey [1]: https://issues.apache.org/jira/browse/HBASE-18276
Re: [DISCUSS] Any committers willing to do maintenance releases over the holidays?
I can do it. -- Appy On Mon, Oct 30, 2017 at 9:35 PM, Sean Busbey wrote: > Hi folks! > > I'm in the process of getting us a 1.2.7-RC, hopefully soon[1]. This > release is badly behind schedule. I'd very much like to get us back on > a regular cadence for our stable release line, with a goal of once a > month. Unfortunately, I'm very unlikely to have time to do release > managing after the 1.2.7 release until spring (luckily it's for > fortunate-to-me reasons). > > Despite the delay in schedule on this release, generally doing a > maintenance release is relatively low effort. These kinds of > "substitute RM" situations are a great chance to get familiar with the > process while having a well-defined end point. If there's anyone who'd > like to give it a try by handling things while I'm gone, both I and > I'm sure the project community would be grateful. > > To take on this particular task, one must be a committer already (due > to ASF policy). It's a great way to "act like a PMC" for those > committers who'd like to make a long term investment in HBase. > > Anyone interested? > > -busbey > > [1]: https://issues.apache.org/jira/browse/HBASE-18276 > -- -- Appy
Re: Moving 2.0 forward
+1 go from my POV. On 10/31/17 10:07 AM, Stack wrote: I want to push an alpha-4 today. A few items didn't make it (HBASE-19092). They need more time. We'll pull them in for beta-1. CP API is basically done. There may be some changes for beta-1 but hopefully only changes informed by experience trying to port an existing Coprocessor to hbase2. Shout if there is anything that needs to make alpha-4. Thanks, St.Ack On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser wrote: Yup, that was going to be my plan, Mike! Making a pass now, and will check back later tonight again. I see others have already done some work today on this front. On 10/27/17 11:38 PM, Mike Drob wrote: Josh - Do you want to kick off a bunch of QA runs? (Do you know how to do it directly on the jenkins job, so you don't have to bother with JIRA uploads) If you're busy, then I can make time tomorrow or Sunday to kick off jobs, but I want to make sure we're not duplicating effort and jenkins cycles. On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser wrote: My turn to bump ;) By my take: HBASE-18770 and HBASE-19092 are the only issues that remain needing some more work. The rest are just awaiting a good QA run. Unless I hear otherwise, I'll try to keep an eye on things over the weekend, bump them along as necessary, and get them committed. Would be great to be able get a vote up on Monday. On 10/24/17 6:03 PM, Stack wrote: Chatting with my coworker Mr. Mike Drob, we were batting back and forth what remains to be done. Surfacing our thoughts here so you all clued inOr if you think otherwise, please speak up. We have ~13 issues to land: https://issues.apache.org/jira/projects/HBASE/versions/12341594 About two are meta-issues that are about process which leaves 11. Duo and Zheng Hu are to merge the FilterList fixes improvements (HBASE-19057, HBASE-18410 et al.). These are blocker because some changed API/semantic that we need to get out earlier rather than later. Once the above is merged, HBASE-13346, change of Filter method names to mention Cell instead of KeyValue can land. HBASE-199048 needs a review (Anoop will probably do it), removing IA.Private objects as params to MasterObserver... Hopefully this goes in soon. Duo is hard at work on trackers for flush and compaction for CPs (HBASE-18905). How is HBASE-19033 looking Duo (facility for Tephra)? I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate after Duo is done w/ his work above. I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after all the purges allowing CPs do direct calls against Regions in same Host. Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil. Another day or two? St.Ack On Mon, Oct 23, 2017 at 2:52 PM, Stack wrote: On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser wrote: +1 I was trying to work on helping out on the outstanding alpha-4 stuff last week -- will be continuing to try to do the same this week. If you need any help, Stack, or if others need reviews where I haven't noticed on my own: feel free to @mention me. Thanks for the offer Josh. All items seem assigned and are being actively worked on. If you get a moment, reviews by you (or anyone else) helps move the process along. We need to merge in HBASE-18410 branch to pick up Filter improvements. Then HBASE-13346 can go in. You are already helping out on HBASE-18906, thanks. Looks like that will be addressed by other alpha-4s about to land. St.Ack TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594 On 10/23/17 12:53 PM, Stack wrote: (Reviving this thread) Lets push out alpha-4 this week. Alpha-4 is the release that has the refactor of the Coprocessor API shutting down access to internals marked InterfaceAudience.Private. The outstanding list is here: https://issues.apache.org/jira/projects/HBASE/versions/12341594 Please push in anything marked alpha-4 that belongs to you. If issue, talk out loud on this thread. If you need a review to land an item, shout on the issue and here; we'll help you out. As is, items are coming along nicely I'd say. We need to merge the filter branch -- HBASE-18410 -- so APIs are finished for hbase2. Post alpha-4, we'll have to hunt down our downstreamers and help them test on top of alpha-4 so rolling into beta-1, we have confidence our downstreamers know what to expect (or we discover what we missed BEFORE we beta-1). Thanks for time, S On Fri, Sep 8, 2017 at 2:04 PM, Stack wrote: I'll put up an alpha3 RC Monday, probably Monday night. That should be time, if we all sprint, for the public-facing API fixes to be done. I had a bunch of Coprocessor refactor and fixup scheduled for alpha3 but it is plain that more time is needed (in spite of valiant effort so far by Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose theme is "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the following week. We should then be ready for beta (beta == no new feature
Re: Moving 2.0 forward
I want to push an alpha-4 today. A few items didn't make it (HBASE-19092). They need more time. We'll pull them in for beta-1. CP API is basically done. There may be some changes for beta-1 but hopefully only changes informed by experience trying to port an existing Coprocessor to hbase2. Shout if there is anything that needs to make alpha-4. Thanks, St.Ack On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser wrote: > Yup, that was going to be my plan, Mike! > > Making a pass now, and will check back later tonight again. I see others > have already done some work today on this front. > > > On 10/27/17 11:38 PM, Mike Drob wrote: > >> Josh - Do you want to kick off a bunch of QA runs? (Do you know how to do >> it directly on the jenkins job, so you don't have to bother with JIRA >> uploads) >> >> If you're busy, then I can make time tomorrow or Sunday to kick off jobs, >> but I want to make sure we're not duplicating effort and jenkins cycles. >> >> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser wrote: >> >> My turn to bump ;) >>> >>> By my take: HBASE-18770 and HBASE-19092 are the only issues that remain >>> needing some more work. The rest are just awaiting a good QA run. >>> >>> Unless I hear otherwise, I'll try to keep an eye on things over the >>> weekend, bump them along as necessary, and get them committed. Would be >>> great to be able get a vote up on Monday. >>> >>> >>> On 10/24/17 6:03 PM, Stack wrote: >>> >>> Chatting with my coworker Mr. Mike Drob, we were batting back and forth what remains to be done. Surfacing our thoughts here so you all clued inOr if you think otherwise, please speak up. We have ~13 issues to land: https://issues.apache.org/jira/projects/HBASE/versions/12341594 About two are meta-issues that are about process which leaves 11. Duo and Zheng Hu are to merge the FilterList fixes improvements (HBASE-19057, HBASE-18410 et al.). These are blocker because some changed API/semantic that we need to get out earlier rather than later. Once the above is merged, HBASE-13346, change of Filter method names to mention Cell instead of KeyValue can land. HBASE-199048 needs a review (Anoop will probably do it), removing IA.Private objects as params to MasterObserver... Hopefully this goes in soon. Duo is hard at work on trackers for flush and compaction for CPs (HBASE-18905). How is HBASE-19033 looking Duo (facility for Tephra)? I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate after Duo is done w/ his work above. I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after all the purges allowing CPs do direct calls against Regions in same Host. Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil. Another day or two? St.Ack On Mon, Oct 23, 2017 at 2:52 PM, Stack wrote: On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser wrote: > > +1 > >> >> I was trying to work on helping out on the outstanding alpha-4 stuff >> last >> week -- will be continuing to try to do the same this week. >> >> If you need any help, Stack, or if others need reviews where I haven't >> noticed on my own: feel free to @mention me. >> >> >> Thanks for the offer Josh. All items seem assigned and are being >> > actively > worked on. If you get a moment, reviews by you (or anyone else) helps > move > the process along. > > We need to merge in HBASE-18410 branch to pick up Filter improvements. > Then HBASE-13346 can go in. > > You are already helping out on HBASE-18906, thanks. Looks like that > will > be addressed by other alpha-4s about to land. > > St.Ack > TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594 > > > > > > > > > > On 10/23/17 12:53 PM, Stack wrote: > >> >> (Reviving this thread) >> >>> >>> Lets push out alpha-4 this week. Alpha-4 is the release that has the >>> refactor of the Coprocessor API shutting down access to internals >>> marked >>> InterfaceAudience.Private. >>> >>> The outstanding list is here: >>> https://issues.apache.org/jira/projects/HBASE/versions/12341594 >>> >>> Please push in anything marked alpha-4 that belongs to you. >>> >>> If issue, talk out loud on this thread. If you need a review to land >>> an >>> item, shout on the issue and here; we'll help you out. >>> >>> As is, items are coming along nicely I'd say. We need to merge the >>> filter >>> branch -- HBASE-18410 -- so APIs are finished for hbase2. >>> >>> Post alpha-4, we'll have to hunt down our downstreamers and help them >>> test >>> on top of alpha-4 so rolling into beta-1, we have confidence our >>> downstreamers know w
[jira] [Created] (HBASE-19134) Make WALKey an Interface; expose Read-Only version to CPs
stack created HBASE-19134: - Summary: Make WALKey an Interface; expose Read-Only version to CPs Key: HBASE-19134 URL: https://issues.apache.org/jira/browse/HBASE-19134 Project: HBase Issue Type: Bug Components: Coprocessors, wal Reporter: stack Assignee: stack Fix For: 2.0.0-beta-1 WALKey has been made IA.Private in hbase2. Even so, given we don't have an alternative to expose at this time, it is exposed to coprocessors still at a few (now deprecated) locations. In review of HBASE-18770, [~chia7712] makes reasonable suggestion that what we expose to CPs be a read-only WALKey. He gets pushback on doing this for hbase2 (Do we even want to expose WALKey to CPs, is WALKey right going forward, etc.). Chia-Ping comes back w/ the below (copied from HBASE-18770): What we want to fix for WALKey are shown below. * expose some methods to CP user safety * refactor/redo As I see it, adding an interface exposed to CP user for WALKey is a right choice because it can bring some benefit. * We can expose part of WALKey's methods to CP users - a read-only interface or an interface with some modifiable setting. * The related CP hooks won't be deprecated * Doing the refactor for WALKey doesn't essentially impact the CP hook after 2.0 release. Although, it will be better to redo WALKey before 2.0 release. In short, I think it warrants such an interface. (We both agree this would be a load of work given WALKey is written to HFiles. Warrants a look though). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19133) Transfer big cells or upserted/appended cells into MSLAB upon flattening to CellChunkMap
Anastasia Braginsky created HBASE-19133: --- Summary: Transfer big cells or upserted/appended cells into MSLAB upon flattening to CellChunkMap Key: HBASE-19133 URL: https://issues.apache.org/jira/browse/HBASE-19133 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky CellChunkMap Segment index requires all cell data to be written in the MSLAB Chunks. Eventhough MSLAB is enabled, cells bigger than chunk size or upserted/incremented/appended cells are still allocated on the JVM stack. If such cells are found in the process of flattening into CellChunkMap (in-memory-flush) they need to be copied into MSLAB. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19132) Better align bypassable coprocessor methods so all have same pattern
stack created HBASE-19132: - Summary: Better align bypassable coprocessor methods so all have same pattern Key: HBASE-19132 URL: https://issues.apache.org/jira/browse/HBASE-19132 Project: HBase Issue Type: Bug Components: Coprocessors Reporter: stack A coprocessor observer hook bypasses -- asks to skip normal processing and instead return a result conjured by the coprocessor -- by calling bypass on the passed in context. How the bypass is conveyed back to the executing code varies. Usually it is a boolean that the execution tests. But it gets a little odd around Scan Coprocessors especially around next. In a few places we used to allow return of a null object to indicate bypass (flush, compact) but these were fixed by HBASE-19122. This issue is about giving a general pass over bypass to unify how it done everywhere. I don't think we too bad now after HBASE-18770 but more cleanup to do (Probably an hbase-3 project). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19131) Remove the ListDeadServers hook in MasterObserver
Chia-Ping Tsai created HBASE-19131:
--
Summary: Remove the ListDeadServers hook in MasterObserver
Key: HBASE-19131
URL: https://issues.apache.org/jira/browse/HBASE-19131
Project: HBase
Issue Type: Sub-task
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
The {{ListDeadServers}}, which is introduced by HBASE-18131, can be replaced by
{{getClusterStatus}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19130) Typo in HStore.initializeRegionInternals for replaying wal
Duo Zhang created HBASE-19130:
-
Summary: Typo in HStore.initializeRegionInternals for replaying wal
Key: HBASE-19130
URL: https://issues.apache.org/jira/browse/HBASE-19130
Project: HBase
Issue Type: Bug
Components: Recovery
Reporter: Duo Zhang
Priority: Critical
Fix For: 2.0.0-alpha4
{code}
} finally {
// update the stores that we are done replaying
stores.forEach(HStore::startReplayingFromWAL);
}
{code}
Should be stopReplayingFromWAL. Found this when implementing HBASE-19095.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
