[jira] [Reopened] (HBASE-19009) implement modifyTable and enable/disableTableReplication for AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-19009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reopened HBASE-19009: > implement modifyTable and enable/disableTableReplication for AsyncAdmin > --- > > Key: HBASE-19009 > URL: https://issues.apache.org/jira/browse/HBASE-19009 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 3.0.0, 2.0.0-beta-1 > > Attachments: HBASE-19009.master.001.patch, > HBASE-19009.master.002.patch, HBASE-19009.master.003.patch, > HBASE-19009.master.004.patch, HBASE-19009.master.005.patch, > HBASE-19009.master.006.patch, HBASE-19009.master.007.patch, > HBASE-19009.master.008.patch, HBASE-19009.master.009.patch, > HBASE-19009.master.010.patch, HBASE-19009.master.011.patch, > HBASE-19009.master.012.patch, HBASE-19009.master.addendum.patch > > > Add 3 methods to AsyncAdmin. > modifyTable() > enableTableReplication() > disableTableReplication() -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19279) Reenable TestMasterBalanceThrottling disabled by HBASE-14614
stack created HBASE-19279: - Summary: Reenable TestMasterBalanceThrottling disabled by HBASE-14614 Key: HBASE-19279 URL: https://issues.apache.org/jira/browse/HBASE-19279 Project: HBase Issue Type: Sub-task Reporter: stack See head of class for a comment. It depends on simpleloadbalancer which seems broke. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19278) Reenable cleanup in test teardown in TestAccessController3 disabled by HBASE-14614
stack created HBASE-19278: - Summary: Reenable cleanup in test teardown in TestAccessController3 disabled by HBASE-14614 Key: HBASE-19278 URL: https://issues.apache.org/jira/browse/HBASE-19278 Project: HBase Issue Type: Sub-task Reporter: stack We'd commented out cleanup in the test because test was stalling on it. This issue is about restoring it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: [DISCUSS] [HBASE-19207] HBase REST API
I think this is a laudable goal, so please proceed, but note: > There is no clear separation of client and server components. The REST client code, as such, is meant just to debug the server components, and is used by unit tests, and is not meant as any kind of "official" REST client, and this explains why there has been no such care taken. We've talked before about moving these classes from src/main/ to src/test/ and that should have been done to avoid this kind of confusion. On Tue, Nov 14, 2017 at 2:58 PM, Rick Kellogg wrote: > Greetings, > > > > About a week ago, I opened a new JIRA ticket [HBASE-19207] - Create a > Minimal HBase REST Client. As I started to study the feasibility of this > task, I determined the project dependencies were a hornets nest. There is > no clear separation of client and server components. For example the > VersionModel class relies upon the javax.servlet.ServletContext class for > construction on the server. There are many other examples of blurred > boundaries. > > > > I initially tried to break out the client specific files plus protocol > buffer related files in the hbase-rest project. This lead me to believe we > could create a clear separation of client and server components. The > trouble started on how to provide access to classes such as Cell or Scan > from the hbase-client. Using the hbase-shaded-client seems to be a > solution > but far from ideal. This failed as well because of complications with > shading. The protocol buffer related stuff was generated twice. Once in > my > project and already in the hbase-shaded-client. > > > > So as an interim solution, I would like to suggest the introduction of a > Apache Maven Bill-of-Materials (BOM) for access to the HBase REST Client. > I > have created a BOM for 2.0.0-alpha4 on Github for comment: > > > > https://github.com/rmkellogg/hbase-rest-client-ex > > > > Using the BOM feature, I was able to reduce the number of dependencies from > 146 to 21. > > > > I think we could allow some flexibility for upgrade to common libraries > (Guava, Apache HttpComponents, etc) using this technique. > > > > Thoughts and comments appreciated. > > Rick Kellogg > > > > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Created] (HBASE-19277) hbase sync between 0.9 and 1.2
SuperbDong created HBASE-19277: -- Summary: hbase sync between 0.9 and 1.2 Key: HBASE-19277 URL: https://issues.apache.org/jira/browse/HBASE-19277 Project: HBase Issue Type: Brainstorming Affects Versions: 1.2.6 Environment: hbase 0.9 hbase1.2 Reporter: SuperbDong I expect synchrodata between hbase 0.9 and hbase 1.2. What's more,I find several ways to do it. Follow : 1.replication (need modify) 2.sync hlog before delete to hdfs .oldlog (need modify) 3.client writes data to two hbase 4.client writes data to kafka and consume to two hbase But, I think the bigest question is one java client how to use two hbase-cliet jar,It must be conflict,How can I do? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19276) RegionPlan should correctly implement equals and hashCode
Andrew Purtell created HBASE-19276: -- Summary: RegionPlan should correctly implement equals and hashCode Key: HBASE-19276 URL: https://issues.apache.org/jira/browse/HBASE-19276 Project: HBase Issue Type: Bug Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.4.0, 2.0.0-beta-1 error-prone identified dodgy code in AssignmentManager where we are relying on reference (object) equality to do the right thing, and are getting lucky, because if we properly used equals() the result is wrong, because RegionPlan does not correctly implement equals and hashCode according to the JDK contracts for same. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19275) TestSnapshotFileCache never worked properly
Andrew Purtell created HBASE-19275:
--
Summary: TestSnapshotFileCache never worked properly
Key: HBASE-19275
URL: https://issues.apache.org/jira/browse/HBASE-19275
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 3.0.0, 1.4.0, 1.5.0
Reporter: Andrew Purtell
Error-prone noticed we were asking Iterables.contains() questions with the
wrong type. I've attached a fixed version of the test. The results suggest the
cache is not evicting entries properly.
{noformat}
java.lang.AssertionError: Cache found
'hdfs://localhost:52867/user/apurtell/test-data/8ce04c85-ce4b-4844-b454-5303482ade95/data/default/snapshot1/9e49edd0ab41657fb0c6ebb4d9dfad15/cf/f132e5b06f66443f8003363ed1535aac',
but it shouldn't have.
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertFalse(Assert.java:64)
at
org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.createAndTestSnapshot(TestSnapshotFileCache.java:260)
at
org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.createAndTestSnapshotV1(TestSnapshotFileCache.java:206)
at
org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.testReloadModifiedDirectory(TestSnapshotFileCache.java:102)
{noformat}
{noformat}
java.lang.AssertionError: Cache found
'hdfs://localhost:52867/user/apurtell/test-data/8ce04c85-ce4b-4844-b454-5303482ade95/data/default/snapshot1a/2e81adb9212c98cff970eafa006fc40b/cf/a2ec478d850e4e348359699c53b732c4',
but it shouldn't have.
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertFalse(Assert.java:64)
at
org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.createAndTestSnapshot(TestSnapshotFileCache.java:260)
at
org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.createAndTestSnapshotV1(TestSnapshotFileCache.java:206)
at
org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.testLoadAndDelete(TestSnapshotFileCache.java:88)
{noformat}
These changes are part of HBASE-19239
I've disabled the offending test cases with @Ignore in that patch, but they
should be reenabled and fixed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19274) Log IOException when unable to determine the size of committed file
Ted Yu created HBASE-19274:
--
Summary: Log IOException when unable to determine the size of
committed file
Key: HBASE-19274
URL: https://issues.apache.org/jira/browse/HBASE-19274
Project: HBase
Issue Type: Improvement
Reporter: Ted Yu
Priority: Minor
During troubleshooting of slow response, I saw the following in region server
log:
{code}
2017-10-26 14:03:53,080 WARN org.apache.hadoop.hbase.regionserver.HRegion:
Failed to find the size of hfile
hdfs://BETA/hbase/data/default/beta_b_history/e514111fae9d7ffc38ed48ad72fa197f/d/04d7c9fce73d4197be114448b1eb295a_SeqId_3766_
{code}
Here is related code:
{code}
} catch (IOException e) {
LOG.warn("Failed to find the size of hfile " + commitedStoreFile);
{code}
The exception should also be logged to facilitate debugging.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19273) IntegrationTestBulkLoad#installSlowingCoproc() uses read-only HTableDescriptor
Ted Yu created HBASE-19273:
--
Summary: IntegrationTestBulkLoad#installSlowingCoproc() uses
read-only HTableDescriptor
Key: HBASE-19273
URL: https://issues.apache.org/jira/browse/HBASE-19273
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
[~romil.choksi] reported the following :
{code}
2017-11-15 23:03:04,455 ERROR [main] util.AbstractHBaseTool: Error running
command-line tool
java.lang.UnsupportedOperationException: HTableDescriptor is read-only
at
org.apache.hadoop.hbase.client.ImmutableHTableDescriptor.getDelegateeForModification(ImmutableHTableDescriptor.java:59)
at
org.apache.hadoop.hbase.HTableDescriptor.addCoprocessor(HTableDescriptor.java:710)
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.installSlowingCoproc(IntegrationTestBulkLoad.java:215)
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.testBulkLoad(IntegrationTestBulkLoad.java:222)
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.runTestFromCommandLine(IntegrationTestBulkLoad.java:790)
at
org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:155)
{code}
This is due to read only descriptor being used.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19272) Reenable HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again...
stack created HBASE-19272: - Summary: Reenable HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again... Key: HBASE-19272 URL: https://issues.apache.org/jira/browse/HBASE-19272 Project: HBase Issue Type: Sub-task Reporter: stack DIsabled by HBASE-14614, enabling AMv2. See HBASE-18110. Here is the list: * TestHBaseFsckTwoRS * TestOfflineMetaRebuildBase * TestHBaseFsckReplicas * TestOfflineMetaRebuildOverlap * TestHBaseFsckOneRS * TestOfflineMetaRebuildHole -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19271) Update ref guide about the async client to reflect the change in HBASE-19251
Duo Zhang created HBASE-19271: - Summary: Update ref guide about the async client to reflect the change in HBASE-19251 Key: HBASE-19271 URL: https://issues.apache.org/jira/browse/HBASE-19271 Project: HBase Issue Type: Task Components: documentation Reporter: Duo Zhang -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19270) Reenable TestRegionMergeTransactionOnCluster#testMergeWithReplicas disable by HBASE-14614
stack created HBASE-19270: - Summary: Reenable TestRegionMergeTransactionOnCluster#testMergeWithReplicas disable by HBASE-14614 Key: HBASE-19270 URL: https://issues.apache.org/jira/browse/HBASE-19270 Project: HBase Issue Type: Sub-task Reporter: stack -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19269) Reenable TestShellRSGroups
stack created HBASE-19269: - Summary: Reenable TestShellRSGroups Key: HBASE-19269 URL: https://issues.apache.org/jira/browse/HBASE-19269 Project: HBase Issue Type: Sub-task Reporter: stack It was disabled by the parent issue because RSGroups was failing. RSGroups now works but this test is still failling. Need to dig in (signal from these jruby tests is murky). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: [DISCUSS] Any committers willing to do maintenance releases over the holidays?
:-) So if 1.2.7 gets out by say around Nov end, should I give around a month for decent amount of changes to pile up for 1.2.8? So maybe start around Christmas. Does that sound right? -- Appy On Wed, Nov 15, 2017 at 9:58 AM, Sean Busbey wrote: > I really hate autocorrect. I followed up with Appy. For whatever reason my > phone thinks he's a huge multinational company. > > On Nov 15, 2017 11:57, "Sean Busbey" wrote: > > > I followed up with Apple and I think he's positioned to get things done. > > I'll leave the timing up to him. > > > > That said, I am already on progress on 1.2.7. currently RCs for that > > release are blocked on getting nightly tests that pass again. > > > > On Nov 15, 2017 10:54, "Nick Dimiduk" wrote: > > > >> Sean, > >> > >> Do you have a target date you'd like to see the community aiming for on > >> this one? > >> > >> If you don't mind, I would propose the Release Wrangler get started in > >> JIRA > >> pruning and test suite kicking over the next couple weeks, and we can > >> target the first or second week of Dec. There's been quite a few commits > >> since the last release, so there may be some surprises lurking for our > >> volunteers to sort out. A good place to start is the compatibility > report. > >> > >> Who will carry this banner? Apekshit, Jan: shall we draw straws? One of > >> you > >> have a bit of extra free time over the next 4 weeks? > >> > >> Thanks, > >> Nick > >> > >> On Wed, Nov 1, 2017 at 11:33 AM Jan Hentschel < > >> jan.hentsc...@ultratendency.com> wrote: > >> > >> > I also like to volunteer, but I don’t think that I can do it on my > own. > >> > > >> > > >> > > >> > Jan > >> > > >> > > >> > > >> > From: Apekshit Sharma > >> > Reply-To: > >> > Date: Wednesday, November 1, 2017 at 12:24 AM > >> > To: > >> > Subject: Re: [DISCUSS] Any committers willing to do maintenance > releases > >> > over the holidays? > >> > > >> > > >> > > >> > Loads of helping hands...woohoo! > >> > > >> > > >> > > >> > -Appy > >> > > >> > > >> > > >> > > >> > > >> > On Oct 31, 2017 4:01 PM, "Nick Dimiduk" wrote: > >> > > >> > > >> > > >> > As 1.1 is rounding down, I will be able to apply that time to helping > >> out > >> > > >> > on another release line, if that's useful to the community. > >> > > >> > > >> > > >> > Thanks, > >> > > >> > Nick > >> > > >> > > >> > > >> > On Mon, Oct 30, 2017 at 9:35 PM, Sean Busbey > wrote: > >> > > >> > > >> > > >> > > Hi folks! > >> > > >> > > > >> > > >> > > I'm in the process of getting us a 1.2.7-RC, hopefully soon[1]. This > >> > > >> > > release is badly behind schedule. I'd very much like to get us back > on > >> > > >> > > a regular cadence for our stable release line, with a goal of once a > >> > > >> > > month. Unfortunately, I'm very unlikely to have time to do release > >> > > >> > > managing after the 1.2.7 release until spring (luckily it's for > >> > > >> > > fortunate-to-me reasons). > >> > > >> > > > >> > > >> > > Despite the delay in schedule on this release, generally doing a > >> > > >> > > maintenance release is relatively low effort. These kinds of > >> > > >> > > "substitute RM" situations are a great chance to get familiar with > the > >> > > >> > > process while having a well-defined end point. If there's anyone > who'd > >> > > >> > > like to give it a try by handling things while I'm gone, both I and > >> > > >> > > I'm sure the project community would be grateful. > >> > > >> > > > >> > > >> > > To take on this particular task, one must be a committer already > (due > >> > > >> > > to ASF policy). It's a great way to "act like a PMC" for those > >> > > >> > > committers who'd like to make a long term investment in HBase. > >> > > >> > > > >> > > >> > > Anyone interested? > >> > > >> > > > >> > > >> > > -busbey > >> > > >> > > > >> > > >> > > [1]: https://issues.apache.org/jira/browse/HBASE-18276 > >> > > >> > > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > > -- -- Appy
[jira] [Created] (HBASE-19268) Enable Replica tests that were disabled by Proc-V2 AM in HBASE-14614
stack created HBASE-19268: - Summary: Enable Replica tests that were disabled by Proc-V2 AM in HBASE-14614 Key: HBASE-19268 URL: https://issues.apache.org/jira/browse/HBASE-19268 Project: HBase Issue Type: Sub-task Components: test Reporter: stack Assignee: stack Fix For: 2.0.0-beta-1 Reenable replica tests disabled so could land AMv2. In particular, reenable... Disabled testRegionReplicasOnMidClusterHighReplication in TestStochasticLoadBalancer2 Disabled testFlushAndCompactionsInPrimary in TestRegionReplicas Enabling these tests used to be part of HBASE-18352 but we broke them out of there so HBASE-18352 is only about fixing/reenabling TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: [DISCUSS] Plan to avoid backup/restore removal from 2.0
On 11/14/17 4:54 PM, Mike Drob wrote: I can see a small section on the documentation update I've already been hacking on to include details on the issue "We can't help you secure where you put the data". Given how many instances of "globally readable S3 bucket" I've seen recently, this strikes me as prudent. I would prefer this to be a giant, hard to miss, red letters, all caps warning; not a small section. I do think it is our responsibility for telling users how to configure the backup/restore process for communicating with secure systems. Or, at a minimum, documenting how we pass arbitrary configuration options that can then be used to communicate with said systems. :D For example, if we support writing backups to S3, then we should have a way to specify an Auth string and maybe even some of the custom headers like x-amz-acl. We don't have to explicitly enumerate best practices, but if the only option is to write to a globally open bucket, then I don't think we should advertise writing to S3 as an available option. Similarly, if we tell people that they can send backups to HDFS, then we should give them the hooks to correctly interface with a kerberized HDFS. Maybe this is already in the proposed patch, I haven't gone looking yet. Nope. I actually meant to include this in the patch I re-rolled today but forgot. Let me update once more. Thanks again, Mike. Good questions/feedback!
Re: [DISCUSS] Any committers willing to do maintenance releases over the holidays?
I really hate autocorrect. I followed up with Appy. For whatever reason my phone thinks he's a huge multinational company. On Nov 15, 2017 11:57, "Sean Busbey" wrote: > I followed up with Apple and I think he's positioned to get things done. > I'll leave the timing up to him. > > That said, I am already on progress on 1.2.7. currently RCs for that > release are blocked on getting nightly tests that pass again. > > On Nov 15, 2017 10:54, "Nick Dimiduk" wrote: > >> Sean, >> >> Do you have a target date you'd like to see the community aiming for on >> this one? >> >> If you don't mind, I would propose the Release Wrangler get started in >> JIRA >> pruning and test suite kicking over the next couple weeks, and we can >> target the first or second week of Dec. There's been quite a few commits >> since the last release, so there may be some surprises lurking for our >> volunteers to sort out. A good place to start is the compatibility report. >> >> Who will carry this banner? Apekshit, Jan: shall we draw straws? One of >> you >> have a bit of extra free time over the next 4 weeks? >> >> Thanks, >> Nick >> >> On Wed, Nov 1, 2017 at 11:33 AM Jan Hentschel < >> jan.hentsc...@ultratendency.com> wrote: >> >> > I also like to volunteer, but I don’t think that I can do it on my own. >> > >> > >> > >> > Jan >> > >> > >> > >> > From: Apekshit Sharma >> > Reply-To: >> > Date: Wednesday, November 1, 2017 at 12:24 AM >> > To: >> > Subject: Re: [DISCUSS] Any committers willing to do maintenance releases >> > over the holidays? >> > >> > >> > >> > Loads of helping hands...woohoo! >> > >> > >> > >> > -Appy >> > >> > >> > >> > >> > >> > On Oct 31, 2017 4:01 PM, "Nick Dimiduk" wrote: >> > >> > >> > >> > As 1.1 is rounding down, I will be able to apply that time to helping >> out >> > >> > on another release line, if that's useful to the community. >> > >> > >> > >> > Thanks, >> > >> > Nick >> > >> > >> > >> > On Mon, Oct 30, 2017 at 9:35 PM, Sean Busbey wrote: >> > >> > >> > >> > > Hi folks! >> > >> > > >> > >> > > I'm in the process of getting us a 1.2.7-RC, hopefully soon[1]. This >> > >> > > release is badly behind schedule. I'd very much like to get us back on >> > >> > > a regular cadence for our stable release line, with a goal of once a >> > >> > > month. Unfortunately, I'm very unlikely to have time to do release >> > >> > > managing after the 1.2.7 release until spring (luckily it's for >> > >> > > fortunate-to-me reasons). >> > >> > > >> > >> > > Despite the delay in schedule on this release, generally doing a >> > >> > > maintenance release is relatively low effort. These kinds of >> > >> > > "substitute RM" situations are a great chance to get familiar with the >> > >> > > process while having a well-defined end point. If there's anyone who'd >> > >> > > like to give it a try by handling things while I'm gone, both I and >> > >> > > I'm sure the project community would be grateful. >> > >> > > >> > >> > > To take on this particular task, one must be a committer already (due >> > >> > > to ASF policy). It's a great way to "act like a PMC" for those >> > >> > > committers who'd like to make a long term investment in HBase. >> > >> > > >> > >> > > Anyone interested? >> > >> > > >> > >> > > -busbey >> > >> > > >> > >> > > [1]: https://issues.apache.org/jira/browse/HBASE-18276 >> > >> > > >> > >> > >> > >> > >> > >> > >> >
Re: [DISCUSS] Any committers willing to do maintenance releases over the holidays?
I followed up with Apple and I think he's positioned to get things done. I'll leave the timing up to him. That said, I am already on progress on 1.2.7. currently RCs for that release are blocked on getting nightly tests that pass again. On Nov 15, 2017 10:54, "Nick Dimiduk" wrote: > Sean, > > Do you have a target date you'd like to see the community aiming for on > this one? > > If you don't mind, I would propose the Release Wrangler get started in JIRA > pruning and test suite kicking over the next couple weeks, and we can > target the first or second week of Dec. There's been quite a few commits > since the last release, so there may be some surprises lurking for our > volunteers to sort out. A good place to start is the compatibility report. > > Who will carry this banner? Apekshit, Jan: shall we draw straws? One of you > have a bit of extra free time over the next 4 weeks? > > Thanks, > Nick > > On Wed, Nov 1, 2017 at 11:33 AM Jan Hentschel < > jan.hentsc...@ultratendency.com> wrote: > > > I also like to volunteer, but I don’t think that I can do it on my own. > > > > > > > > Jan > > > > > > > > From: Apekshit Sharma > > Reply-To: > > Date: Wednesday, November 1, 2017 at 12:24 AM > > To: > > Subject: Re: [DISCUSS] Any committers willing to do maintenance releases > > over the holidays? > > > > > > > > Loads of helping hands...woohoo! > > > > > > > > -Appy > > > > > > > > > > > > On Oct 31, 2017 4:01 PM, "Nick Dimiduk" wrote: > > > > > > > > As 1.1 is rounding down, I will be able to apply that time to helping out > > > > on another release line, if that's useful to the community. > > > > > > > > Thanks, > > > > Nick > > > > > > > > On Mon, Oct 30, 2017 at 9:35 PM, Sean Busbey wrote: > > > > > > > > > Hi folks! > > > > > > > > > > I'm in the process of getting us a 1.2.7-RC, hopefully soon[1]. This > > > > > release is badly behind schedule. I'd very much like to get us back on > > > > > a regular cadence for our stable release line, with a goal of once a > > > > > month. Unfortunately, I'm very unlikely to have time to do release > > > > > managing after the 1.2.7 release until spring (luckily it's for > > > > > fortunate-to-me reasons). > > > > > > > > > > Despite the delay in schedule on this release, generally doing a > > > > > maintenance release is relatively low effort. These kinds of > > > > > "substitute RM" situations are a great chance to get familiar with the > > > > > process while having a well-defined end point. If there's anyone who'd > > > > > like to give it a try by handling things while I'm gone, both I and > > > > > I'm sure the project community would be grateful. > > > > > > > > > > To take on this particular task, one must be a committer already (due > > > > > to ASF policy). It's a great way to "act like a PMC" for those > > > > > committers who'd like to make a long term investment in HBase. > > > > > > > > > > Anyone interested? > > > > > > > > > > -busbey > > > > > > > > > > [1]: https://issues.apache.org/jira/browse/HBASE-18276 > > > > > > > > > > > > > > > > > >
Re: [DISCUSS] [HBASE-19207] HBase REST API
On Tue, Nov 14, 2017 at 2:58 PM, Rick Kellogg wrote: > Greetings, > > > > About a week ago, I opened a new JIRA ticket [HBASE-19207] - Create a > Minimal HBase REST Client. As I started to study the feasibility of this > task, I determined the project dependencies were a hornets nest. There is > no clear separation of client and server components. For example the > VersionModel class relies upon the javax.servlet.ServletContext class for > construction on the server. There are many other examples of blurred > boundaries. > > > Indeed. Separation of client and server, like the separation of church and state, ebbs and flows is a long way from being done. > > I initially tried to break out the client specific files plus protocol > buffer related files in the hbase-rest project. This lead me to believe we > could create a clear separation of client and server components. The > trouble started on how to provide access to classes such as Cell or Scan > from the hbase-client. Cell should be hbase-common; no need of the hbase-client dependency. Scan is another story. You need access to these Operation classes? > Using the hbase-shaded-client seems to be a solution > but far from ideal. This failed as well because of complications with > shading. The protocol buffer related stuff was generated twice. Once in > my > project and already in the hbase-shaded-client. > > Your project could not access the shaded classes inside hbase-client? Is the double-execution of shading an issue? (Perhaps you are saying that we shaded already-shaded classes?) This is great feedback though. Please dump the litany of your troubles into HBASE-19207 if you have a list. This will help with the aforementioned laggard separation project. > > So as an interim solution, I would like to suggest the introduction of a > Apache Maven Bill-of-Materials (BOM) for access to the HBase REST Client. > I > have created a BOM for 2.0.0-alpha4 on Github for comment: > > > > https://github.com/rmkellogg/hbase-rest-client-ex > > > > Using the BOM feature, I was able to reduce the number of dependencies from > 146 to 21. > > > Good. Let me comment up on github. Thanks for writing back to the dev list Richard, St > > I think we could allow some flexibility for upgrade to common libraries > (Guava, Apache HttpComponents, etc) using this technique. > > > > > Thoughts and comments appreciated. > > Rick Kellogg > > > >
[jira] [Created] (HBASE-19267) Eclipse project import issues on 2.0
Josh Elser created HBASE-19267: -- Summary: Eclipse project import issues on 2.0 Key: HBASE-19267 URL: https://issues.apache.org/jira/browse/HBASE-19267 Project: HBase Issue Type: Task Components: build Reporter: Josh Elser Assignee: Josh Elser Fix For: 2.0.0-beta-1 Trying to do a fresh import of branch-2 nets some errors.. It seems like a previous change I made to clean up errors (HBASE-13236), specifically adding the maven-compiler-plugin lifecycle mapping for m2eclipse, is now causing Eclipse to not compile HBase as Java8. Removing the lifecycle mapping fixes this. I assume this only needs to happen for 2.0. I keep having issues with the JavaNature being ignored. Not yet sure if this is a result of something we're doing wrong (or just Eclipse being Eclipse). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19266) TestAcidGuarantees should cover adaptive in-memory compaction
Ted Yu created HBASE-19266: -- Summary: TestAcidGuarantees should cover adaptive in-memory compaction Key: HBASE-19266 URL: https://issues.apache.org/jira/browse/HBASE-19266 Project: HBase Issue Type: Test Reporter: Ted Yu Priority: Minor Currently TestAcidGuarantees populates 3 policies of (in-memory) compaction. Adaptive in-memory compaction is new and should be added as 4th compaction policy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: [DISCUSS] Any committers willing to do maintenance releases over the holidays?
Sean, Do you have a target date you'd like to see the community aiming for on this one? If you don't mind, I would propose the Release Wrangler get started in JIRA pruning and test suite kicking over the next couple weeks, and we can target the first or second week of Dec. There's been quite a few commits since the last release, so there may be some surprises lurking for our volunteers to sort out. A good place to start is the compatibility report. Who will carry this banner? Apekshit, Jan: shall we draw straws? One of you have a bit of extra free time over the next 4 weeks? Thanks, Nick On Wed, Nov 1, 2017 at 11:33 AM Jan Hentschel < jan.hentsc...@ultratendency.com> wrote: > I also like to volunteer, but I don’t think that I can do it on my own. > > > > Jan > > > > From: Apekshit Sharma > Reply-To: > Date: Wednesday, November 1, 2017 at 12:24 AM > To: > Subject: Re: [DISCUSS] Any committers willing to do maintenance releases > over the holidays? > > > > Loads of helping hands...woohoo! > > > > -Appy > > > > > > On Oct 31, 2017 4:01 PM, "Nick Dimiduk" wrote: > > > > As 1.1 is rounding down, I will be able to apply that time to helping out > > on another release line, if that's useful to the community. > > > > Thanks, > > Nick > > > > On Mon, Oct 30, 2017 at 9:35 PM, Sean Busbey wrote: > > > > > Hi folks! > > > > > > I'm in the process of getting us a 1.2.7-RC, hopefully soon[1]. This > > > release is badly behind schedule. I'd very much like to get us back on > > > a regular cadence for our stable release line, with a goal of once a > > > month. Unfortunately, I'm very unlikely to have time to do release > > > managing after the 1.2.7 release until spring (luckily it's for > > > fortunate-to-me reasons). > > > > > > Despite the delay in schedule on this release, generally doing a > > > maintenance release is relatively low effort. These kinds of > > > "substitute RM" situations are a great chance to get familiar with the > > > process while having a well-defined end point. If there's anyone who'd > > > like to give it a try by handling things while I'm gone, both I and > > > I'm sure the project community would be grateful. > > > > > > To take on this particular task, one must be a committer already (due > > > to ASF policy). It's a great way to "act like a PMC" for those > > > committers who'd like to make a long term investment in HBase. > > > > > > Anyone interested? > > > > > > -busbey > > > > > > [1]: https://issues.apache.org/jira/browse/HBASE-18276 > > > > > > > > >
Final 1.1 release
Hello! By (extremely) lazy consensus, it's looking like the final release of HBase 1.1 will be lucky number 1.1.13. I'd like to publish a release candidate after the US Thanksgiving holiday, let's say on or around Monday, Nov 27. That said, there is one think that I think may delay, or at least be good reason to make a subsequent release. If the conclusion of the thread on upgrades to 2.0 finds some minor issue excluding 1.1 as a viable upgrade path, I would like to get them fixed. That would be reason enough to warrant a follow up release. Thanks for your attention. Nick
[jira] [Resolved] (HBASE-13850) Check for dead server on CallTimeoutException
[
https://issues.apache.org/jira/browse/HBASE-13850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-13850.
---
Resolution: Duplicate
> Check for dead server on CallTimeoutException
> -
>
> Key: HBASE-13850
> URL: https://issues.apache.org/jira/browse/HBASE-13850
> Project: HBase
> Issue Type: Improvement
> Components: Client, MTTR
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Matteo Bertozzi
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-13850-v0.patch, TestGetPerf.java
>
>
> WARN this may be a misconf, so let me know if there is a timeout param to set.
> {noformat}
> hbase-site.xml
> zookeeper.session.timeout 1
> hbase.regionserver.storefile.refresh.period 1
> hbase.client.operation.timeout 5000
> hbase.client.meta.operation.timeout 5000
> hbase.client.scanner.timeout.period 1
> hbase.regionserver.lease.period 1
> {noformat}
> I have a test that does a kill STOP on a RS and tries to query it.
> From the conf the zk lease is 10sec, and the master is correctly doing the
> reassign after 10sec and meta is updated.
> the client keep trying to query the RS for a specific row until it get a
> response. The table.get(row) in the loop throws a CallTimeoutException every
> 5sec (which is the configured settings). but instead of succeed after 2/3
> retries (> 10sec where the master reassign) it keeps retrying up to 60sec (I
> don't know what that 60sec is, maybe a conf param that I'm not able to find)
> one simple fix in the code is handling the CallTimeoutException in
> RegionServerCallable and clear the meta cache for that RS that is not
> responding. (but maybe there is already a conf to set to reduce that 60sec
> period)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19265) If a test is changed in a patch, then we should ignore the fact that it is in the flaky list when testing.
Sean Busbey created HBASE-19265: --- Summary: If a test is changed in a patch, then we should ignore the fact that it is in the flaky list when testing. Key: HBASE-19265 URL: https://issues.apache.org/jira/browse/HBASE-19265 Project: HBase Issue Type: Improvement Components: test Reporter: Sean Busbey Priority: Critical [~chia7712] noticed on HBASE-19248 that the test being fixed there didn't run in precommit due to being listed as untrusted. we should update our handling of the flaky list to remove any test that show up as changed in the patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19264) Use a thread pool to execute the open/close region operation in execProcedures
Duo Zhang created HBASE-19264: - Summary: Use a thread pool to execute the open/close region operation in execProcedures Key: HBASE-19264 URL: https://issues.apache.org/jira/browse/HBASE-19264 Project: HBase Issue Type: Improvement Components: proc-v2 Reporter: Duo Zhang The return value of execProcedure is just ignored at master side, and we will rely on the reportRegionTransitionState to determine whether the open/close is succeeded, so it is not necessary to open/close region synchronously in the execProcedure method, we can use a thread pool to execute the actual operations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19263) Hbase BaseLoadBalancer
SuperbDong created HBASE-19263: -- Summary: Hbase BaseLoadBalancer Key: HBASE-19263 URL: https://issues.apache.org/jira/browse/HBASE-19263 Project: HBase Issue Type: Bug Environment: hbase 1.2.6 Reporter: SuperbDong hbase master as a regionserver,BaseLoadBalancer throw Exception ArrayIndexOutOfBoundsException because of count of region is contain hbase master.However,hbase master can not resign any region except hbase-site.xml config. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: DISCUSSION: Minimum hbase1 version from which you can upgrade to hbase2 (1.2.x?)
bq. Don't know. Will shout if I find anything. Will try it soon (next week or so). Thank you sir, will wait for your note. Will also try the upgrade if time allows and shout if find anything. Best Regards, Yu On 12 November 2017 at 04:46, Stack wrote: > On Thu, Nov 9, 2017 at 9:06 PM, Yu Li wrote: > > > Sorry for the late response boss. We're still on 1.1 and have been > keeping > > a close watch on 2.0 progress (silently though, sorry about this, > occupied > > by singles day). If 1.2 could rolling upgrade to 2.0, anything special > that > > prevents 1.1 to (could you please refer me to some JIRA)? Thanks. > > > > > Don't know. Will shout if I find anything. Will try it soon (next week or > so). > > > > Rolling upgrade is a must-have for us when choosing the next version, and > > since we have already backported the offheap work, 2.0 would be the first > > choice for us than 1.4 (to avoid the pain of another round patch porting) > > (smile) > > > > Good to know. > > Going to an intermediate version would be a PITA for you I'm sure. > > St.Ack > > > > > Best Regards, > > Yu > > > > On 9 November 2017 at 14:08, Stack wrote: > > > > > FYI, I'm resolving HBASE-13631 "Migration from 0.94 to 2.0.0" because > of > > > the discussion here on this thread. > > > > > > Sounds like 1.2 is minimum but lets try and see if we can go from 0.98. > > > > > > Thanks, > > > S > > > > > > On Sat, Nov 4, 2017 at 9:41 PM, Stack wrote: > > > > > > > On Sat, Nov 4, 2017 at 6:19 PM, Guanghao Zhang > > > wrote: > > > > > > > >> Our internal branch is based on 0.98. And we plan rolling to 2.0. > So I > > > >> will > > > >> take a try for rolling from 0.98 to 2.0. But we take a lot backport > to > > > our > > > >> internal branch, like async client, netty rpc client, serial > > > replication, > > > >> throttling, some replication improvements and so on. So our rolling > > > >> experience may not apply to community totally. I will post our > rolling > > > >> experience (which can apply to community 0.98 branch) after we > rolling > > > to > > > >> 2.0 :-). > > > >> > > > >> > > > > Let me try going from 0.98 then and see what is broke. Would be good > if > > > > you fellows could do one step rather than two. > > > > S > > > > > > > > > > > > > > > > > > > >> 2017-11-05 2:41 GMT+08:00 Stack : > > > >> > > > >> > On Fri, Nov 3, 2017 at 9:01 PM, Guanghao Zhang < > zghao...@gmail.com> > > > >> wrote: > > > >> > > > > >> > > Can we rolling from 0.98 and 1.1 to 1.2? If this rolling is ok, > > user > > > >> can > > > >> > > rolling to 2.0 by two steps, 0.98 to 1.2, then 1.2 to 2.0. > > > >> > > > > > >> > > > > > >> > Yes. They could do that. Would be a pain. Might be able to go from > > > 0.98 > > > >> to > > > >> > 2.0 though... I've not tried it. > > > >> > St.Ack > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > 2017-11-04 11:25 GMT+08:00 Nick Dimiduk : > > > >> > > > > > >> > > > 1.2 is good, but are we aware of anything that precludes 1.1? > > > 0.98? > > > >> On > > > >> > > disk > > > >> > > > compatibility (HFile, WAL, AMv2) should be the limiting factor > > > here, > > > >> > > right? > > > >> > > > Wire protocols have been compatible all the while... > > > >> > > > > > > >> > > > On Fri, Nov 3, 2017 at 5:56 PM Zach York < > > > >> zyork.contribut...@gmail.com > > > >> > > > > > >> > > > wrote: > > > >> > > > > > > >> > > > > +1 for having the minimum (supported) hbase1 version be > 1.2.x. > > > >> > > > > > > > >> > > > > On Fri, Nov 3, 2017 at 5:30 PM, Stack > > wrote: > > > >> > > > > > > > >> > > > > > Over in the adjacent "[DISCUSS] hbase-2.0.0 compatibility > > > >> > > expectations" > > > >> > > > > > thread, we chatted some on what would be the minimum > > hbase-1.x > > > >> > > version > > > >> > > > > from > > > >> > > > > > which you can upgrade to hbase-2. > > > >> > > > > > > > > >> > > > > > The last statement made on this topic by Sean was that > only > > > >> > upgrades > > > >> > > > from > > > >> > > > > > 1.2.x, our current stable offering, or later should be > > > >> supported. > > > >> > > > > > > > > >> > > > > > There was no dissent. > > > >> > > > > > > > > >> > > > > > We all good w/ this? Speak up if you disagree else 1.2.x > > > becomes > > > >> > the > > > >> > > > > > 'official' minimum. > > > >> > > > > > > > > >> > > > > > NOTES: > > > >> > > > > > > > > >> > > > > > + We need to agree on a minimum so we know what migrations > > to > > > >> test. > > > >> > > > > > + It might be possible to upgrade from versions before > 1.2.x > > > >> but we > > > >> > > (or > > > >> > > > > at > > > >> > > > > > least I -- smile) won't have tried it or run verifications > > to > > > >> > ensure > > > >> > > > all > > > >> > > > > > made it over (let us know if you successfully migrate > from a > > > >> > baseline > > > >> > > > > that > > > >> > > > > > precedes 1.2). > > > >> > > > > > + Hopefully we can avoid requiring Users move to the > latest > > on > > > >> the > > > >> > > 1.2 > > > >> > > > >
