Re: [DISCUSS] InterfaceStability with InterfaceAudience.Public
unsubscribe
On Wed, Sep 8, 2021 at 8:45 AM Bryan Beaudreault
wrote:
> Thank you both for your input. That seems reasonable to me. In terms of the
> PR I was working on, I decided to keep it as the more restrictive
> InterfaceAudience.Public. I also created
>
> https://urldefense.com/v3/__https://issues.apache.org/jira/browse/HBASE-26266__;!!DCbAVzZNrAf4!QOyfoMjEhLOeMPVcgbU4AmTIvLamD9jdOj0hz45u_1CCG85JEOAkYJVpOyG3YJqwOw$
> to improve our docs on
> this topic. I'll try to get to that when I have time, if no one else gets
> to it before me.
>
> On Thu, Sep 2, 2021 at 11:38 AM Sean Busbey wrote:
>
> > Our API is already too big to audit by hand for breakage. The limited
> > tooling we have for automatically scanning as a part of the release
> > process[1] only has the ability to cope with a single set of
> > annotation (i.e. it can do "filter to things that are IA.Public" and
> > it can't do "filter to things that are IA.Public and IS.Stable"). As a
> > practical matter I don't think we can reliably meet promises beyond
> > "everything IA.Public is stable".
> >
> > That practical limitation is why the current HBase dev docs call out
> > our difference from yetus javadocs.
> >
> > I like the idea of a IA.LimitedPrivate experimental as a way to have a
> > proving ground for APIs we intend to make public but we want a test
> > out period in a user facing release. It's a relatively low risk way
> > for us as a community to see if we and our users find the approach
> > useful compared to how we currently do things (front load API
> > discussions and then mark things public; if needed deprecate/remove if
> > it doesn't work out).
> >
> > On Thu, Sep 2, 2021 at 9:38 AM 张铎(Duo Zhang)
> > wrote:
> > >
> > > I think the current Compatibility Matrix for our IA.Public APIs is
> > already
> > > complicated enough, so adding IS annotation to the IA.Public APIs will
> > be a
> > > huge pain for our end users, so I suppose we should not do this.
> > > And it is a bit strange that, an IA.Public API is also marked as
> > > IS.Unsable, right? It seems to just tell users do not use it, as it
> will
> > be
> > > broken even in a patch release...
> > > So in general, I think we should change the javadoc for the IS
> > annotation,
> > > to mention that we do not IS annotation for IA.Public APIs, it should
> > > always be IS.Stable.
> > >
> > > But looking from the developer side, it is a true pain that, seems
> there
> > is
> > > no way for us to introduce 'experimental' APIs.
> > > So maybe we could add a new LP type called experimental, so these APIs
> > > could be marked IA.LimitedPrivate("Experimental") and we could use the
> IS
> > > annotation then.
> > >
> > > This could make developers life easier, but I still a bit worry that,
> > will
> > > end users actually use these 'Experimental' APIs? If no one will use it
> > > until it becomes IA.Public, then what's the value for doing this...
> > >
> > > Just my simple thoughts.
> > >
> > > Thanks.
> > >
> > > Bryan Beaudreault 于2021年9月1日周三
> > 上午9:41写道：
> > >
> > > > Hello devs,
> > > >
> > > > A recent discussion came up on slack related to a PR I'm working on
> > which
> > > > adds a new class annotated with InterfaceAudience.Public. It seems
> like
> > > > there's some disagreement in terms of what the
> > > > current documented expectations are for InterfaceStability in this
> > case,
> > > > and what expectations we might actually want. Specifically, should we
> > allow
> > > > annotating IA.Public classes with IS.Evolving or IS.Unstable?
> > > >
> > > > Below I quote two conflicting documents, and I'm curious how the
> group
> > > > thinks we should reconcile them. Before I do, I just wanted to put
> out
> > my
> > > > opinion that it feels like we should have some ability to push new
> > public
> > > > classes that might evolve; basically beta features that are part of a
> > > > normal release.
> > > >
> > > > In the dev docs (
> > > >
> https://urldefense.com/v3/__https://hbase.apache.org/book.html*hbase.client.api.surface__;Iw!!DCbAVzZNrAf4!QOyfoMjEhLOeMPVcgbU4AmTIvLamD9jdOj0hz45u_1CCG85JEOAkYJVpOyHUjfiiaQ$
> > <
> https://urldefense.com/v3/__https://hbase.apache.org/book.html*hbase.client.api.surface__;Iw!!DCbAVzZNrAf4!QOyfoMjEhLOeMPVcgbU4AmTIvLamD9jdOj0hz45u_1CCG85JEOAkYJVpOyHUjfiiaQ$
> >
> > ),
> > > > there is this quote:
> > > >
> > > > IA.Public classes are inherently stable and adhere to our stability
> > > > guarantees relating to the type of upgrade (major, minor, or patch).
> > > > IA.LimitedPrivate classes should always be annotated with one of the
> > given
> > > > InterfaceStability values. If they are not, you should presume they
> are
> > > > IS.Unstable.
> > > > IA.Private classes should be considered implicitly unstable, with no
> > > > guarantee of stability between releases.
> > > >
> > > > On the other hand, the actual javadoc (
> > > >
> > > >
> >
>
[jira] [Resolved] (HBASE-26243) Fix typo for file 'hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java'
[
https://issues.apache.org/jira/browse/HBASE-26243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-26243.
---
Fix Version/s: 2.4.7
2.3.7
3.0.0-alpha-2
2.5.0
Hadoop Flags: Reviewed
Resolution: Fixed
Pushed to branch-2.3+.
Thanks [~hapihu] for contributing.
> Fix typo for file
> 'hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java'
> --
>
> Key: HBASE-26243
> URL: https://issues.apache.org/jira/browse/HBASE-26243
> Project: HBase
> Issue Type: Improvement
>Reporter: wuguihu
>Assignee: wuguihu
>Priority: Trivial
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.3.7, 2.4.7
>
> Attachments: HBASE-26243.patch
>
>
> Fix typo for file
> 'hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java'
> {code:bash}
> hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java:1823:
> mulitple ==> multiple
> hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java:3562:
> compatiblity ==> compatibility
> hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java:3864:
> supportted ==> supported
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-26269) Shell no longer supports 'load' command
Nick Dimiduk created HBASE-26269:
Summary: Shell no longer supports 'load' command
Key: HBASE-26269
URL: https://issues.apache.org/jira/browse/HBASE-26269
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.4.5, 3.0.0-alpha-2
Reporter: Nick Dimiduk
After upgrade fro 2.3.x to 2.4.x, we noticed that the behavior of the shell's
{{load}} command has changed.
Given a file of commands, i.e.
{noformat}
$ echo 'list' > /tmp/file
{noformat}
2.3.x:
{noformat}
$ hbase shell
hbase(main):001:0> load '/tmp/file'
TABLE
TestTable
1 row(s)
Took 0.3076 seconds
=> true
hbase(main):002:0>
{noformat}
branch-2.4:
{noformat}
$ hbase shell
hbase:001:0> load '/tmp/file'
Traceback (most recent call last):
NameError (undefined local variable or method `list' for main:Object)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-26023) Overhaul of test cluster set up for table skew
[
https://issues.apache.org/jira/browse/HBASE-26023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Clara Xiong resolved HBASE-26023.
-
Fix Version/s: 3.0.0-alpha-1
2.3.6
2.4.5
Resolution: Fixed
This was fixed in the same PRs for
https://issues.apache.org/jira/browse/HBASE-25739
> Overhaul of test cluster set up for table skew
> --
>
> Key: HBASE-26023
> URL: https://issues.apache.org/jira/browse/HBASE-26023
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer, test
>Reporter: Clara Xiong
>Priority: Major
> Fix For: 2.4.5, 2.3.6, 3.0.0-alpha-1
>
>
> There is another bug in the original tableSkew cost function for aggregation
> of the cost per table:
> If we have 10 regions, one per table, evenly distributed on 10 nodes, the
> cost is scale to 1.0.
> The more tables we have, the closer the value will be to 1.0. The cost
> function becomes useless.
> All the balancer tests were set up with large numbers of tables with minimal
> regions per table. This artificially inflates the total cost and trigger
> balancer runs. With this fix on TableSkewFunction, we need to overhaul the
> tests too. We also need to add tests that reflect more diversified scenarios
> for table distribution such as large tables with large numbers of regions.
> {code:java}
> protected double cost() {
> double max = cluster.numRegions;
> double min = ((double) cluster.numRegions) / cluster.numServers;
> double value = 0;
> for (int i = 0; i < cluster.numMaxRegionsPerTable.length; i++) {
> value += cluster.numMaxRegionsPerTable[i];
> }
> LOG.info("min = {}, max = {}, cost= {}", min, max, value);
> return scale(min, max, value);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-26237) Improve computation complexity for primaryRegionCountSkewCostFunctio
[ https://issues.apache.org/jira/browse/HBASE-26237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clara Xiong resolved HBASE-26237. - Fix Version/s: 3.0.0-alpha-2 Resolution: Fixed > Improve computation complexity for primaryRegionCountSkewCostFunctio > > > Key: HBASE-26237 > URL: https://issues.apache.org/jira/browse/HBASE-26237 > Project: HBase > Issue Type: Sub-task > Components: Balancer >Reporter: Clara Xiong >Priority: Minor > Fix For: 3.0.0-alpha-2 > > > Recomputation of primaryRegionCountSkewCostFunction can be reduced from O(n ) > to O(1) by only incrementing the destination and decrementing the source > instead of full recompute. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26268) AccessController is missing a couple AdminService methods
Bryan Beaudreault created HBASE-26268: - Summary: AccessController is missing a couple AdminService methods Key: HBASE-26268 URL: https://issues.apache.org/jira/browse/HBASE-26268 Project: HBase Issue Type: Bug Reporter: Bryan Beaudreault Sorry for the vague title, not sure if it'd be better to create individual jiras. I discovered that the following two AdminService endpoints are not covered by AccessController: * updateConfiguration * clearRegionBlockCache There may be others, I haven't done a full audit. We should add coprocessor hooks for these and wrap with AccessController, especially since they can both have an affect on the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26267) Master initialization fails if Master Region WAL dir is missing
Josh Elser created HBASE-26267:
--
Summary: Master initialization fails if Master Region WAL dir is
missing
Key: HBASE-26267
URL: https://issues.apache.org/jira/browse/HBASE-26267
Project: HBase
Issue Type: Improvement
Components: master
Affects Versions: 2.4.6
Reporter: Josh Elser
Assignee: Josh Elser
Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.7
>From a recent branch-2.4 build:
{noformat}
2021-09-07 19:31:19,666 ERROR [master/localhost:16000:becomeActiveMaster]
master.HMaster(159): * ABORTING master localhost,16000,1631057476442:
Unhandled exception. Starting shutdown. *
java.io.FileNotFoundException: File
hdfs://localhost:8020/hbase-2.4-wals/MasterData/WALs does not exist.
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:1059)
at
org.apache.hadoop.hdfs.DistributedFileSystem.access$1000(DistributedFileSystem.java:131)
at
org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1119)
at
org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1116)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:1126)
at
org.apache.hadoop.hbase.master.region.MasterRegion.open(MasterRegion.java:226)
at
org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:303)
at
org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:839)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2189)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:512)
at java.lang.Thread.run(Thread.java:748)
{noformat}
If the WAL directory is missing but the Master Region already exists, we will
try to list the contents of the Master Region's WAL directory which may or may
not exist. If we simply check to make sure the directory exists and then the
rest of the initialization code works as expected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-26195) Data is present in replicated cluster but not present in primary cluster.
[
https://issues.apache.org/jira/browse/HBASE-26195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rushabh Shah resolved HBASE-26195.
--
Resolution: Fixed
> Data is present in replicated cluster but not present in primary cluster.
> -
>
> Key: HBASE-26195
> URL: https://issues.apache.org/jira/browse/HBASE-26195
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 1.7.0
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Major
> Fix For: 1.8.0
>
>
> We encountered a case where we are seeing some rows (via Phoenix) in
> replicated cluster but they are not present in source/active cluster.
> Triaging further we found memstore rollback logs in few of the region servers.
> {noformat}
> 2021-07-28 14:17:59,353 DEBUG [3,queue=3,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,353 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,354 DEBUG [3,queue=3,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,354 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,354 DEBUG [3,queue=3,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,354 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,355 DEBUG [3,queue=3,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,355 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> 2021-07-28 14:17:59,356 DEBUG [,queue=25,port=60020] regionserver.HRegion -
> rollbackMemstore rolled back 23
> {noformat}
> Looking more into logs, found that there were some hdfs layer issues sync'ing
> wal to hdfs.
> It was taking around 6 mins to sync wal. Logs below
> {noformat}
> 2021-07-28 14:19:30,511 WARN [sync.0] hdfs.DataStreamer - Slow
> waitForAckedSeqno took 391210ms (threshold=3ms). File being written:
> /hbase/WALs/,60020,1626191371499/%2C60020%2C1626191371499.1627480615620,
> block: BP-958889176--1567030695029:blk_1689647875_616028364, Write
> pipeline datanodes:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]].
> 2021-07-28 14:19:30,589 WARN [sync.1] hdfs.DataStreamer - Slow
> waitForAckedSeqno took 391148ms (threshold=3ms). File being written:
> /hbase/WALs/,60020,1626191371499/%2C60020%2C1626191371499.1627480615620,
> block: BP-958889176--1567030695029:blk_1689647875_616028364, Write
> pipeline datanodes:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]].
> 2021-07-28 14:19:30,589 WARN [sync.2] hdfs.DataStreamer - Slow
> waitForAckedSeqno took 391147ms (threshold=3ms). File being written:
> /hbase/WALs/,60020,1626191371499/%2C60020%2C1626191371499.1627480615620,
> block: BP-958889176--1567030695029:blk_1689647875_616028364, Write
> pipeline datanodes:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]].
> 2021-07-28 14:19:30,591 INFO [sync.0] wal.FSHLog - Slow sync cost: 391289
> ms, current pipeline:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]]
> 2021-07-28 14:19:30,591 INFO [sync.1] wal.FSHLog - Slow sync cost: 391227
> ms, current pipeline:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]]
> 2021-07-28 14:19:30,591 WARN [sync.1] wal.FSHLog - Requesting log roll
> because we exceeded slow sync threshold; time=391227 ms, threshold=1 ms,
> current pipeline:
> [DatanodeInfoWithStorage[:50010,DS-b5747702-8ab9-4a5e-916e-5fae6e305738,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-505dabb0-0fd6-42d9-b25d-f25e249fe504,DISK],
>
> DatanodeInfoWithStorage[:50010,DS-6c585673-d4d0-4ec6-bafe-ad4cd861fb4b,DISK]]
>
Re: [VOTE] First release candidate for HBase 2.4.6 (RC0) is available
+1 (Non-binding) * Signature: ok * Checksum : ok * Rat check (1.8.0_282): ok - mvn clean apache-rat:check -D hadoop.profile=3.0 * Built from source (1.8.0_282): ok - mvn clean install -D hadoop.profile=3.0 -DskipTests * Unit tests pass (1.8.0_282): ok - mvn package -P runSmallTests -D hadoop.profile=3.0 -Dsurefire.rerunFailingTestsCount=3 * Installed a single node cluster and exercised basic client operations through HBase shell. * Checked HMaster and RegionServers WEB UI. LGTM. Regards, Pankaj On Wed, Sep 8, 2021 at 4:47 PM Guanghao Zhang wrote: > +1 (binding) > > * Signature: ok > * Checksum : ok > * Rat check (1.8.0_301): ok > - mvn clean apache-rat:check > * Built from source (1.8.0_301): ok > - mvn clean install -DskipTests > * Unit tests pass (1.8.0_301): ok > - mvn package -P runSmallTests > * HMaster, RS, Table, Region UI: ok > * HBase shell (list, create, disable, drop, get, put, scan, > delete): ok > > > > 张铎(Duo Zhang) 于2021年9月8日周三 上午10:12写道： > > > +1(binding) > > > > Checked sigs and sums: Matched > > Rat check: Passed > > Built from source(AdoptOpenJDK-11.0.11+9): OK with -Dhadoop.profile=3.0 > > Run UTs: Got some trouble in the hbase-server module, > > TestMasterAddressRefresher always failed but when running on other > > machines, it was fine, so should not be a blocker. > > Started a mini cluster: Run ./start-hbase.sh. succeeded, the web page > looks > > fine > > Run basic shell commands: list, create, disable, drop, all fine > > Run ltt with 1 rows: Passed, count in hbase shell also returned 1 > > > > Jan Hentschel 于2021年9月8日周三 上午5:11写道： > > > > > +1 (binding) > > > > > > * Signature: ok > > > * Checksum : ok > > > * Rat check (1.8.0_202-ea): ok > > > - mvn clean apache-rat:check > > > * Built from source (1.8.0_202-ea): ok > > > - mvn clean install -DskipTests > > > * Unit tests pass (1.8.0_202-ea): ok > > > - mvn package -P runSmallTests > > > -Dsurefire.rerunFailingTestsCount=3 > > > > > > From: Andrew Purtell > > > Date: Saturday, September 4, 2021 at 3:31 AM > > > To: dev > > > Subject: [VOTE] First release candidate for HBase 2.4.6 (RC0) is > > available > > > Please vote on this Apache hbase release candidate, > > > hbase-2.4.6RC0 > > > > > > The VOTE will remain open for at least 72 hours. > > > > > > [ ] +1 Release this package as Apache hbase 2.4.6 > > > [ ] -1 Do not release this package because ... > > > > > > The tag to be voted on is 2.4.6RC0: > > > > > > https://github.com/apache/hbase/tree/2.4.6RC0 > > > > > > This tag currently points to git reference 7374d396c2. > > > > > > The release files, including signatures, digests, as well as CHANGES.md > > > and RELEASENOTES.md included in this RC can be found at: > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.4.6RC0/ > > > > > > The API compatibility report can be found at: > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.4.6RC0/api_compare_2.4.5_to_2.4.6RC0.html > > > > > > There are no reported compatibility issues. > > > > > > Maven artifacts are available in a staging repository at: > > > > > > > > https://repository.apache.org/content/repositories/orgapachehbase-1464/ > > > > > > Artifacts were signed with the code signing key 0xD5365CCD which can be > > > found in: > > > > > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > > > > > Some pre-flight checks were made prior to release candidate generation, > > > including unit test execution. Some flaky tests were found, see > > > HBASE-26254. > > > > > > To learn more about Apache hbase, please see > > > > > > http://hbase.apache.org/ > > > > > > Thanks, > > > Your HBase Release Manager > > > > > >
Re: [DISCUSS] InterfaceStability with InterfaceAudience.Public
Thank you both for your input. That seems reasonable to me. In terms of the
PR I was working on, I decided to keep it as the more restrictive
InterfaceAudience.Public. I also created
https://issues.apache.org/jira/browse/HBASE-26266 to improve our docs on
this topic. I'll try to get to that when I have time, if no one else gets
to it before me.
On Thu, Sep 2, 2021 at 11:38 AM Sean Busbey wrote:
> Our API is already too big to audit by hand for breakage. The limited
> tooling we have for automatically scanning as a part of the release
> process[1] only has the ability to cope with a single set of
> annotation (i.e. it can do "filter to things that are IA.Public" and
> it can't do "filter to things that are IA.Public and IS.Stable"). As a
> practical matter I don't think we can reliably meet promises beyond
> "everything IA.Public is stable".
>
> That practical limitation is why the current HBase dev docs call out
> our difference from yetus javadocs.
>
> I like the idea of a IA.LimitedPrivate experimental as a way to have a
> proving ground for APIs we intend to make public but we want a test
> out period in a user facing release. It's a relatively low risk way
> for us as a community to see if we and our users find the approach
> useful compared to how we currently do things (front load API
> discussions and then mark things public; if needed deprecate/remove if
> it doesn't work out).
>
> On Thu, Sep 2, 2021 at 9:38 AM 张铎(Duo Zhang)
> wrote:
> >
> > I think the current Compatibility Matrix for our IA.Public APIs is
> already
> > complicated enough, so adding IS annotation to the IA.Public APIs will
> be a
> > huge pain for our end users, so I suppose we should not do this.
> > And it is a bit strange that, an IA.Public API is also marked as
> > IS.Unsable, right? It seems to just tell users do not use it, as it will
> be
> > broken even in a patch release...
> > So in general, I think we should change the javadoc for the IS
> annotation,
> > to mention that we do not IS annotation for IA.Public APIs, it should
> > always be IS.Stable.
> >
> > But looking from the developer side, it is a true pain that, seems there
> is
> > no way for us to introduce 'experimental' APIs.
> > So maybe we could add a new LP type called experimental, so these APIs
> > could be marked IA.LimitedPrivate("Experimental") and we could use the IS
> > annotation then.
> >
> > This could make developers life easier, but I still a bit worry that,
> will
> > end users actually use these 'Experimental' APIs? If no one will use it
> > until it becomes IA.Public, then what's the value for doing this...
> >
> > Just my simple thoughts.
> >
> > Thanks.
> >
> > Bryan Beaudreault 于2021年9月1日周三
> 上午9:41写道：
> >
> > > Hello devs,
> > >
> > > A recent discussion came up on slack related to a PR I'm working on
> which
> > > adds a new class annotated with InterfaceAudience.Public. It seems like
> > > there's some disagreement in terms of what the
> > > current documented expectations are for InterfaceStability in this
> case,
> > > and what expectations we might actually want. Specifically, should we
> allow
> > > annotating IA.Public classes with IS.Evolving or IS.Unstable?
> > >
> > > Below I quote two conflicting documents, and I'm curious how the group
> > > thinks we should reconcile them. Before I do, I just wanted to put out
> my
> > > opinion that it feels like we should have some ability to push new
> public
> > > classes that might evolve; basically beta features that are part of a
> > > normal release.
> > >
> > > In the dev docs (
> > > https://hbase.apache.org/book.html#hbase.client.api.surface
>
> ),
> > > there is this quote:
> > >
> > > IA.Public classes are inherently stable and adhere to our stability
> > > guarantees relating to the type of upgrade (major, minor, or patch).
> > > IA.LimitedPrivate classes should always be annotated with one of the
> given
> > > InterfaceStability values. If they are not, you should presume they are
> > > IS.Unstable.
> > > IA.Private classes should be considered implicitly unstable, with no
> > > guarantee of stability between releases.
> > >
> > > On the other hand, the actual javadoc (
> > >
> > >
> https://yetus.apache.org/documentation/in-progress/javadocs/org/apache/yetus/audience/InterfaceStability.htm
>
> > > )
> > > for InterfaceStability states:
> > >
> > > All classes that are annotated with InterfaceAudience.Public or
> > > InterfaceAudience.LimitedPrivate must have InterfaceStability
> annotation.
> > > Classes that are InterfaceAudience.Private are to be considered
> unstable
> > > unless a different InterfaceStability annotation states otherwise.
> > > Incompatible changes must not be made to classes marked as stable.
> > >
> > > One interpretation is that these are not in conflict, since one should
> > >
[jira] [Created] (HBASE-26266) Improve developer docs for InterfaceAudience of public experimental APIs
Bryan Beaudreault created HBASE-26266: - Summary: Improve developer docs for InterfaceAudience of public experimental APIs Key: HBASE-26266 URL: https://issues.apache.org/jira/browse/HBASE-26266 Project: HBase Issue Type: Task Reporter: Bryan Beaudreault Per [https://lists.apache.org/thread.html/re7736008a7619029bf34ed81f5931c537281d22788719f2dcf37f0c0%40%3Cdev.hbase.apache.org%3E,] our InterfaceAudience.Public restrictions intentionally deviate from the default yetus docs, and we would like to keep it that way. It was acknowledged that there may be a reason to expose early access to certain APIs that may need to evolve in breaking ways. The suggested approach is to annotate these as InterfaceAudience.LimitedPrivate. We should update developer docs to clarify both points. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26265) Update ref guide to mention the new store file tracker implementations
Duo Zhang created HBASE-26265: - Summary: Update ref guide to mention the new store file tracker implementations Key: HBASE-26265 URL: https://issues.apache.org/jira/browse/HBASE-26265 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Duo Zhang For example, when to use these store file trackers. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26264) Add more checks to prevent misconfiguration on store file tracker
Duo Zhang created HBASE-26264: - Summary: Add more checks to prevent misconfiguration on store file tracker Key: HBASE-26264 URL: https://issues.apache.org/jira/browse/HBASE-26264 Project: HBase Issue Type: Sub-task Components: conf, HFile Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26263) [Rolling Upgrading] Persist the StoreEngine configurations to TableDescriptor for existing tables
Duo Zhang created HBASE-26263: - Summary: [Rolling Upgrading] Persist the StoreEngine configurations to TableDescriptor for existing tables Key: HBASE-26263 URL: https://issues.apache.org/jira/browse/HBASE-26263 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] First release candidate for HBase 2.4.6 (RC0) is available
+1 (binding) * Signature: ok * Checksum : ok * Rat check (1.8.0_301): ok - mvn clean apache-rat:check * Built from source (1.8.0_301): ok - mvn clean install -DskipTests * Unit tests pass (1.8.0_301): ok - mvn package -P runSmallTests * HMaster, RS, Table, Region UI: ok * HBase shell (list, create, disable, drop, get, put, scan, delete): ok 张铎(Duo Zhang) 于2021年9月8日周三 上午10:12写道： > +1(binding) > > Checked sigs and sums: Matched > Rat check: Passed > Built from source(AdoptOpenJDK-11.0.11+9): OK with -Dhadoop.profile=3.0 > Run UTs: Got some trouble in the hbase-server module, > TestMasterAddressRefresher always failed but when running on other > machines, it was fine, so should not be a blocker. > Started a mini cluster: Run ./start-hbase.sh. succeeded, the web page looks > fine > Run basic shell commands: list, create, disable, drop, all fine > Run ltt with 1 rows: Passed, count in hbase shell also returned 1 > > Jan Hentschel 于2021年9月8日周三 上午5:11写道： > > > +1 (binding) > > > > * Signature: ok > > * Checksum : ok > > * Rat check (1.8.0_202-ea): ok > > - mvn clean apache-rat:check > > * Built from source (1.8.0_202-ea): ok > > - mvn clean install -DskipTests > > * Unit tests pass (1.8.0_202-ea): ok > > - mvn package -P runSmallTests > > -Dsurefire.rerunFailingTestsCount=3 > > > > From: Andrew Purtell > > Date: Saturday, September 4, 2021 at 3:31 AM > > To: dev > > Subject: [VOTE] First release candidate for HBase 2.4.6 (RC0) is > available > > Please vote on this Apache hbase release candidate, > > hbase-2.4.6RC0 > > > > The VOTE will remain open for at least 72 hours. > > > > [ ] +1 Release this package as Apache hbase 2.4.6 > > [ ] -1 Do not release this package because ... > > > > The tag to be voted on is 2.4.6RC0: > > > > https://github.com/apache/hbase/tree/2.4.6RC0 > > > > This tag currently points to git reference 7374d396c2. > > > > The release files, including signatures, digests, as well as CHANGES.md > > and RELEASENOTES.md included in this RC can be found at: > > > > https://dist.apache.org/repos/dist/dev/hbase/2.4.6RC0/ > > > > The API compatibility report can be found at: > > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.4.6RC0/api_compare_2.4.5_to_2.4.6RC0.html > > > > There are no reported compatibility issues. > > > > Maven artifacts are available in a staging repository at: > > > > > https://repository.apache.org/content/repositories/orgapachehbase-1464/ > > > > Artifacts were signed with the code signing key 0xD5365CCD which can be > > found in: > > > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > > > Some pre-flight checks were made prior to release candidate generation, > > including unit test execution. Some flaky tests were found, see > > HBASE-26254. > > > > To learn more about Apache hbase, please see > > > > http://hbase.apache.org/ > > > > Thanks, > > Your HBase Release Manager > > >
[jira] [Created] (HBASE-26262) Implement region based store file tracker
Duo Zhang created HBASE-26262: - Summary: Implement region based store file tracker Key: HBASE-26262 URL: https://issues.apache.org/jira/browse/HBASE-26262 Project: HBase Issue Type: Sub-task Components: HFile Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26261) Store configuration loss when use update_config
Yi Mei created HBASE-26261:
--
Summary: Store configuration loss when use update_config
Key: HBASE-26261
URL: https://issues.apache.org/jira/browse/HBASE-26261
Project: HBase
Issue Type: Bug
Reporter: Yi Mei
When use update_config shell command, some store configuration is loss.
When initialize store, the conf is set by:
{code:java}
this.conf = new CompoundConfiguration()
.add(confParam)
.addBytesMap(region.getTableDescriptor().getValues())
.addStringMap(family.getConfiguration())
.addBytesMap(family.getValues());
{code}
when change configuration, the conf is set by:
{code:java}
this.conf = new CompoundConfiguration()
.add(conf)
.addBytesMap(getColumnFamilyDescriptor().getValues());
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-26079) Use StoreFileTracker when splitting and merging
[ https://issues.apache.org/jira/browse/HBASE-26079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil resolved HBASE-26079. -- Release Note: Changes MergeTableRegionProcedure and SplitTableRegionProcedure to use the underlying StoreFileTracker implementation to select the valid files on regions to be merged/split, rather than direct listing those from file system. It also changes merge/split commit logic in HRegionFileSystem, to add the links/refs on resulting split/merged regions to the StoreFileTracker. Resolution: Resolved > Use StoreFileTracker when splitting and merging > --- > > Key: HBASE-26079 > URL: https://issues.apache.org/jira/browse/HBASE-26079 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Wellington Chevreuil >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
