HBase quarterly report Apr - June 2018

[Misty Linville]

All,

HBase submits a report to the ASF board once a quarter, to inform the board
about project health. I'm sending the report to the user@ and dev@ mailing
lists because you are the project, and for transparency. If you have any
questions about the report or the running of the project, you can pose them
to me or any other PMC member or committer, or send an email to
priv...@hbase.apache.org, which every PMC member subscribes to.

Thanks,
Misty

--

---
Apache HBase is an open-source, distributed, versioned, non-relational
database. Apache HBase gives you low latency random access to billions of
rows with millions of columns atop non-specialized hardware.

hbase-thirdparty is a set of internal artifacts used by the project to
mitigate the impact of our dependency choices on the wider ecosystem.

ISSUES FOR THE BOARD’S ATTENTION

Board-only information removed from public report.

RELEASES

HBase 1.4.4 was released on April 30 2018.

HBase 2.0.0 was released on April 30 2018.

HBASE 1.2.6.1 was released on June 12 2018.

HBase 1.3.2.1 was released on June 13 2018.

HBase 2.0.1 was released on June 20 2018.

HBase 1.4.5 was released on June 20, 2018.

The "stable" pointer for downstream users is now HBase 1.2.6.1.


ACTIVITY

HBaseCon US took place at Joseph McEnery Convention Center in San Jose, CA,
on June 18, 2018, simultaneous with PhoenixCon 2018. There were 180
attendees including 27 women. 120 of these attendees were there
specifically for HBaseCon. Some relevant quotes from the post-event
feedback:
- "I appreciate all talks, especially those showing unexpected use-cases or
developments in the community."
- "20 minute sessions are great since folks from the community can share
short stories of their experience using HBase."
- "Focus on technical context was refreshing"
Thanks again to Josh Elser and his team at Hortonworks for organizing and
facilitating HBaseCon.

An HBase meet-up took place in Beijing on June 6th in preparation for
HBaseCon Asia 2018. Around 100 people attended in person, and the event was
live-streamed to over 10,000 audience members. Several PMC members were in
attendance. Three remote presenters recorded videos to be shown at the
event.

The second annual HBaseCon Asia is being held in Beijing, China, on Aug.
17, 2018. The event will be hosted by Alibaba, sponsored by Didi, Huawei,
and Xiaomi. Admission will be free to attendees. The call for papers closed
June 30, and resulted in 28 submissions, currently under review. Thanks to
Yu Li for coordinating the event.

HBase 2.0 was released on May 4, 2018, followed up HBase 2.0.1 on June 20.
The HBase 2.0.x line represents the work of over four years and includes
new features, improvements, and bug fixes encompassing 4623 JIRAs. Some of
the new features include the compacting memstore, a rewrite of the
assignment manager, quotas, big performance improvements to the bucket
cache, lots of stability improvements, and much better test coverage.
Thanks to Michael Stack for shepherding this release into the world.

CVE-2018-8025 was addressed by releasing HBase 1.2.6.1, 1.3.2.1, 1.4.5, and
2.0.1. For more information about this vulnerability, see
https://lists.apache.org/thread.html/a919e38f587c714c386a01d40fc8f45bd4219a65aaf2dc0bb4eccc96@%3Cdev.hbase.apache.org%3E.
To report an HBase vulnerability, send an email to priv...@hbase.apache.org
so that we can investigate and address the vulnerability in a responsible
way.

In total, HBase had 6 releases over this quarter.

Francis Liu agreed to step up as a PMC member on Tue Apr 10 2018.

Two contributors became committers during this reporting period:
- Guangxu Cheng was added as a committer on Thu May 31 2018
- Reid Chan was added as a committer on Mon Jun 25 2018

Thanks to the new committers and PMC members for agreeing to take on more
responsibilities in the project.

STATS

The dev@ mailing list saw a slight increase in subscribers over the last
quarter, and a 35% decrease in traffic, consistent with the summer quarter.

The user@ mailing list saw a a slight decline in subscribers, and a 13%
decrease in traffic, consistent with the summer quarter.

75 committers (+2 from last quarter)
42 PMC members (+1 from last quarter)
497 JIRA tickets created (down from 664 last quarter)
428 JIRA tickets closed/resolved (down from 651 last quarter)

[RESPONSE REQUESTED] HBase Committer / PMC pronouns

[Misty Linville]

All,

I'd like to give you the opportunity to state your pronouns, for inclusion
in the project POM and the members list. *This is completely optional.*

If you'd like your pronouns to be listed, please fill out this survey:
https://docs.google.com/forms/d/e/1FAIpQLSfRMJxZfsELIquhyu8rRcmCivDCyE2vX6U9rvoaSfswjvo7xQ/viewform.
*The survey will be open until July 31.*

For more information about pronouns, see http://pronoun.is/.

Thanks,
Misty

Re: Failure: HBase Generate Website

[Misty Linville]

In addition to the build problems probably caused by a lack of Jenkins
workers, there is a bug in the script somewhere that causes the git hash of
the commit that failed to build not to be expanded. I'm not going to be on
a computer all weekend so can someone file and / or investigate?

On Jul 21, 2018 7:30 AM, "Apache Jenkins Server" 
wrote:

Build status: Failure

The HBase website has not been updated to incorporate HBase commit
${CURRENT_HBASE_COMMIT}.

See https://builds.apache.org/job/hbase_generate_website/1406/console

Re: [DISCUSS] Kafka Connection, HBASE-15320

[Misty Linville]

I like the idea of a separate connectors repo/release vehicle, but I'm a
little concerned about the need to release all together to update just one
of the connectors. How would that work? What kind of compatibility
guarantees are we signing up for?

On Tue, Jul 24, 2018, 9:41 PM Stack  wrote:

> Grand. I filed https://issues.apache.org/jira/browse/HBASE-20934. Let me
> have a go at making the easy one work first (the kafka proxy). Lets see how
> it goes. I'll report back here.
> S
>
> On Tue, Jul 24, 2018 at 2:43 PM Sean Busbey  wrote:
>
> > Key functionality for the project's adoption should be in the project.
> > Please do not suggest we donate things to Bahir.
> >
> > I apologize if this is brisk. I have had previous negative experiences
> > with folks that span our communities trying to move work I spent a lot
> > of time contributing to within HBase over to Bahir in an attempt to
> > bypass an agreed upon standard of quality.
> >
> > On Tue, Jul 24, 2018 at 3:38 PM, Artem Ervits 
> > wrote:
> > > Why not just donating the connector to http://bahir.apache.org/ ?
> > >
> > > On Tue, Jul 24, 2018, 12:51 PM Lars Francke 
> > wrote:
> > >
> > >> I'd love to have the Kafka Connector included.
> > >>
> > >> @Mike thanks so much for the contribution (and your planned ones)
> > >>
> > >> I'm +1 on adding it to the core but I'm also +1 on having a separate
> > >> repository under Apache governance
> > >>
> > >> On Tue, Jul 24, 2018 at 6:01 PM, Josh Elser 
> wrote:
> > >>
> > >> > +1 to the great point by Duo about use of non-IA.Public classes
> > >> >
> > >> > +1 for Apache for the governance (although, I wouldn't care if we
> use
> > >> > Github PRs to try to encourage more folks to contribute), a repo
> with
> > the
> > >> > theme of "connectors" (to include Thrift, REST, and the like). Spark
> > too
> > >> --
> > >> > I think we had suggested that prior, but it could be a mental
> > invention
> > >> of
> > >> > mine..
> > >> >
> > >> >
> > >> > On 7/24/18 10:16 AM, Hbase Janitor wrote:
> > >> >
> > >> >> Hi everyone,
> > >> >>
> > >> >> I'm the author of the patch.  A separate repo for all the
> connectors
> > is
> > >> a
> > >> >> great idea! I can make whatever changes necessary to the patch to
> > help.
> > >> >>
> > >> >> I have several other integration type projects like this planned.
> > >> >>
> > >> >> Mike
> > >> >>
> > >> >>
> > >> >> On Tue, Jul 24, 2018, 00:03 Mike Drob  wrote:
> > >> >>
> > >> >> I would be ok with all of the connectors in a single repo. Doing a
> > repo
> > >> >>> per
> > >> >>> connector seems like a large amount of overhead work.
> > >> >>>
> > >> >>> On Mon, Jul 23, 2018, 9:12 PM Clay B.  wrote:
> > >> >>>
> > >> >>> [Non-binding]
> > >> 
> > >>  I am all for the Kafka Connect(er) as indeed it makes HBase "more
> > >>  relevant" and generates buzz to help me sell HBase adoption in my
> > >>  endeavors.
> > >> 
> > >>  Also, I would like to see a connectors repo a lot as I would
> > expect it
> > >> 
> > >> >>> can
> > >> >>>
> > >>  make the HBase source and releases more obvious in what is
> > changing.
> > >> Not
> > >>  to distract from Kafka, but Spark has in the past been a hang-up
> > and
> > >> 
> > >> >>> seems
> > >> >>>
> > >>  a good fit in such a repo too; as such, I would prefer Apache
> over
> > >> 
> > >> >>> GitHub.
> > >> >>>
> > >> 
> > >>  -Clay
> > >> 
> > >>  On Mon, 23 Jul 2018, Andrew Purtell wrote:
> > >> 
> > >>  Would we make a new repo called hbase-connectors and move REST,
> > >> >>
> > >> > thrift,
> > >> >>>
> > >>  and this new patch there?
> > >> >
> > >> > I like this idea. We are already releasing hbase-thirdparty like
> > >> this.
> > >> >
> > >> >
> > >> > On Mon, Jul 23, 2018 at 5:47 PM Stack  wrote:
> > >> >
> > >> > (Thanks for the good discussion)
> > >> >>
> > >> >> Where we think  'outside of HBase' would be?
> > >> >>
> > >> >> Github seems too 'remote' from project and from Apache? Would
> we
> > >> make
> > >> >>
> > >> > a
> > >> >>>
> > >>  new
> > >> 
> > >> > repo called hbase-connectors and move REST, thrift, and this new
> > >> patch
> > >> >> there?
> > >> >>
> > >> >> Thanks,
> > >> >> S
> > >> >>
> > >> >> On Mon, Jul 23, 2018 at 3:50 PM Josh Elser 
> > >> wrote:
> > >> >>
> > >> >> I'm -0 for including this into the main hbase tree. I feel like
> > >> we've
> > >> >>> made a bit of progress in cleaning up our core, and this
> > strikes me
> > >> >>>
> > >> >> as
> > >> >>>
> > >>  a
> > >> 
> > >> > step in the wrong direction.
> > >> >>>
> > >> >>> At the same time, the integration seems nice enough (for the
> > same
> > >> >>> reasons Andrew points out). Is there a reason this couldn't
> > exist
> > >> >>> outside of HBase (at the ASF or otherwise)? Given a quick
> > glance at
> > >> >>>
> > >> 

Re: [DRAFT] draft announcement of stable pointer change

[Misty Linville]

I wouldn't offer them email as a way to communicate about releases. Maybe
JIRA or the list. Though I think JIRA is best.

On Wed, Aug 15, 2018, 12:16 PM Sean Busbey  wrote:

> If anyone has feedback before I send this, please let me know. I've
> already updated dist.a.o for the stable line change. I'd like to send
> this announcement tomorrow once the mirrors are updated.
>
> ---
>
> Hi HBase Community!
>
> The HBase developers would like you to know that we have updated the
> "stable" release pointer to the 1.4.6 release. As future 1.4.z
> releases are made the stable pointer will track them. All users of
> releases prior to 1.4.6 are advised to upgrade to the new stable
> release line.
>
> The prior stable release line was 1.2.z. I'm in the process of getting
> releases on that branch going again. Users of that release line are
> advised that once releases start again they will continue monthly for
> a period of about 6 months before the branch moves to end-of-life.
>
> If you are currently a user of a release line older than 1.4 and
> there's anything that would make your transition to the stable release
> line easier please let us know either via email or JIRAs.
>

Re: Rough notes from dev meetup, day after hbaseconasia 2018, saturday morning

[Misty Linville]

Thanks for the report and for attending, and thanks to everyone who made
HbaseCon Asia 2018 and the dev meet-up happen!

On Sun, Aug 19, 2018, 3:50 AM Stack  wrote:

> There were about 30 of us. I didn't take roll. See photos below [1][2].
> PMCers, committers, contributors, and speakers from the day before. There
> is no attribution of comments or ideas. Please excuse. No agenda.
>
> TESTING
> What do people do testing?
> Allan Yang is finding good stuff when he tests AMv2 compared to me. Why?
> slowDeterministic does more op types than serverKilling.
> What do others do for testing?
> Add more variety to the ITBLLs, more chaos?
> What for performance testing?
> YCSB.
> Batch is important. Its what our users do. Recent addition of batch in YCSB
> (and in PerformanceEvaluation). Size of batch matters too. And number of
> clients.
> Alibaba described what they do.
> Advocate that we all try different test types rather than all do same runs.
> Need to add new async client into YCSB. Alibaba use it for their testing of
> new SEDA core (upstreaming soon).
> Understanding each others benchmarks can take a while. Common understanding
> takes some effort, communication.
> New hbase-operation-tools will be good place to put perf and testing
> tooling.
>
> GITHUB
> Can hbase adopt the github dev flow? Support PRs?
> Its a case of just starting the discussion on the dev list?
> Do we lose review/commentary information if we go github route? Brief
> overview of what is possible w/ the new gitbox repos follows ultimately
> answering that no, there should be no loss (github comments show as jira
> comments).
> Most have github but not apache accounts. PRs are easier. Could encourage
> more contribution, lower the barrier to contrib.
> Other tools for hbase-operation-tools would be stuff like the alibaba
> tooling for shutting down servers... moving regions to new one.
>
> PERF ACROSS VERSIONS
> Lucent (lucene?) has a perf curve on home page with markings for when large
> features arrived and when releases were cut so can see if increase/decrease
> in perf.
> There was a big slowdown going from 0.98 to 1.1.2 hbase.
> We talked about doing such a perf curve on hbase home page. Would be a big
> project. Asked if anyone interested?
> Perhaps a dedicated cluster up on Apache. We could do a whip-around to pay
> for it.
>
> USER FRIENDLY
> Small/new users have a hard time. Is there a UI for users to see data in
> cells or to change schema, or to create/drop tables. Is there anything we
> can do here?
> Much back and forth.
> Xiaomi don't let users have access to shell. Have a web ui where you click
> to build command that is run for you. Afraid that users will mistakenly
> destroy the database so shudown access.
> It turns out that most of the bigger players present have some form of UI
> built against hbase. Alibaba have something. The DiDi folks have howto wiki
> pages.
> Talked about upstreaming.
> Where to put it? hbase-operator-tools?
> What about Docker file to give devs their own hbase easily. Can throw away
> when done.
> One attendee talked of Hue from CDH, how it is good for simple insert and
> view.
> Can check the data. For testing and feel-good getting to know system, it
> helps.
> Another uses Apache Drill but tough when types.
> New users need to be able to import data from a csv.
> How hard to have a few pages of clicky, clicky, wizard to create/drop
> tables or for small query...
> A stripped-down version of Hue to come with HBase how hard to do this?
>
> Next we went over backburner items mention on previous day staring with
> SQL-like access.
> What about lightweight SQL support?
> At Huawei... they have a project going for lightweight SQL support in hbase
> based-on calcite.
> For big queries, they'd go to sparksql.
> Did you look at phoenix?
> Phoenix is complicated, difficult. Calcite migration not done in Phoenix
> (Sparksql is not calcite-based).
> Talk to phoenix project about generating a lightweight artifact. We could
> help with build. One nice idea was building with a cut-down grammar, one
> that removed all the "big stuff" and problematics. Could return to the user
> a nice "not supported" if they try to do a 10Bx10B join.
> An interesting idea about a facade query analyzer making transfer to
> sparksql if big query. Would need stats.
>
> COPROCESSORS
> Can we add some identifiers to distinguish whether request from CP or from
> client. Can we calculate stats on CP resources used? Limit? Can we update
> CPs more gracefully. If heavy usage, when update. Have to disable the
> table. Short answer was no.
> Move CPs to another process. A sidecar process is way to go.
> The Huawei effort at lightweight would also use CPs (like Phoenix).
> Bring the types into hbase, the phoenix types for spark to use etc.
>
> SECONDARY INDICES
> Full support is hard, can we do step-by-step...
> Seperate into several steps?
> Push back that this is a well covered space. Problems known. Contribs in
> tier above welc

Re: HBaseConAsia2018 successfully held

[Misty Linville]

Congratulations to you and all involved in organizing this successful
event, Yu Li! Thank you for your leadership.

It would be great if any attendees on these lists would to share their
experiences from event. It's awesome to see the growth of the HBase
community in Asia.

Misty

On Mon, Aug 20, 2018, 11:54 PM Yu Li  wrote:

> Dear all,
>
> As chairman of the conference, I'm glad to announce the success of
> HBaseConAsia2018 (Aug. 17th, Beijing, China). There were totally 494
> attendees at scene and 14796 views from live broadcast. Shared are part of
> the pictures and more coming, slides and videos will be uploaded later
> (will use another thread for notification when it's ready).
>  HBaseConAsia2018_Pictures
> 
> Please also allow me to express my great thanks to all PC members (both
> community PC and Alibaba local PC). We wouldn't have made it without your
> help!
>
> Just let us know if you have any suggestions about the conference, and
> let's expect the next year!
>
> Best Regards,
> Yu
>

Re: [DISCUSS] Move to github flow (WAS subtopic of "Rough notes from dev meetup, day after hbaseconasia 2018, saturday morning")

[Misty Linville]

Is GitHub integration going to make patch review more palatable and
efficient for reviewers? Maybe we can increase our roster of regular
reviewers by using GitHub.

We can make a bot that looks for things in PR comment text. That way you
could add the JIRA number in a way the bit can pick it up and use it. Maybe
the bot could even attach the patch to the JIRA automatically on new
commits. Kubernetes uses something called Prow and something else called
Blunderbuss for this type of thing.

On Tue, Aug 21, 2018, 9:03 AM Josh Elser  wrote:

> Summarizing my feelings: A first-step might be to inch towards some
> middle ground where we allow code-review via PRs, but we still require a
> Jira issue and a patch to trigger QA (and avoid authorship issues,
> release note issues, etc) to "gate" the commit.
>
> That gives us clear next steps:
> 1) Once QA works for PRs automatically, we can remove patch step
> 2) Once Yetus doc-maker can pull from GH, we can skip Jira issues
> ...
>
> There's some precedence here with already allowing reviewboard and
> phabricator (in my mind). I would be OK with doing code-review on GH and
> would personally prefer it over all other tools at our disposal.
>
> (some things inline too)
>
> On 8/21/18 12:30 AM, Sean Busbey wrote:
> > A couple of other contribution concerns:
> >
> > a) Yetus Precommit works with github PRs, but the ASF Jenkins admin
> > job that sends requests over to our job that runs the precommit tests
> > does not. It's a minor issue (in that collectively we know what has to
> > change), but someone will have to do the work.
>
> Thanks, Sean. I expected something like this to be an immediate blocker.
>
> > b) We've just started getting better release notes together by using
> > Yetus Release Doc Maker. AFAIK it currently only works off of JIRA. I
> > know we haven't said anything about not having a JIRA associated with
> > changes just because we support github PRs, but it'll be around the
> > corner because we'll be duplicating work for those PRs. The two may
> > very well stay side-by-site for those who don't have/want GitHub
> > accounts, but if anything that makes the situation for "how do I
> > gather release notes" more complicated.
>
> Agreed, that would be the next thing to come down the pipe.
>
> Like point-A, I would guess that this is something we can fix by
> expanding Yetus release-doc maker, but requires someone to do it.
>
> > c) Are more casual PRs a boon? In addition to HBase I spend time on an
> > open source project that relies exclusively on GitHub tooling and I
> > lurk on several others. One thing I've noticed is that while the
> > number of casual PRs is certainly higher they tend to be "drop off
> > PRs"; the engagement for follow up is much lower. Many folks who get
> > that PR up on GitHub then don't come back to address requests from
> > reviewers. We'll have to pick one or more of closing unresponsive PRs,
> > more proactively having committers "fix up" contributions, providing
> > more feedback as "follow-on work" instead of something that gets done
> > during the review. I personally would favor closing unresponsive PRs
> > because it has the least overhead for our already sparse reviewing
> > bandwidth.
>
> That's a good point. I don't have strong feelings on how to interpret
> these.
>
> > d) All this said, we don't need to move to gitbox to accept PRs. We
> > can do anything today that we could do after moving to gitbox except
> > have committers merge the PR directly from the GitHub interface.
> > That's not easier for contributors, that's easier for committers. I'm
> > definitely not saying this is a bad thing. I do a non-trivial amount
> > of reviews from my phone and the github UI is definitely worlds
> > better.
>
> Agreed. I think these are two separate issues.
>
> > On Mon, Aug 20, 2018 at 10:08 PM, Stack  wrote:
> >> This came up at the recent devs meeting: could we move to github flow
> >> committing to Apache HBase? Do folks want this? If so, what would it
> take?
> >> What would it look like?
> >>
> >> The new gitbox repos at apache allow contribution back into apache via
> >> github tooling: PRs can be merged into apache repos with a click of a
> >> button, github-based comments can show as comments in apache JIRA. The
> new
> >> hbase-operator-tools and hbase-connector repos are gitbox based. We can
> run
> >> experiments there with fear of damage to the core.
> >>
> >> The justification is that if our project supported PRs and contribution
> via
> >> github, we could glean more contributors.
> >>
> >> Below I repeat two follow-on comments taken from the "Rough notes from
> dev
> >> meetup, day after hbaseconasia 2018, saturday morning" thread by way of
> a
> >> kickstart:
> >>
> >>  From our Josh Elser:
> >>
> >>> This [supporting PRs] is something the PMC should take to heart. If we
> >> are excluding
> >>> contributions because of how we choose accept them, we're limiting our
> own
> >>> growth. Do we have technical re

Re: [DISCUSS] Hadoop version compatibility table

[Misty Linville]

Let's change that table to positive statements. Production, supported,
recommended, tested. WDYT?

On Tue, Aug 21, 2018, 8:12 AM Josh Elser  wrote:

> Mich T had cross-posted a question to users@{hbase,phoenix} the only
> day. After some more information, we were able to find out that Mich was
> trying to use Hadoop 3.1 with HBase 1.2.6
>
> After pointing Mich to the compatibility table[1], I was about to puff
> out my chest and say "look! this table could've told you that HBase
> 1.2.6 wouldn't work with Hadoop 3.x!"
>
> But, then I realized that we don't have a single entry for HBase that
> implies it would even work for Hadoop 3. We presently have the following:
>
>"S" = supported
>"X" = not supported
>"NT" = Not tested
>
> I propose that would should add another "value" for cells in the table
> to better represent "Works, but not battle-tested" or similar. That
> would make possible values:
>
>"S" = supported
>"NP" = not production ready
>"X" = not supported
>"NT" = not tested
>
> Furthermore, the word "supported" drives me up a wall (as I think it
> implies the wrong mindset for an open source community), and I would
> rather see "functional". e.g.
>
>"F" = Fully functional, production ready
>"NP" = Functional, but not production ready/has known issues
>"X" = Not functional, lacking basic ability
>"NT" = Not tested, functionality is unknown
>
> Thoughts? Things that I've missed?
>
> - Josh
>
> [1] http://hbase.apache.org/book.html#hadoop
>

Re: [DISCUSS] Hadoop version compatibility table

[Misty Linville]

To be clear, I'm good with a symbolic representation, but I think we should
be making positive statements about configs we stand behind rather than
those we don't. More of a bounded set.

On Tue, Aug 21, 2018, 11:01 AM Misty Linville  wrote:

> Let's change that table to positive statements. Production, supported,
> recommended, tested. WDYT?
>
> On Tue, Aug 21, 2018, 8:12 AM Josh Elser  wrote:
>
>> Mich T had cross-posted a question to users@{hbase,phoenix} the only
>> day. After some more information, we were able to find out that Mich was
>> trying to use Hadoop 3.1 with HBase 1.2.6
>>
>> After pointing Mich to the compatibility table[1], I was about to puff
>> out my chest and say "look! this table could've told you that HBase
>> 1.2.6 wouldn't work with Hadoop 3.x!"
>>
>> But, then I realized that we don't have a single entry for HBase that
>> implies it would even work for Hadoop 3. We presently have the following:
>>
>>"S" = supported
>>"X" = not supported
>>"NT" = Not tested
>>
>> I propose that would should add another "value" for cells in the table
>> to better represent "Works, but not battle-tested" or similar. That
>> would make possible values:
>>
>>"S" = supported
>>"NP" = not production ready
>>"X" = not supported
>>"NT" = not tested
>>
>> Furthermore, the word "supported" drives me up a wall (as I think it
>> implies the wrong mindset for an open source community), and I would
>> rather see "functional". e.g.
>>
>>"F" = Fully functional, production ready
>>"NP" = Functional, but not production ready/has known issues
>>"X" = Not functional, lacking basic ability
>>"NT" = Not tested, functionality is unknown
>>
>> Thoughts? Things that I've missed?
>>
>> - Josh
>>
>> [1] http://hbase.apache.org/book.html#hadoop
>>
>

Re: [DISCUSS] automated checks while asf jenkins is down

[Misty Linville]

I'm concerned that getting the logs will take a lot of your time. Is there
a way to have the automation put them somewhere where the patch author can
get to them when needed? Also how much time will this take to set up? Will
it be worth it? Will this initial setup be useful long-term in any way?

On Thu, Aug 30, 2018, 7:36 AM Sean Busbey  wrote:

> Hi folks!
>
> As background, the ASF jenkins master (what you see when you go to
> builds.apache.org) went down Tuesday night[1]. ASF Infra is dutifully
> working to restore it to full capabilities without losing information
> on prior builds. Their most recent estimate has a return to normal
> sometime tonight or early tomorrow morning.
>
> I have a few patches I'd like to still see precommit results for
> before they get signed off on. I was going to set up checking them
> with test-patch myself, but it would only be marginally more work for
> me to automate it.
>
> What do folks think about me doing this as a stop-gap while ASF Jenkins is
> down?
>
> It would be on transient hosts that essentially only I had access to
> (and I guess my employer?). So we'd get results posted to JIRA but
> getting to actual logs would be a bit more of a pain.
>
> Does anyone feel strongly about me using the existing QABot JIRA
> credentials vs making a "I'm a placeholder QABot" account with its own
> credentials?
>
> [1]: https://status.apache.org/incidents/4zl6mkyrg8qt
>

Re: [DISCUSS] Move to github flow (WAS subtopic of "Rough notes from dev meetup, day after hbaseconasia 2018, saturday morning")

[Misty Linville]

I like this suggestion, Yu, along with enthusiastic closing of GitHub PRs
that don't follow the established procedures. Could we create an automated
test for such compliance, like scanning the PR comments for specific text?
CNCF projects use automation to parse a specific syntax in comments and set
GitHub labels based on it. It could be as simple as /jira HBASE-foo, and
automation could check that the issue actually exists and is open. The PR
author could get a grace period of say a week before the PR is
automatically closed, with warnings at regular intervals. Even if the PR is
closed they can easily reopen it when they have created the issue. If they
don't, we can't take their contribution, no matter how good it is.

This approach front-loads the work of creating the automation, but reduces
overall load of reviewers verifying compliance of the PR. It also enforces
the policy with more consistency than relying on humans to do it.

I suspect that ASF will eventually need to offer account holders to
associate their account with a GitHub account, and that would alleviate
some of this. There could even be a chain of trust by committing a
cryptographically unique file generated by ASF processes into the user's
GitHub account maybe (as a gist?) as part of that connection, or by using
GitHub's authorization system.

On Sun, Sep 2, 2018, 11:32 PM Yu Li  wrote:

> Maybe we could follow some other projects' process? For example I could
> find below lines of Flink's Code of Conduct
> , JFYI:
>
> *Before you start coding…*
>
> *…please make sure there is a Jira issue that corresponds to your
> contribution. This is a general rule that the Flink community follows for
> all code contributions, including bug fixes, improvements, or new features,
> with an exception for trivial hot fixes. If you would like to fix a bug
> that you found or if you would like to add a new feature or improvement to
> Flink, please follow the File a bug report
>  or
> Propose
> an improvement or a new feature
> <
> https://flink.apache.org/how-to-contribute.html#propose-an-improvement-or-a-new-feature
> >
> guidelines
> to open an issue in Flink’s Jira
>  before starting with the
> implementation.*
>
> Best Regards,
> Yu
>
>
> On Mon, 3 Sep 2018 at 00:52, Andrew Purtell 
> wrote:
>
> > It's on us as committers to ensure Apache policies on code contribution
> > are followed. I think that is the bottom line. Our identities are known
> to
> > Apache infrastructure and recorded in commit history. By committing we
> are
> > asserting we believe the provenance of the contribution is known and
> > conforms to Apache policy. If you have concerns then don't commit.
> > Otherwise, it is the same as it always was. Both of Apache's JIRA
> instance
> > and Github offer self service account signups without any sort of
> identity
> > verification, only verification that the email account provided is valid
> > and under the control of the contributor.
> >
> > I think opening placeholder JIRAs as committers because we need it for
> > tracking is fine. With my PMC hat on I'd like to wonder aloud what is the
> > problem with signing up to Apache's JIRA instance. Any reason to be
> > concerned? I take it you don't think so. I also take it you believe you
> > know the identity of the contributor. Assuming you answer in the
> > affirmative I don't see that we have any issues.
> >
> >
> > > On Sep 2, 2018, at 9:00 AM, Sean Busbey  wrote:
> > >
> > >> On Mon, Aug 20, 2018, 22:09 Stack  wrote:
> > >>
> > >> This came up at the recent devs meeting: could we move to github flow
> > >> committing to Apache HBase? Do folks want this? If so, what would it
> > take?
> > >> What would it look like?
> > >>
> > >> The new gitbox repos at apache allow contribution back into apache via
> > >> github tooling: PRs can be merged into apache repos with a click of a
> > >> button, github-based comments can show as comments in apache JIRA. The
> > new
> > >> hbase-operator-tools and hbase-connector repos are gitbox based. We
> can
> > run
> > >> experiments there with fear of damage to the core.
> > >>
> > >> The justification is that if our project supported PRs and
> contribution
> > via
> > >> github, we could glean more contributors.
> > >>
> > >> Below I repeat two follow-on comments taken from the "Rough notes from
> > dev
> > >> meetup, day after hbaseconasia 2018, saturday morning" thread by way
> of
> > a
> > >> kickstart:
> > >>
> > >> From our Josh Elser:
> > >>
> > >>> This [supporting PRs] is something the PMC should take to heart. If
> we
> > >> are excluding
> > >>> contributions because of how we choose accept them, we're limiting
> our
> > >> own
> > >>> growth. Do we have technical reasons (e.g. PreCommit) which we cannot
> > >> accept
> > >>> PR's or is it just because "we do patches because we do patches"?
> 

Re: [DISCUSS] Move to github flow (WAS subtopic of "Rough notes from dev meetup, day after hbaseconasia 2018, saturday morning")

[Misty Linville]

The below snip isn't true. Somehow CNCF has tooling that does it (it is a
bot called fejtabot). I can try to find out more about it if this is a
mechanism we are interested in pursuing.

On Tue, Sep 4, 2018 at 7:08 AM, Sean Busbey  wrote:
>
>
> We can't automate closing PRs because the only way to close a PR is by
> pushing a commit to our master branch and ASF policy doesn't allow for
> non-commiters pushing into the repository if the repository is used to
> make release artifacts.
>
>

Re: [DISCUSS] Move to github flow (WAS subtopic of "Rough notes from dev meetup, day after hbaseconasia 2018, saturday morning")

[Misty Linville]

Here's the code to the bot:
https://github.com/kubernetes/test-infra/tree/master/robots/commenter

On Tue, Sep 4, 2018 at 10:21 AM, Sean Busbey  wrote:

> Do you have a link to an instance of the bot you're talking about?
> On Tue, Sep 4, 2018 at 11:18 AM Misty Linville  wrote:
> >
> > The below snip isn't true. Somehow CNCF has tooling that does it (it is a
> > bot called fejtabot). I can try to find out more about it if this is a
> > mechanism we are interested in pursuing.
> >
> > On Tue, Sep 4, 2018 at 7:08 AM, Sean Busbey  wrote:
> > >
> > >
> > > We can't automate closing PRs because the only way to close a PR is by
> > > pushing a commit to our master branch and ASF policy doesn't allow for
> > > non-commiters pushing into the repository if the repository is used to
> > > make release artifacts.
> > >
> > >
>

Re: [DISCUSS] Move to github flow (WAS subtopic of "Rough notes from dev meetup, day after hbaseconasia 2018, saturday morning")

[Misty Linville]

The code I posted above is an abstraction that is configured by YAML stuff
like the following (where the test exists for closing PRs):
https://github.com/kubernetes/test-infra/blob/a8cee5a60a2d9476341cf843867221a8bd18a3e8/config/jobs/kubernetes/test-infra/test-infra-periodics.yaml#L47

I am not up on Go, but others pointed me there.

On Tue, Sep 4, 2018 at 4:12 PM, Misty Linville  wrote:

> Here's the code to the bot: https://github.com/kubernetes/test-infra/tree/
> master/robots/commenter
>
> On Tue, Sep 4, 2018 at 10:21 AM, Sean Busbey  wrote:
>
>> Do you have a link to an instance of the bot you're talking about?
>> On Tue, Sep 4, 2018 at 11:18 AM Misty Linville  wrote:
>> >
>> > The below snip isn't true. Somehow CNCF has tooling that does it (it is
>> a
>> > bot called fejtabot). I can try to find out more about it if this is a
>> > mechanism we are interested in pursuing.
>> >
>> > On Tue, Sep 4, 2018 at 7:08 AM, Sean Busbey  wrote:
>> > >
>> > >
>> > > We can't automate closing PRs because the only way to close a PR is by
>> > > pushing a commit to our master branch and ASF policy doesn't allow for
>> > > non-commiters pushing into the repository if the repository is used to
>> > > make release artifacts.
>> > >
>> > >
>>
>
>

Re: HBaseCon 2016 and Archives

[Misty Linville]

Then it will get clobbered the next time the site is built by CI. Please
update it in the main master.

On Mon, Sep 17, 2018, 10:01 AM Josh Elser  wrote:

> Any chance you can open up a Jira issue and provide a patch to update
> hbase-site.git?
>
> https://git-wip-us.apache.org/repos/asf?p=hbase-site.git
>
> You would want to modify the file hbasecon-archives.html (if that wasn't
> clear) :)
>
> On 9/12/18 9:07 AM, J. Javier Maestro wrote:
> > Sorry, the final link for the slides is
> >
> >
> https://speakerdeck.com/jjmaestro/hbasecon-2016-west-containerizing-apache-hbase-clusters
> >
> > I've just updated it to reflect that the conference was the West
> conference.
> >
> > Thanks,
> >
> > Javier
> > On Wed, Sep 12, 2018 at 2:04 PM J. Javier Maestro 
> wrote:
> >>
> >> Hi there!
> >>
> >> In HBaseCon 2016 West (San Francisco) I presented, together with David
> >> Pope "Containerizing Apache HBase Clusters". I think there was some
> >> issue with the recording, we were told that due to past issues
> >> recording Facebook talks, they didn't publish the video this time.
> >> That was a pity.
> >>
> >> However, I've now checked the archive of HBaseCon and we are
> >> completely absent from there. There were other presentations by
> >> Facebookers in there and they are listed (Matt Mullins on the panel,
> >> for example) and in HBaseCon 2016 East, Mikhail Antonov's talk too,
> >> both with slides and recording.
> >>
> >> I'd like to know if you can:
> >>
> >> - Fix this, by adding the talk to HBaseCon 2016 East. Here are the
> >> slides, so you can at least link that:
> >>
> https://speakerdeck.com/jjmaestro/hbasecon2016-containerizing-apache-hbase-clusters
> >>
> >> - Find out if the recording took place.
> >>
> >> Thanks a ton for all your help!
> >>
> >> Regards,
> >>
> >> Javier
> >
> >
> >
>

Re: HBaseCon 2016 and Archives

[Misty Linville]

Nope. Our content is built via an automated and documented Jenkins job from
our source, via Maven. Jenkins commits those artifacts to the hbase-site
repo. It is all above board.

On Mon, Sep 17, 2018, 4:53 PM Josh Elser  wrote:

> It will? Maybe what we have going on right now is dangerous/in risk of
> disappearing magically?
>
> I'm pretty sure all of the HBaseCon content only exists in the
> hbase-site.git repository.
>
> Thanks Misty.
>
> On 9/17/18 3:12 PM, Misty Linville wrote:
> > Then it will get clobbered the next time the site is built by CI. Please
> > update it in the main master.
> >
> > On Mon, Sep 17, 2018, 10:01 AM Josh Elser  wrote:
> >
> >> Any chance you can open up a Jira issue and provide a patch to update
> >> hbase-site.git?
> >>
> >> https://git-wip-us.apache.org/repos/asf?p=hbase-site.git
> >>
> >> You would want to modify the file hbasecon-archives.html (if that wasn't
> >> clear) :)
> >>
> >> On 9/12/18 9:07 AM, J. Javier Maestro wrote:
> >>> Sorry, the final link for the slides is
> >>>
> >>>
> >>
> https://speakerdeck.com/jjmaestro/hbasecon-2016-west-containerizing-apache-hbase-clusters
> >>>
> >>> I've just updated it to reflect that the conference was the West
> >> conference.
> >>>
> >>> Thanks,
> >>>
> >>> Javier
> >>> On Wed, Sep 12, 2018 at 2:04 PM J. Javier Maestro 
> >> wrote:
> >>>>
> >>>> Hi there!
> >>>>
> >>>> In HBaseCon 2016 West (San Francisco) I presented, together with David
> >>>> Pope "Containerizing Apache HBase Clusters". I think there was some
> >>>> issue with the recording, we were told that due to past issues
> >>>> recording Facebook talks, they didn't publish the video this time.
> >>>> That was a pity.
> >>>>
> >>>> However, I've now checked the archive of HBaseCon and we are
> >>>> completely absent from there. There were other presentations by
> >>>> Facebookers in there and they are listed (Matt Mullins on the panel,
> >>>> for example) and in HBaseCon 2016 East, Mikhail Antonov's talk too,
> >>>> both with slides and recording.
> >>>>
> >>>> I'd like to know if you can:
> >>>>
> >>>> - Fix this, by adding the talk to HBaseCon 2016 East. Here are the
> >>>> slides, so you can at least link that:
> >>>>
> >>
> https://speakerdeck.com/jjmaestro/hbasecon2016-containerizing-apache-hbase-clusters
> >>>>
> >>>> - Find out if the recording took place.
> >>>>
> >>>> Thanks a ton for all your help!
> >>>>
> >>>> Regards,
> >>>>
> >>>> Javier
> >>>
> >>>
> >>>
> >>
> >
>

HBase quarterly report Jul-Sep 2018

[Misty Linville]

All,

HBase submits a report to the ASF board once a quarter, to inform the board
about project health. I'm sending the report to the user@ and dev@ mailing
lists because you are the project. If you have any questions about the
report or the running of the project, you can pose them to me or any other
PMC member or committer, or send an email to priv...@hbase.apache.org,
which every PMC member subscribes to.

Thanks,
Misty

--
Apache HBase is an open-source, distributed, versioned, non-relational
database. Apache HBase gives you low latency random access to billions of
rows with millions of columns atop non-specialized hardware.

hbase-thirdparty is a set of internal artifacts used by the project to
mitigate the impact of our dependency choices on the wider ecosystem.

ISSUES FOR THE BOARD’S ATTENTION

Board-only information removed from public report.

RELEASES

- 2.1.0 was released on Tue Jul 17 2018
- 1.4.6 was released on Sun Jul 29 2018
- 2.0.2 was released on Sat Sep 01 2018
- 1.4.7 was released on Sun Sep 02 2018
- 1.2.7 was released on Sat Sep 15 2018


ACTIVITY

The second annual HBaseCon Asia was held in Beijing, China, on Aug. 17,
2018. The event was hosted by Alibaba, sponsored by Didi, Huawei, and
Xiaomi. Admission was free to attendees, and a live stream allowed
worldwide participation. In total, 494 individuals attended locally and the
livestream accumulated 14796 views. Thanks to Yu Li and all organizers for
coordinating the event. [Event summary](
http://mail-archives.apache.org/mod_mbox/hbase-user/201808.mbox/) [Photos,
videos, slides](
http://mail-archives.apache.org/mod_mbox/hbase-user/201808.mbox/)

A developer meet-up occurred in conjunction with HBaseCon Asia on August
18, 2018. Around 30 people attended. [Michael Stack took notes](
http://mail-archives.apache.org/mod_mbox/hbase-dev/201808.mbox/browser).
Key discussion points included testing, proposals for using Github in our
workflow, a discussion about user experienc for new HBase users, and more.

Three more developer meet-ups occurred in September in China:

- September 1 in HangZhou
- September 8 in ShangHai
- September 15 in ShenZhen

More than 200 people attended locally and hundreds participated via
livestreams.
[Details and recordings](https://s.apache.org/MdSV) are available (in
Chinese).

In total, HBase had 5 releases over this quarter, across 4 release lines.

Toshihiro Suzuki was added as a committer on Wed Aug 01 2018.

Zach York joined the PMC on October 5, 2018 (out of this reporting period).

Stay tuned for more committer and PMC member announcements next month, as
they just missed the report cut-off date!

Thanks to the new committers and PMC members for agreeing to take on more
responsibilities in the project.

STATS

The dev@ mailing list saw a negligible decline in subscribers and nearly
identical traffic as the last quarter.

The user@ mailing list lost 23 subscribers along with a negligible increase
in traffic over the last quarter.

76 committers (+1 from last quarter)
42 PMC members (+0 from last quarter)
413 JIRA tickets created (down from 497 last quarter)
343 JIRA tickets closed/resolved (down from 428 last quarter)

Re: Site edit and rebuild howto?

[Misty Linville]

I'll review this JIRA this weekend if you add me as a reviewer.

On Fri, Oct 19, 2018, 12:35 PM Andrew Purtell  wrote:

> https://issues.apache.org/jira/browse/HBASE-21346
>
> On Fri, Oct 19, 2018 at 12:29 PM Andrew Purtell 
> wrote:
>
> > Roger that, try this, push, and pray. Thanks Mike.
> >
> >
> > On Fri, Oct 19, 2018 at 12:08 PM Mike Drob  wrote:
> >
> >> To update the download links, on master branch edit
> >> src/site/xdoc/downloads.xml
> >> After you commit and push, jenkins will build the site and publish it
> for
> >> you.
> >>
> >> To update the API Docs and version specific reference guide, update
> >> src/site/site.xml with a new section to link to the docs in the drop
> down
> >> list. (only necessary the first time, but it hasn't been done yet for
> >> 1.4.x)
> >>
> >> Then git clone https://git-wip-us.apache.org/repos/asf/hbase-site.git
> and
> >> make a 1.4 directory there.
> >> Copy contents of the docs/ directory from the release tarball to the
> >> version directory.
> >> Copy target/site/devapidocs and testapidocs from a local build of the
> tag,
> >> since those don't get published in the release tarball.
> >> Commit your changes, then do an empty commit with message "INFRA-10751
> >> Empty commit"
> >> Push your changes, then pray.
> >>
> >>
> >> Also, if these work for you then consider updating either
> >> http://hbase.apache.org/book.html#website_publish or the post-release
> >> instructions for people in the future.
> >>
> >>
> >> Mike
> >>
> >> On Fri, Oct 19, 2018 at 1:46 PM Mike Drob  wrote:
> >>
> >> > I'll write up some docs for you, putting this placeholder out so that
> >> > somebody else doesn't get stuck duplicating effort since this might
> >> take me
> >> > 30-60 min to gather notes and send them out.
> >> >
> >> > On Fri, Oct 19, 2018 at 1:30 PM Andrew Purtell 
> >> > wrote:
> >> >
> >> >> I need to update hbase.apache.org/downloads.html to add an entry for
> >> the
> >> >> 1.4.8 release. It's been ages since I've updated our website. I don't
> >> >> actually remember how to do it and imagine the procedure has changed
> >> >> anyhow. How do I go about editing the site, building it, and
> deploying
> >> it?
> >> >> The book doesn't seem to include instructions.
> >> >>
> >> >> I apologize for the lack of release announcement for 1.4.8. I can't
> >> >> announce the release until there is a download link and this has been
> >> >> blocking me. I also apologize about asking for help only now. I keep
> >> >> coming
> >> >> back to it, realizing it is going to take an inordinate amount of
> time
> >> to
> >> >> get sorted, and then put it off again.
> >> >>
> >> >>
> >> >> --
> >> >> Best regards,
> >> >> Andrew
> >> >>
> >> >> Words like orphans lost among the crosstalk, meaning torn from
> truth's
> >> >> decrepit hands
> >> >>- A23, Crosstalk
> >> >>
> >> >
> >>
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >- A23, Crosstalk
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>

Re: [DISCUSS] Release cadence for HBase 2.y

[Misty Linville]

This makes me wonder if we have, or have a way to get, analytics about the
version people are running? It's not great to guess about things like this.
We're making a big assumption that our users actually pay attention to user@
or more often dev@ in order to complain about a branch being retired too
quickly in time for us to listen.

On Sun, Nov 11, 2018, 8:04 PM Stack  Yes. Was suggesting retiring branch-2.0 and suggesting that we throw the
> troops against the branch-2.2 flank. Agree though that if there are folks
> who want more releases, lets do them (please speak up if this is so). 2.0.3
> will be good since it close to 2.1.1. Unless demand, 2.0.4 will likely
> drag.
>
> In my limited testing (10B ITBLL), branch-2.1 (2.1.1) was making a good
> showing; better than what we've shipped previous in the past (in my
> estimation).
>
> Thanks Allan.
>
> (FYI branch-1.0 as short-lived if any consolation).
>
> S
>
>
>
>
>
>
>
> On Sun, Nov 11, 2018 at 6:12 PM Allan Yang  wrote:
>
> > Stack, are you suggest about retiring branch-2.0? I think it is OK, since
> > branch-2.0 is almost the same with branch-2.1 now(except some new feature
> > on replication). Yes, agree that we should help out on branch-2.2. AMv2
> > changed a lot in branch-2, there may still have some work to do to make
> > branch-2.2 stable. But at same time, I think we can mark branch-2.1 as
> > stable. We have done tremendous work on this branch, and recently ITBLLs
> > shows it is already stable enough(based on our internal version, but most
> > of patches in branch-2.1 was backported)
> > Best Regards
> > Allan Yang
> >
> >
> > Stack  于2018年11月12日周一 上午6:57写道：
> >
> > > Agree w/ Duo that the 2.x releases have been gated on stability
> > watersheds
> > > rather than features.
> > >
> > > What else do we need to add to HBCK2 Duo (apart from a release)?
> > >
> > > Related, I was going to work on a 2.0.3 release. It has been a while
> and
> > a
> > > bunch of good stability work has made it into branch-2.0. Thereafter
> > > though, I was going to let branch-2.0 go unless demand -- Allan Yang?
> --
> > > and switch instead to helping out on branch-2.2.
> > >
> > > S
> > >
> > > On Thu, Nov 8, 2018 at 6:10 PM 张铎(Duo Zhang) 
> > > wrote:
> > >
> > > > I think for the 2.x release the problem is that we are still busy on
> > > making
> > > > the code stable, or speak more clearly, to make the procedure v2
> > > framework
> > > > stable... And another big problem is lacking of HBCK2 support. These
> > > things
> > > > are all big issues which prevent people to upgrade to 2.x.
> > > >
> > > > Once these things are done, I think a monthly release will not be a
> big
> > > > problem to the RMs. Just simply run an ITBLL(for now it is not easy
> to
> > > get
> > > > a successful run and then we need to find out why...), and then the
> > > > make_rc.sh can not everything for you...
> > > >
> > > > Sean Busbey  于2018年11月9日周五 上午9:45写道：
> > > >
> > > > > I think it just shifts the RM burden, no? Like instead of watching
> > e.g.
> > > > > branch-2.2 I instead need to watch branch-2.
> > > > >
> > > > > On Thu, Nov 8, 2018, 17:28 Josh Elser  > > > >
> > > > > > I think what I'd be concerned about WRT time-based releases is
> the
> > > > > > burden on RM to keep the branch in a good state. Perhaps we need
> to
> > > not
> > > > > > push that onto an RM and do better about sharing that load
> (looking
> > > in
> > > > > > the mirror).
> > > > > >
> > > > > > However, I do like time-based releases as a means to avoid "hurt
> > > > > > feelings" (e.g. the personal ties of a developer to a feature.
> "The
> > > > > > release goes out on /yy/xx, this feature is not yet ready,
> can
> > go
> > > > > > out one month later.." etc)
> > > > > >
> > > > > > On 11/7/18 2:31 PM, Sean Busbey wrote:
> > > > > > > Hi folks!
> > > > > > >
> > > > > > > Some time ago we talked about trying to get back on track for a
> > > more
> > > > > > > regular cadence of minor releases rather than maintenance
> > releases
> > > > > > > (like how we did back pre-1.0). That never quite worked out for
> > the
> > > > > > > HBase 1.y line, but is still something we could make happen for
> > > HBase
> > > > > > > 2.
> > > > > > >
> > > > > > > We're coming up on 4 months since the 2.1 release line started.
> > ATM
> > > > > > > there are 63 issues in JIRA that claim to be in 2.2.0 and not
> in
> > > any
> > > > > > > 2.1.z version[1].
> > > > > > >
> > > > > > > The main argument against starting to do a 2.2.0 release is
> that
> > > > > > > nothing springs out of that list as a "feature" that would
> entice
> > > > > > > users to upgrade. Waiting for these kinds of selling points to
> > > drive
> > > > a
> > > > > > > release is commonly referred to as "feature based releases." I
> > > think
> > > > > > > it would be fair to characterize the HBase 2.0 release as
> feature
> > > > > > > based centered on AMv2.
> > > > > > >
> > > > > > > An alternative to feature based releases is date based re

Re: [DISCUSS] Release cadence for HBase 2.y

[Misty Linville]

I’ll start another thread about metrics gathering.

On Tue, Nov 13, 2018 at 8:15 AM Stack  wrote:

> On Sun, Nov 11, 2018 at 8:18 PM Misty Linville  wrote:
>
> >  It's not great to guess about things like this.
> > We're making a big assumption that our users actually pay attention to
> > user@
> > or more often dev@ in order to complain about a branch being retired too
> > quickly in time for us to listen.
> >
> >
> True.
>
> A phone home would be sweet so we had a bit of data on what versions and
> where.
>
> I was thinking of pushing the 2.0.3 and then just waiting until an ugly bug
> or someone spoke up before doing a 2.0.4... slo-mo death.
>
> On general topic, yeah, we should try to be time-based once over the
> stability humps (IMO branch-2.1 is candidate now for time-based releases).
> Lets try and get branch-2.2 stabilized so can push out a 2.2.0.
>
> Thanks,
> S
>
>
>
>
> > On Sun, Nov 11, 2018, 8:04 PM Stack  >
> > > Yes. Was suggesting retiring branch-2.0 and suggesting that we throw
> the
> > > troops against the branch-2.2 flank. Agree though that if there are
> folks
> > > who want more releases, lets do them (please speak up if this is so).
> > 2.0.3
> > > will be good since it close to 2.1.1. Unless demand, 2.0.4 will likely
> > > drag.
> > >
> > > In my limited testing (10B ITBLL), branch-2.1 (2.1.1) was making a good
> > > showing; better than what we've shipped previous in the past (in my
> > > estimation).
> > >
> > > Thanks Allan.
> > >
> > > (FYI branch-1.0 as short-lived if any consolation).
> > >
> > > S
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Sun, Nov 11, 2018 at 6:12 PM Allan Yang 
> wrote:
> > >
> > > > Stack, are you suggest about retiring branch-2.0? I think it is OK,
> > since
> > > > branch-2.0 is almost the same with branch-2.1 now(except some new
> > feature
> > > > on replication). Yes, agree that we should help out on branch-2.2.
> AMv2
> > > > changed a lot in branch-2, there may still have some work to do to
> make
> > > > branch-2.2 stable. But at same time, I think we can mark branch-2.1
> as
> > > > stable. We have done tremendous work on this branch, and recently
> > ITBLLs
> > > > shows it is already stable enough(based on our internal version, but
> > most
> > > > of patches in branch-2.1 was backported)
> > > > Best Regards
> > > > Allan Yang
> > > >
> > > >
> > > > Stack  于2018年11月12日周一 上午6:57写道：
> > > >
> > > > > Agree w/ Duo that the 2.x releases have been gated on stability
> > > > watersheds
> > > > > rather than features.
> > > > >
> > > > > What else do we need to add to HBCK2 Duo (apart from a release)?
> > > > >
> > > > > Related, I was going to work on a 2.0.3 release. It has been a
> while
> > > and
> > > > a
> > > > > bunch of good stability work has made it into branch-2.0.
> Thereafter
> > > > > though, I was going to let branch-2.0 go unless demand -- Allan
> Yang?
> > > --
> > > > > and switch instead to helping out on branch-2.2.
> > > > >
> > > > > S
> > > > >
> > > > > On Thu, Nov 8, 2018 at 6:10 PM 张铎(Duo Zhang) <
> palomino...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > I think for the 2.x release the problem is that we are still busy
> > on
> > > > > making
> > > > > > the code stable, or speak more clearly, to make the procedure v2
> > > > > framework
> > > > > > stable... And another big problem is lacking of HBCK2 support.
> > These
> > > > > things
> > > > > > are all big issues which prevent people to upgrade to 2.x.
> > > > > >
> > > > > > Once these things are done, I think a monthly release will not
> be a
> > > big
> > > > > > problem to the RMs. Just simply run an ITBLL(for now it is not
> easy
> > > to
> > > > > get
> > > > > > a successful run and then we need to find out why...), and then
> the
> > > > > > make_rc.sh can not everything for you...
> > > > > >
> > > > > > Sean Busbey  于20

[DISCUSS] Gathering metrics on HBase versions in use

[Misty Linville]

When discussing the 2.0.x branch in another thread, it came up that we
don’t have a good way to understand the version skew of HBase across the
user base. Metrics gathering can be tricky. You don’t want to capture
personally identifiable information (PII) and you need to be transparent
about what you gather, for what purpose, how long the data will be
retained, etc. The data can also be sensitive, for instance if a large
number of installations are running a version with a CVE or known
vulnerability against it. If you gather metrics, it really needs to be
opt-out rather than opt-in so that you actually get a reasonable amount of
data. You also need to stand up some kind of metrics-gathering service and
run it somewhere, and some kind of reporting / visualization tooling. The
flip side of all these difficulties is a more intelligent way to decide
when to retire a branch or when to communicate more broadly / loudly asking
people in a certain version stream to upgrade, as well as where to
concentrate our efforts.

I’m not sticking my hand up to implement such a monster. I only wanted to
open a discussion and see what y’all think. It seems to me that a few
must-haves are:

- Transparency: Release notes, logging about the status of
metrics-gathering (on or off) at master or RS start-up, logging about
exactly when and what metrics are sent
- Low frequency: Would we really need to wake up and send metrics more
often than weekly?
- Conservative approach: Only collect what we can find useful today, don’t
collect the world.
- Minimize PII: This probably means not trying to group together
time-series results for a given server or cluster at all, but could make
the data look like there were a lot more clusters running in the world than
really are.
- Who has access to the data? Do we make it public or limit access to the
PMC? Making it public would bolster our discipline about transparency and
minimizing PII.

I’m sure I’m missing a ton so I leave the discussion to y’all.

Re: [ANNOUNCE] New HBase committer Jingyun Tian

[Misty Linville]

Congratulations, and thanks for opting to spend more time making HBase the
software and the project better!

On Wed, Nov 14, 2018, 3:26 AM Anoop John  Congrats Jingyun !!!
>
> -Anoop-
>
> On Wed, Nov 14, 2018 at 8:18 AM Jingyun Tian  wrote:
>
> > Thank you all!
> >
> > Sincerely,
> > Jingyun Tian
> >
> > On Wed, Nov 14, 2018 at 8:59 AM stack  wrote:
> >
> > > Welcome jingyun.
> > > S
> > >
> > > On Mon, Nov 12, 2018, 11:54 PM 张铎(Duo Zhang)  > wrote:
> > >
> > > > On behalf of the Apache HBase PMC, I am pleased to announce that
> > Jingyun
> > > > Tian has accepted the PMC's invitation to become a committer on the
> > > > project. We appreciate all of Jingyun's generous contributions thus
> far
> > > and
> > > > look forward to his continued involvement.
> > > >
> > > > Congratulations and welcome, Jingyun!
> > > >
> > >
> >
>

Re: [ANNOUNCE] Please welcome Zach York to the HBase PMC

[Misty Linville]

Thanks for your continuing and increasing commitment to the project and the
community!

On Wed, Oct 17, 2018, 11:14 PM OpenInx  Congratulations Zach!
>
> On Wed, Oct 17, 2018 at 8:14 PM ramkrishna vasudevan <
> ramkrishna.s.vasude...@gmail.com> wrote:
>
> > Congratulations Zach!!!
> >
> > On Wed, Oct 17, 2018 at 11:15 AM Yu Li  wrote:
> >
> > > Congratulations and welcome, Zach!
> > >
> > > Best Regards,
> > > Yu
> > >
> > >
> > > On Wed, 17 Oct 2018 at 13:02, Pankaj kr  wrote:
> > >
> > > > Congratulations Zach..!! :)
> > > >
> > > > Regards,
> > > > Pankaj
> > > >
> > > > -Original Message-
> > > > From: Sean Busbey [mailto:bus...@apache.org]
> > > > Sent: 12 October 2018 01:32
> > > > To: dev 
> > > > Subject: [ANNOUNCE] Please welcome Zach York to the HBase PMC
> > > >
> > > > On behalf of the Apache HBase PMC I am pleased to announce that Zach
> > York
> > > > has accepted our invitation to become a PMC member on the Apache
> HBase
> > > > project. We appreciate Zach stepping up to take more responsibility
> in
> > > the
> > > > HBase project.
> > > >
> > > > Please join me in welcoming Zach to the HBase PMC!
> > > >
> > > > As a reminder, if anyone would like to nominate another person as a
> > > > committer or PMC member, even if you are not currently a committer or
> > PMC
> > > > member, you can always drop a note to priv...@hbase.apache.org to
> let
> > us
> > > > know.
> > > >
> > >
> >
>

Re: The child procedure is scheduled in the reversed order in Procedure V2 Framework

[Misty Linville]

I like your solution in principle, but I think it would be better to loop
until the queue reports that it's empty.

On Wed, Nov 28, 2018, 8:18 PM OpenInx  Thanks Allan's reply
>
> So if we can ensure that all children are independent, then it won't be a
> problem.
> But the reversed shcedule order is some counterintuitive.   I think a minor
> change can
> help fix this:
>
> diff --git
>
> a/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
>
> b/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
> index cbdb9b8..b688ec3 100644
> ---
>
> a/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
> +++
>
> b/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
> @@ -1794,7 +1794,7 @@ public class ProcedureExecutor {
>}
>
>private void submitChildrenProcedures(Procedure[]
> subprocs) {
> -for (int i = 0; i < subprocs.length; ++i) {
> +for (int i = subprocs.length - 1; i >= 0; i--) {
>Procedure subproc = subprocs[i];
>subproc.updateMetricsOnSubmit(getEnvironment());
>assert !procedures.containsKey(subproc.getProcId());
>
> Thanks.
>
> On Wed, Nov 28, 2018 at 11:13 PM Allan Yang  wrote:
>
> > Yes, you are right, every procedure will be add to front, so the final
> > execution order may be reversed, But actually, there will be more than
> one
> > worker threads, so likely, they will be executed at the same time. I
> think
> > the design is unintentionally, since all the child procedure should be
> > independent and won't depend on each other, so they can be executed at
> any
> > order. And more, after HBASE-21375
> >  checked in all 2.x
> > branch, the worker thread will execute every possible procedure in the
> > queue, so the front ones won't block, so this won't be a problem.
> > Best Regards
> > Allan Yang
> >
> >
> > OpenInx  于2018年11月28日周三 下午10:42写道：
> >
> > > Hi :
> > >
> > >  I read parts of the procedure v2 framework, and found that  if a
> > procedure
> > > has  3 added child procedure,
> > >  then it's children will be schedued in the reversed order.
> > >
> > > Let me give an example. if  a procedure A added 3 child procedure: B,
> C ,
> > > D.
> > >
> > > a.addChildProcedure(B, C, D)
> > >
> > > The procedure framework will add the B,C,D child produre in a dequeue
> to
> > > schedule
> > >
> > > ( Code Path  --- ProcedureExecutor#execProcedure
> > > -- submitChildrenProcedures  -- dequeue#addFront )
> > >
> > > So the dequeue will be :(front)   D, C, B  (back)
> > >
> > > Then we will poll each procedure from the dequeue, and dispatch to the
> > > executor to run ..
> > >
> > > In the final,   the procedure executing order will be :  D, C, B,
> which
> > is
> > > the revered order  compared to the addChildProcedure order.
> > >
> > > My question is :  is it designed intentionally ?  Or unintentionally
> > doing
> > > the wrong implementation ?
> > >
> > > Seems most the child procedure are region assign or unassign, looks
> like
> > no
> > > problem now..
> > >
> > > Please correct me if I am wrong or missing something.
> > >
> > > Thanks.
> > >
> >
>

Re: The child procedure is scheduled in the reversed order in Procedure V2 Framework

[Misty Linville]

People could come to rely on the order.

The reason I suggested you should rely on the queue being empty to break
the loop instead of getting the number of elements at the start is to avoid
falling off the end of the queue in the presence of multiple threads.

On Sat, Dec 1, 2018, 1:44 AM 张铎(Duo Zhang)  I think it is fine to keep the current implementation? What is the problem
> if we schedule in reverse order? We do not guarantee any order when
> executing the sub procedures...
>
> Misty Linville  于2018年12月1日周六 下午1:56写道：
>
> > I like your solution in principle, but I think it would be better to loop
> > until the queue reports that it's empty.
> >
> > On Wed, Nov 28, 2018, 8:18 PM OpenInx  >
> > > Thanks Allan's reply
> > >
> > > So if we can ensure that all children are independent, then it won't
> be a
> > > problem.
> > > But the reversed shcedule order is some counterintuitive.   I think a
> > minor
> > > change can
> > > help fix this:
> > >
> > > diff --git
> > >
> > >
> >
> a/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
> > >
> > >
> >
> b/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
> > > index cbdb9b8..b688ec3 100644
> > > ---
> > >
> > >
> >
> a/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
> > > +++
> > >
> > >
> >
> b/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
> > > @@ -1794,7 +1794,7 @@ public class ProcedureExecutor {
> > >}
> > >
> > >private void submitChildrenProcedures(Procedure[]
> > > subprocs) {
> > > -for (int i = 0; i < subprocs.length; ++i) {
> > > +for (int i = subprocs.length - 1; i >= 0; i--) {
> > >Procedure subproc = subprocs[i];
> > >subproc.updateMetricsOnSubmit(getEnvironment());
> > >assert !procedures.containsKey(subproc.getProcId());
> > >
> > > Thanks.
> > >
> > > On Wed, Nov 28, 2018 at 11:13 PM Allan Yang 
> wrote:
> > >
> > > > Yes, you are right, every procedure will be add to front, so the
> final
> > > > execution order may be reversed, But actually, there will be more
> than
> > > one
> > > > worker threads, so likely, they will be executed at the same time. I
> > > think
> > > > the design is unintentionally, since all the child procedure should
> be
> > > > independent and won't depend on each other, so they can be executed
> at
> > > any
> > > > order. And more, after HBASE-21375
> > > > <https://issues.apache.org/jira/browse/HBASE-21375> checked in all
> 2.x
> > > > branch, the worker thread will execute every possible procedure in
> the
> > > > queue, so the front ones won't block, so this won't be a problem.
> > > > Best Regards
> > > > Allan Yang
> > > >
> > > >
> > > > OpenInx  于2018年11月28日周三 下午10:42写道：
> > > >
> > > > > Hi :
> > > > >
> > > > >  I read parts of the procedure v2 framework, and found that  if a
> > > > procedure
> > > > > has  3 added child procedure,
> > > > >  then it's children will be schedued in the reversed order.
> > > > >
> > > > > Let me give an example. if  a procedure A added 3 child procedure:
> B,
> > > C ,
> > > > > D.
> > > > >
> > > > > a.addChildProcedure(B, C, D)
> > > > >
> > > > > The procedure framework will add the B,C,D child produre in a
> dequeue
> > > to
> > > > > schedule
> > > > >
> > > > > ( Code Path  --- ProcedureExecutor#execProcedure
> > > > > -- submitChildrenProcedures  -- dequeue#addFront )
> > > > >
> > > > > So the dequeue will be :(front)   D, C, B  (back)
> > > > >
> > > > > Then we will poll each procedure from the dequeue, and dispatch to
> > the
> > > > > executor to run ..
> > > > >
> > > > > In the final,   the procedure executing order will be :  D, C, B,
> > > which
> > > > is
> > > > > the revered order  compared to the addChildProcedure order.
> > > > >
> > > > > My question is :  is it designed intentionally ?  Or
> unintentionally
> > > > doing
> > > > > the wrong implementation ?
> > > > >
> > > > > Seems most the child procedure are region assign or unassign, looks
> > > like
> > > > no
> > > > > problem now..
> > > > >
> > > > > Please correct me if I am wrong or missing something.
> > > > >
> > > > > Thanks.
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] move to gitbox

[Misty Linville]

+1 What needs to happen next?

On Fri, Dec 7, 2018, 9:03 AM Sean Busbey  Hi folks!
>
> Per the email from infra "[NOTICE] Mandatory relocation of Apache git
> repositories on git-wip-us.apache.org" ( https://s.apache.org/0sfe ) it
> looks like the future of interacting directly with PRs is coming sooner
> rather than later.
>
> I think we should move to gitbox ASAP rather than wait for the crunch. If
> we hit a rough spot we're more likely to get some help when things aren't
> busy. Maybe we wait until our open RCs close so that folks that need to tag
> those releases don't need to update their workflow first?
>
> Presuming everyone still agrees that we get value out of JIRA, I think we
> need update our committer guidelines to expressly remind folks to check on
> things like commit messages before merging PRs, as well as to ensure folks
> use the "squash and merge" option to keep the git history less complicated.
> Probably a good time to add text about the importance of backporting, since
> there isn't a github UI for doing that.
>
>
>
>
>

HBase quarterly report Oct-Dec 2018

[Misty Linville]

All,

HBase submits a report to the ASF board once a quarter, to inform the board
about project health. I'm sending the report to the user@ and dev@ mailing
lists because you are the project, and for transparency. If you have any
questions about the report or the running of the project, you can pose them
to me or any other PMC member or committer, or send an email to
priv...@hbase.apache.org, which every PMC member subscribes to.

Thanks,
Misty

--
## Description:

Apache HBase is an open-source, distributed, versioned, non-relational
database. Apache HBase gives you low latency random access to billions of
rows with millions of columns atop non-specialized hardware.

hbase-thirdparty is a set of internal artifacts used by the project to
mitigate the impact of our dependency choices on the wider ecosystem.

## Issues:

Board-only information removed from public report.

## Activity:

There have been lots of interesting discussions on the dev@ mailing list.
Highlights:
- Discussion is ongoing about plans to move to Gitbox and ways the project
can accommodate users who prefer to contribute using Github. (
https://lists.apache.org/thread.html/3496568d6cc002f74f5c3bcce46ed44b7ee9e90d7d53af2c65b6f785@%3Cdev.hbase.apache.org%3E
)
- An interesting discussion occurred about the frequency of releases in the
2.1.x line (
https://lists.apache.org/thread.html/222aacbf74d9788287ef5606fe66a06c1a3aa31a299a469d124a5ee0@%3Cdev.hbase.apache.org%3E).
The discussion included a sub-discussion about what it will take to
separate the hbck took from the main project, and another sub-discussion
about time-based releases and release cadence in general. It's worth a read.
- A proposal to retire the 1.3 line resulted in Francis Liu stepping in to
be the release manage for the line, because his team relies on that branch.
Francis is aiming for a quarterly release cadence. In the same discussion,
Andrew Purtell and Sean Busbey discuss how they have been managing 1.2 and
1.4 lines, and plans for the 1.5 line. (
https://lists.apache.org/thread.html/dd655a21cb5eb23a983648d71f5d08ca567b110ab9128a70f06eddcc@%3Cdev.hbase.apache.org%3E
)
- Another discussion of 1.x release cadences was started by Andrew Purtell,
including specific plans for the 1.5 branch. (
https://lists.apache.org/thread.html/e18266c8aa1e1406f3e652a3a1ee52d498aaad506537621cc754499b@%3Cdev.hbase.apache.org%3E
)
- Another interesting discussion resulted in improved understanding and
documentation of the process for editing and rebuilding pages on the HBase
website that don't result from building the documentation. (
https://lists.apache.org/thread.html/18010d0897752db7f9d91f4f75694831a7ed9db28739ce4a91e546e4@%3Cdev.hbase.apache.org%3E
)
- Speaking of 1.5, the first snapshot is available for testing. (
https://lists.apache.org/thread.html/bbf7b47920da532535bb8ce493df3265fd91211b227f681b2cc3c3a2@%3Cdev.hbase.apache.org%3E
)

Josh Elser started a discussion on the PMC list about the definition of and
parameters for an HBaseCon event, who can run one, and other guidelines,
informed by his work organizing HBaseCon US 2018. He's working on some
project-facing documentation, which promises to be helpful for future
organizers of HBase-related events worldwide.

The HBase project is working toward the goal of more frequent minor
releases and fewer unprompted maintenance releases. In total this quarter,
HBase had 8 releases over this quarter, across 5 release lines.

Another goal, spanning several quarters, is to solicit more non-binding
votes on release candidates, as non-binding votes are one signal of broader
community engagement. Here are a few examples:
- HBase 1.3.3 RC0 had 40% non-binding votes (
https://lists.apache.org/thread.html/9e3a472e142faa9a48ed5ab0e1e1baaaeb9ddfe7a9fdcfa547c2cd20@%3Cdev.hbase.apache.org%3E
)
- HBase 1.4.8 RC0 had 50% non-binding votes (
https://lists.apache.org/thread.html/2f8ad79f6da4d2ac223a61b82aeb29a2322479257f271da90f0687e0@%3Cdev.hbase.apache.org%3E
)
- HBase 2.1.1 RC0 also had 50% non-binding votes (
https://lists.apache.org/thread.html/f9ccc377dbeac4b3a7a0cddcef0cd7369675f5b1cc570b6c19a52864@%3Cdev.hbase.apache.org%3E
)

As a reminder, anyone on the dev@ list can test and provide feedback on a
release candidate, and cast a non-binding vote.

Lars Hofhansl is in the process of organizing a bay-area HBase developer
meet-up at Salesforce. If you're interested, participate in the thread. (
https://lists.apache.org/thread.html/e814054e8d8ad1c5ebb30b9c50d97b5b8b5edada121f184fdf045e3d@%3Cdev.hbase.apache.org%3E
)

Two new committers were added this quarter, and two committers joined the
PMC. More details below. Thanks to the new committers and PMC members for
agreeing to take on more responsibilities in the project.

## Health report:

Despite a slow holiday quarter, the HBase project shows a high level of
engagement, as seen through release cadence, deep and meaningful discussion
on mailing lists, discussion and documentation of processes, and
non-bin

Re: Seek for help to create an account on review.apache

[Misty Linville]

I think you may need to contact infra directly. I'm sorry you're facing
this trouble and I appreciate the extra effort you're going through to
contribute. I wonder if you can try creating a pull request on GitHub
instead of using Reviewboard until you get a resolution. I confess I am not
as familiar with the Gitbox functionality as I should be. Can anyone else
help out here? I've added the PMC list just in case it helps speed things
along.

On Tue, Jan 15, 2019, 6:59 PM wang huan  Hi, Zhang: https://reviews.apache.org have enabled LDAP,  can not log in
> with account on jira.
>
> On 16/1/19, 10:17 AM, "张铎(Duo Zhang)"  wrote:
>
> I think it uses the same account with jira? i.e, issues.apache.org.
>
> wang huan  于2019年1月16日周三 上午10:13写道：
>
> > Hi, All:
> > I want to login https://reviews.apache.org to upload my patch
> > onhttps://jira.apache.org/jira/browse/HBASE-21699,  all apache
> projects
> > told me go to https://reviews.apache.org and create an account, but
> no
> > register page on there !
> >  My colleague told me he got his account from dev@mesos, so I
> want to
> > know if someone can
> > help me create an account
> >
> > Best Regards,
> > huan
> >
> >
>
>

Re: The first HBase 1.5.0 release candidate (RC0) is available

[Misty Linville]

Andrew, maybe you can turn your ITBLL notes into docs in the reference
guide. There’s a whole section in there about testing.

On Sat, Feb 2, 2019 at 4:03 PM Andrew Purtell 
wrote:

> Thanks. As I am not able to produce those unit test results we will need
> your help to diagnose the issues. Please file
>
> The ITBLL results may be a tool usage problem. The numbers in the failure
> messages you posted are too round. I expect real failures to produce more
> irregular numbers. ITBLL can a bit hard to use. Contact me offline and I’ll
> give you notes on how I ran the tests myself.
>
>
> > On Feb 2, 2019, at 3:45 PM, Xu Cang 
> wrote:
> >
> > 2 jars sha12 verification: pass.
> > Basic UI check: pass.
> > Unit test. Some failures in  hbase - server package. (see details below,
> > not sure if these are flaky tests)
> > ITBLL tests with slowDeterministic and serverKilling monky. Both got some
> > failures. (Not sure if this is my environment issue since I am using *my
> > laptop* to conduct this testing)
> > Not voting for now since I have some doubts regarding my testing result.
> > Will keep looking.
> >
> >
> >   - *Unit test failure: (failures are reproducable)*
> >
> > [INFO] Results:
> > [INFO]
> > [ERROR] Failures:
> > [ERROR]
> >
> TestRegionLocationCaching.testCachingForHTableMultiPut:133->checkRegionLocationIsCached:148
> > Expected non-zero number of cached region locations. Actual: 0
> > [ERROR]
> >
> TestRegionLocationCaching.testCachingForHTableMultiplexerMultiPut:95->checkRegionLocationIsCached:148
> > Expected non-zero number of cached region locations. Actual: 0
> > [ERROR]
> >
> TestRegionLocationCaching.testCachingForHTableMultiplexerSinglePut:73->checkRegionLocationIsCached:148
> > Expected non-zero number of cached region locations. Actual: 0
> > [ERROR]
> >
> TestRegionLocationCaching.testCachingForHTableSinglePut:116->checkRegionLocationIsCached:148
> > Expected non-zero number of cached region locations. Actual: 0
> > [ERROR]   TestReplicasClient.testHedgedRead:595 expected:<0> but was:<1>
> > [ERROR]
> >
> TestFilterListOrOperatorWithBlkCnt.testMultiRowRangeWithFilterListOrOperatorWithBlkCnt:127
> > expected:<4> but was:<5>
> > [ERROR]   TestRegionServerMetrics.testRequestCount:137 Metrics Counters
> > should be equal expected:<59> but was:<89>
> > [INFO]
> > [ERROR] Tests run: 1870, Failures: 7, Errors: 0, Skipped: 17
> >
> >
> >   -
> > *ITBLL testing result: (failures are reproducable) *
> >
> > ./bin/hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList
> Loop
> > 1 1 2500 /tmp/itbll 1 -m slowDeterministic
> >
> > 2019-02-01 23:31:03,746 INFO  [main] mapreduce.Job:  map 100% reduce 100%
> > 2019-02-01 23:31:03,746 INFO  [main] mapreduce.Job: Job
> > job_local221831554_0003 completed successfully
> > 2019-02-01 23:31:03,758 INFO  [main] mapreduce.Job: 17518
> >Input split bytes=679
> >Combine input records=0
> >Combine output records=0
> >Reduce input groups=7500
> >Reduce shuffle bytes=517518
> >Reduce input records=15000
> >Reduce output records=0
> >Spilled Records=561948580
> >Shuffled Maps =3
> >Failed Shuffles=0
> >Merged Map outputs=3
> >GC time elapsed (ms)=1859
> >Total committed heap usage (bytes)=1846542336
> >HBase Counters
> >BYTES_IN_REMOTE_RESULTS=0
> >BYTES_IN_RESULTS=31125001574
> >MILLIS_BETWEEN_NEXTS=934178
> >NOT_SERVING_REGION_EXCEPTION=7
> >NUM_SCANNER_RESTARTS=0
> >NUM_SCAN_RESULTS_STALE=0
> >REGIONS_SCANNED=12
> >REMOTE_RPC_CALLS=0
> >REMOTE_RPC_RETRIES=0
> >ROWS_FILTERED=12
> >ROWS_SCANNED=7512
> >RPC_CALLS=14607
> >RPC_RETRIES=7
> >Shuffle Errors
> >BAD_ID=0
> >CONNECTION=0
> >IO_ERROR=0
> >WRONG_LENGTH=0
> >WRONG_MAP=0
> >WRONG_REDUCE=0
> >
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Verify$Counts
> >REFERENCED=7500
> >File Input Format Counters
> >Bytes Read=0
> >File Output Format Counters
> >Bytes Written=108
> > 2019-02-01 23:31:03,764 ERROR [main]
> > test.IntegrationTestBigLinkedList$Verify: *Expected referenced count does
> > not match with actual referenced count. expected referenced=2500
> > ,actual=7500*
> >
> >
> > ./bin/hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList
> Loop
> > 1 4 100 /tmp/itbll 4 -m slowDeterministic
> > 2019-02-02 00:34:27,009 ERROR [main]
> > test.IntegrationTestBigLinkedList$Verify: Expected referenced count does
> > not match with actual referenced count. expected referenced=400
> > ,actual=7900
> >
> >
> >> On Fri, Feb 1, 2019 at 2:17 PM Andrew Purtell 
> wrote:
> >>
> >> The first HBase 1.5.0 release candidate (RC0) is available for download
> at
> >> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.5.0RC0/ and Maven
> >> artifacts are ava

Re: HBase Website Cleanup task

[Misty Linville]

+1

On Mon, Feb 4, 2019 at 10:04 AM Stack  wrote:

> Sounds good by me.
> S
>
> On Mon, Feb 4, 2019 at 7:40 AM Peter Somogyi  wrote:
>
> > Does anyone have any concern with removing 0.94 documentation completely
> > from hbase.apache.org? Currently it is hosted at
> > https://hbase.apache.org/0.94/ to which we don't have any direct link
> but
> > with a few hops a user can reach this area. This release is almost 4
> years
> > old!
> >
> > I had a chat with Sean Busbey and he mentioned 0.94 could be removed from
> > the site and we could send a NOTICE to user@ that the documentation can
> be
> > found in the final 0.94 tarball in case anyone still need it. The
> > tarball[1] has the same content in docs directory.
> >
> > What do you think?
> >
> > Peter
> >
> > [1] http://archive.apache.org/dist/hbase/hbase-0.94.27/
> >
> > On Tue, Jan 29, 2019 at 10:18 PM Sakthi 
> > wrote:
> >
> > > FYI: This is the job I'm talking about:
> > > https://builds.apache.org/job/HBase%20Website%20Link%20Checker/
> > >
> > > Regards,
> > > Sakthi
> > >
> > > On Tue, Jan 29, 2019 at 11:17 AM Sakthi 
> > > wrote:
> > >
> > > > Hello all,
> > > >
> > > > The most recent (and many of the previous) "*HBase Website Link
> > Checker*"
> > > > job of ours has reported several issues with the website. The job
> takes
> > > > roughly 20+ hours to finish and has reported "*1458 missing files*"
> and
> > > "*7143
> > > > missing named anchors*".
> > > >
> > > > *Quick facts:* Of the 1400+ missing files issues, around 1200 are
> from
> > > > 1.2, 90 from 0.94, 40 from 2.0, 30 from 2.1 and rest from the master.
> > > >
> > > > I have filed an *Umbrella JIRA:* HBASE-21803
> > > >  ("HBase Website
> > > > Cleanup"). If folks find any other website related issues that we
> would
> > > > like to address, we can use this umbrella to track it.
> > > >
> > > > Any suggestions are welcome!
> > > >
> > > > Regards,
> > > > Sakthi
> > > >
> > >
> >
>

Re: HBase Website Cleanup task

[Misty Linville]

We should add rel=canonical info to all the headers and always point to the
latest as canonical.

On Mon, Feb 4, 2019 at 2:56 PM William Shen 
wrote:

> If we are concerned about SEO, we should try to set up 301 Redirect to the
> latest doc when we remove 0.94. That should alleviate some of the problems
> with users landing on dead links. Over time the result should adjust, or we
> could set up Google Search Console for HBase site, and trigger a reindex.
> Not sure if we have someone already owning the SEO aspect of the site.
>
> On Mon, Feb 4, 2019 at 2:52 PM Nick Dimiduk  wrote:
>
> > My only concern is that all my searches targeting our user guide land in
> > 0.94 docs. I don't know why the SEO lands us thusly, but I fear that
> > suddenly it'll look to user searches like the project is dead.
> >
> > On Mon, Feb 4, 2019 at 11:46 AM Misty Linville  wrote:
> >
> > > +1
> > >
> > > On Mon, Feb 4, 2019 at 10:04 AM Stack  wrote:
> > >
> > > > Sounds good by me.
> > > > S
> > > >
> > > > On Mon, Feb 4, 2019 at 7:40 AM Peter Somogyi 
> > > wrote:
> > > >
> > > > > Does anyone have any concern with removing 0.94 documentation
> > > completely
> > > > > from hbase.apache.org? Currently it is hosted at
> > > > > https://hbase.apache.org/0.94/ to which we don't have any direct
> > link
> > > > but
> > > > > with a few hops a user can reach this area. This release is almost
> 4
> > > > years
> > > > > old!
> > > > >
> > > > > I had a chat with Sean Busbey and he mentioned 0.94 could be
> removed
> > > from
> > > > > the site and we could send a NOTICE to user@ that the
> documentation
> > > can
> > > > be
> > > > > found in the final 0.94 tarball in case anyone still need it. The
> > > > > tarball[1] has the same content in docs directory.
> > > > >
> > > > > What do you think?
> > > > >
> > > > > Peter
> > > > >
> > > > > [1] http://archive.apache.org/dist/hbase/hbase-0.94.27/
> > > > >
> > > > > On Tue, Jan 29, 2019 at 10:18 PM Sakthi <
> sakthivel.azh...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > FYI: This is the job I'm talking about:
> > > > > > https://builds.apache.org/job/HBase%20Website%20Link%20Checker/
> > > > > >
> > > > > > Regards,
> > > > > > Sakthi
> > > > > >
> > > > > > On Tue, Jan 29, 2019 at 11:17 AM Sakthi <
> > sakthivel.azh...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hello all,
> > > > > > >
> > > > > > > The most recent (and many of the previous) "*HBase Website Link
> > > > > Checker*"
> > > > > > > job of ours has reported several issues with the website. The
> job
> > > > takes
> > > > > > > roughly 20+ hours to finish and has reported "*1458 missing
> > files*"
> > > > and
> > > > > > "*7143
> > > > > > > missing named anchors*".
> > > > > > >
> > > > > > > *Quick facts:* Of the 1400+ missing files issues, around 1200
> are
> > > > from
> > > > > > > 1.2, 90 from 0.94, 40 from 2.0, 30 from 2.1 and rest from the
> > > master.
> > > > > > >
> > > > > > > I have filed an *Umbrella JIRA:* HBASE-21803
> > > > > > > <https://issues.apache.org/jira/browse/HBASE-21803> ("HBase
> > > Website
> > > > > > > Cleanup"). If folks find any other website related issues that
> we
> > > > would
> > > > > > > like to address, we can use this umbrella to track it.
> > > > > > >
> > > > > > > Any suggestions are welcome!
> > > > > > >
> > > > > > > Regards,
> > > > > > > Sakthi
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: HBase Website Cleanup task

[Misty Linville]

For 0.94 files we should add a permanent redirect to htaccess manually. For
the rest we should modify the javadoc template to add the correct metadata
for older branches. There is probably a way to pass arguments to the
javadoc command and maybe that could be toggled on or off depending on
whether we were building the latest or not. You’ll need to look up the
actual syntax you need and how to achieve it in the template. We should
avoid modifying generated content manually so we don’t blow our hashes in
released bundles.

Before you do any of this, please outline at least a rough plan so we can
help spot problems. :)

Misty

On Tue, Feb 5, 2019 at 4:44 AM Peter Somogyi  wrote:

> That's a very good point Nick!
>
> I'd know how we could add rel=canonical tag to 0.94 pages if we remove
> those. Also, how could we add this tag to generated javadoc files (e.g.
> canonical link between 1.4 and 3.0 javadoc)? I did not find any solutions
> online to automate this process.
> What we can do is to add a redirect rule to .htaccess with something like
> this:
>
> RedirectMatch permanent ^/0.94/(.*)$ /$1
>
> I think it could work for most of the links. An example:
>
> https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/HTable.html
>
> https://hbase.apache.org/0.94/devapidocs/org/apache/hadoop/hbase/client/HTable.html
>
> On Tue, Feb 5, 2019 at 3:58 AM Misty Linville  wrote:
>
> > We should add rel=canonical info to all the headers and always point to
> the
> > latest as canonical.
> >
> > On Mon, Feb 4, 2019 at 2:56 PM William Shen 
> > wrote:
> >
> > > If we are concerned about SEO, we should try to set up 301 Redirect to
> > the
> > > latest doc when we remove 0.94. That should alleviate some of the
> > problems
> > > with users landing on dead links. Over time the result should adjust,
> or
> > we
> > > could set up Google Search Console for HBase site, and trigger a
> reindex.
> > > Not sure if we have someone already owning the SEO aspect of the site.
> > >
> > > On Mon, Feb 4, 2019 at 2:52 PM Nick Dimiduk 
> wrote:
> > >
> > > > My only concern is that all my searches targeting our user guide land
> > in
> > > > 0.94 docs. I don't know why the SEO lands us thusly, but I fear that
> > > > suddenly it'll look to user searches like the project is dead.
> > > >
> > > > On Mon, Feb 4, 2019 at 11:46 AM Misty Linville 
> > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > On Mon, Feb 4, 2019 at 10:04 AM Stack  wrote:
> > > > >
> > > > > > Sounds good by me.
> > > > > > S
> > > > > >
> > > > > > On Mon, Feb 4, 2019 at 7:40 AM Peter Somogyi <
> psomo...@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > Does anyone have any concern with removing 0.94 documentation
> > > > > completely
> > > > > > > from hbase.apache.org? Currently it is hosted at
> > > > > > > https://hbase.apache.org/0.94/ to which we don't have any
> direct
> > > > link
> > > > > > but
> > > > > > > with a few hops a user can reach this area. This release is
> > almost
> > > 4
> > > > > > years
> > > > > > > old!
> > > > > > >
> > > > > > > I had a chat with Sean Busbey and he mentioned 0.94 could be
> > > removed
> > > > > from
> > > > > > > the site and we could send a NOTICE to user@ that the
> > > documentation
> > > > > can
> > > > > > be
> > > > > > > found in the final 0.94 tarball in case anyone still need it.
> The
> > > > > > > tarball[1] has the same content in docs directory.
> > > > > > >
> > > > > > > What do you think?
> > > > > > >
> > > > > > > Peter
> > > > > > >
> > > > > > > [1] http://archive.apache.org/dist/hbase/hbase-0.94.27/
> > > > > > >
> > > > > > > On Tue, Jan 29, 2019 at 10:18 PM Sakthi <
> > > sakthivel.azh...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > FYI: This is the job I'm talking about:
> > > > > > > >
> > https://builds.apache.org/job/HBase%20Website%20Link%20Checker/
> >

Re: [DISCUSS] what's our biggest source of friction for getting new contributors going?

[Misty Linville]

#1 this is difficult code. That’s probably the one we can’t surmount but I
wanted to put it out there.

On Tue, Mar 19, 2019 at 5:03 PM Sean Busbey  wrote:

> I have my own opinions, obvs. But I'm curious what other folks think are
> the biggest impediments to new contributors?
>

Re: [DISCUSS] what's our biggest source of friction for getting new contributors going?

[Misty Linville]

Another open source project I’m on is exploring the idea of a mentoring
rotation where each mentor serves for a week and especially looks out for
and provides practical assistance and review to new contributors for that
week, on an office-hours type of basis. Many hands make light work. WDYT?
I’d say that this type of mentoring, if done well, would be another way to
put one on the road to recognition for leadership within the project.

On Wed, Mar 20, 2019 at 9:45 AM Tak-Lon (Stephen) Wu 
wrote:

> +1 one for the mentoring if one existing member who knows the feature plan
> and is willing to create tasks for pickup.
>
> my current barrier is to find a continuous topic/component and focus on it,
> but I'm a bit too random with my approach.
>
> Currently, not sure this is the right way, I lookup JIRAs by filtering tags
> (e.g. BUG) with priority and created dates to find
> newly reported issues and somehow didn't know which JIRA should be resolved
> next release. meanwhile, I may miss
> older JIRA which potentially is critical (probably the longer it has no one
> look at it, should it considered as less critical?).
>
> Thanks,
> Stephen
>
>
>
>
> On Wed, Mar 20, 2019 at 12:12 AM Artem Ervits 
> wrote:
>
> > Would be great to have mentors. Ted used to review my patches and provide
> > feedback on timely basis, understand everyone is busy but would be nice
> to
> > have a point person (per feature, bug, branch, module, etc). I have
> cycles
> > to burn but currently it feels a bit overwhelming as community may feel
> > there's a certain level of familiarity with the process necessary.
> >
> > On Wed, Mar 20, 2019, 4:59 AM Misty Linville  wrote:
> >
> > > #1 this is difficult code. That’s probably the one we can’t surmount
> but
> > I
> > > wanted to put it out there.
> > >
> > > On Tue, Mar 19, 2019 at 5:03 PM Sean Busbey  wrote:
> > >
> > > > I have my own opinions, obvs. But I'm curious what other folks think
> > are
> > > > the biggest impediments to new contributors?
> > > >
> > >
> >
>

Re: The GitHub PR integration basically works

[Misty Linville]

Can we protect the GitHub branches from direct merges? That’s a repo-level
setting and we may not be able to change it. It seems potentially dangerous
for people to be able to merge their own changes especially if it only
takes one successful reviewer. Other communities use mechanisms like Prow
[1] for this kind of thing. I imagine it requires some infrastructure
though.

[1] https://github.com/kubernetes/test-infra/tree/master/prow

On Fri, Apr 5, 2019 at 7:04 PM 张铎(Duo Zhang)  wrote:

> Yes, at least there should be a relevant JIRA issue.
>
> And on the retesting, we need to find a way to re-trigger the webhook. But
> anyway, we can fall back to use the old pre commit way, just checkout the
> branch and make a patch and upload it to the jira issue...
>
> I'm trying to make use of GitHub in the recent works. And yesterday, I
> added Zheng Hu as a reviewer for the addendum of HBASE-22152, and he posted
> a LGTM and then just merged the PR... In fact I just want him to approve
> the PR, this is the correct way to '+1' on GitHub. So I think we need to
> write something done in the tell committers how to make use of the GitHub
> PR...
>
>
>
> Sean Busbey  于2019年4月6日周六 上午9:43写道：
>
> > Excellent to see Duo!
> >
> > Do we have any guidelines for committers in the ref guide? I think we had
> > previously discussed calling out that they should make sure there's a
> JIRA
> > for anything merged?
> >
> > Does retesting work from the github UI or is it like before where one
> > resubmits the jenkins job?
> >
> > On 2019/04/04 06:15:39, 张铎(Duo Zhang)  wrote:
> > > Please see here
> > >
> > > https://github.com/apache/hbase/pull/110
> > >
> > > Still need to polish the jenkinsfile so we can keep the same experience
> > > with the old hadoop QA, but anyway, it basically works.
> > >
> > > So I think it is time to set up our github based workflow. Need to
> > discuss
> > > how to work together with our jira.
> > >
> > > Thanks.
> > >
> >
>

Re: The GitHub PR integration basically works

[Misty Linville]

Yes, but you were doing the merge, unless they were a committer. I
understood (perhaps incorrectly) Duo was describing a situation where a
chang was merged by someone who shouldn’t have been able to do so
otherwise.

On Fri, Apr 5, 2019 at 9:52 PM Sean Busbey  wrote:

>
> This seems like a difference in ease compared to jira and not
> something wildly different. There have certainly been times where a
> committer posted a patch to jira for review and I merged it at a part
> of giving my +1.
>
> We should make sure things default to squash-and-rebase instead of
> merge for PRs in the UI, but I think we did that already.
>
> On Fri, Apr 5, 2019 at 10:54 PM 张铎(Duo Zhang) 
> wrote:
> >
> > IIRC we have filed an infra ticket to disable several operations related
> to
> > PR, and for merging, I think we should only allow committers to merge
> PRs.
> >
> > Misty Linville  于2019年4月6日周六 上午10:11写道：
> >
> > > Can we protect the GitHub branches from direct merges? That’s a
> repo-level
> > > setting and we may not be able to change it. It seems potentially
> dangerous
> > > for people to be able to merge their own changes especially if it only
> > > takes one successful reviewer. Other communities use mechanisms like
> Prow
> > > [1] for this kind of thing. I imagine it requires some infrastructure
> > > though.
> > >
> > > [1] https://github.com/kubernetes/test-infra/tree/master/prow
> > >
> > > On Fri, Apr 5, 2019 at 7:04 PM 张铎(Duo Zhang) 
> > > wrote:
> > >
> > > > Yes, at least there should be a relevant JIRA issue.
> > > >
> > > > And on the retesting, we need to find a way to re-trigger the
> webhook.
> > > But
> > > > anyway, we can fall back to use the old pre commit way, just
> checkout the
> > > > branch and make a patch and upload it to the jira issue...
> > > >
> > > > I'm trying to make use of GitHub in the recent works. And yesterday,
> I
> > > > added Zheng Hu as a reviewer for the addendum of HBASE-22152, and he
> > > posted
> > > > a LGTM and then just merged the PR... In fact I just want him to
> approve
> > > > the PR, this is the correct way to '+1' on GitHub. So I think we
> need to
> > > > write something done in the tell committers how to make use of the
> GitHub
> > > > PR...
> > > >
> > > >
> > > >
> > > > Sean Busbey  于2019年4月6日周六 上午9:43写道：
> > > >
> > > > > Excellent to see Duo!
> > > > >
> > > > > Do we have any guidelines for committers in the ref guide? I think
> we
> > > had
> > > > > previously discussed calling out that they should make sure
> there's a
> > > > JIRA
> > > > > for anything merged?
> > > > >
> > > > > Does retesting work from the github UI or is it like before where
> one
> > > > > resubmits the jenkins job?
> > > > >
> > > > > On 2019/04/04 06:15:39, 张铎(Duo Zhang) 
> wrote:
> > > > > > Please see here
> > > > > >
> > > > > > https://github.com/apache/hbase/pull/110
> > > > > >
> > > > > > Still need to polish the jenkinsfile so we can keep the same
> > > experience
> > > > > > with the old hadoop QA, but anyway, it basically works.
> > > > > >
> > > > > > So I think it is time to set up our github based workflow. Need
> to
> > > > > discuss
> > > > > > how to work together with our jira.
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > >
> > > >
> > >
>

Re: The GitHub PR integration basically works

[Misty Linville]

Ah, I see. Thanks for explaining. That makes more sense!

On Fri, Apr 5, 2019 at 10:29 PM 张铎(Duo Zhang)  wrote:

> Oh, Zheng Hu is a committer so he has the permission to merge... What I
> said is that, he should approve first before merging...
>
> Misty Linville  于2019年4月6日周六 下午1:19写道：
>
> > Yes, but you were doing the merge, unless they were a committer. I
> > understood (perhaps incorrectly) Duo was describing a situation where a
> > chang was merged by someone who shouldn’t have been able to do so
> > otherwise.
> >
> > On Fri, Apr 5, 2019 at 9:52 PM Sean Busbey  wrote:
> >
> > >
> > > This seems like a difference in ease compared to jira and not
> > > something wildly different. There have certainly been times where a
> > > committer posted a patch to jira for review and I merged it at a part
> > > of giving my +1.
> > >
> > > We should make sure things default to squash-and-rebase instead of
> > > merge for PRs in the UI, but I think we did that already.
> > >
> > > On Fri, Apr 5, 2019 at 10:54 PM 张铎(Duo Zhang) 
> > > wrote:
> > > >
> > > > IIRC we have filed an infra ticket to disable several operations
> > related
> > > to
> > > > PR, and for merging, I think we should only allow committers to merge
> > > PRs.
> > > >
> > > > Misty Linville  于2019年4月6日周六 上午10:11写道：
> > > >
> > > > > Can we protect the GitHub branches from direct merges? That’s a
> > > repo-level
> > > > > setting and we may not be able to change it. It seems potentially
> > > dangerous
> > > > > for people to be able to merge their own changes especially if it
> > only
> > > > > takes one successful reviewer. Other communities use mechanisms
> like
> > > Prow
> > > > > [1] for this kind of thing. I imagine it requires some
> infrastructure
> > > > > though.
> > > > >
> > > > > [1] https://github.com/kubernetes/test-infra/tree/master/prow
> > > > >
> > > > > On Fri, Apr 5, 2019 at 7:04 PM 张铎(Duo Zhang) <
> palomino...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Yes, at least there should be a relevant JIRA issue.
> > > > > >
> > > > > > And on the retesting, we need to find a way to re-trigger the
> > > webhook.
> > > > > But
> > > > > > anyway, we can fall back to use the old pre commit way, just
> > > checkout the
> > > > > > branch and make a patch and upload it to the jira issue...
> > > > > >
> > > > > > I'm trying to make use of GitHub in the recent works. And
> > yesterday,
> > > I
> > > > > > added Zheng Hu as a reviewer for the addendum of HBASE-22152, and
> > he
> > > > > posted
> > > > > > a LGTM and then just merged the PR... In fact I just want him to
> > > approve
> > > > > > the PR, this is the correct way to '+1' on GitHub. So I think we
> > > need to
> > > > > > write something done in the tell committers how to make use of
> the
> > > GitHub
> > > > > > PR...
> > > > > >
> > > > > >
> > > > > >
> > > > > > Sean Busbey  于2019年4月6日周六 上午9:43写道：
> > > > > >
> > > > > > > Excellent to see Duo!
> > > > > > >
> > > > > > > Do we have any guidelines for committers in the ref guide? I
> > think
> > > we
> > > > > had
> > > > > > > previously discussed calling out that they should make sure
> > > there's a
> > > > > > JIRA
> > > > > > > for anything merged?
> > > > > > >
> > > > > > > Does retesting work from the github UI or is it like before
> where
> > > one
> > > > > > > resubmits the jenkins job?
> > > > > > >
> > > > > > > On 2019/04/04 06:15:39, 张铎(Duo Zhang) 
> > > wrote:
> > > > > > > > Please see here
> > > > > > > >
> > > > > > > > https://github.com/apache/hbase/pull/110
> > > > > > > >
> > > > > > > > Still need to polish the jenkinsfile so we can keep the same
> > > > > experience
> > > > > > > > with the old hadoop QA, but anyway, it basically works.
> > > > > > > >
> > > > > > > > So I think it is time to set up our github based workflow.
> Need
> > > to
> > > > > > > discuss
> > > > > > > > how to work together with our jira.
> > > > > > > >
> > > > > > > > Thanks.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
>

Re: [NOTICE] Renumbering branch-1 from 1.5.0 to 1.6.0-SNAPSHOT

[Misty Linville]

+1 on renumbering.

On Wed, Apr 10, 2019 at 6:23 PM Andrew Purtell  wrote:

> I am not anticipating a 1.5.1 release unless we need a patch release for
> some reason. After 1.5.0 will be 1.6.0.
>
> If we do need to make a 1.5.1 patch release, we can make a new branch from
> rel/1.5.0 named "branch-1.5", but not before this is necessary. Hopefully
> this will not be necessary.
>
> Therefore I plan to renumber branch-1 from 1.5.0 to 1.6.0-SNAPSHOT and also
> plan to go through JIRA and retarget any fixVersion of 1.5.1 to 1.6.0.
>
> Any concerns, please let me know.
>
> If no concerns, I will do this tomorrow.
>
> Also, please be advised voting is still open on 1.5.0RC3, please go to the
> vote thread on dev@ if you can spare some time to vote on it, thank you.
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>

[REPORT] HBase quarterly report Jan-Mar 2019

[Misty Linville]

All,

HBase submits a report to the ASF board once a quarter, to inform the board
about project health. I'm sending the report to the user@ and dev@ mailing
lists because you are the project, and for transparency. If you have any
questions about the report or the running of the project, you can pose them
to me or any other PMC member or committer, or send an email to
priv...@hbase.apache.org, which every PMC member subscribes to.

Thanks,
Misty

--
## Description:

Apache HBase is an open-source, distributed, versioned, non-relational
database. Apache HBase gives you low latency random access to billions of
rows with millions of columns atop non-specialized hardware.

hbase-thirdparty is a set of internal artifacts used by the project to
mitigate the impact of our dependency choices on the wider ecosystem.

## Issues:

Board-only information removed from public report.

## Activity:

There have been lots of interesting discussions on the dev@ mailing list.
Highlights:
- We have nearly all the infrastructure ready to complete the move to
Gitbox. (
https://lists.apache.org/thread.html/3496568d6cc002f74f5c3bcce46ed44b7ee9e90d7d53af2c65b6f785@%3Cdev.hbase.apache.org%3E
)
- An interesting discussion occurred about the frequency of releases in the
2.1.x line (
https://lists.apache.org/thread.html/222aacbf74d9788287ef5606fe66a06c1a3aa31a299a469d124a5ee0@%3Cdev.hbase.apache.org%3E).
The discussion included a sub-discussion about what it will take to
separate the hbck took from the main project, and another sub-discussion
about time-based releases and release cadence in general. It's worth a read.
- A conversation is underway about getting the branch-2 line ready for the
"stable" pointer. (
https://lists.apache.org/thread.html/81b2917eba3fce2872b96526635a5ba11535c73641cb3d38271ee353@%3Cdev.hbase.apache.org%3E
)
- We asked ourselves what our greatest friction point is when attracting
new contributors. We'd love for you to weigh in. (
https://lists.apache.org/thread.html/887a253ccce44b51e4cfc57276545fcea123fa780a73ffc9fa279cd2@%3Cdev.hbase.apache.org%3E
)
- Sean Busbey, the release manager for 1.2, proposes that 1.2.12 (currently
a release candidate) be the last release in the 1.2 line. (
https://lists.apache.org/thread.html/3cf19458b739a612fbe1f72d17c14124ff536b0cb8678a63d80cf019@%3Cdev.hbase.apache.org%3E
)

Planning is underway for the next HBaseCon Asia! Contact Duo Zhang <
zhang...@apache.org> to get involved.

The HBase project is working toward the goal of more frequent minor
releases and fewer unprompted maintenance releases. In total this quarter,
HBase had 7 releases over this quarter, across 3 release lines, and
hbase-thirdparty had 1 release.

Another goal, spanning several quarters, is to solicit more non-binding
votes on release candidates, as non-binding votes are one signal of broader
community engagement.
- HBase 2.1.3 RC1 had 40% non-binding votes (
https://lists.apache.org/thread.html/baef78476204a476b45f7e273b006f19d057c6d9123d8a34178539a9@%3Cdev.hbase.apache.org%3E
)
- We voted on and failed several release candidates this quarter. We had
less participation from non-binding voters this quarter.
As a reminder, anyone on the dev@ list can test and provide feedback on a
release candidate, and cast a non-binding vote.

One new committer was added this quarter, and one committer joined the PMC.
More details below. Thanks to the new committers and PMC members for
agreeing to take on more responsibilities in the project.

## Health report:

We've bounced back from the holiday dip in activity. Specifically, we have
fixed a lot of JIRA issues this quarter!

## PMC changes:

 - Currently 45 PMC members.
 - Peter Somogyi was added to the PMC on Mon Jan 21 2019.

## Committer base changes:

 - Currently 79 committers.
 - Xu Cang was added as a committer on Fri Feb 01 2019

## Releases:

- 2.0.4 was released on Thu Jan 3 2019
- 2.1.2 was released on Tue Jan 8 2019
- 1.2.10 was released on Wed Jan 16 2019
- 2.1.3 was released on Wed Feb 13 2019
- 1.2.11 was released on Sun Feb 24 2019
- 2.0.5 was released on Sun Mar 24 2019
- 2.1.4 was released on Mon Mar 25 2019
- hbase-thirdparty 2.2.0 was released on Tue Apr 2, 2019

## Mailing list activity:

 - dev@hbase.apache.org:
- 1051 subscribers (down -5 in the last 3 months):
- 1115 emails sent to list (997 in previous quarter)

 - u...@hbase.apache.org:
- 2236 subscribers (down -17 in the last 3 months):
- 117 emails sent to list (119 in previous quarter)

## JIRA activity:

 - 494 JIRA tickets created in the last 3 months  (up from 405 last quarter)
 - 436 JIRA tickets closed/resolved in the last 3 months  (up from 332 last
quarter)

Release process docs (was Re: [VOTE] First release candidate for HBASE 2.2.0 is available)

[Misty Linville]

Hi Guanghao Zhang,

I feel your pain! The RMs are the only ones using those docs. Please create
friction logs that make it into JIRAs so someone can update the docs, or
make the doc updates as you find errors during the release. I’m sorry to
put more work back on the RM, but it really is the most efficient way.

Those docs were super old before I fixed them (they were basically Stack’s
notes to himself, pasted into the ref guide) and then they rotted again.
It’s a project-wide effort to keep process docs in sync with processes. :)

Thanks for all you do, and that goes for each of our release managers!

Misty

On Fri, Mar 8, 2019 at 12:09 AM Guanghao Zhang  wrote:

> Ok. But we need to update the ref guide about how to release. It has too
> many mistakes now...
>

Re: [DISCUSS] Handling removal of deprecated methods and classes

[Misty Linville]

I’m late on this, but I had this thought. Why not have a minimum
deprecation period of, say, 6 months, after which the API is eligible for
depreciation. At that point, a deprecation can be proposed on a DISCUSS
thread, with plenty of time given for discussion.

On Wed, Apr 24, 2019 at 9:56 PM Stack  wrote:

> On Wed, Apr 24, 2019 at 9:04 PM Yu Li  wrote:
>
> > ...
> > Thirdly, besides API deprecation, I think we should also discuss about
> the
> > rule for feature deprecation. We ever deprecated DLR, I could observe
> some
> > discussion on deprecating preemptive fast fail recently, and personally I
> > think there're some more features with few usage but making code base
> > complicated.
> >
> >
> I like this suggestion. Lets start a new topic on stuff to remove.
>
> Thanks Yu Li,
> S
>
>
>
>
> > Lastly, I think we need to emphasize on adding the incompatible flag and
> > release note to JIRA and updating document whenever a deprecated
> > feature/API is removed, which IMHO is not followed quite well recently.
> >
> > Thanks.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Thu, 25 Apr 2019 at 10:24, 张铎(Duo Zhang) 
> wrote:
> >
> > > Let's get a conclusion here?
> > >
> > > By default, we should try our best to keep the deprecated APIs for at
> > least
> > > a whole major release. All committers should have this in mind, when
> > there
> > > is patch which deprecates a public API, we should make sure that there
> is
> > > a @deprecated javadoc comment says on which version it is deprecated,
> and
> > > on which version it is expected to be removed.
> > >
> > > And due to the difficulties in real world, if you think the above rule
> > > makes it really hard to maintain a clean and stable code base, you can
> > > break the rule. But you should have a strong enough reason, and also, a
> > > nice written release note, and also update the upgrading section in our
> > ref
> > > guide.
> > >
> > > Is this OK?
> > >
> > > Thanks.
> > >
> > > 张铎(Duo Zhang)  于2019年4月24日周三 下午2:38写道：
> > >
> > > > Phoenix is a good example I'd say, we have done a lot of CP related
> > > > changes before releasing 2.0.0. But until now, Phoenix still has
> > troubles
> > > > to work together with 2.x, although we have already fixed several
> > > > compatibility issues... It shows that, usually people will take much
> > care
> > > > on the alpha and beta releases...
> > > >
> > > > Reid Chan  于2019年4月24日周三 上午10:58写道：
> > > >
> > > >>
> > > >> As a strong coupling system to HBase, Phoenix has always been the
> > first
> > > >> sufferer, not only client APIs, but also some internal APIs.
> > > >> Maybe we can ask Phoenix dev team what do they think, and get some
> > > >> thoughts from them.
> > > >>
> > > >>
> > > >> --
> > > >>
> > > >> Best regards,
> > > >> R.C
> > > >>
> > > >>
> > > >>
> > > >> 
> > > >> From: Sean Busbey 
> > > >> Sent: 24 April 2019 10:16
> > > >> To: dev
> > > >> Subject: Re: [DISCUSS] Handling removal of deprecated methods and
> > > classes
> > > >>
> > > >> Isn't API stabilization what we have alpha and beta releases for?
> > > >>
> > > >> There was nearly a year between the first alpha version of 2.0.0 and
> > the
> > > >> GA
> > > >> release. Surely that should be a reasonable window for "trying out"
> a
> > > new
> > > >> major release.
> > > >>
> > > >> I don't like the idea of telling our downstreamers that they can't
> > > >> "really"
> > > >> use or rely on our major releases until several minor releases in.
> > > Either
> > > >> we should say "API locks at X.0.0" or we should say "any API change
> > > might
> > > >> happen at a major version".
> > > >>
> > > >> We can always document what went away between minor releases even if
> > we
> > > >> can't give a compile time warning in advance (e.g. for the 2.0.0 to
> > > 3.0.0
> > > >> user in your example Duo).
> > > >>
> > > >> We ended up trying to do that for 2.0.0 because so much had changed
> > from
> > > >> 1.0.0.
> > > >>
> > > >>
> > > >> On Tue, Apr 23, 2019, 20:51 张铎(Duo Zhang) 
> > > wrote:
> > > >>
> > > >> > Ideally, we'd better keep the deprecated APIs for at least a whole
> > > major
> > > >> > version, that means user can upgrade to a new major release from
> any
> > > >> minor
> > > >> > releases, without seeing the removal of any non-deprecated APIs.
> > > >> >
> > > >> > For example, if we deprecate a method in 2.1.0, and remove it in
> > > 3.0.0,
> > > >> > then the users who upgrade to 3.0.0 from 2.0.0 will see the
> removal
> > of
> > > >> this
> > > >> > method, although the method is not deprecated on 2.0.0.
> > > >> >
> > > >> > But considering the current status of our project(as Sean
> mentioned
> > > >> above),
> > > >> > I do not think it is a good idea to fully follow the rule. One
> more
> > > >> thing
> > > >> > is that, for a new major release, usually the code base is not
> > stable
> > > >> > enough, need to be polished. Ideally it will be stable enough
> after
> >

Re: [DISCUSS] Handling removal of deprecated methods and classes

[Misty Linville]

For features, I think we would need to understand the popularity of a given
feature by our user base, and that might vary wildly by feature. Do you
think it would be possible to generalize?

On Wed, Apr 24, 2019 at 9:56 PM Stack  wrote:

> On Wed, Apr 24, 2019 at 9:04 PM Yu Li  wrote:
>
> > ...
> > Thirdly, besides API deprecation, I think we should also discuss about
> the
> > rule for feature deprecation. We ever deprecated DLR, I could observe
> some
> > discussion on deprecating preemptive fast fail recently, and personally I
> > think there're some more features with few usage but making code base
> > complicated.
> >
> >
> I like this suggestion. Lets start a new topic on stuff to remove.
>
> Thanks Yu Li,
> S
>
>
>
>
> > Lastly, I think we need to emphasize on adding the incompatible flag and
> > release note to JIRA and updating document whenever a deprecated
> > feature/API is removed, which IMHO is not followed quite well recently.
> >
> > Thanks.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Thu, 25 Apr 2019 at 10:24, 张铎(Duo Zhang) 
> wrote:
> >
> > > Let's get a conclusion here?
> > >
> > > By default, we should try our best to keep the deprecated APIs for at
> > least
> > > a whole major release. All committers should have this in mind, when
> > there
> > > is patch which deprecates a public API, we should make sure that there
> is
> > > a @deprecated javadoc comment says on which version it is deprecated,
> and
> > > on which version it is expected to be removed.
> > >
> > > And due to the difficulties in real world, if you think the above rule
> > > makes it really hard to maintain a clean and stable code base, you can
> > > break the rule. But you should have a strong enough reason, and also, a
> > > nice written release note, and also update the upgrading section in our
> > ref
> > > guide.
> > >
> > > Is this OK?
> > >
> > > Thanks.
> > >
> > > 张铎(Duo Zhang)  于2019年4月24日周三 下午2:38写道：
> > >
> > > > Phoenix is a good example I'd say, we have done a lot of CP related
> > > > changes before releasing 2.0.0. But until now, Phoenix still has
> > troubles
> > > > to work together with 2.x, although we have already fixed several
> > > > compatibility issues... It shows that, usually people will take much
> > care
> > > > on the alpha and beta releases...
> > > >
> > > > Reid Chan  于2019年4月24日周三 上午10:58写道：
> > > >
> > > >>
> > > >> As a strong coupling system to HBase, Phoenix has always been the
> > first
> > > >> sufferer, not only client APIs, but also some internal APIs.
> > > >> Maybe we can ask Phoenix dev team what do they think, and get some
> > > >> thoughts from them.
> > > >>
> > > >>
> > > >> --
> > > >>
> > > >> Best regards,
> > > >> R.C
> > > >>
> > > >>
> > > >>
> > > >> 
> > > >> From: Sean Busbey 
> > > >> Sent: 24 April 2019 10:16
> > > >> To: dev
> > > >> Subject: Re: [DISCUSS] Handling removal of deprecated methods and
> > > classes
> > > >>
> > > >> Isn't API stabilization what we have alpha and beta releases for?
> > > >>
> > > >> There was nearly a year between the first alpha version of 2.0.0 and
> > the
> > > >> GA
> > > >> release. Surely that should be a reasonable window for "trying out"
> a
> > > new
> > > >> major release.
> > > >>
> > > >> I don't like the idea of telling our downstreamers that they can't
> > > >> "really"
> > > >> use or rely on our major releases until several minor releases in.
> > > Either
> > > >> we should say "API locks at X.0.0" or we should say "any API change
> > > might
> > > >> happen at a major version".
> > > >>
> > > >> We can always document what went away between minor releases even if
> > we
> > > >> can't give a compile time warning in advance (e.g. for the 2.0.0 to
> > > 3.0.0
> > > >> user in your example Duo).
> > > >>
> > > >> We ended up trying to do that for 2.0.0 because so much had changed
> > from
> > > >> 1.0.0.
> > > >>
> > > >>
> > > >> On Tue, Apr 23, 2019, 20:51 张铎(Duo Zhang) 
> > > wrote:
> > > >>
> > > >> > Ideally, we'd better keep the deprecated APIs for at least a whole
> > > major
> > > >> > version, that means user can upgrade to a new major release from
> any
> > > >> minor
> > > >> > releases, without seeing the removal of any non-deprecated APIs.
> > > >> >
> > > >> > For example, if we deprecate a method in 2.1.0, and remove it in
> > > 3.0.0,
> > > >> > then the users who upgrade to 3.0.0 from 2.0.0 will see the
> removal
> > of
> > > >> this
> > > >> > method, although the method is not deprecated on 2.0.0.
> > > >> >
> > > >> > But considering the current status of our project(as Sean
> mentioned
> > > >> above),
> > > >> > I do not think it is a good idea to fully follow the rule. One
> more
> > > >> thing
> > > >> > is that, for a new major release, usually the code base is not
> > stable
> > > >> > enough, need to be polished. Ideally it will be stable enough
> after
> > > one
> > > >> or
> > > >> > two minor releases. This means we will in

Re: [DISCUSS] EOL 2.0 release line

[Misty Linville]

Thank you for prioritizing the upgrade experience for the cautious
upgrader!

On Wed, Apr 24, 2019 at 6:38 AM 张铎(Duo Zhang)  wrote:

> I think we'd better make a 2.0.6 release after 2.2.0 it out, and make sure
> that it is fine to upgrade to 2.2.0 from 2.0.6, and then EOL the 2.0.x
> release line.
>
> OpenInx  于2019年4月24日周三 下午9:00写道：
>
> > I'v checked the issues in version 2.0.6 [1], seems no critial issues, so
> > for me +1.
> >
> > 1.
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20fixVersion%20%3D%202.0.6
> >
> > On Wed, Apr 24, 2019 at 8:37 PM Sean Busbey  wrote:
> >
> > > Hi folks!
> > >
> > > Tomorrow will be a month since the 2.0.5 release, which was noted:
> > >
> > > > NOTICE: We plan on this being the last release on branch-2.0 (unless
> > > critical issues found).
> > >
> > >
> > > Any objections to me doing the cleanup + announcement for formally
> > > shutting down the release line?
> > >
> >
>

Re: [FYI] HBase review queue

[Misty Linville]

I’ll review these by the end of the week unless anyone wants to jump in
sooner.

On Tue, May 28, 2019 at 8:59 AM Biju N  wrote:

> Here are some simple ones (all documentation related issues). Hope these
> can be looked at quickly.
>
> https://issues.apache.org/jira/browse/HBASE-7129
> https://issues.apache.org/jira/browse/HBASE-12169
> https://issues.apache.org/jira/browse/HBASE-15898
> https://issues.apache.org/jira/browse/HBASE-17550
> https://issues.apache.org/jira/browse/HBASE-21231
> https://issues.apache.org/jira/browse/HBASE-21415
>
>
>
> On Thu, May 23, 2019 at 10:04 PM Sean Busbey  wrote:
>
> > What would help folks prioritize doing reviews?
> >
> > A report that showed target branch and wether the patch still applied?
> > (similar to how in github PRs there's a notice about if there are
> > current conflicts)
> >
> > On Thu, May 23, 2019 at 6:03 PM Biju N  wrote:
> > >
> > > Analogy being "to gardening" :-)
> > >
> > > On Thu, May 23, 2019 at 7:01 PM Biju N  wrote:
> > >
> > > > Hi all,
> > > >As you may have noticed, there are 378 issues in the review queue
> > and a
> > > > large portion of them are in the queue for a long time. It would be
> > good to
> > > > clear at least the old ones before all valid patches become unusable.
> > It
> > > > will also help with trimming the JIRA to borrow Stack's analogy.
> > > >
> > > > Thanks,
> > > > Biju
> > > >
> >
>

Re: Someone checked in 85.80 MB RELEASENOTES.md

[Misty Linville]

I’m confused how an automatically generated file can vary so much in size.
Can the release notes file with an archive just target the release in
question and leave off the older stuff? What’s the difference in practice
between the release noted and changelog?

A pre-commit and possibly a presubmit would help.

On Tue, Jun 4, 2019 at 10:55 AM Andrew Purtell  wrote:

> The various release notes and changes.txt come up frequently in a listing
> of large-ish objects committed in the repo, along with autogenerated
> protobuf and thrift files. It's fine if we tolerate them all in return for
> something. Not requiring local IDL compilers to build is a reasonable
> tradeoff for checking in what can be (and is) generated. I'm less convinced
> about release notes, given they can be made readily available online, and
> already come in an online form on JIRA with no work required from us beyond
> proper attention to fix versions, but I don't have a strong opinion about
> it.
>
> On Tue, Jun 4, 2019 at 10:35 AM Sean Busbey  wrote:
>
> > presuming you mean the two files from your original email:
> >
> > cb0e9bb95599 86MiB RELEASENOTES.md
> > 61e9de9b82a9   14MiB RELEASENOTES.md
> >
> > these are both for HBase 2.2.0. Since branch-2.2 currently shows:
> >
> >
> > Busbey-MBA:hbase busbey$ git checkout branch-2.2
> > Already on 'branch-2.2'
> > Your branch is up to date with 'origin/branch-2.2'.
> > Busbey-MBA:hbase busbey$ ls -lah *.md
> > -rw-r--r--  1 busbey  staff   100K Jun  4 12:24 CHANGES.md
> > -rw-r--r--  1 busbey  staff69K Jun  4 12:24 RELEASENOTES.md
> >
> > Yes, I'd wager it was also a mistake.
> >
> > If we don't want files over some threshold size, we should do what we
> > normally do when we want to change committer behavior: offer people
> > tools that help them do the right thing without thinking about it too
> > much. In this case I would guess a client side git pre-commit hook
> > that stopped things when files are too big, e.g. 10MiB. When a case
> > comes up that there's something bigger we need to commit then folks
> > can use their judgement and discuss it on dev@ if they think it's
> > contentious.
> >
> > On Tue, Jun 4, 2019 at 12:18 PM Andrew Purtell 
> > wrote:
> > >
> > > There's an 80 MB release notes file, possibly a mistake; the next
> largest
> > > object is an 11 MB release notes file. Also a mistake?
> > >
> > > On Tue, Jun 4, 2019 at 10:11 AM Sean Busbey  wrote:
> > >
> > > > Currently putting them in the repo is how we get release notes into
> the
> > > > source and binary artifacts we vote on. It's really convenient for
> > making
> > > > sure folks who download things have some version of the notes.
> > > >
> > > > We've been including the bare file in the dist area as well, so we'd
> > face
> > > > the same issue around distributing a large file (probably more likely
> > to
> > > > face it since it's compressed in the tarballs).
> > > >
> > > > I agree we should have an up to date rendered version on the website
> > (it's
> > > > been an outstanding doc jira). But the files usually aren't very
> large
> > and
> > > > having them shipped in the release is nice.
> > > >
> > > > On Tue, Jun 4, 2019, 12:04 Andrew Purtell 
> wrote:
> > > >
> > > > > What do you think about linking to a remote site-based release
> notes
> > file
> > > > > instead of checking them into the main repo?
> > > > >
> > > > > On Mon, Jun 3, 2019 at 10:12 PM Guanghao Zhang  >
> > > > wrote:
> > > > >
> > > > > > Sorry. This large RELEASENOTES.md may be introduced by me. I
> > committed
> > > > a
> > > > > > big RELEASENOTES.md to branch-2.2 yesterday. I fixed it today.
> And
> > > > roll a
> > > > > > new 2.2.0RC5. Thanks.
> > > > > >
> > > > > > Andrew Purtell  于2019年6月4日周二 上午3:24写道：
> > > > > >
> > > > > > > remote: warning: GH001: Large files detected. You may want to
> > try Git
> > > > > > Large
> > > > > > > File Storage - https://git-lfs.github.com.
> > > > > > > remote: warning: See http://git.io/iEPt8g for more
> information.
> > > > > > > remote: warning: File RELEASENOTES.md is 85.80 MB; this is
> larger
> > > > than
> > > > > > > GitHub's recommended maximum file size of 50.00 MB
> > > > > > >
> > > > > > > The object is cb0e9bb95599 86MiB RELEASENOTES.md
> > > > > > >
> > > > > > > Incidentially, the next largest file is also a markup release
> > notes
> > > > > file.
> > > > > > >
> > > > > > > 61e9de9b82a9   14MiB RELEASENOTES.md
> > > > > > >
> > > > > > > As far as I can tell cb0e9bb95599 is not referenced from
> > anywhere. So
> > > > > > > eventually garbage collection both on the github side and in
> > local
> > > > > > > repositories will clear it out?
> > > > > > >
> > > > > > > This leads to the natural question of whether we should be
> > checking
> > > > in
> > > > > > such
> > > > > > > really large autogenerated files. There are release policy
> > > > > implications.
> > > > > > > Perhaps we can check in very small release notes files
> containing
> > > > only
> > > > > a
> > > > > > 

Re: Someone checked in 85.80 MB RELEASENOTES.md

[Misty Linville]

So what was in the 85MB file? I’m confused about the nature of the mistake
that caused it.

On Tue, Jun 4, 2019 at 1:10 PM Sean Busbey  wrote:

> changelog is a list of all the fixes. release notes is supposed to be
> the things we as a community think downstream users need to pay
> attention to.
>
> The release notes in an archive should be related to the release in
> question. Historically that has meant "all the maintenance releases in
> this minor release".
>
> It's still not a problematic amount. We are prematurely optimizing IMHO.
>
> HBase 1.2 was around for 3+ years. its change documentation predates
> our use of Yetus Release Doc Maker so it's basically the report JIRA
> produces for each maintenance release. the final 1.2.12 release had a
> changelog of 154KiB.
>
> https://github.com/apache/hbase/blob/rel/1.2.12/CHANGES.txt
>
> HBase 2.0.0 had like 3 years of backlog fixes and the 2.0.5 summary of
> to date has 827KiB for CHANGES and 460KiB for RELEASENOTES.
>
> https://github.com/apache/hbase/blob/rel/2.0.5/CHANGES.md
> https://github.com/apache/hbase/blob/rel/2.0.5/RELEASENOTES.md
>
>
> On Tue, Jun 4, 2019 at 2:55 PM Misty Linville  wrote:
> >
> > I’m confused how an automatically generated file can vary so much in
> size.
> > Can the release notes file with an archive just target the release in
> > question and leave off the older stuff? What’s the difference in practice
> > between the release noted and changelog?
> >
> > A pre-commit and possibly a presubmit would help.
> >
> > On Tue, Jun 4, 2019 at 10:55 AM Andrew Purtell 
> wrote:
> >
> > > The various release notes and changes.txt come up frequently in a
> listing
> > > of large-ish objects committed in the repo, along with autogenerated
> > > protobuf and thrift files. It's fine if we tolerate them all in return
> for
> > > something. Not requiring local IDL compilers to build is a reasonable
> > > tradeoff for checking in what can be (and is) generated. I'm less
> convinced
> > > about release notes, given they can be made readily available online,
> and
> > > already come in an online form on JIRA with no work required from us
> beyond
> > > proper attention to fix versions, but I don't have a strong opinion
> about
> > > it.
> > >
> > > On Tue, Jun 4, 2019 at 10:35 AM Sean Busbey  wrote:
> > >
> > > > presuming you mean the two files from your original email:
> > > >
> > > > cb0e9bb95599 86MiB RELEASENOTES.md
> > > > 61e9de9b82a9   14MiB RELEASENOTES.md
> > > >
> > > > these are both for HBase 2.2.0. Since branch-2.2 currently shows:
> > > >
> > > >
> > > > Busbey-MBA:hbase busbey$ git checkout branch-2.2
> > > > Already on 'branch-2.2'
> > > > Your branch is up to date with 'origin/branch-2.2'.
> > > > Busbey-MBA:hbase busbey$ ls -lah *.md
> > > > -rw-r--r--  1 busbey  staff   100K Jun  4 12:24 CHANGES.md
> > > > -rw-r--r--  1 busbey  staff69K Jun  4 12:24 RELEASENOTES.md
> > > >
> > > > Yes, I'd wager it was also a mistake.
> > > >
> > > > If we don't want files over some threshold size, we should do what we
> > > > normally do when we want to change committer behavior: offer people
> > > > tools that help them do the right thing without thinking about it too
> > > > much. In this case I would guess a client side git pre-commit hook
> > > > that stopped things when files are too big, e.g. 10MiB. When a case
> > > > comes up that there's something bigger we need to commit then folks
> > > > can use their judgement and discuss it on dev@ if they think it's
> > > > contentious.
> > > >
> > > > On Tue, Jun 4, 2019 at 12:18 PM Andrew Purtell 
> > > > wrote:
> > > > >
> > > > > There's an 80 MB release notes file, possibly a mistake; the next
> > > largest
> > > > > object is an 11 MB release notes file. Also a mistake?
> > > > >
> > > > > On Tue, Jun 4, 2019 at 10:11 AM Sean Busbey 
> wrote:
> > > > >
> > > > > > Currently putting them in the repo is how we get release notes
> into
> > > the
> > > > > > source and binary artifacts we vote on. It's really convenient
> for
> > > > making
> > > > > > sure folks who download things have some version of the notes.
> >

Re: Someone checked in 85.80 MB RELEASENOTES.md

[Misty Linville]

Was the duplication due to a script that generates the release note or is
this something that happened manually? If the former, it would be best for
that script to itself check for duplicates, I reckon. I’m not trying to
detail the original discussion but understand how we can prevent mistakes
without giving the RM more to do. :)

On Tue, Jun 4, 2019 at 7:46 PM Guanghao Zhang  wrote:

> >
> > So what was in the 85MB file? I’m confused about the nature of the
> mistake
> > that caused it.
> >
> There are very duplicate content. Every issue's release note repeated
> hundreds of times. So it may be heplful to check this in precommit job.
> Grep the issue number from release note and check whether there are
> duplicate.
>
> Sean Busbey  于2019年6月5日周三 上午4:15写道：
>
> > Busbey-MBA:hbase busbey$ git show cb0e9bb95599 | grep "Release Notes" |
> > sort -u
> > # HBASE  2.2.0 Release Notes
> > Busbey-MBA:hbase busbey$ git show 61e9de9b82a9 | grep "Release Notes" |
> > sort -u
> > # HBASE  2.2.0 Release Notes
> > Busbey-MBA:hbase busbey$ git checkout branch-2.2
> > Switched to branch 'branch-2.2'
> > Your branch is up to date with 'origin/branch-2.2'.
> > Busbey-MBA:hbase busbey$ cat RELEASENOTES.md | grep "Release Notes" |
> sort
> > -u
> > # HBASE  2.2.0 Release Notes
> > Busbey-MBA:hbase busbey$ cat RELEASENOTES.md | grep "Release Notes" | wc
> -l
> >1
> > Busbey-MBA:hbase busbey$ git show cb0e9bb95599 | grep "Release Notes" |
> wc
> > -l
> > 1296
> > Busbey-MBA:hbase busbey$ git show 61e9de9b82a9 | grep "Release Notes" |
> wc
> > -l
> >  216
> >
> > Seems like a simple mistake. No big deal; maybe some instructions are
> > unclear. I haven't tried to drive a 2.y.z release yet.
> >
> > On Tue, Jun 4, 2019 at 3:12 PM Misty Linville  wrote:
> > >
> > > So what was in the 85MB file? I’m confused about the nature of the
> > mistake
> > > that caused it.
> > >
> > > On Tue, Jun 4, 2019 at 1:10 PM Sean Busbey  wrote:
> > >
> > > > changelog is a list of all the fixes. release notes is supposed to be
> > > > the things we as a community think downstream users need to pay
> > > > attention to.
> > > >
> > > > The release notes in an archive should be related to the release in
> > > > question. Historically that has meant "all the maintenance releases
> in
> > > > this minor release".
> > > >
> > > > It's still not a problematic amount. We are prematurely optimizing
> > IMHO.
> > > >
> > > > HBase 1.2 was around for 3+ years. its change documentation predates
> > > > our use of Yetus Release Doc Maker so it's basically the report JIRA
> > > > produces for each maintenance release. the final 1.2.12 release had a
> > > > changelog of 154KiB.
> > > >
> > > > https://github.com/apache/hbase/blob/rel/1.2.12/CHANGES.txt
> > > >
> > > > HBase 2.0.0 had like 3 years of backlog fixes and the 2.0.5 summary
> of
> > > > to date has 827KiB for CHANGES and 460KiB for RELEASENOTES.
> > > >
> > > > https://github.com/apache/hbase/blob/rel/2.0.5/CHANGES.md
> > > > https://github.com/apache/hbase/blob/rel/2.0.5/RELEASENOTES.md
> > > >
> > > >
> > > > On Tue, Jun 4, 2019 at 2:55 PM Misty Linville 
> > wrote:
> > > > >
> > > > > I’m confused how an automatically generated file can vary so much
> in
> > > > size.
> > > > > Can the release notes file with an archive just target the release
> in
> > > > > question and leave off the older stuff? What’s the difference in
> > practice
> > > > > between the release noted and changelog?
> > > > >
> > > > > A pre-commit and possibly a presubmit would help.
> > > > >
> > > > > On Tue, Jun 4, 2019 at 10:55 AM Andrew Purtell <
> apurt...@apache.org>
> > > > wrote:
> > > > >
> > > > > > The various release notes and changes.txt come up frequently in a
> > > > listing
> > > > > > of large-ish objects committed in the repo, along with
> > autogenerated
> > > > > > protobuf and thrift files. It's fine if we tolerate them all in
> > return
> > > > for
> > > > > > something. Not requiring local IDL compil

Re: Fwd: ACTION REQUIRED: disk space on jenkins master nearly full

[Misty Linville]

Keeping artifacts and keeping build logs are two separate things. I don’t
see a need to keep any artifacts past the most recent green and most recent
red builds. Alternately if we need the artifacts let’s have Jenkins put
them somewhere rather than keeping them there. You can get back to whatever
hash you need within git to reproduce a build problem.

On Mon, Jun 10, 2019 at 2:26 PM Josh Elser  wrote:

> https://issues.apache.org/jira/browse/HBASE-22563 for a quick bandaid (I
> hope).
>
> On 6/10/19 4:31 PM, Josh Elser wrote:
> > Eyes on.
> >
> > Looking at master, we already have the linked configuration, set to
> > retain 30 builds.
> >
> > We have some extra branches which we can lop off (branch-1.2,
> > branch-2.0, maybe some feature branches too). A quick fix might be to
> > just pull back that 30 to 10.
> >
> > Largely figuring out how this stuff works now, give me a shout in Slack
> > if anyone else has cycles.
> >
> > On 6/10/19 2:34 PM, Peter Somogyi wrote:
> >> Hi,
> >>
> >> HBase jobs are using more than 400GB based on this list.
> >> Could someone take a look at the job configurations today? Otherwise, I
> >> will look into it tomorrow morning.
> >>
> >> Thanks,
> >> Peter
> >>
> >> -- Forwarded message -
> >> From: Chris Lambertus 
> >> Date: Mon, Jun 10, 2019 at 7:57 PM
> >> Subject: ACTION REQUIRED: disk space on jenkins master nearly full
> >> To: 
> >> Cc: , 
> >>
> >>
> >> Hello,
> >>
> >> The jenkins master is nearly full.
> >>
> >> The workspaces listed below need significant size reduction within 24
> >> hours
> >> or Infra will need to perform some manual pruning of old builds to
> >> keep the
> >> jenkins system running. The Mesos “Packaging” job also needs to be
> >> corrected to include the project name (mesos-packaging) please.
> >>
> >> It appears that the typical ‘Discard Old Builds’ checkbox in the job
> >> configuration may not be working for multibranch pipeline jobs. Please
> >> refer to these articles for information on discarding builds in
> >> multibranch
> >> jobs:
> >>
> >>
> https://support.cloudbees.com/hc/en-us/articles/115000237071-How-do-I-set-discard-old-builds-for-a-Multi-Branch-Pipeline-Job-
> >>
> >> https://issues.jenkins-ci.org/browse/JENKINS-35642
> >>
> https://issues.jenkins-ci.org/browse/JENKINS-34738?focusedCommentId=263489&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-263489
> >>
> >>
> >>
> >>
> >> NB: I have not fully vetted the above information, I just notice that
> >> many
> >> of these jobs have ‘Discard old builds’ checked, but it is clearly not
> >> working.
> >>
> >>
> >> If you are unable to reduce your disk usage beyond what is listed,
> please
> >> let me know what the reasons are and we’ll see if we can find a
> solution.
> >> If you believe you’ve configured your job properly and the space usage
> is
> >> more than you expect, please comment here and we’ll take a look at what
> >> might be going on.
> >>
> >> I cut this list off arbitrarily at 40GB workspaces and larger. There are
> >> many which are between 20 and 30GB which also need to be addressed, but
> >> these are the current top contributors to the disk space situation.
> >>
> >>
> >> 594GPackaging
> >> 425Gpulsar-website-build
> >> 274Gpulsar-master
> >> 195Ghadoop-multibranch
> >> 173GHBase Nightly
> >> 138GHBase-Flaky-Tests
> >> 119Gnetbeans-release
> >> 108GAny23-trunk
> >> 101Gnetbeans-linux-experiment
> >> 96G Jackrabbit-Oak-Windows
> >> 94G HBase-Find-Flaky-Tests
> >> 88G PreCommit-ZOOKEEPER-github-pr-build
> >> 74G netbeans-windows
> >> 71G stanbol-0.12
> >> 68G Sling
> >> 63G Atlas-master-NoTests
> >> 48G FlexJS Framework (maven)
> >> 45G HBase-PreCommit-GitHub-PR
> >> 42G pulsar-pull-request
> >> 40G Atlas-1.0-NoTests
> >>
> >>
> >>
> >> Thanks,
> >> Chris
> >> ASF Infra
> >>
>

[REPORT] HBase Quarterly report Apr-Jun 2019

[Misty Linville]

All,

HBase submits a report to the ASF board once a quarter, to inform the board
about project health. I'm sending the report to the user@ and dev@ mailing
lists because you are the project, and for transparency. If you have any
questions about the report or the running of the project, you can pose them
to any PMC member or committer, or send an email to priv...@hbase.apache.org,
which every PMC member subscribes to.

Thanks,
Misty
--
## Description:

Apache HBase is an open-source, distributed, versioned, non-relational
database. Apache HBase gives you low latency random access to billions of
rows with millions of columns atop non-specialized hardware.

hbase-connectors is a collection of integration points with other projects.
The initial release includes artifacts for use with Apache Kafka and Apache
Spark.

hbase-filesystem contains HBase project-specific implementations of the
Apache Hadoop FileSystem API. It is currently experimental and internal to
the project.

## Issues:


The HBase PMC previously requested mediation from the ASF membership on an
internal PMC matter, and are withdrawing the request. The individual is not
currently active in the HBase project. This item will be omitted from
future reports, unless new issues arise. See
https://lists.apache.org/thread.html/e5b2d756033821e4c7162647259a673c91fcd581b00ca16e8280c13a@%3Cboard.apache.org%3E
.

Board-only information removed from public report.

## Activity:

There have been lots of interesting discussions on the dev@ mailing list.
Highlights:
- HBase 1.2.x is at the end of maintenance (EOM). 1.2.x users are
encouraged to upgrade to the current stable line (1.4.9) or even newer if
they are able.
- We had more discussions of EOLing the 2.0.x line, and decided to release
a 2.0.6 release first. (
https://lists.apache.org/thread.html/804ee14d830ab41bdf5846f6f86d3db7f1af36f6304d82ef89d67f44@%3Cdev.hbase.apache.org%3E
)
We started talking about HBase 3 and we'd like to hear your thoughts. (
https://lists.apache.org/thread.html/e12dd0805d2dc99ddb713f72e8978977e406681716059944a141fca9@%3Cdev.hbase.apache.org%3E
)
- We're looking for users who are using "Preemptive Fast Fail." If that's
you, please weigh in. (
https://lists.apache.org/thread.html/e234283ff115bd74d018178052886ad5494932b8070317d7b9268d86@%3Cdev.hbase.apache.org%3E
)
- We mostly wrapped up the 6+ month move to git-box. (
https://lists.apache.org/thread.html/3496568d6cc002f74f5c3bcce46ed44b7ee9e90d7d53af2c65b6f785@%3Cdev.hbase.apache.org%3E,
https://lists.apache.org/thread.html/d05defa96237dca49783c1b615964b6b09ff90482c4873b390141488@%3Cdev.hbase.apache.org%3E)
and improved the documentation for reviewers and committers (
https://lists.apache.org/thread.html/11ade0cc981913997d719c7138bbb930bf528c2d2f5c048b7d4cd4f3@%3Cdev.hbase.apache.org%3E
).
- Toward modularizing HBase code for ease of maintenance and releases, we
created new repositories for hbase-connectors, hbase-native-client,
hbase-filesystem. (https://issues.apache.org/jira/browse/HBASE-20934,
https://lists.apache.org/thread.html/c6d4717d5569de7cf89125b253b2126d9ec87ed8a895d7cf3c46dff2@%3Cdev.hbase.apache.org%3E,
https://github.com/apache/hbase-native-client,
https://lists.apache.org/thread.html/a6403bc04be7bf04ba90196b44f1a4cdc0364cd2e7ca1927250a8f1b@%3Cdev.hbase.apache.org%3E,
https://lists.apache.org/thread.html/93fd0bd79cb54bc14015fa7e17ddb11e02a4b44ecaf75f916f232833@%3Cdev.hbase.apache.org%3E
)
- We improved the Downloads page by removing all references to
dist.apache.org from it. (https://issues.apache.org/jira/browse/HBASE-22206)
- We had several heart-felt discussions, on dev@ and private@, about the
responsibility of the PMC in voting for release candidates, Release Manager
burn-out, the meaning of +0 votes, whether to continue testing a RC after
it is sunk, and other related topics. Some of those discussions are
available to the public. (
https://lists.apache.org/thread.html/a4f8e44bef8ceac7faf7759c3c8e66d495ce484216978f71548b4e31@%3Cdev.hbase.apache.org%3E,
https://lists.apache.org/thread.html/9de91f65ecd4290bf82c52393078ac15a8933adb67bb22719ca6744b@%3Cdev.hbase.apache.org%3E,
https://lists.apache.org/thread.html/8cf2f7d8651f0975d413355641ea8ca4e6ac1204b5c4532c027f5208@%3Cdev.hbase.apache.org%3E).
See the items below about voting.
- Anoop started a Google Group for people interested in hosting or
attending HBase meet-ups in India (
https://lists.apache.org/thread.html/cfca58e62ceb2120808dbe62c9860bf9bdb7e3a023edfeffdcef83ed@%3Cdev.hbase.apache.org%3E
)
- Josh Elser organized and attended the first NoSQL Day on May 21 in
Washington, DC, and wrote up a report. (
https://lists.apache.org/thread.html/a30a42f34cc6581aca68944551604b18b90fab5147ff69705e543081@%3Cdev.hbase.apache.org%3E
)
- A team of developers expressed interest in getting HBase running on ARM,
and offered to help manage some testing infrastructure. (
https://lists.apache.org/thread.html/e2c1a628491d6d659ee9ef948d3e9d760a781c2ce2d657cda1d9ab14@%3Cdev.hbase.

[Announce] 张铎 (Duo Zhang) is Apache HBase PMC chair

[Misty Linville]

Each Apache project has a project management committee (PMC) that oversees
governance of the project, votes on new committers and PMC members, and
ensures that the software we produce adheres to the standards of the
Foundation. One of the roles on the PMC is the PMC chair. The PMC chair
represents the project as a Vice President of the Foundation and
communicates to the board about the project's health, once per quarter and
at other times as needed.

It's been my honor to serve as your PMC chair since 2017, when I took over
from Andrew Purtell. I've decided to step back from my volunteer ASF
activities to leave room in my life for other things. The HBase PMC
nominated Duo for this role, and Duo has kindly agreed! The board passed
this resolution in its meeting yesterday[1] and it is already official[2].
Congratulations, Duo, and thank you for continuing to honor the project
with your dedication.

Misty

[1] The minutes have not yet posted at the time of this email, but will be
available at http://www.apache.org/foundation/records/minutes/2019/.
[2] https://www.apache.org/foundation/#who-runs-the-asf

Re: The slides for HBaseConAsia2019 are ready

[Misty Linville]

+users@hbase, -users@infra to BCC

Thanks Duo for the great report and congratulations on such a successful
event.


On Tue, Aug 20, 2019 at 10:56 AM Stack  wrote:

> Thanks for the writeup Duo. I added a synopsis here to the hbasecon
> archives (which should show up tonight after site build):
> https://hbase.apache.org/hbasecon-archives.html  I liked this picture of
> the handsome devils in attendance...
> https://drive.google.com/file/d/1OSE5fQ_H-oClWW_8DFh_SwnuNM0k6nh_/view
> (We need more women committers!).
>
> S
>
> On Mon, Aug 19, 2019 at 11:10 PM 张铎(Duo Zhang) 
> wrote:
>
>> Thanks Stack for uploading them to slideshare
>>
>>
>> https://www.slideshare.net/search/slideshow?searchfrom=header&q=hbaseconasia2019&ud=any&ft=all&lang=en&sort=
>>
>> And for HBaseConAsia 2019, it is a successful conference. There are
>> roughly
>> 500+ attendees and more than 20 presentations. We also had a developer
>> meeting at the day after the conference, and the notes have already been
>> sent out to the dev list.
>>
>> There are some photos:
>>
>> All the speakers and some guests from Xiaomi.
>>  8F3A9596.JPG
>> <
>> https://drive.google.com/file/d/11_fQJ2Uh5Gz3Bcw3KzurF5g01UGzbPAH/view?usp=drive_web
>> >
>>
>> All the committers came to the conference(except Yi Mei, she had left
>> before we took this picture...).
>>  ZQC_5908.JPG
>> <
>> https://drive.google.com/file/d/1OSE5fQ_H-oClWW_8DFh_SwnuNM0k6nh_/view?usp=drive_web
>> >
>>
>> From left to right: Guanghao Zhang, Zheng Hu, Phil Yang(Zhe Yang), Allan
>> Yang(Wenlong Yang), Duo Zhang, Yu Li, Wellington Chevreuil, Anoop Sam
>> John,
>> Ramkrishna S. Vasudevan, Chunhui Shen, Guangxu Cheng, Heng Chen
>>
>> This is the main hall of the conference.
>>  8F3A9382.JPG
>> <
>> https://drive.google.com/file/d/1Y8lTV-vM7NWROMCPXPYWzx5n_9rmqu4k/view?usp=drive_web
>> >
>
>
>>
>> As there is a 1M limit for the email size so I can not include the
>> pictures
>> in the mail content... And more pictures can be found here:
>> https://live.photoplus.cn/activity/live/pc/66810531/#/
>>
>> And there is also article from Xiaomi tells about the conference:
>>
>> http://blog.mi.com/en/2019/07/21/xiaomi-amplifies-commitment-to-open-source-software
>>
>> Thanks the HBase community and thanks all the speakers&attendees. See you
>> next year on HBaseConAsia 2020!
>>
>

Fwd: [NOTICE] Introducing .asf.yaml for enhanced automation of git repository services

[Misty Linville]

Per-branch website / docs staging and publishing if the project wants it.
:) Other useful stuff below too.

-- Forwarded message -
From: Daniel Gruno 
Date: Wed, Sep 4, 2019 at 5:33 PM
Subject: [NOTICE] Introducing .asf.yaml for enhanced automation of git
repository services
To: 


Hello, fellow Apache committers and enthusiasts!

Today, the Apache Infrastructure Team is launching new self-serve
features to help augment the productivity of Apache projects through a
series of simple configurations for enabling automation of various
service offerings.

We call this new initiative '.asf.yaml', and as the name implies, it
consists of a new file that projects can add to their git repositories
to control various aspects that were previously done through JIRA
tickets and manual operations by the Infrastructure staff.

For detailed information about these features and how to enable them,
please visit our documentation page at: https://s.apache.org/asfyaml

At launch time, .asf.yaml has three features enabled that projects can
make use of: web site staging, web site publishing, and github meta-data
settings:

web site staging:
   Much like with the Apache CMS system, projects using git can now get
   their web sites staged for previews at a specific (HTTPS-enabled)
   staging domain. Staging supports multi-tenancy, which allows for
   multiple staging web sites from different site repository branches.
   For more information on multi-tenancy, see the canonical documentation
   linked above.

web site publishing:
   Projects can now automatically set up publishing for their own main
   web site ($project.apache.org) and change at will, without the need to
   wait for assistance from the Infrastructure team. New podlings and
   projects can also get web sites bootstrapped and ready immediately.

github meta-data settings:
   Projects can now, via .asf.yaml, specify their repositories' meta-data
   on github, such as the description, homepage URL, and which topics to
   add to a repository's front page.

In the coming months, we will extend this feature with many new,
exciting features such as automated buildbot integration for web site
generation (think CMS but via git) and enhanced JIRA integration
automation.

We hope projects will appreciate and make use of these new features. If
you or your project have any questions, please feel free to reach out to
us at: us...@infra.apache.org - but please read through the
documentation first :)

With regards, and on behalf of the Apache Infrastructure Team,
Daniel.

Re: [DISCUSS] EOM branch-1.3

[Misty Linville]

Should the user list be allowed to weigh in?

On Mon, Dec 2, 2019 at 7:33 AM Andrew Purtell 
wrote:

> I think there is a consensus on moving the stable pointer, based on
> earlier discussion. What I would suggest is a separate thread to propose
> it, and if nobody objects, do it.
>
> > On Dec 2, 2019, at 5:14 AM, 张铎(Duo Zhang)  wrote:
> >
> > +1.
> >
> > And I think it is time to move the stable pointer to 2.2.x? I know that
> > 2.2.x still has some bugs, especially on the procedure store, but anyway,
> > we have HBCK2 to fix them.
> >
> > And for the current stable release line, 1.4.x, the assignment manager
> also
> > has bugs, as it is the reason why we introduced AMv2.
> >
> > So I do not think bug free is the 'must have' for a stable release line.
> >
> > Jan Hentschel  于2019年12月2日周一 下午4:57写道：
> >
> >> +1
> >>
> >> From: Sakthi 
> >> Reply-To: "dev@hbase.apache.org" 
> >> Date: Monday, December 2, 2019 at 3:32 AM
> >> To: "dev@hbase.apache.org" 
> >> Subject: Re: [DISCUSS] EOM branch-1.3
> >>
> >> +1
> >>
> >> On Sun, Dec 1, 2019 at 6:28 PM Andrew Purtell  >> >
> >> wrote:
> >>
> >> +1 for EOL of 1.3.
> >>
> >> Onward to 1.6!
> >>
> >>
> >>> On Dec 1, 2019, at 5:38 PM, Sean Busbey  >> bus...@apache.org>> wrote:
> >>>
> >>> Hi folks!
> >>>
> >>> It's been about a month since the last 1.3.z release came out. We've
> >>> been talking about EOM for branch-1.3 for about a year. Most recently,
> >>> we had a growing consensus[1] to EOM after getting the 1.3.6 release
> >>> out with the fixes for Jackson in HBASE-22728 out.
> >>>
> >>> Looking at the things that have since landed in branch-1.3 and nothing
> >>> looks critical (these are all Major or Minor)[2]:
> >>>
> >>> - HBASE-23149 hbase shouldPerformMajorCompaction logic is not correct
> >>> - HBASE-23185 High cpu usage because getTable()#put() gets config
> >>> value every time
> >>> - HBASE-23261 Region stuck in transition while splitting
> >>> - HBASE-18439 Subclasses of o.a.h.h.chaos.actions.Action all use the
> >>> same logger
> >>> - HBASE-23207 Log a region open journal
> >>> - HBASE-23250 Log message about CleanerChore delegate initialization
> >>> should be at INFO
> >>>
> >>> Someone on 1.3.6 can get all these same things fixed by upgrading to
> >>> our current stable release.
> >>>
> >>> Releases on 1.3.z started in 2017. The branch has only averaged ~2
> >>> maintenance releases a year; I think reflecting a lack of community
> >>> interest in maintaining the branch. For comparison 1.4 started about a
> >>> year later and has already had twice as many maintenance releases.
> >>>
> >>> - 1.3.0: 2017-01-16
> >>> - 1.3.1: 2017-04-21
> >>> - 1.3.2: 2018-03-07
> >>> - 1.3.2.1: 2018-06-13
> >>> - 1.3.3: 2018-12-21
> >>> - 1.3.5: 2019-06-10
> >>> - 1.3.6: 2019-10-20
> >>>
> >>> Any objections to shutting branch-1.3 down? If folks show up down the
> >>> road and want to do the work of maintaining it for some reason, we can
> >>> always spin it up again.
> >>>
> >>> [1]:
> >>>
> >>> There's more background if you search farther back, but most recently:
> >>>
> >>> * "Considering immediate EOL of branch-1.3 and branch-1.4"
> >>> https://s.apache.org/f32d0
> >>> * https://issues.apache.org/jira/browse/HBASE-22728
> >>> * https://issues.apache.org/jira/browse/HBASE-22835
> >>> * ANNOUNCE for 1.3.6 included a warning
> >>> "This is ought to be the last release in the 1.3 line unless something
> >>> critical comes up within in the next month or so."
> >>>
> >>> [2]:
> >>>
> >>> https://issues.apache.org/jira/projects/HBASE/versions/12346250
> >>
> >>
> >>
>

Re: [DISCUSS] EOM branch-1.3

[Misty Linville]

Whether any non-dev users are unable to move off 1.3, I suppose.

On Mon, Dec 2, 2019 at 11:04 AM Sean Busbey  wrote:

> On what, specifically?
>
> On Mon, Dec 2, 2019, 11:24 Misty Linville  wrote:
>
> > Should the user list be allowed to weigh in?
> >
> > On Mon, Dec 2, 2019 at 7:33 AM Andrew Purtell 
> > wrote:
> >
> > > I think there is a consensus on moving the stable pointer, based on
> > > earlier discussion. What I would suggest is a separate thread to
> propose
> > > it, and if nobody objects, do it.
> > >
> > > > On Dec 2, 2019, at 5:14 AM, 张铎(Duo Zhang) 
> > wrote:
> > > >
> > > > +1.
> > > >
> > > > And I think it is time to move the stable pointer to 2.2.x? I know
> that
> > > > 2.2.x still has some bugs, especially on the procedure store, but
> > anyway,
> > > > we have HBCK2 to fix them.
> > > >
> > > > And for the current stable release line, 1.4.x, the assignment
> manager
> > > also
> > > > has bugs, as it is the reason why we introduced AMv2.
> > > >
> > > > So I do not think bug free is the 'must have' for a stable release
> > line.
> > > >
> > > > Jan Hentschel  于2019年12月2日周一
> > 下午4:57写道：
> > > >
> > > >> +1
> > > >>
> > > >> From: Sakthi 
> > > >> Reply-To: "dev@hbase.apache.org" 
> > > >> Date: Monday, December 2, 2019 at 3:32 AM
> > > >> To: "dev@hbase.apache.org" 
> > > >> Subject: Re: [DISCUSS] EOM branch-1.3
> > > >>
> > > >> +1
> > > >>
> > > >> On Sun, Dec 1, 2019 at 6:28 PM Andrew Purtell <
> > andrew.purt...@gmail.com
> > > >> <mailto:andrew.purt...@gmail.com>>
> > > >> wrote:
> > > >>
> > > >> +1 for EOL of 1.3.
> > > >>
> > > >> Onward to 1.6!
> > > >>
> > > >>
> > > >>> On Dec 1, 2019, at 5:38 PM, Sean Busbey  > > >> bus...@apache.org>> wrote:
> > > >>>
> > > >>> Hi folks!
> > > >>>
> > > >>> It's been about a month since the last 1.3.z release came out.
> We've
> > > >>> been talking about EOM for branch-1.3 for about a year. Most
> > recently,
> > > >>> we had a growing consensus[1] to EOM after getting the 1.3.6
> release
> > > >>> out with the fixes for Jackson in HBASE-22728 out.
> > > >>>
> > > >>> Looking at the things that have since landed in branch-1.3 and
> > nothing
> > > >>> looks critical (these are all Major or Minor)[2]:
> > > >>>
> > > >>> - HBASE-23149 hbase shouldPerformMajorCompaction logic is not
> correct
> > > >>> - HBASE-23185 High cpu usage because getTable()#put() gets config
> > > >>> value every time
> > > >>> - HBASE-23261 Region stuck in transition while splitting
> > > >>> - HBASE-18439 Subclasses of o.a.h.h.chaos.actions.Action all use
> the
> > > >>> same logger
> > > >>> - HBASE-23207 Log a region open journal
> > > >>> - HBASE-23250 Log message about CleanerChore delegate
> initialization
> > > >>> should be at INFO
> > > >>>
> > > >>> Someone on 1.3.6 can get all these same things fixed by upgrading
> to
> > > >>> our current stable release.
> > > >>>
> > > >>> Releases on 1.3.z started in 2017. The branch has only averaged ~2
> > > >>> maintenance releases a year; I think reflecting a lack of community
> > > >>> interest in maintaining the branch. For comparison 1.4 started
> about
> > a
> > > >>> year later and has already had twice as many maintenance releases.
> > > >>>
> > > >>> - 1.3.0: 2017-01-16
> > > >>> - 1.3.1: 2017-04-21
> > > >>> - 1.3.2: 2018-03-07
> > > >>> - 1.3.2.1: 2018-06-13
> > > >>> - 1.3.3: 2018-12-21
> > > >>> - 1.3.5: 2019-06-10
> > > >>> - 1.3.6: 2019-10-20
> > > >>>
> > > >>> Any objections to shutting branch-1.3 down? If folks show up down
> the
> > > >>> road and want to do the work of maintaining it for some reason, we
> > can
> > > >>> always spin it up again.
> > > >>>
> > > >>> [1]:
> > > >>>
> > > >>> There's more background if you search farther back, but most
> > recently:
> > > >>>
> > > >>> * "Considering immediate EOL of branch-1.3 and branch-1.4"
> > > >>> https://s.apache.org/f32d0
> > > >>> * https://issues.apache.org/jira/browse/HBASE-22728
> > > >>> * https://issues.apache.org/jira/browse/HBASE-22835
> > > >>> * ANNOUNCE for 1.3.6 included a warning
> > > >>> "This is ought to be the last release in the 1.3 line unless
> > something
> > > >>> critical comes up within in the next month or so."
> > > >>>
> > > >>> [2]:
> > > >>>
> > > >>> https://issues.apache.org/jira/projects/HBASE/versions/12346250
> > > >>
> > > >>
> > > >>
> > >
> >
>

[jira] [Created] (HBASE-21184) Update site-wide references from http to https

[Misty Linville (JIRA)]

Misty Linville created HBASE-21184:
--

 Summary: Update site-wide references from http to https
 Key: HBASE-21184
 URL: https://issues.apache.org/jira/browse/HBASE-21184
 Project: HBase
  Issue Type: Task
  Components: scripts, website
Affects Versions: 2.0.2
Reporter: Misty Linville
Assignee: Misty Linville


This is a naive approach! I basically replaced http:// with https:// everywhere 
in plaintext and source files. I'm not on an appropriate host to build this 
today so I will try it later, unless someone else wants to try it first.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)