Re: [ANNOUNCE] New HBase committer Rushabh Shah

[Geoffrey Jacoby]

Congratulations, Rushabh!

Geoffrey

On Thu, Dec 15, 2022 at 10:26 AM Tak Lon (Stephen) Wu 
wrote:

> Congratulations, and welcome!
>
> -Stephen
>
> On Thu, Dec 15, 2022 at 5:39 AM Viraj Jasani  wrote:
>
> > Very well deserved! Congratulations and Welcome, Rushabh!!
> >
> >
> > On Wed, Dec 14, 2022 at 10:57 PM 张铎(Duo Zhang) 
> > wrote:
> >
> > > On behalf of the Apache HBase PMC, I am pleased to announce that
> > > Rushabh Shah(shahrs87)
> > > has accepted the PMC's invitation to become a committer on the
> > > project. We appreciate all
> > > of Rushabh's generous contributions thus far and look forward to his
> > > continued involvement.
> > >
> > > Congratulations and welcome, Rushabh Shah!
> > >
> > > 我很高兴代表 Apache HBase PMC 宣布 Rushabh Shah 已接受我们的邀请，成
> > > 为 Apache HBase 项目的 Committer。感谢 Rushabh Shah 一直以来为 HBase 项目
> > > 做出的贡献，并期待他在未来继续承担更多的责任。
> > >
> > > 欢迎 Rushabh Shah！
> > >
> >
>

Re: [ANNOUNCE] New HBase committer Rushabh Shah

[Geoffrey Jacoby]

Congratulations, Rushabh!

Geoffrey

On Thu, Dec 15, 2022 at 10:26 AM Tak Lon (Stephen) Wu 
wrote:

> Congratulations, and welcome!
>
> -Stephen
>
> On Thu, Dec 15, 2022 at 5:39 AM Viraj Jasani  wrote:
>
> > Very well deserved! Congratulations and Welcome, Rushabh!!
> >
> >
> > On Wed, Dec 14, 2022 at 10:57 PM 张铎(Duo Zhang) 
> > wrote:
> >
> > > On behalf of the Apache HBase PMC, I am pleased to announce that
> > > Rushabh Shah(shahrs87)
> > > has accepted the PMC's invitation to become a committer on the
> > > project. We appreciate all
> > > of Rushabh's generous contributions thus far and look forward to his
> > > continued involvement.
> > >
> > > Congratulations and welcome, Rushabh Shah!
> > >
> > > 我很高兴代表 Apache HBase PMC 宣布 Rushabh Shah 已接受我们的邀请，成
> > > 为 Apache HBase 项目的 Committer。感谢 Rushabh Shah 一直以来为 HBase 项目
> > > 做出的贡献，并期待他在未来继续承担更多的责任。
> > >
> > > 欢迎 Rushabh Shah！
> > >
> >
>

Re: [ANNOUNCE] New HBase committer Zhuoyue Huang(GeorryHuang)

[Geoffrey Jacoby]

Congratulations, Zhuoyue!

Geoffrey

On Thu, Oct 14, 2021 at 11:37 AM Andrew Purtell 
wrote:

> Congratulations and welcome!
>
> > On Oct 14, 2021, at 12:14 AM, Guanghao Zhang  wrote:
> >
> > Folks,
> >
> > On behalf of the Apache HBase PMC I am pleased to announce that Zhuoyue
> > Huang has accepted the PMC's invitation to become a committer on the
> > project.
> >
> > We appreciate all of the great contributions Zhuoyue Huang has made to
> the
> > community thus far and we look forward to his continued involvement.
> >
> > Allow me to be the first to congratulate Zhuoyue Huang on his new role!
> >
> > Thanks.
>

Re: [ANNOUNCE] New HBase PMC Bharath Vissapragada

[Geoffrey Jacoby]

Congratulations, Bharath!

On Mon, Aug 2, 2021 at 4:36 AM 张铎(Duo Zhang)  wrote:

> Congratulations!
>
> Peter Somogyi  于2021年8月2日周一 下午4:10写道：
> >
> > Congratulations!
> >
> > On Fri, Jul 30, 2021 at 3:26 PM Viraj Jasani  wrote:
> >
> > > On behalf of the Apache HBase PMC I am pleased to announce that Bharath
> > > Vissapragada has accepted our invitation to become a PMC member on the
> > > HBase project. We appreciate Bharath stepping up to take more
> > > responsibility for the project.
> > >
> > > Congratulations and welcome, Bharath!
> > >
>

Re: [NOTICE] HBaseTestingUtility is now deprecated, starting from 3.0.0

[Geoffrey Jacoby]

It would of course be possible to solve any particular gap with a special
LP method or interface. I see three problems with that:

1. These special-purpose methods would increase in number over time, and
make the new interface less clean and more cluttered. I really like how
clean it is right now.
2. This requires us to know in advance what special-purpose methods are
needed. There are probably other developers out there with a very different
set of testing needs than mine that I'd never think of. This is
particularly difficult in an Apache-licensed project, where APIs are built
in public but often consumed in private organizations.
3. Some developers who need special-purpose LP methods won't be paying
attention to this thread, or only realize a gap later when building a new
feature, and will have to file a JIRA and either wait for the next HBase
release or just use the old minicluster anyway. That's not a good developer
experience.

Providing a normal mode (the new interface) and an "expert" mode (the
pointer to the inner HBaseTestingUtil for server-side code testing) keeps
the interface clean with just a single additional accessor, and allows
developers to adopt the new interface with confidence, knowing that if
there's a gap they can always work around it easily without a JIRA. Marking
the expert mode LP(COPROC, REPLICATION) makes it clear that it _is_ an
expert mode, just for server-side testing.

Or if you prefer, there could be a ServerTestingHBaseCluster interface that
extends TestingHBaseCluster and adds the extra HBTU accessor, with an
LP(COPROC, REPLICATION).

Could you please explain more about what you don't like about exposing HBTU
even for limited cases? There seems to be an assumption that downstream
developers will follow directions for an IA.Private on HBTU by not using
it, but not follow directions for IA.Private annotations on classes
reachable from HBTU?

Thanks,

Geoffrey



On Wed, Jul 21, 2021 at 1:21 AM 张铎(Duo Zhang)  wrote:

> Inline
>
> Geoffrey Jacoby  于2021年7月21日周三 上午4:43写道：
>
> > Thanks for creating the JIRA for adding the Configuration object.
> >
> > To make the discussion more concrete, I did an initial pass through many
> of
> > the minicluster tests at my dayjob, and here are the gaps I've found so
> > far. These don't count many cases where convenience methods on the HBTU
> are
> > used, but where the tests could be easily modified to use existing client
> > APIs instead.
> >
> > * An enormous number of both internal and Phoenix tests execute MapReduce
> > jobs (for example, Phoenix builds its secondary indexes via MapReduce).
> It
> > doesn't appear possible to run MapReduce jobs through the new testing
> > framework. (I filed HBASE-26102 for this.)
> >
> Usually, you do not need to start a mini MR client for running MR job, it
> will execute locally. Most HBase tests in hbase-mapreduce module do not
> start an actual mini MR cluster.
> And the current MiniMRCluster has already been deprecated in hadoop...
>
> >
> > * A test which uses two miniclusters which share the same mini-ZK quorum
> > under different znodes
> >
> What is the goal for these tests? In HBase, we do this when testing
> replication, but I think it is also OK to use different zk clusters? Unless
> there are problems when starting two zk clusters, then we should try to fix
> this problem.
>
> >
> > * An end-to-end test that verifies that a batch job that appends Cell
> Tags
> > to Delete markers it creates via a coproc in fact writes the Tags. This
> > requires getting a reference to the Region from the minicluster and
> > creating a RegionScanner because the HBase client APIs expressly prevent
> > reading Cell Tags. Supporting this would go against the
> > TestingHBaseCluster's design philosophy, so the old minicluster is still
> > necessary for it.
> >
> This is indeed a problem. We should find a way to let users verify the
> tags, as we do not leak it to client side. We could have something with
> IA.LP to expose the Region interface, maybe.
>
> >
> > So far, I think that with HBASE-26098 and HBASE-26102 the overwhelming
> > majority of internal tests would be fine with the new streamlined
> > TestingHBaseCluster. But not all of them, and there are valid reasons a
> > downstream developer might sometimes need the lower-level control the
> HBTU
> > gives.
> >
> > That's why I still believe that the premise that only HBase and Phoenix
> > need to directly test server-side components is incorrect, because
> > _downstream developers can create server-side components_ like coprocs
> and
> > replication endpoints.
> >
> > My suggested solution:
> > 1. Expose the inner HBaseTest

Re: [NOTICE] HBaseTestingUtility is now deprecated, starting from 3.0.0

[Geoffrey Jacoby]

c/main directory. You can
> > > still use it for several years...
> > >
> > > Thanks.
> > >
> > > Andrew Purtell  于2021年7月20日周二 上午3:26写道：
> > >
> > >>> Just leaving a reference to the old, lower-level HBTU as a public
> > >> property of the new
> > >>> interface seems lower-risk to me. What are the gains from hiding the
> > >>> existing HBTU?
> > >>
> > >> This would be similar to the strategy we adopted for the Admin
> > >> interface. Admin there for new users, and as a migration target, but
> > >> HBaseAdmin still available with deprecation annotations.
> > >>
> > >> I guess the question is if the consensus is HBTU was meant to be
> > >> adopted and consumed by downstream projects.
> > >>
> > >> In my opinion, nothing in any project's test/* source directories
> > >> should be considered public, supported, and supportable. Test
> > >> resources within a project exist to test that project, including its
> > >> private internals.
> > >>
> > >> What I would recommend, fwiw is two things:
> > >>
> > >> 1. Explicitly release a supported hbase-testing-util artifact, with
> > >> Public and LimitedPrivate interfaces, with code in src/ not test/.
> > >> 2. Bring back HBTU, but as a compatibility shim. Provide deprecated
> > >> access to HBTU for Phoenix, marked LP(Phoenix), with this deprecated
> > >> accessor to be removed in HBase 4.0, along with the HBTU interface.
> > >>
> > >>> On Mon, Jul 19, 2021 at 12:15 PM Geoffrey Jacoby  >
> > >>> wrote:
> > >>>
> > >>> Can this be a [DISCUSS] rather than a [NOTICE]? The implications for
> > >>> downstream projects (both Phoenix and many internal projects) are
> > large,
> > >>> and it seems like something that needs broader discussion before
> being
> > >> set
> > >>> in stone. The HBaseTestingUtility is used extensively in Phoenix, as
> > well
> > >>> as in many internal projects at my dayjob (some directly and some
> > through
> > >>> Phoenix's BaseTest wrapping of HBTU) -- it's quite useful.
> > >>>
> > >>> The idea of a better-encapsulated, easier-to-use HBase testing
> utility
> > >> is a
> > >>> good one, and the TestingHBaseCluster interface looks like a definite
> > >>> improvement. However, I notice at least one large gap right away:
> there
> > >>> doesn't appear to be a way to inject a custom Configuration object
> into
> > >> the
> > >>> test cluster, which is a very common pattern. (Example: run a test
> > suite
> > >>> twice with a new minicluster each time, once with a flag off and then
> > >> on.)
> > >>> This seems like a simple fix.
> > >>>
> > >>> More concerning is the underlying assumption of the change, that only
> > the
> > >>> HBase project, and perhaps Phoenix, will ever need to write a test of
> > >>> server-side components. That's simply not the case, because HBase has
> > >> many
> > >>> integration points that allow downstream developed code to run in
> > >>> server-side processes.
> > >>>
> > >>> These include:
> > >>> Coprocessor Observers and Endpoints
> > >>> Replication Endpoints
> > >>> MapReduce integration (which acts as a client from HBase's
> perspective
> > >> but
> > >>> runs within YARN services)
> > >>>
> > >>> In addition, Phoenix supports user-defined functions (UDFs) which I
> > >> believe
> > >>> can run server-side within a coproc in certain query plans.
> > >>>
> > >>> The change assumes that no one will ever need direct access to the
> > >> testing
> > >>> utility's internal ZooKeeper, MR, or DFS services, but this seems
> > >> relevant
> > >>> to failure scenario tests of both Replication Endpoints and MapReduce
> > >> jobs.
> > >>> The Admin API may be able to replace quite a lot of existing logic
> > going
> > >>> forward, and many existing tests already use it rather than the test
> > >>> utility directly.  But there are literally thousands of downstream
> > tests
> > >> to
> > >>> a

Re: [NOTICE] HBaseTestingUtility is now deprecated, starting from 3.0.0

[Geoffrey Jacoby]

Can this be a [DISCUSS] rather than a [NOTICE]? The implications for
downstream projects (both Phoenix and many internal projects) are large,
and it seems like something that needs broader discussion before being set
in stone. The HBaseTestingUtility is used extensively in Phoenix, as well
as in many internal projects at my dayjob (some directly and some through
Phoenix's BaseTest wrapping of HBTU) -- it's quite useful.

The idea of a better-encapsulated, easier-to-use HBase testing utility is a
good one, and the TestingHBaseCluster interface looks like a definite
improvement. However, I notice at least one large gap right away: there
doesn't appear to be a way to inject a custom Configuration object into the
test cluster, which is a very common pattern. (Example: run a test suite
twice with a new minicluster each time, once with a flag off and then on.)
This seems like a simple fix.

More concerning is the underlying assumption of the change, that only the
HBase project, and perhaps Phoenix, will ever need to write a test of
server-side components. That's simply not the case, because HBase has many
integration points that allow downstream developed code to run in
server-side processes.

These include:
Coprocessor Observers and Endpoints
Replication Endpoints
MapReduce integration (which acts as a client from HBase's perspective but
runs within YARN services)

In addition, Phoenix supports user-defined functions (UDFs) which I believe
can run server-side within a coproc in certain query plans.

The change assumes that no one will ever need direct access to the testing
utility's internal ZooKeeper, MR, or DFS services, but this seems relevant
to failure scenario tests of both Replication Endpoints and MapReduce jobs.
The Admin API may be able to replace quite a lot of existing logic going
forward, and many existing tests already use it rather than the test
utility directly.  But there are literally thousands of downstream tests to
analyze across many different organizations and institutions to verify that
nothing important is being lost, and that will take time. Just leaving a
reference to the old, lower-level HBTU as a public property of the new
interface seems lower-risk to me. What are the gains from hiding the
existing HBTU?

Geoffrey



On Sun, Jul 18, 2021 at 9:44 PM 张铎(Duo Zhang)  wrote:

> Please see the discussion in
> https://issues.apache.org/jira/browse/HBASE-13126
>
> And final work is done in
> https://issues.apache.org/jira/browse/HBASE-26081
> https://github.com/apache/hbase/pull/3478
>
> The original HBaseTestingUtility has been renamed to HBaseTestingUtil, and
> MiniHBaseCluster has been renamed to SingleProcessHBaseCluster. Now they
> are not expected to be used by end users any more. We marked it as
> IA.LimitedPrivate("Phoenix"), as maybe the Phoenix project may still need
> to test something internal to HBase.
>
> Anyway, we encourage every downstream projects(including Phoenix) to try to
> make use of the new TestingHBaseCluster introduced in
> https://issues.apache.org/jira/browse/HBASE-26080
>
> We can keep improving it if the current API set is not enough.
>
>  简略的中文版通知，非直译 
>
> HBaseTestingUtility 已经在 3.0.0 中被标记为 Deprecated，请所有用户尽量尝试使用在 HBASE-26080
> 中引入的 TestingHBaseCluster。有任何需求请随时反馈，我们会持续优化。
>

Re: [DISCUSS] Removing problematic terms from our project

[Geoffrey Jacoby]

For most of the proposals (slave -> worker, blacklist -> denylist,
whitelist-> allowlist), I'm +1 (nonbinding). Denylist and acceptlist even
have the advantage of being clearer than the terms they're replacing.

However, I'm not convinced about changing "master" to "coordinator", or
something similar. Unlike "slave", which is negative in any context,
"master" has many definitions, including some common ones which do not
appear problematic. See https://www.merriam-webster.com/dictionary/master for
examples. In particular, the progression of an artisan was from
"apprentice" to "journeyman" to "master". A master smith, carpenter, or
artist would run a shop managing lots of workers and apprentices who would
hope to become masters of their own someday. So "master" and "worker" can
still go together.

Since it's the least problematic term, and by far the hardest term to
change (both within HBase and with effects on downstream projects such as
Ambari), I'm -0 (nonbinding) on changing "master".

Geoffrey

On Mon, Jun 22, 2020 at 1:32 PM Rushabh Shah
 wrote:

> +1 to renaming.
>
>
> Rushabh Shah
>
>- Software Engineering SMTS | Salesforce
>-
>   - Mobile: 213 422 9052
>
>
>
> On Mon, Jun 22, 2020 at 1:18 PM Josh Elser  wrote:
>
> > +1
> >
> > On 6/22/20 4:03 PM, Sean Busbey wrote:
> > > We should change our use of these terms. We can be equally or more
> clear
> > in
> > > what we are trying to convey where they are present.
> > >
> > > That they have been used historically is only useful if the advantage
> we
> > > gain from using them through that shared context outweighs the
> potential
> > > friction they add. They make me personally less enthusiastic about
> > > contributing. That's enough friction for me to advocate removing them.
> > >
> > > AFAICT reworking our replication stuff in terms of "active" and
> "passive"
> > > clusters did not result in a big spike of folks asking new questions
> > about
> > > where authority for state was.
> > >
> > > On Mon, Jun 22, 2020, 13:39 Andrew Purtell 
> wrote:
> > >
> > >> In response to renewed attention at the Foundation toward addressing
> > >> culturally problematic language and terms often used in technical
> > >> documentation and discussion, several projects have begun discussions,
> > or
> > >> made proposals, or started work along these lines.
> > >>
> > >> The HBase PMC began its own discussion on private@ on June 9, 2020
> > with an
> > >> observation of this activity and this suggestion:
> > >>
> > >> There is a renewed push back against classic technology industry terms
> > that
> > >> have negative modern connotations.
> > >>
> > >> In the case of HBase, the following substitutions might be proposed:
> > >>
> > >> - Coordinator instead of master
> > >>
> > >> - Worker instead of slave
> > >>
> > >> Recommendations for these additional substitutions also come up in
> this
> > >> type of discussion:
> > >>
> > >> - Accept list instead of white list
> > >>
> > >> - Deny list instead of black list
> > >>
> > >> Unfortunately we have Master all over our code base, baked into
> various
> > >> APIs and configuration variable names, so for us the necessary changes
> > >> amount to a new major release and deprecation cycle. It could well be
> > worth
> > >> it in the long run. We exist only as long as we draw a willing and
> > >> sufficient contributor community. It also wouldn’t be great to have an
> > >> activist fork appear somewhere, even if unlikely to be successful.
> > >>
> > >> Relevant JIRAs are:
> > >>
> > >> - HBASE-12677  >:
> > >> Update replication docs to clarify terminology
> > >> - HBASE-13852  >:
> > >> Replace master-slave terminology in book, site, and javadoc with a
> > more
> > >> modern vocabulary
> > >> - HBASE-24576  >:
> > >> Changing "whitelist" and "blacklist" in our docs and project
> > >>
> > >> In response to this proposal, a member of the PMC asked if the term
> > >> 'master' used by itself would be fine, because we only have use of
> > 'slave'
> > >> in replication documentation and that is easily addressed. In response
> > to
> > >> this question, others on the PMC suggested that even if only 'master'
> is
> > >> used, in this context it is still a problem.
> > >>
> > >> For folks who are surprised or lacking context on the details of this
> > >> discussion, one PMC member offered a link to this draft RFC as
> > background:
> > >> https://tools.ietf.org/id/draft-knodel-terminology-00.html
> > >>
> > >> There was general support for removing the term "master" / "hmaster"
> > from
> > >> our code base and using the terms "coordinator" or "leader" instead.
> In
> > the
> > >> context of replication, "worker" makes less sense and perhaps
> > "destination"
> > >> or "follower" would be more appropriate terms.
> > >>
> > >> One PMC member's 

Re: [ANNOUNCE] New HBase committer Bharath Vissapragada

[Geoffrey Jacoby]

Congratulations, Bharath!

On Wed, Feb 5, 2020 at 7:41 PM Viraj Jasani 
wrote:

> Congratulations Bharath..!! Well done :)
>
> On Thu, 6 Feb 2020 at 8:36 AM, Nick Dimiduk  wrote:
>
> > On behalf of the Apache HBase PMC I am pleased to announce that Bharath
> > Vissapragada has accepted the PMC's invitation to become a commiter on
> the
> > project. We appreciate all of Bharath's generous contributions thus far
> and
> > look forward to his continued involvement.
> >
> > Allow me to be the first to congratulate and welcome Bharath into his new
> > role!
> >
> > Thanks,
> > Nick
> >
> --
> Thanks,
> Viraj
>