Re: [DISCUSS] Using the new 'help wanted' tool from comdev

[Stack]

On Thu, May 12, 2016 at 8:37 PM, Daniel Vimont  wrote:

> ..
> I'm relatively new not only to HBase and big-data in general, but also to
> the whole open-source scene (coming from several decades of working solely
> in the proprietary realm with various legacy database technologies), so I
> very much appreciate the #beginners tagged issues, and above all I am very
> appreciative of the great, friendly, professional attitude that everybody
> I've worked with here has thus far displayed. There's always a natural
> tendency for a newby to feel intimidated by the inherent complexities of
> technologies like this; such sheer complexity is likely the chief "barrier
> to entry" of anyone considering joining in as a contributor. To me, this
> makes the positive attitudes on display here all the more valuable and
> appreciated -- so *thanks*!
>
>

Welcome Dan. Didn't realize you were a newbie judging by the way you've
been going about contrib.



> With regard to recruiting new contributors, I'll just toss out a question
> which I hope does not give offense: What can be done to encourage more
> gender-diversity in this project (and projects like it)?
>
>
Thank you for raising this 'awkward' question up here on the dev list.

Our ratio is awful. I'm ashamed to cite numbers.

I don't know what we can do to encourage participation. I'd be interested
in any ideas others might have and would be up for acting them to try and
make redress.

St.Ack




> As I said, I come from a background of working in 
> and with  companies in the proprietary
> software
> realm, and as such I'm accustomed to working in environments in which males
> outnumber females (like with 60/40 or 70/30 majorities). But one shocking
> aspect of getting involved with this project was coming in and seeing only
> ONE woman's name come up among the current, active contributors. Perhaps
> all-male or almost-all-male projects are just the norm in most open-source
> projects? I don't imagine for a second that this project is actively and
> purposefully *excluding* women. I'm just wondering why (with one exception
> that I'm aware of) they're not here, and what could be done to effectively
> encourage them to join.
>
-- Dan
>
> On Fri, May 13, 2016 at 8:25 AM, Sean Busbey  wrote:
>
> > There's a bit of extra information they'd like to describe the work
> > for new comers. I'd be happy if all of our beginner jiras ended up
> > listed, but that will take a fair bit of work. I suspect we'll get
> > better ROI if we can prioritize a little.
> >
> > On Thu, May 12, 2016 at 4:13 PM, Nick Dimiduk 
> wrote:
> > > Nifty. We already do this with the beginners (neé noobies) tag, right?
> > Just
> > > pipe them through?
> > >
> > > On Thu, May 12, 2016 at 1:13 PM, Sean Busbey 
> wrote:
> > >
> > >> Hi folks!
> > >>
> > >> ASF comdev has put up a great new tool for funneling in new folks:
> > >>
> > >> https://helpwanted.apache.org/
> > >>
> > >> How about we brainstorm a few things here (maybe some beginner JIRAs
> > >> we can flesh out a little?) and then file?
> > >>
> > >> -Sean
> > >>
> >
> >
> >
> > --
> > busbey
> >
>

Re: [DISCUSS] Using the new 'help wanted' tool from comdev

[Daniel Vimont]

JIRA is in some sort of "spam lockdown" state right now, so that affords me
a few minutes to chime in on this thread.

I'm relatively new not only to HBase and big-data in general, but also to
the whole open-source scene (coming from several decades of working solely
in the proprietary realm with various legacy database technologies), so I
very much appreciate the #beginners tagged issues, and above all I am very
appreciative of the great, friendly, professional attitude that everybody
I've worked with here has thus far displayed. There's always a natural
tendency for a newby to feel intimidated by the inherent complexities of
technologies like this; such sheer complexity is likely the chief "barrier
to entry" of anyone considering joining in as a contributor. To me, this
makes the positive attitudes on display here all the more valuable and
appreciated -- so *thanks*!

With regard to recruiting new contributors, I'll just toss out a question
which I hope does not give offense: What can be done to encourage more
gender-diversity in this project (and projects like it)?

As I said, I come from a background of working in 
and with  companies in the proprietary software
realm, and as such I'm accustomed to working in environments in which males
outnumber females (like with 60/40 or 70/30 majorities). But one shocking
aspect of getting involved with this project was coming in and seeing only
ONE woman's name come up among the current, active contributors. Perhaps
all-male or almost-all-male projects are just the norm in most open-source
projects? I don't imagine for a second that this project is actively and
purposefully *excluding* women. I'm just wondering why (with one exception
that I'm aware of) they're not here, and what could be done to effectively
encourage them to join.

-- Dan

On Fri, May 13, 2016 at 8:25 AM, Sean Busbey  wrote:

> There's a bit of extra information they'd like to describe the work
> for new comers. I'd be happy if all of our beginner jiras ended up
> listed, but that will take a fair bit of work. I suspect we'll get
> better ROI if we can prioritize a little.
>
> On Thu, May 12, 2016 at 4:13 PM, Nick Dimiduk  wrote:
> > Nifty. We already do this with the beginners (neé noobies) tag, right?
> Just
> > pipe them through?
> >
> > On Thu, May 12, 2016 at 1:13 PM, Sean Busbey  wrote:
> >
> >> Hi folks!
> >>
> >> ASF comdev has put up a great new tool for funneling in new folks:
> >>
> >> https://helpwanted.apache.org/
> >>
> >> How about we brainstorm a few things here (maybe some beginner JIRAs
> >> we can flesh out a little?) and then file?
> >>
> >> -Sean
> >>
>
>
>
> --
> busbey
>

[jira] [Resolved] (HBASE-15403) Performance Evaluation tool isn't working as expected

[Appy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy resolved HBASE-15403.
--
Resolution: Invalid

> Performance Evaluation tool isn't working as expected
> -
>
> Key: HBASE-15403
> URL: https://issues.apache.org/jira/browse/HBASE-15403
> Project: HBase
>  Issue Type: Bug
>  Components: Performance
>Affects Versions: 1.2.0
>Reporter: Appy
>Priority: Critical
>
> hbase pe --nomapred --rows=100 --table='t4' randomWrite 10
> # count on t4 gives 620 rows
> hbase pe --nomapred --rows=200 --table='t5' randomWrite 10
> # count on t5 gives 1257 rows
> hbase pe --nomapred --table='t6' --rows=200 randomWrite 1
> # count on t6 gives 126 rows
> I was working with 1.2.0, but it's likely that it'll also be affecting master.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [VOTE] First release candidate for HBase 1.1.5 (RC0) is available

[Sean Busbey]

+1

* checked signatures / sums[1]
* spot checked file header compliance using apache-rat plugin on clean
source unpack
* build from source[2]
* source matches commit 239b80456118175b340b2e562a5568b5c744252e
(which the signed tag mentioned pointed to when I checked)[3]

[1]:

I now realize the issue with using gpg printed md5: the md5 files end up in
a different format for dist.apache than the md5 files pushed into
repository.apache.
the latter match md5sum / md5 -r output.

It's not a big deal, just some added noise in the scripting I use to
go through checking
all of the checksum files.

[2]:

I couldn't get a clean mvn verify run, but all the complaints looked
like I'm just
on an underpowered laptop (messages are about minicluster misfiring / timeout)

[3]:

the module hbase-native-client is found in git but isn't included in
the source artifact.
AFAIK, this is as it has always been and will be until we add a new one.


On Sun, May 8, 2016 at 9:23 PM, Nick Dimiduk  wrote:
> *** Please note that my key expired since the previous release. I have
> updated its expiration, pushed to pgp.mit.edu, updated the KEYS file linked
> below, and attempted to force an update on id.apache.org. I don't know how
> long it will take for people.apache.org to refresh. ***
>
> *** Please note that this voting window is slightly shorter than the
> customary one week so that we have time for an RC1 before HBaseCon, if
> necessary. ***
>
> I'm happy to announce the first release candidate of HBase 1.1.5 (HBase-1.1
> .5RC0) is available for download at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.1.5RC0/
>
> Maven artifacts are also available in the staging repository
> https://repository.apache.org/content/repositories/orgapachehbase-1136/
>
> Artifacts are signed with my code signing subkey 0xAD9039071C3489BD,
> available in the Apache keys directory
> https://people.apache.org/keys/committer/ndimiduk.asc and in our KEYS file
> http://www-us.apache.org/dist/hbase/KEYS.
>
> There's also a signed tag for this release at
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=tag;h=92323e8e630e46d277ab2e8ebd34b91ab5d597d5
>
> The detailed source and binary compatibility report vs 1.1.4 has been
> published for your review, at
> http://home.apache.org/~ndimiduk/1.1.4_1.1.5RC0_compat_report.html
>
> HBase 1.1.5 is the fifth patch release in the HBase 1.1 line, continuing on
> the theme of bringing a stable, reliable database to the Hadoop and NoSQL
> communities. This release includes over 20 bug fixes since the 1.1.4
> release. Notable correctness fixes
> include HBASE-15234, HBASE-15295, HBASE-15325, HBASE-15622, and HBASE-15645.
>
> The full list of fixes included in this release is available at
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753=12335058
> and and in the CHANGES.txt file included in the distribution.
>
> Please try out this candidate and vote +/-1 by 23:59 Pacific time on
> Thursday, 2016-05-12 as to whether we should release these artifacts as
> HBase 1.1.5.
>
> Thanks,
> Nick

Re: [DISCUSS] Using the new 'help wanted' tool from comdev

[Sean Busbey]

There's a bit of extra information they'd like to describe the work
for new comers. I'd be happy if all of our beginner jiras ended up
listed, but that will take a fair bit of work. I suspect we'll get
better ROI if we can prioritize a little.

On Thu, May 12, 2016 at 4:13 PM, Nick Dimiduk  wrote:
> Nifty. We already do this with the beginners (neé noobies) tag, right? Just
> pipe them through?
>
> On Thu, May 12, 2016 at 1:13 PM, Sean Busbey  wrote:
>
>> Hi folks!
>>
>> ASF comdev has put up a great new tool for funneling in new folks:
>>
>> https://helpwanted.apache.org/
>>
>> How about we brainstorm a few things here (maybe some beginner JIRAs
>> we can flesh out a little?) and then file?
>>
>> -Sean
>>



-- 
busbey

Re: [DISCUSS] Using the new 'help wanted' tool from comdev

[Nick Dimiduk]

Nifty. We already do this with the beginners (neé noobies) tag, right? Just
pipe them through?

On Thu, May 12, 2016 at 1:13 PM, Sean Busbey  wrote:

> Hi folks!
>
> ASF comdev has put up a great new tool for funneling in new folks:
>
> https://helpwanted.apache.org/
>
> How about we brainstorm a few things here (maybe some beginner JIRAs
> we can flesh out a little?) and then file?
>
> -Sean
>

[jira] [Created] (HBASE-15821) Document TestUtil

[Elliott Clark (JIRA)]

Elliott Clark created HBASE-15821:
-

 Summary: Document TestUtil
 Key: HBASE-15821
 URL: https://issues.apache.org/jira/browse/HBASE-15821
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark


Trying to keep everything documented well. Document TestUtil and how it should 
be used.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-15590) Add ACL for requesting table backup

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu reopened HBASE-15590:


Thinking more about giving table(s) owner permission, that would be more user 
friendly.

> Add ACL for requesting table backup
> ---
>
> Key: HBASE-15590
> URL: https://issues.apache.org/jira/browse/HBASE-15590
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15590.v1.patch, 15590.v2.txt, 15590.v3.txt, 15590.v4.txt
>
>
> This issue adds necessary coprocessor hooks for table backup request along 
> with enforcing permission check in AccessController through the new hooks.
> To perform backup, admin privilege is required in secure deployment. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [DISCUSS] Make AsyncFSWAL the default WAL in 2.0

[Sean Busbey]

On Wed, May 11, 2016 at 7:53 PM, 张铎  wrote:
> I think at that time I will start a new project called AsyncDFSClient which
> will implement the whole client side logic of HDFS without using reflection
> :)
>

If we end up in this dystopian future, then please have that project
live as a subproject of HBase.

-- 
busbey

[DISCUSS] Using the new 'help wanted' tool from comdev

[Sean Busbey]

Hi folks!

ASF comdev has put up a great new tool for funneling in new folks:

https://helpwanted.apache.org/

How about we brainstorm a few things here (maybe some beginner JIRAs
we can flesh out a little?) and then file?

-Sean

Re: [DISCUSS] Make AsyncFSWAL the default WAL in 2.0

[Nick Dimiduk]

On Wed, May 11, 2016 at 10:28 PM, Andrew Purtell 
wrote:

> All you have to do is stick around long enough. Hadoop 0.20-append v2 :-)
>

*palm-all-the-faces*

> On May 11, 2016, at 9:46 PM, Stack  wrote:
> >
> >> On Wed, May 11, 2016 at 7:53 PM, 张铎  wrote:
> >>
> >> I think at that time I will start a new project called AsyncDFSClient
> which
> >> will implement the whole client side logic of HDFS without using
> reflection
> >> :)
> > Haven't I seen this movie before? (smile)
> > St.Ack
> >
> >
> >
> >> 2016-05-12 10:27 GMT+08:00 Andrew Purtell :
> >>
> >>> If Hadoop refuses the changes before we release, we can change the
> >> default
> >>> back.
> >>>
> >>>
> >>> On May 11, 2016, at 6:50 PM, Gary Helmling 
> wrote:
> >>>
> >
> >
> > I was trying to avoid the below oft-repeated pattern at least for the
> >>> case
> > of critical developments:
> >
> > + New feature arrives after much work by developer, reviewers and
> >>> testers
> > accompanied by fanfare (blog, talks).
> > + Developers and reviewers move on after getting it committed or it
> >> gets
> > hacked into a deploy so it works in a frankenstein form
> > + It sits in our code base across one or more releases marked as
> >>> optional,
> > 'experimental'
> > + The 'experimental' bleamish discourages its exercise by users
> > + The feature lags, rots
> > + Or, the odd time, we go ahead and enable it as default in spite of
> >> the
> > fact it was never tried when experimental.
> >
> > Distributed Log Replay sat in hbase across a few major versions. Only
> >>> when
> > the threat of our making an actual release with it on by default did
> >> it
> >>> get
> > serious attention where it was found flawed and is now being actively
> > purged. This was after it made it past reviews, multiple attempts at
> > testing at scale, and so on; i.e. we'd done it all by the book. The
> >>> time in
> > an 'experimental' state added nothing.
>  Those are all valid concerns as well. It's certainly a pattern that
> >> we've
>  seen repeated. That's also a broader concern I have about the farther
> >> we
>  push out 2.0, then the less exercised master is.
> 
>  I don't really know how best to balance this with concerns about user
>  stability.  Enabling by default in master would certainly be a forcing
>  function and would help it get more testing before release.  I hear
> >> that
>  argument.  But I'm worried about the impact after release, where
> >>> something
>  as simple as a bug-fix point release upgrade of Hadoop could result in
>  runtime breakage of an HBase install.  Will this happen in practice?
> I
>  don't know.  It seems unlikely that the private variable names being
> >> used
>  for example would change in a point release.  But we're violating the
>  abstraction that Hadoop provides us which guarantees such breakage
> >> won't
>  occur.
> 
> 
> >> Yes. 2.0 is a bit out there so we have some time to iron out issues
> >> is
> > the
> > thought. Yes, it could push out delivery of 2.0.
>  Having this on by default in an unreleased master doesn't actually
> >> worry
> >>> me
>  that much.  It's just the question of what happens when we do release.
> >>> At
>  that point, this discussion will be ancient history and I don't think
> >>> we'll
>  give any renewed consideration to what the impact of this change might
> >>> be.
>  Ideally it would be great to see this work in HDFS by that point and
> >> for
>  that HDFS version this becomes a non-issue.
> 
> 
> >
> > I think the discussion here has been helpful. Holes have been found
> >> (and
> > plugged), the risk involved has gotten a good airing out here on dev,
> >>> and
> > in spite of the back and forth, one of our experts in good standing
> is
> > still against it being on by default.
> >
> > If you are not down w/ the arguments, I'd be fine not making it the
> > default.
> > St.Ack
> 
>  I don't think it's right to block this by myself, since I'm clearly in
> >>> the
>  minority.  Since others clearly support this change, have at it.
> 
>  But let me pose an alternate question: what if HDFS flat out refuses
> to
>  adopt this change?  What are our options then with this already
> >> shipping
> >>> as
>  a default?  Would we continue to endure breakage due to the use of
> HDFS
>  private internals?  Do we switch the default back?  Do we do something
> >>> else?
> 
>  Thanks for the discussion.
> >>
>

Successful: HBase Generate Website

[Apache Jenkins Server]

Build status: Successful

If successful, the website and docs have been generated. If failed, skip to the 
bottom of this email.

Use the following commands to download the patch and apply it to a clean branch 
based on origin/asf-site. If you prefer to keep the hbase-site repo around 
permanently, you can skip the clone step.

  git clone https://git-wip-us.apache.org/repos/asf/hbase-site.git

  cd hbase-site
  wget -O- 
https://builds.apache.org/job/hbase_generate_website/227/artifact/website.patch.zip
 | funzip > 1267f76e9aa9591b462f6e1ba2e50ec4dbf92f8b.patch
  git fetch
  git checkout -b asf-site-1267f76e9aa9591b462f6e1ba2e50ec4dbf92f8b 
origin/asf-site
  git am --whitespace=fix 1267f76e9aa9591b462f6e1ba2e50ec4dbf92f8b.patch

At this point, you can preview the changes by opening index.html or any of the 
other HTML pages in your local 
asf-site-1267f76e9aa9591b462f6e1ba2e50ec4dbf92f8b branch, and you can review 
the differences by running:

  git diff origin/asf-site

There are lots of spurious changes, such as timestamps and CSS styles in 
tables. To see a list of files that have been added, deleted, renamed, changed 
type, or are otherwise interesting, use the following command:

  git diff --name-status --diff-filter=ADCRTXUB origin/asf-site

To see only files that had 100 or more lines changed:

  git diff --stat origin/asf-site | grep -E '[1-9][0-9]{2,}'

When you are satisfied, publish your changes to origin/asf-site using this 
command:

  git push origin asf-site-1267f76e9aa9591b462f6e1ba2e50ec4dbf92f8b:asf-site

Changes take a couple of minutes to be propagated. You can then remove your 
asf-site-1267f76e9aa9591b462f6e1ba2e50ec4dbf92f8b branch:

  git checkout asf-site && git branch -d 
asf-site-1267f76e9aa9591b462f6e1ba2e50ec4dbf92f8b



If failed, see https://builds.apache.org/job/hbase_generate_website/227/console

[jira] [Created] (HBASE-15820) Re-using byte[] from Bytes can save some objects creations

[Jean-Marc Spaggiari (JIRA)]

Jean-Marc Spaggiari created HBASE-15820:
---

 Summary: Re-using byte[] from Bytes can save some objects creations
 Key: HBASE-15820
 URL: https://issues.apache.org/jira/browse/HBASE-15820
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Jean-Marc Spaggiari
Assignee: Jean-Marc Spaggiari
Priority: Minor


HBASE-15233 will allow byte arrays re-usage for Bytes.toBytes() methods. 
CellCodec can benefit from that in many places.

Patch will follow as soon as there is one in HBASE-15233.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)