Re: Fwd: ACTION REQUIRED: disk space on jenkins master nearly full

2019-06-11 Thread Peter Somogyi
Thanks Josh for acting on this!

On Tue, Jun 11, 2019 at 3:15 AM Sean Busbey  wrote:

> We used to have a build step that compressed our logs for us. I don't think
> Jenkins can read the test results if we do the xml files from surefire, so
> I'm not sure how much space we can save. That's where I'd start though.
>
> On Mon, Jun 10, 2019, 19:46 张铎(Duo Zhang)  wrote:
>
> > Does surefire have some options to truncate the test output if it is too
> > large? Or jenkins has some options to truncate or compress a file when
> > archiving?
> >
> > Josh Elser  于2019年6月11日周二 上午8:40写道:
> >
> > > Just a cursory glance at some build artifacts showed just test output
> > > which sometimes extended into the multiple megabytes.
> > >
> > > So everyone else knows, I just chatted with ChrisL in Slack and he
> > > confirmed that our disk utilization is down already (after
> HBASE-22563).
> > > He thanked us for the quick response.
> > >
> > > We should keep pulling on this thread now that we're looking at it :)
> > >
> > > On 6/10/19 8:36 PM, 张铎(Duo Zhang) wrote:
> > > > Oh, it is the build artifacts, not the jars...
> > > >
> > > > Most of our build artifacts are build logs, but maybe the problem is
> > that
> > > > some of the logs are very large if the test hangs...
> > > >
> > > > 张铎(Duo Zhang)  于2019年6月11日周二 上午8:16写道:
> > > >
> > > >> For flakey we just need the commit id in the console output then we
> > can
> > > >> build the artifacts locally. +1 on removing artifacts caching.
> > > >>
> > > >> Josh Elser  于2019年6月11日周二 上午7:50写道:
> > > >>
> > > >>> Sure, Misty. No arguments here.
> > > >>>
> > > >>> I think that might be a bigger untangling. Maybe Peter or Busbey
> know
> > > >>> better about how these could be de-coupled (e.g. I think flakies
> > > >>> actually look back at old artifacts), but I'm not sure off the top
> of
> > > my
> > > >>> head. I was just going for a quick fix to keep Infra from doing
> > > >>> something super-destructive.
> > > >>>
> > > >>> For context, I've dropped them a note in Slack to make sure what
> I'm
> > > >>> doing is having a positive effect.
> > > >>>
> > > >>> On 6/10/19 7:34 PM, Misty Linville wrote:
> > >  Keeping artifacts and keeping build logs are two separate things.
> I
> > > >>> don’t
> > >  see a need to keep any artifacts past the most recent green and
> most
> > > >>> recent
> > >  red builds. Alternately if we need the artifacts let’s have
> Jenkins
> > > put
> > >  them somewhere rather than keeping them there. You can get back to
> > > >>> whatever
> > >  hash you need within git to reproduce a build problem.
> > > 
> > >  On Mon, Jun 10, 2019 at 2:26 PM Josh Elser 
> > wrote:
> > > 
> > > > https://issues.apache.org/jira/browse/HBASE-22563 for a quick
> > > bandaid
> > > >>> (I
> > > > hope).
> > > >
> > > > On 6/10/19 4:31 PM, Josh Elser wrote:
> > > >> Eyes on.
> > > >>
> > > >> Looking at master, we already have the linked configuration, set
> > to
> > > >> retain 30 builds.
> > > >>
> > > >> We have some extra branches which we can lop off (branch-1.2,
> > > >> branch-2.0, maybe some feature branches too). A quick fix might
> be
> > > to
> > > >> just pull back that 30 to 10.
> > > >>
> > > >> Largely figuring out how this stuff works now, give me a shout
> in
> > > >>> Slack
> > > >> if anyone else has cycles.
> > > >>
> > > >> On 6/10/19 2:34 PM, Peter Somogyi wrote:
> > > >>> Hi,
> > > >>>
> > > >>> HBase jobs are using more than 400GB based on this list.
> > > >>> Could someone take a look at the job configurations today?
> > > >>> Otherwise, I
> > > >>> will look into it tomorrow morning.
> > > >>>
> > > >>> Thanks,
> > > >>> Peter
> > > >>>
> > > >>> -- Forwarded message -
> > > >>> From: Chris Lambertus 
> > > >>> Date: Mon, Jun 10, 2019 at 7:57 PM
> > > >>> Subject: ACTION REQUIRED: disk space on jenkins master nearly
> > full
> > > >>> To: 
> > > >>> Cc: , 
> > > >>>
> > > >>>
> > > >>> Hello,
> > > >>>
> > > >>> The jenkins master is nearly full.
> > > >>>
> > > >>> The workspaces listed below need significant size reduction
> > within
> > > 24
> > > >>> hours
> > > >>> or Infra will need to perform some manual pruning of old builds
> > to
> > > >>> keep the
> > > >>> jenkins system running. The Mesos “Packaging” job also needs to
> > be
> > > >>> corrected to include the project name (mesos-packaging) please.
> > > >>>
> > > >>> It appears that the typical ‘Discard Old Builds’ checkbox in
> the
> > > job
> > > >>> configuration may not be working for multibranch pipeline jobs.
> > > >>> Please
> > > >>> refer to these articles for information on discarding builds in
> > > >>> multibranch
> > > >>> jobs:
> > > >>>
> > > >>>
> > > >
> > > >>>
> > >
> >
> https://support.cloudbees.com/hc/en-us/artic

Re: [VOTE] Merge branch HBASE-21512 back to master

2019-06-11 Thread Duo Zhang
Let me do a YCSB test about the performance.

Stack  于2019年6月11日周二 下午1:15写道:

> +1 on merge from me.
>
> It removes the complicated multi-threaded edifice we'd built client-side to
> fake an async behavior replacing it with an actual async implementation.
> Users will immediately notice a radical plummet in working thread count on
> the client side.
>
> For the cleanup of old idioms alone, in test code in particular, the patch
> is worth merging.
>
> Any perf numbers to share comparing old sync and async?
>
> What about difference in operation? Is there any commentary or doc or
> release note to point at?
>
> Thanks,
> S
>
>
>
> On Mon, Jun 10, 2019 at 6:59 PM 张铎(Duo Zhang) 
> wrote:
>
> > https://issues.apache.org/jira/browse/HBASE-21512
> >
> > "Reimplement sync client based on async client"
> >
> > The jira title tells everything. This is what I promised when I first
> > introduced the async client in HBase, about three years ago, that the
> sync
> > client can be implemented on top of the async client, so we can remove
> the
> > old sync client implementation, which can reduce our client code base a
> > lot.
> >
> > I've already opened a PR here, and received several feedback(thanks
> stack!)
> >
> > https://github.com/apache/hbase/pull/287
> >
> > It shows that we add 8,663 lines and remove 31,386 lines.
> >
> > This is the flaky dashboard for this branch
> >
> >
> >
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/HBASE-21512/lastSuccessfulBuild/artifact/dashboard.html
> >
> > With the recent efforts I think it is getting better.
> >
> > Will fill the release note soon, it will be a fat one.
> >
> > Please vote
> >
> > [] +1
> > [] +0/-0
> > [] -1 Do not merge the branch back because ...
> >
> > Thanks. Any suggestions are welcomed.
> >
>


[jira] [Created] (HBASE-22564) Compare performance number with old sync client implementation

2019-06-11 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-22564:
-

 Summary: Compare performance number with old sync client 
implementation
 Key: HBASE-22564
 URL: https://issues.apache.org/jira/browse/HBASE-22564
 Project: HBase
  Issue Type: Sub-task
Reporter: Duo Zhang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[VOTE] The sixth HBase 2.2.0 release candidate (RC6) is available

2019-06-11 Thread Guanghao Zhang
Please vote on this release candidate (RC) for Apache HBase 2.2.0.
This is the first release of the branch-2.2 line.

RC6 is almost same with RC5. The biggest change is the release note, which
added the release note of HBASE-21970 about how to upgrade 2.0 or 2.1
cluster to 2.2+. So this VOTE will only open 24 hours.

[] +1
[] +0/-0
[] -1 Do not release this package because ...

The tag to be voted on is 2.2.0RC6. The release files, including
signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC6/

Maven artifacts are available in a staging repository at:
https://repository.apache.org/content/repositories/orgapachehbase-1324

Signatures used for HBase RCs can be found in this file:
https://dist.apache.org/repos/dist/release/hbase/KEYS

The list of bug fixes going into 2.2.0 can be found in included
CHANGES.md and RELEASENOTES.md available here:
https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC6/CHANGES.md
https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC6/RELEASENOTES.md

A detailed source and binary compatibility report for this release is
available at
https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC6/api_compare_2.2.0RC6_to_2.1.5.html

To learn more about Apache HBase, please see http://hbase.apache.org/

Thanks,
Guanghao Zhang


Re: [VOTE] Merge branch HBASE-21512 back to master

2019-06-11 Thread Duo Zhang
Filed  https://issues.apache.org/jira/browse/HBASE-22564

张铎(Duo Zhang)  于2019年6月11日周二 下午3:53写道:

> Let me do a YCSB test about the performance.
>
> Stack  于2019年6月11日周二 下午1:15写道:
>
>> +1 on merge from me.
>>
>> It removes the complicated multi-threaded edifice we'd built client-side
>> to
>> fake an async behavior replacing it with an actual async implementation.
>> Users will immediately notice a radical plummet in working thread count on
>> the client side.
>>
>> For the cleanup of old idioms alone, in test code in particular, the patch
>> is worth merging.
>>
>> Any perf numbers to share comparing old sync and async?
>>
>> What about difference in operation? Is there any commentary or doc or
>> release note to point at?
>>
>> Thanks,
>> S
>>
>>
>>
>> On Mon, Jun 10, 2019 at 6:59 PM 张铎(Duo Zhang) 
>> wrote:
>>
>> > https://issues.apache.org/jira/browse/HBASE-21512
>> >
>> > "Reimplement sync client based on async client"
>> >
>> > The jira title tells everything. This is what I promised when I first
>> > introduced the async client in HBase, about three years ago, that the
>> sync
>> > client can be implemented on top of the async client, so we can remove
>> the
>> > old sync client implementation, which can reduce our client code base a
>> > lot.
>> >
>> > I've already opened a PR here, and received several feedback(thanks
>> stack!)
>> >
>> > https://github.com/apache/hbase/pull/287
>> >
>> > It shows that we add 8,663 lines and remove 31,386 lines.
>> >
>> > This is the flaky dashboard for this branch
>> >
>> >
>> >
>> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/HBASE-21512/lastSuccessfulBuild/artifact/dashboard.html
>> >
>> > With the recent efforts I think it is getting better.
>> >
>> > Will fill the release note soon, it will be a fat one.
>> >
>> > Please vote
>> >
>> > [] +1
>> > [] +0/-0
>> > [] -1 Do not merge the branch back because ...
>> >
>> > Thanks. Any suggestions are welcomed.
>> >
>>
>


Re: [VOTE] The sixth HBase 2.2.0 release candidate (RC6) is available

2019-06-11 Thread Duo Zhang
Better adding a note at the bottom of CHANGES.md and RELEASENOTES.md to
mention that all the changes in 2.1.0 are also included in 2.2.0, as
discussed in another thread?

Guanghao Zhang  于2019年6月11日周二 下午5:48写道:

> Please vote on this release candidate (RC) for Apache HBase 2.2.0.
> This is the first release of the branch-2.2 line.
>
> RC6 is almost same with RC5. The biggest change is the release note, which
> added the release note of HBASE-21970 about how to upgrade 2.0 or 2.1
> cluster to 2.2+. So this VOTE will only open 24 hours.
>
> [] +1
> [] +0/-0
> [] -1 Do not release this package because ...
>
> The tag to be voted on is 2.2.0RC6. The release files, including
> signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC6/
>
> Maven artifacts are available in a staging repository at:
> https://repository.apache.org/content/repositories/orgapachehbase-1324
>
> Signatures used for HBase RCs can be found in this file:
> https://dist.apache.org/repos/dist/release/hbase/KEYS
>
> The list of bug fixes going into 2.2.0 can be found in included
> CHANGES.md and RELEASENOTES.md available here:
> https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC6/CHANGES.md
> https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC6/RELEASENOTES.md
>
> A detailed source and binary compatibility report for this release is
> available at
>
> https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC6/api_compare_2.2.0RC6_to_2.1.5.html
>
> To learn more about Apache HBase, please see http://hbase.apache.org/
>
> Thanks,
> Guanghao Zhang
>


Re: [VOTE] The sixth HBase 2.2.0 release candidate (RC6) is available

2019-06-11 Thread Guanghao Zhang
RC6 is almost same with RC5. So we can carry the RC5's three binding +1 and
one non-binding +1 here?  And one more +1 from me.


Re: [VOTE] The sixth HBase 2.2.0 release candidate (RC6) is available

2019-06-11 Thread Guanghao Zhang
>
> Better adding a note at the bottom of CHANGES.md and RELEASENOTES.md to
> mention that all the changes in 2.1.0 are also included in 2.2.0, as
> discussed in another thread?
>
We didn't reached a conclusion yet in the DISCUSS thread? 2.2.0 has been
delayed too long to release. I thought we can fix it when release 2.2.1 if
we got a conclusion about how to generate RELEASENOTES. Thanks.

Guanghao Zhang  于2019年6月11日周二 下午5:54写道:

> RC6 is almost same with RC5. So we can carry the RC5's three binding +1
> and one non-binding +1 here?  And one more +1 from me.
>


Re: [VOTE] The sixth HBase 2.2.0 release candidate (RC6) is available

2019-06-11 Thread Duo Zhang
OK...

Guanghao Zhang  于2019年6月11日周二 下午6:07写道:

> >
> > Better adding a note at the bottom of CHANGES.md and RELEASENOTES.md to
> > mention that all the changes in 2.1.0 are also included in 2.2.0, as
> > discussed in another thread?
> >
> We didn't reached a conclusion yet in the DISCUSS thread? 2.2.0 has been
> delayed too long to release. I thought we can fix it when release 2.2.1 if
> we got a conclusion about how to generate RELEASENOTES. Thanks.
>
> Guanghao Zhang  于2019年6月11日周二 下午5:54写道:
>
> > RC6 is almost same with RC5. So we can carry the RC5's three binding +1
> > and one non-binding +1 here?  And one more +1 from me.
> >
>


Re: [VOTE] First HBase File FileSystem 1.0.0-alpha1-RC0

2019-06-11 Thread Wellington Chevreuil
After three binding +1s, plus two non-binding +1s and no 0 or -1 votes,
this "vote" has passed!

Thank you to all who voted on this first alpha release candidate for
hbase-filesystem!


Em seg, 10 de jun de 2019 às 20:17, Esteban Gutierrez
 escreveu:

> +1
>
> - Build from source: OK
> - Tests run, 160: OK
> - Checksums and signatures: OK
> - RAT: OK
>
>
> --
> Cloudera, Inc.
>
>
>
> On Mon, Jun 10, 2019 at 2:08 PM Andrew Purtell 
> wrote:
>
> > +1
> >
> > Sums and signatures ok
> > Built from source, ok
> > RAT check passes
> > Tests pass, Hadoop 2
> > Tests pass, Hadoop 3
> >
> >
> > On Fri, Jun 7, 2019 at 7:25 AM Wellington Chevreuil <
> > wellington.chevre...@gmail.com> wrote:
> >
> > > Please vote on this first HBase FileSystem 1.0.0-alpha1 release
> > candidate.
> > >
> > > The release files including signatures can be found at:
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/hbase/hbase-filesystem-1.0.0-alpha1-RC0/
> > >
> > > Maven artifacts are available here:
> > >
> https://repository.apache.org/content/repositories/orgapachehbase-1321/
> > >
> > > Artifacts were signed with wellington.chevre...@gmail.com key
> available
> > > at:
> > > https://dist.apache.org/repos/dist/release/hbase/KEYS
> > >
> > > Compatibility information:
> > > - hadoop 2.9.2
> > > - hadoop 3.2.0
> > > - hbase 2.1.4
> > >
> > > As the first alpha release for hbase-filesystem project, it only
> includes
> > > hbase-oss module, which includes implementation of Hadoop FileSystem
> > > interface semantics for Object Stores. Further implementation and
> > > configuration details are available on hbase-oss/README.md file.
> > >
> > > Please try out the candidate and vote +1/0/-1.
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >- A23, Crosstalk
> >
>


[jira] [Created] (HBASE-22565) Javadoc Warnings: @see cannot be used in inline documentation

2019-06-11 Thread Murtaza Hassan (JIRA)
Murtaza Hassan created HBASE-22565:
--

 Summary: Javadoc Warnings: @see cannot be used in inline 
documentation
 Key: HBASE-22565
 URL: https://issues.apache.org/jira/browse/HBASE-22565
 Project: HBase
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Murtaza Hassan
Assignee: Murtaza Hassan






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HBASE-22566) Call out default compaction and flush throttling for 2.x in Book

2019-06-11 Thread Josh Elser (JIRA)
Josh Elser created HBASE-22566:
--

 Summary: Call out default compaction and flush throttling for 2.x 
in Book
 Key: HBASE-22566
 URL: https://issues.apache.org/jira/browse/HBASE-22566
 Project: HBase
  Issue Type: Task
  Components: documentation
Reporter: Josh Elser
Assignee: Josh Elser


We had it in the release notes, but it didn't make it into the upgrade path 
chapter of the Book ([https://hbase.apache.org/book.html#upgrade2.0]) that 
compactions and flushes are now limited in throughput per RegionServer.

Came as a surprise to me the other day - I'm sure I won't be the last.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [DISCUSS] What versions should be mentioned in CHANGES.md/RELEASENOTES.md per release?

2019-06-11 Thread Andrew Purtell
> have the foot of CHANGES and RELEASENOTES point to hosted, next older
RELEASENOTES/CHANGES hosted in our release dir

+1


On Mon, Jun 10, 2019 at 10:05 PM Stack  wrote:

> HBASE-21935 tries to automate the generation of RELEASENOTES.md and
> CHANGES.md. It has yetus interpolates those that made the current RC (if
> new RC, it scrubs the old and regenerates them). It is trying to make the
> upkeep of RELEASENOTES and CHANGES painless.
>
> Could do a version of what Duo (and Andy) suggest and have the foot of
> CHANGES and RELEASENOTES point to hosted, next older RELEASENOTES/CHANGES
> hosted in our release dir? (Or to JIRA).
>
> S
>
> On Mon, Jun 10, 2019 at 8:11 PM Andrew Purtell 
> wrote:
>
> > No it is just the way it’s been done on branch-1 and going back to 0.x,
> > long before Yetus was a thing. The practice of using Yetus for change
> logs
> > in 2.x releases is an improvement for sure.
> >
> > On Jun 10, 2019, at 8:08 PM, Guanghao Zhang  wrote:
> >
> > >>
> > >> We use JIRA’s change log generation feature instead.
> > >>
> > > Do we have any document about this?
> > >
> > > Andrew Purtell  于2019年6月11日周二 上午10:44写道:
> > >
> > >> We don’t use Yetus to generate release notes in 1.x releases. We use
> > >> JIRA’s change log generation feature instead. There is no overlap in
> the
> > >> 1.5.0 release candidate changes file. I managed fix versions in JIRA
> for
> > >> 1.5.0 for that purpose if you recall hundreds of fix version updates a
> > >> couple of months ago for the first 1.5.0 RC as discussed on dev@ at
> the
> > >> time. Should be available in list archives.
> > >>
> > >>
> > >> On Jun 10, 2019, at 5:46 PM, Guanghao Zhang 
> wrote:
> > >>
> > 
> >  The change log there is based on the 1.4.9 one and contains everyone
> > >> later
> >  than 1.4.9 or new to 1.5.0.
> > 
> > >>> So only generate the release note of 1.5.0 and append it to 1.4.9's
> > >> release
> > >>> note, then get a new release note for 1.5.0? If I am not wrong, the
> > yetus
> > >>> use issue's fix version to generate release note. There are duplicate
> > >>> issues number if a issus's fix versions has both 1.4.9 and 1.5.0?
> > >>>
> > >>> 张铎(Duo Zhang)  于2019年6月11日周二 上午8:31写道:
> > >>>
> >  Maybe we could add a note at the bottom of the release note for each
> > >> minor
> >  release line, to mention that this release line contains all the
> > >> changes in
> >  the previous minor or major release?
> > 
> >  For example, 2.1.0 contains all the changes in 2.0.0, and 2.0.0
> > contains
> >  all the changes in 1.0.0. If users are interested they can go to see
> > the
> >  release note for the previous major or minor release line.
> > 
> >  Andrew Purtell  于2019年6月11日周二 上午12:08写道:
> > 
> > > 1.5.0 will continue the practice. The change log there is based on
> > the
> > > 1.4.9 one and contains everyone later than 1.4.9 or new to 1.5.0.
> > >
> > > The branch-1 releases use the old practice of JIRA generated change
> > >> logs,
> > > not the far more verbose Yetus ones, and even then a list of
> objects
> > > ordered by size is dominated in the largest of sizes by these auto
> > > generated CHANGES files, mixed in with generated protobuf and
> thrift
> > > support files. How big would a Yetus generated release notes file
> be
> > if
> >  it
> > > includes changes all the way back to HBASE-1?
> > >
> > >>> On Jun 10, 2019, at 8:16 AM, Stack  wrote:
> > >>>
> > >>> On Mon, Jun 10, 2019 at 8:05 AM Sean Busbey 
> >  wrote:
> > >>>
> > >>> Back for the 1.2 release line I tried to include enough
> information
> > >>> that someone looking at the given 1.2 release coming from the
> prior
> > >>> major version would have everything.
> > >>>
> > >>> That meant:
> > >>>
> > >>> * 1.0.0 release notes
> > >>> * 1.1.0 release notes
> > >>> * 1.2.z (for all z 0-12) release notes
> > >>>
> > >>> https://github.com/apache/hbase/blob/branch-1.2/CHANGES.txt
> > >>
> > >>
> > >>
> > >> Yeah, this is how it has been in all releases until 2.1 where I
> seem
> > >> to
> > >> have broken the practice (I just looked at the 1.4.10 RC and
> notice
> >  that
> > >> Andrew follows the above practice. 2.0.x has all CHANGES).
> > >>
> > >>
> > >>
> > >>> I do not know if this was actually useful. This seems like a
> > >>> conversation better had on user@hbase, tbh.
> > >>>
> > >>>
> > >> I can ask over there too.
> > >>
> > >>
> > >> S
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>> (folks interested in background material, the last time we talked
> > >>> about this was in HBASE-14025 in 2015 and 2016)
> > >>>
> > >>
> > >>
> > >>
> >  On Mon, Jun 10, 2019 at 9:54 AM Stack  wrote:
> > 
> >  I was under the impression that our CHANGES.md was a list of all
> > > change

[jira] [Created] (HBASE-22567) HBCK2 addMissingRegionsToMeta

2019-06-11 Thread Wellington Chevreuil (JIRA)
Wellington Chevreuil created HBASE-22567:


 Summary: HBCK2 addMissingRegionsToMeta
 Key: HBASE-22567
 URL: https://issues.apache.org/jira/browse/HBASE-22567
 Project: HBase
  Issue Type: New Feature
Reporter: Wellington Chevreuil
Assignee: Wellington Chevreuil


Following latest discussion on HBASE-21745, this proposes an hbck2 command that 
allows for inserting back regions missing in META that still have *regioninfo* 
available in HDFS. Although this is still an interactive and simpler version 
than the old _OfflineMetaRepair_, it still relies on hdfs state as the source 
of truth, and performs META updates mostly independently from Master (apart 
from requiring Meta table been online).

For a more detailed explanation on this command behaviour, pasting _command 
usage_ text:
{noformat}
To be used for scenarios where some regions may be missing in META,
but there's still a valid 'regioninfo' metadata file on HDFS.
This is a lighter version of 'OfflineMetaRepair' tool commonly used for
similar issues on 1.x release line.
This command needs META to be online. For each table name passed as
parameter, it performs a diff between regions available in META,
against existing regions dirs on HDFS. Then, for region dirs with
no matches in META, it reads regioninfo metadata file and
re-creates given region in META. Regions are re-created in 'CLOSED'
state at META table only, but not in Masters' cache, and are not
assigned either. A rolling Masters restart, followed by a
hbck2 'assigns' command with all re-inserted regions is required.
This hbck2 'assigns' command is printed for user convenience.
WARNING: To avoid potential region overlapping problems due to ongoing
splits, this command disables given tables while re-inserting regions.
An example adding missing regions for tables 'table_1' and 'table_2':
$ HBCK2 addMissingRegionsInMeta table_1 table_2
Returns hbck2 'assigns' command with all re-inserted regions.{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HBASE-22568) [2.1] Upgrade to Jetty 9.3.latest and Jackson 2.9.latest

2019-06-11 Thread Josh Elser (JIRA)
Josh Elser created HBASE-22568:
--

 Summary: [2.1] Upgrade to Jetty 9.3.latest and Jackson 2.9.latest
 Key: HBASE-22568
 URL: https://issues.apache.org/jira/browse/HBASE-22568
 Project: HBase
  Issue Type: Task
  Components: dependencies
Reporter: Josh Elser
Assignee: Josh Elser
 Fix For: 3.0.0, 2.3.0, 2.2.1


Worked through some static analysis scans and these two popped up as a result.

Upgrades are:
 * Jetty 9.3.25->9.3.27
 * Jackson 2.9.2->2.9.9

Not expecting any pain with these, but we'll find out what QA thinks! :)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [VOTE] The sixth HBase 2.2.0 release candidate (RC6) is available

2019-06-11 Thread Stack
On Tue, Jun 11, 2019 at 2:54 AM Guanghao Zhang  wrote:

> RC6 is almost same with RC5. So we can carry the RC5's three binding +1 and
> one non-binding +1 here?  And one more +1 from me.
>


No vote carrying Guanghao. If same as RC5, shouldn't take long to gather up
the votes again.

+1 from me on RC6.

Here is output from hbase-vote.sh.
* Signature: ok
* Checksum : ok
* Rat check (1.8.0_102): ok
 - mvn clean apache-rat:check
* Built from source (1.8.0_102): ok
 - mvn clean install -DskipTests
* Unit tests pass (1.8.0_102): failed
 - mvn test -P runAllTests

Failed on these UTs:

[ERROR] Errors:



 [ERROR]
TestSuperUserQuotaPermissions.tearDown:101 » TestTimedOut test timed out
after...
[ERROR]
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
[ERROR]   Run 1: TestSplitTransactionOnCluster.tearDown:162 » TestTimedOut
test timed out after...
[ERROR]   Run 2:
TestSplitTransactionOnCluster.org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
»
[ERROR]   Run 3: TestSplitTransactionOnCluster.before:144 » IllegalState A
mini-cluster is alre...
[INFO]
[ERROR]
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testRITStateForRollback(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster)
[ERROR]   Run 1: TestSplitTransactionOnCluster.testRITStateForRollback:229
» TimeoutIO java.uti...
[ERROR]   Run 2: TestSplitTransactionOnCluster.tearDown:162 » InterruptedIO
Interrupt while wai...


  [INFO]
[ERROR]   TestRaceWhenCreatingReplicationSource.setUpBeforeClass:137 » IO
Waiting for st...


Thanks,
S


[jira] [Created] (HBASE-22569) Should treat null consistency as Consistency.STRONG in ConnectionUtils.timelineConsistentRead

2019-06-11 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-22569:
-

 Summary: Should treat null consistency as Consistency.STRONG in 
ConnectionUtils.timelineConsistentRead
 Key: HBASE-22569
 URL: https://issues.apache.org/jira/browse/HBASE-22569
 Project: HBase
  Issue Type: Sub-task
Reporter: Duo Zhang


In our PE we may set the consistency to null and in 
ConnectionUtils.timelineConsistentRead, we will treat it as TIMELINE.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [VOTE] The sixth HBase 2.2.0 release candidate (RC6) is available

2019-06-11 Thread Duo Zhang
+1(binding)

Checked sigs and sums: Matched
Rat check: Passed
Built from src: OK

Stack  于2019年6月12日周三 上午4:15写道:

> On Tue, Jun 11, 2019 at 2:54 AM Guanghao Zhang  wrote:
>
> > RC6 is almost same with RC5. So we can carry the RC5's three binding +1
> and
> > one non-binding +1 here?  And one more +1 from me.
> >
>
>
> No vote carrying Guanghao. If same as RC5, shouldn't take long to gather up
> the votes again.
>
> +1 from me on RC6.
>
> Here is output from hbase-vote.sh.
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_102): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_102): ok
>  - mvn clean install -DskipTests
> * Unit tests pass (1.8.0_102): failed
>  - mvn test -P runAllTests
>
> Failed on these UTs:
>
> [ERROR] Errors:
>
>
>
>  [ERROR]
> TestSuperUserQuotaPermissions.tearDown:101 » TestTimedOut test timed out
> after...
> [ERROR]
>
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
> [ERROR]   Run 1: TestSplitTransactionOnCluster.tearDown:162 » TestTimedOut
> test timed out after...
> [ERROR]   Run 2:
>
> TestSplitTransactionOnCluster.org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
> »
> [ERROR]   Run 3: TestSplitTransactionOnCluster.before:144 » IllegalState A
> mini-cluster is alre...
> [INFO]
> [ERROR]
>
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testRITStateForRollback(org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster)
> [ERROR]   Run 1: TestSplitTransactionOnCluster.testRITStateForRollback:229
> » TimeoutIO java.uti...
> [ERROR]   Run 2: TestSplitTransactionOnCluster.tearDown:162 » InterruptedIO
> Interrupt while wai...
>
>
>   [INFO]
> [ERROR]   TestRaceWhenCreatingReplicationSource.setUpBeforeClass:137 » IO
> Waiting for st...
>
>
> Thanks,
> S
>


[jira] [Created] (HBASE-22570) test-util.sh failed when executing AbstractHBaseToolTest

2019-06-11 Thread star (JIRA)
star created HBASE-22570:


 Summary: test-util.sh failed when executing AbstractHBaseToolTest
 Key: HBASE-22570
 URL: https://issues.apache.org/jira/browse/HBASE-22570
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.2.0
Reporter: star
Assignee: star


HBASE-16982 added a unit test with neither Catogory anatation nor ClassRule 
static member. Then testing with script test-util.sh will be stopped throwing 
out ArrayIndexOutOfRangeException.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)