[jira] [Created] (HBASE-23023) upgrade shellcheck used to test branch-1

[Sean Busbey (Jira)]

Sean Busbey created HBASE-23023:
---

 Summary: upgrade shellcheck used to test branch-1
 Key: HBASE-23023
 URL: https://issues.apache.org/jira/browse/HBASE-23023
 Project: HBase
  Issue Type: Task
  Components: build
Reporter: Sean Busbey
Assignee: Sean Busbey


branches-1 are using an out of date version of shellcheck.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (HBASE-23022) download page should use HTTPS for links to archive.a.o

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-23022.
-
Fix Version/s: 3.0.0
   Resolution: Fixed

> download page should use HTTPS for links to archive.a.o
> ---
>
> Key: HBASE-23022
> URL: https://issues.apache.org/jira/browse/HBASE-23022
> Project: HBase
>  Issue Type: Bug
>  Components: website
>Reporter: Sean Busbey
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0
>
>
> Noticed this while working on some automation. We're https everywhere else in 
> this page, just not this link.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (HBASE-23022) download page should use HTTPS for links to archive.a.o

[Sean Busbey (Jira)]

Sean Busbey created HBASE-23022:
---

 Summary: download page should use HTTPS for links to archive.a.o
 Key: HBASE-23022
 URL: https://issues.apache.org/jira/browse/HBASE-23022
 Project: HBase
  Issue Type: Bug
  Components: website
Reporter: Sean Busbey
Assignee: Nick Dimiduk


Noticed this while working on some automation. We're https everywhere else in 
this page, just not this link.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [DISCUSS] Storing images in source repository

[Andrew Purtell]

Sounds good to me. We have been separating out resources for site and 
documentation from the main repo to others for a while. Let’s continue doing 
that. So probably the big hbtop readme and associated resources should go into 
the online book and the docs in the source tree should refer to the new book 
section. 


> On Sep 13, 2019, at 4:26 PM, Peter Somogyi  wrote:
> 
> Hi,
> 
> Recently the hbtop tool (HBASE-11062) was merged and it also includes a
> nice README documentation with demos in GIF format.
> 
> Although I think we should have more documentation like this, probably it
> is not a good idea to store the images in the source repository. The size
> of these GIF images is 31MB which is not huge, however, it will be included
> in the release tarballs from now on and every time we pull the repository
> for the builds it adds +31MB every time.
> 
> My recommendation is to keep similar images in a different place, for
> example in the hbase-site repository and link the images from there.
> 
> Thanks,
> Peter

Re: [DISCUSS] Moving towards a branch-2 line that can get the 'stable' pointer.

[Andrew Purtell]

For what it’s worth I had previously been concerned about the disparity between 
hbck capability in 1.x and 2.x but after review of the recent work I believe 
that is no longer true. Put another way, it is reasonable to claim it on par.

As for moving the stable pointer I don’t personally have enough experience with 
HBase 2 to weigh in but will trust the opinions of those that do. 


> On Sep 14, 2019, at 8:44 AM, Stack  wrote:
> 
> HBASE-21745 , the issue
> addressing gaps between hbck1 and hbck2 was closed a few days back after a
> bunch of work by a kaleidoscope of folks. The release notes section tries
> to describe what was added by HBASE-21745. Shout if you think the claim at
> the end of the release notes section that hbck2 now is on par or beyond
> what hbck1 offered is problematic. Otherwise, will proceed as though it is
> the case.
> 
> Suggestion: Given that hbase 2.2.1 will ship soon and hbase-operator-tools
> 1.0.0 with latest hbase-hbck2 should get an RC inside the next week or so,
> if feedback that 2.2.1 looks good, give 2.2.2 (with bug fixes only) the
> stable pointer?
> 
> Thanks,
> S
> 
>> On Sat, Jan 26, 2019 at 11:31 AM Stack  wrote:
>> 
>> As per Sean, bypass with optional 'force' (override) and recurse for case
>> where a procedure had spawned children was the mechanism Allan implemented
>> after a chat about merits of procedure delete. I found it of use doing
>> fixup to clusters I'd intentionally damaged testing candidates. Procedures
>> are usually part of a fabric with relations that an operator might have
>> trouble unraveling. It was thought that the bypass would be safer than a
>> delete, likely to cause more damage than solution.
>> 
>> Interested in the issues you are seeing on Master branch Sergey.
>> 
>> Thanks,
>> S
>> 
>>> On Fri, Jan 25, 2019 at 1:54 PM Sean Busbey  wrote:
>>> 
>>> that's already present, see the README for the "bypass" command:
>>> 
>>> https://github.com/apache/hbase-operator-tools/tree/master/hbase-hbck2
>>> 
>>> On Fri, Jan 25, 2019 at 3:40 PM Sergey Shelukhin
>>>  wrote:
 
 I think one thing that is needed for HBCK2 for AMv2 is to be able to
>>> delete single procedures from store.
 We are evaluating master (whose assignment is very similar to branch-2)
>>> right now and I have to delete proc WAL pretty much every day because some
>>> procedure(s) are in bad state, but deleting the entire WAL also causes
>>> other issues.
 It should be possible to remove some offending procedure while master
>>> is offline and/or online.
 
 -Original Message-
 From: 张铎(Duo Zhang) 
 Sent: Friday, January 18, 2019 5:52 PM
 To: HBase Dev List 
 Subject: Re: [DISCUSS] Moving towards a branch-2 line that can get the
>>> 'stable' pointer.
 
 OK, the original issue is HBCK2 for AMv2, but here we need to do more,
>>> not only for AMv2.
 
 Let me open a new issue and post what Andrew said above there.
 
>>> 
>> 

Re: [DISCUSS] Moving towards a branch-2 line that can get the 'stable' pointer.

[Stack]

HBASE-21745 , the issue
addressing gaps between hbck1 and hbck2 was closed a few days back after a
bunch of work by a kaleidoscope of folks. The release notes section tries
to describe what was added by HBASE-21745. Shout if you think the claim at
the end of the release notes section that hbck2 now is on par or beyond
what hbck1 offered is problematic. Otherwise, will proceed as though it is
the case.

Suggestion: Given that hbase 2.2.1 will ship soon and hbase-operator-tools
1.0.0 with latest hbase-hbck2 should get an RC inside the next week or so,
if feedback that 2.2.1 looks good, give 2.2.2 (with bug fixes only) the
stable pointer?

Thanks,
S

On Sat, Jan 26, 2019 at 11:31 AM Stack  wrote:

> As per Sean, bypass with optional 'force' (override) and recurse for case
> where a procedure had spawned children was the mechanism Allan implemented
> after a chat about merits of procedure delete. I found it of use doing
> fixup to clusters I'd intentionally damaged testing candidates. Procedures
> are usually part of a fabric with relations that an operator might have
> trouble unraveling. It was thought that the bypass would be safer than a
> delete, likely to cause more damage than solution.
>
> Interested in the issues you are seeing on Master branch Sergey.
>
> Thanks,
> S
>
> On Fri, Jan 25, 2019 at 1:54 PM Sean Busbey  wrote:
>
>> that's already present, see the README for the "bypass" command:
>>
>> https://github.com/apache/hbase-operator-tools/tree/master/hbase-hbck2
>>
>> On Fri, Jan 25, 2019 at 3:40 PM Sergey Shelukhin
>>  wrote:
>> >
>> > I think one thing that is needed for HBCK2 for AMv2 is to be able to
>> delete single procedures from store.
>> > We are evaluating master (whose assignment is very similar to branch-2)
>> right now and I have to delete proc WAL pretty much every day because some
>> procedure(s) are in bad state, but deleting the entire WAL also causes
>> other issues.
>> > It should be possible to remove some offending procedure while master
>> is offline and/or online.
>> >
>> > -Original Message-
>> > From: 张铎(Duo Zhang) 
>> > Sent: Friday, January 18, 2019 5:52 PM
>> > To: HBase Dev List 
>> > Subject: Re: [DISCUSS] Moving towards a branch-2 line that can get the
>> 'stable' pointer.
>> >
>> > OK, the original issue is HBCK2 for AMv2, but here we need to do more,
>> not only for AMv2.
>> >
>> > Let me open a new issue and post what Andrew said above there.
>> >
>>
>

Re: [VOTE] The third HBase 2.2.1 release candidate (RC2) is available

[Stack]

+1
IMO, CHANGES and RELEASENOTES should accumulate all changes, not just show
what is in current release.
Compat report looks good.
Built from src.
Ran a loading and verified it there across restart.
Checked sigs and hash on src tgz (Duo, sign Guanghao's key and vice-versa
-- key is not 'a trusted signature!').
S



On Tue, Sep 10, 2019 at 4:06 AM Guanghao Zhang  wrote:

> Please vote on this release candidate (RC2) for Apache HBase 2.2.1.
>
> As this RC only changed the tar ball revison, this VOTE will remain open
> for 24 hours.
>
> [ ] +1 Release this package as Apache HBase 2.2.1
> [ ] -1 Do not release this package because ...
>
> The tag to be voted on is 2.2.1RC2. The release files, including
> signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/hbase/2.2.1RC2/
>
> Maven artifacts are available in a staging repository at:
> https://repository.apache.org/content/repositories/orgapachehbase-1328/
>
> Signatures used for HBase RCs can be found in this file:
> https://dist.apache.org/repos/dist/release/hbase/KEYS
>
> The list of bug fixes going into 2.2.1 can be found in included
> CHANGES.md and RELEASENOTES.md available here:
> https://dist.apache.org/repos/dist/dev/hbase/2.2.1RC2/CHANGES.md
> https://dist.apache.org/repos/dist/dev/hbase/2.2.1RC2/RELEASENOTES.md
>
> A detailed source and binary compatibility report for this release is
> available at:
>
> https://dist.apache.org/repos/dist/dev/hbase/2.2.1RC2/api_compare_2.2.1RC2_to_2.2.0.html
>
> To learn more about Apache HBase, please see http://hbase.apache.org/
>
> Thanks,
> Guanghao Zhang
>

[jira] [Resolved] (HBASE-23021) [hbase-operator-tools] README edits in prep for release

[stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-23021.
---
Fix Version/s: hbase-operator-tools-1.0.0
 Hadoop Flags: Reviewed
 Assignee: stack
   Resolution: Fixed

Pushed from local checkout and closed PR. Thanks for review [~busbey]

> [hbase-operator-tools] README edits in prep for release
> ---
>
> Key: HBASE-23021
> URL: https://issues.apache.org/jira/browse/HBASE-23021
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: hbase-operator-tools-1.0.0
>
>
> Did an edit of the READMEs to get us ready for 1.0.0 release.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (HBASE-23003) [HBCK2/hbase-operator-tools] Release-making scripts

[stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-23003.
---
Fix Version/s: hbase-operator-tools-1.0.0
 Hadoop Flags: Reviewed
 Assignee: stack
   Resolution: Fixed

Pushed this. Thanks for reviews [~psomogyi].

> [HBCK2/hbase-operator-tools] Release-making scripts
> ---
>
> Key: HBASE-23003
> URL: https://issues.apache.org/jira/browse/HBASE-23003
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: hbase-operator-tools-1.0.0
>
> Attachments: 
> 0001-HBASE-23003-HBCK2-hbase-operator-tools-Release-makin.patch
>
>
> Make some scripts for creating hbase-operator-tools releases so its easy to 
> do and not subject to vagaries of environment or RM's attention-to-detail (or 
> not).



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (HBASE-22691) [hbase-operator-tools] Move Checkstyle suppression file to different location

[stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-22691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-22691.
---
Fix Version/s: hbase-operator-tools-1.0.0
   Resolution: Fixed

This was merged. Resolving.

 commit cf92e611adff3b215dce0292912b8764082fd421
 Author: Jan Hentschel 
 Date:   Sun Jul 14 18:24:52 2019 +0200

 HBASE-22691 Moved Checkstyle suppression file to a different location

 Signed-off-by: Sean Busbey 

> [hbase-operator-tools] Move Checkstyle suppression file to different location
> -
>
> Key: HBASE-22691
> URL: https://issues.apache.org/jira/browse/HBASE-22691
> Project: HBase
>  Issue Type: Improvement
>  Components: hbck2
>Affects Versions: hbase-operator-tools-1.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Trivial
> Fix For: hbase-operator-tools-1.0.0
>
>
> The Checkstyle suppression file in the {{hbase-operator-tools}} is currently 
> defined in the {{dev-support}} directory. It should be moved to a different 
> location.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (HBASE-23021) [hbase-operator-tools] README edits in prep for release

[stack (Jira)]

stack created HBASE-23021:
-

 Summary: [hbase-operator-tools] README edits in prep for release
 Key: HBASE-23021
 URL: https://issues.apache.org/jira/browse/HBASE-23021
 Project: HBase
  Issue Type: Sub-task
Reporter: stack


Did an edit of the READMEs to get us ready for 1.0.0 release.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: Documenting how to do cluster operations when things have gone wrong

[Sean Busbey]

Hi folks!

Was chatting in the project slack earlier today with Stack and Clay about
how unwieldy the hbck2 README is getting.

Heads up that I'm going to try a POC where we have operator centric docs
(like the thing I described in this earlier email, and the stuff for hbck2)
on the hbase.a.o website.

Basics so far:

* Won't be in the ref guide
* Source as markdown
* Source in hbase-operator-tools repo
* Published on website as an "operations" menu item under our top level
documentation menu.

hmu on HBASE-23020 if you have ideas or want to help out.

On Thu, Sep 5, 2019, 23:40 Sean Busbey  wrote:

> Hi folks!
>
> I've had to work through the steps to do various cluster operations in
> the face of a table with a couple of million mob hfiles enough times
> lately that I took the time to write down both a) how to stand up an
> approximation elsewhere to get reproduction and timing information,
> and b) examples of step-by-step with how to track how things are
> going.
>
> To be clear, it's still a draft. It's also currently markdown. If it
> helps to see the specifics of what I'm talking about, it's here:
>
> https://gist.github.com/busbey/5ff88e31705e52a392392b4fb2eadac2
>
> I'd like to get this kind of stuff folded into our community docs
> somewhere. But I'm not sure where something like this would fit. I
> don't think it works for the current one-ref-guide-to-rule-them-all.
>
> What do y'all think?
>

[jira] [Created] (HBASE-23020) Operations docs for HBase website

[Sean Busbey (Jira)]

Sean Busbey created HBASE-23020:
---

 Summary: Operations docs for HBase website
 Key: HBASE-23020
 URL: https://issues.apache.org/jira/browse/HBASE-23020
 Project: HBase
  Issue Type: Umbrella
  Components: hbase-operator-tools, hbck2, Operability, website
Reporter: Sean Busbey
Assignee: Sean Busbey


Basics so far:

* Won't be in the ref guide
* Source as markdown
* Source in hbase-operator-tools repo
* Published on website as an "operations" menu item under our top level 
documentation menu.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [DISCUSS] Storing images in source repository

[Sean Busbey]

Sounds good to me.

Maybe a comment in the hbtop readme source with a pointer about how to
update the image content since folks might not know to look at hbase-site?

On Fri, Sep 13, 2019, 02:27 Peter Somogyi  wrote:

> Hi,
>
> Recently the hbtop tool (HBASE-11062) was merged and it also includes a
> nice README documentation with demos in GIF format.
>
> Although I think we should have more documentation like this, probably it
> is not a good idea to store the images in the source repository. The size
> of these GIF images is 31MB which is not huge, however, it will be included
> in the release tarballs from now on and every time we pull the repository
> for the builds it adds +31MB every time.
>
> My recommendation is to keep similar images in a different place, for
> example in the hbase-site repository and link the images from there.
>
> Thanks,
> Peter
>

[jira] [Resolved] (HBASE-23002) [HBCK2/hbase-operator-tools] Create an assembly that builds an hbase-operator-tools tgz

[stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-23002.
---
Fix Version/s: hbase-operator-tools-1.0.0
 Release Note: First cut at an assembly for hbase-operator-tools project.
 Assignee: stack
   Resolution: Fixed

Merged PR. Thanks for review [~psomogyi]

> [HBCK2/hbase-operator-tools] Create an assembly that builds an 
> hbase-operator-tools tgz
> ---
>
> Key: HBASE-23002
> URL: https://issues.apache.org/jira/browse/HBASE-23002
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: hbase-operator-tools-1.0.0
>
>
> Was going to build a convenience binary tgz as part of the first release of 
> hbase-operator-tools. Not sure how just yet; best would be if it were a 
> fatjar with all dependencies but that'd be kinda insane at same time since 
> the tgz would be massive.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (HBASE-23019) Handle --skip-errorprone on branch-1

[Peter Somogyi (Jira)]

Peter Somogyi created HBASE-23019:
-

 Summary: Handle --skip-errorprone on branch-1
 Key: HBASE-23019
 URL: https://issues.apache.org/jira/browse/HBASE-23019
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 1.5.0, 1.3.6, 1.4.11
Reporter: Peter Somogyi
Assignee: Peter Somogyi
 Fix For: 1.5.0, 1.3.6, 1.4.11


Precommit builds fail on branch-1 with "Unprocessed flag(s): --skip-errorprone".



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[DISCUSS] Storing images in source repository

[Peter Somogyi]

Hi,

Recently the hbtop tool (HBASE-11062) was merged and it also includes a
nice README documentation with demos in GIF format.

Although I think we should have more documentation like this, probably it
is not a good idea to store the images in the source repository. The size
of these GIF images is 31MB which is not huge, however, it will be included
in the release tarballs from now on and every time we pull the repository
for the builds it adds +31MB every time.

My recommendation is to keep similar images in a different place, for
example in the hbase-site repository and link the images from there.

Thanks,
Peter

[jira] [Resolved] (HBASE-23018) [HBCK2] Add useful messages when report/fixing missing regions in meta

[Peter Somogyi (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Somogyi resolved HBASE-23018.
---
Resolution: Fixed

> [HBCK2] Add useful messages when report/fixing missing regions in meta
> --
>
> Key: HBASE-23018
> URL: https://issues.apache.org/jira/browse/HBASE-23018
> Project: HBase
>  Issue Type: Improvement
>  Components: hbase-operator-tools, hbck2, Operability
>Affects Versions: hbase-operator-tools-1.0.0
>Reporter: Daisuke Kobayashi
>Assignee: Daisuke Kobayashi
>Priority: Minor
> Fix For: hbase-operator-tools-1.0.0
>
>
> When a table name is wrongly passed without the namespace portion, 
> *reportMissingRegionsInMeta* just reports:
> {code}
> Missing Regions for each table:
> {code}
> similarly, *addFsRegionsMissingInMeta* just reports:
> {code}
> Regions re-added into Meta: 0
> {code}
> Let' make it more friendly by letting the user know that the table name is 
> wrongly passed while it's mentioned in the command usage.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)