[jira] [Created] (HBASE-26593) [hbase-thirdparty] Set version as 4.0.1 in prep for first RC

2021-12-16 Thread Duo Zhang (Jira)
Duo Zhang created HBASE-26593:
-

 Summary: [hbase-thirdparty] Set version as 4.0.1 in prep for first 
RC
 Key: HBASE-26593
 URL: https://issues.apache.org/jira/browse/HBASE-26593
 Project: HBase
  Issue Type: Sub-task
  Components: build, pom, thirdparty
Reporter: Duo Zhang
 Fix For: thirdparty-4.0.1






--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Created] (HBASE-26594) [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 4.0.1

2021-12-16 Thread Duo Zhang (Jira)
Duo Zhang created HBASE-26594:
-

 Summary: [hbase-thirdparty] Generate CHANGES.md and 
RELEASENOTES.md for 4.0.1
 Key: HBASE-26594
 URL: https://issues.apache.org/jira/browse/HBASE-26594
 Project: HBase
  Issue Type: Sub-task
  Components: thirdparty
Reporter: Duo Zhang
 Fix For: thirdparty-4.0.1






--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Created] (HBASE-26595) Put up 4.0.1RC0

2021-12-16 Thread Duo Zhang (Jira)
Duo Zhang created HBASE-26595:
-

 Summary: Put up 4.0.1RC0
 Key: HBASE-26595
 URL: https://issues.apache.org/jira/browse/HBASE-26595
 Project: HBase
  Issue Type: Sub-task
Reporter: Duo Zhang






--
This message was sent by Atlassian Jira
(v8.20.1#820001)


Re: [VOTE] The second release condidate for hbase-thirdparty 4.0.0 is available for download

2021-12-16 Thread Duo Zhang
Filed HBASE-26592.

张铎(Duo Zhang)  于2021年12月17日周五 12:25写道:

> Oh, when backporting to branch-2, I found that there is a serious problem
> with this release.
>
> We do not include the proto files in protobuf-java jar, I think this is a
> side effect of HBASE-26496.
>
> And on the master branch, hbase-shaded-protobuf also pulls in
> protobuf-java 3.19.1 dependency so everything is fine. But for branch-2, we
> will need to depend on protobuf 2.5.0 then the compilation is broken.
>
> Anyway, we should include the proto files in hbase-shaded-protobuf,  and
> also we should not pull in the original protobuf-java dependency.
>
> Let me fix this and do a 4.0.1 release.
>
> 张铎(Duo Zhang)  于2021年12月10日周五 13:41写道:
>
>> With 4 binding +1s, no -1, the vote passes.
>>
>> Let me push out the release.
>>
>> Thanks all for voting!
>>
>> 张铎(Duo Zhang)  于2021年12月10日周五 13:40写道:
>>
>>> Here is my +1, I've been testing it by opening a PR against hbase master
>>> branch and it works.
>>>
>>> See https://github.com/apache/hbase/pull/3910
>>>
>>> Yu Li  于2021年12月10日周五 09:37写道:
>>>
 +1

 Checked the diff between 3.5.1 and 4.0.0-rc1: OK (
 https://github.com/apache/hbase-thirdparty/compare/rel/3.5.1...4.0.0RC1
 )
 Checked release note and changes: OK
 Checked sums and signatures: OK
 Maven clean install from source (1.8.0_121): OK
 - Minor: I tried to build a tarball from source following README but
 failed
 with "No assembly descriptors found" error
 Checked the jars in the staging repo: OK

 btw, I haven't followed up for a while and could anyone kindly let me
 know
 where to find this fancy hbase-vote.sh script, so next time I could also
 try it out? Thanks :-)

 Best Regards,
 Yu


 On Fri, 10 Dec 2021 at 05:46, Nick Dimiduk  wrote:

 > +1
 >
 > I've used the hbase-vote.sh script to evaluate this artifact and
 there's a
 > problem in the final `run_tests` , executed after `build_from_source`.
 >
 > * Signature: ok
 > * Checksum : ok
 > * Rat check (11.0.11): ok
 >  - mvn clean apache-rat:check
 > * Built from source (11.0.11): ok
 >  - mvn clean install  -DskipTests
 > * Unit tests pass (11.0.11): failed
 >  - mvn package -P runAllTests
 -Dsurefire.rerunFailingTestsCount=3
 >
 > [WARNING] The requested profile "runAllTests" could not be activated
 > because it does not exist.
 > [ERROR] Failed to execute goal
 > org.apache.maven.plugins:maven-shade-plugin:3.2.4:shade (default) on
 > project hbase-shaded-jackson-jaxrs-json-provider: Error creating
 shaded
 > jar: duplicate entry:
 > META-INF/services/org.apache.hbase.thirdparty.javax.ws.rs
 > .ext.MessageBodyWriter
 >
 > Manually running `mvn clean package` within the source tarball, we
 succeed.
 >
 > I have also triggered a PR build of HBASE-25864 / PR#3243 that uses
 this
 > RC. The tests are still running.
 >
 >
 https://ci-hadoop.apache.org/blue/organizations/jenkins/HBase%2FHBase-PreCommit-GitHub-PR/detail/PR-3243/5/pipeline/
 >
 > On Tue, Dec 7, 2021 at 6:02 PM 张铎(Duo Zhang) 
 > wrote:
 >
 > > Ah, Thanks Nick for explaining and thanks Andrew for testing.
 > >
 > > We still need one more +1 to close this vote.
 > >
 > > Andrew Purtell  于2021年12月7日周二 05:50写道:
 > >
 > > > Ok, change my vote to +1 (binding). The hbase-thirdparty build and
 > > > artifacts are good.
 > > >
 > > > > On Dec 6, 2021, at 1:18 PM, Nick Dimiduk 
 > wrote:
 > > > >
 > > > > On Mon, Dec 6, 2021 at 11:49 AM Andrew Purtell <
 apurt...@apache.org
 > >
 > > > wrote:
 > > > >
 > > > >> -1 (binding)
 > > > >>
 > > > >> Checked sums and signature, ok
 > > > >> RAT check passed, ok
 > > > >> Built from source, ok
 > > > >> Built HEAD of master (d9315fa043) with
 > > -Dhbase-thirdparty.version=4.0.0,
 > > > >> hbase-http module tests fail
 > > > >>
 > > > >
 > > > > Adoption of this dependency will require changes to master. I
 had
 > > posted
 > > > > necessary changes on https://github.com/apache/hbase/pull/3243
 and
 > Duo
 > > > did
 > > > > his own on https://github.com/apache/hbase/pull/3910.
 > > > >
 > > > > [ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 2, Time
 > > elapsed:
 > > > >> 2.29 s <<< FAILURE! - in
 org.apache.hadoop.hbase.http.TestHttpServer
 > > > >> [ERROR]
 org.apache.hadoop.hbase.http.TestHttpServer.testJersey  Time
 > > > >> elapsed: 0.123 s  <<< ERROR!
 > > > >> java.io.FileNotFoundException:
 > > http://localhost:55106/jersey/foo?op=bar
 > > > >> at
 > > > >>
 > > > >>
 > > >
 > >
 >
 

[jira] [Created] (HBASE-26592) Fix the broken shaded protobuf module

2021-12-16 Thread Duo Zhang (Jira)
Duo Zhang created HBASE-26592:
-

 Summary: Fix the broken shaded protobuf module
 Key: HBASE-26592
 URL: https://issues.apache.org/jira/browse/HBASE-26592
 Project: HBase
  Issue Type: Sub-task
  Components: hbase-thirdparty, Protobufs
Reporter: Duo Zhang
Assignee: Duo Zhang
 Fix For: thirdparty-4.0.1


First, we should include the proto files in the final uber jar.

Second, we should not pull in the protobuf-java dependency as we always include 
everything in the uber jar.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Created] (HBASE-26591) [hbase-thirdparty] Make a 4.0.1 release for hbase-thirdparty

2021-12-16 Thread Duo Zhang (Jira)
Duo Zhang created HBASE-26591:
-

 Summary: [hbase-thirdparty] Make a 4.0.1 release for 
hbase-thirdparty
 Key: HBASE-26591
 URL: https://issues.apache.org/jira/browse/HBASE-26591
 Project: HBase
  Issue Type: Umbrella
  Components: community, hbase-thirdparty
Reporter: Duo Zhang


Copy the content in mailing list here

{noformat}
Oh, when backporting to branch-2, I found that there is a serious problem with 
this release.

We do not include the proto files in protobuf-java jar, I think this is a side 
effect of HBASE-26496.

And on the master branch, hbase-shaded-protobuf also pulls in protobuf-java 
3.19.1 dependency so everything is fine. But for branch-2, we will need to 
depend on protobuf 2.5.0 then the compilation is broken.

Anyway, we should include the proto files in hbase-shaded-protobuf,  and also 
we should not pull in the original protobuf-java dependency.

Let me fix this and do a 4.0.1 release.
{noformat}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Resolved] (HBASE-26498) [hbase-thirdparty] Make a 4.0.0 release for hbase-thirdparty

2021-12-16 Thread Duo Zhang (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-26498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-26498.
---
Resolution: Fixed

> [hbase-thirdparty] Make a 4.0.0 release for hbase-thirdparty
> 
>
> Key: HBASE-26498
> URL: https://issues.apache.org/jira/browse/HBASE-26498
> Project: HBase
>  Issue Type: Umbrella
>  Components: community, hbase-thirdparty
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


Re: [VOTE] The second release condidate for hbase-thirdparty 4.0.0 is available for download

2021-12-16 Thread Duo Zhang
Oh, when backporting to branch-2, I found that there is a serious problem
with this release.

We do not include the proto files in protobuf-java jar, I think this is a
side effect of HBASE-26496.

And on the master branch, hbase-shaded-protobuf also pulls in protobuf-java
3.19.1 dependency so everything is fine. But for branch-2, we will need to
depend on protobuf 2.5.0 then the compilation is broken.

Anyway, we should include the proto files in hbase-shaded-protobuf,  and
also we should not pull in the original protobuf-java dependency.

Let me fix this and do a 4.0.1 release.

张铎(Duo Zhang)  于2021年12月10日周五 13:41写道:

> With 4 binding +1s, no -1, the vote passes.
>
> Let me push out the release.
>
> Thanks all for voting!
>
> 张铎(Duo Zhang)  于2021年12月10日周五 13:40写道:
>
>> Here is my +1, I've been testing it by opening a PR against hbase master
>> branch and it works.
>>
>> See https://github.com/apache/hbase/pull/3910
>>
>> Yu Li  于2021年12月10日周五 09:37写道:
>>
>>> +1
>>>
>>> Checked the diff between 3.5.1 and 4.0.0-rc1: OK (
>>> https://github.com/apache/hbase-thirdparty/compare/rel/3.5.1...4.0.0RC1)
>>> Checked release note and changes: OK
>>> Checked sums and signatures: OK
>>> Maven clean install from source (1.8.0_121): OK
>>> - Minor: I tried to build a tarball from source following README but
>>> failed
>>> with "No assembly descriptors found" error
>>> Checked the jars in the staging repo: OK
>>>
>>> btw, I haven't followed up for a while and could anyone kindly let me
>>> know
>>> where to find this fancy hbase-vote.sh script, so next time I could also
>>> try it out? Thanks :-)
>>>
>>> Best Regards,
>>> Yu
>>>
>>>
>>> On Fri, 10 Dec 2021 at 05:46, Nick Dimiduk  wrote:
>>>
>>> > +1
>>> >
>>> > I've used the hbase-vote.sh script to evaluate this artifact and
>>> there's a
>>> > problem in the final `run_tests` , executed after `build_from_source`.
>>> >
>>> > * Signature: ok
>>> > * Checksum : ok
>>> > * Rat check (11.0.11): ok
>>> >  - mvn clean apache-rat:check
>>> > * Built from source (11.0.11): ok
>>> >  - mvn clean install  -DskipTests
>>> > * Unit tests pass (11.0.11): failed
>>> >  - mvn package -P runAllTests
>>> -Dsurefire.rerunFailingTestsCount=3
>>> >
>>> > [WARNING] The requested profile "runAllTests" could not be activated
>>> > because it does not exist.
>>> > [ERROR] Failed to execute goal
>>> > org.apache.maven.plugins:maven-shade-plugin:3.2.4:shade (default) on
>>> > project hbase-shaded-jackson-jaxrs-json-provider: Error creating shaded
>>> > jar: duplicate entry:
>>> > META-INF/services/org.apache.hbase.thirdparty.javax.ws.rs
>>> > .ext.MessageBodyWriter
>>> >
>>> > Manually running `mvn clean package` within the source tarball, we
>>> succeed.
>>> >
>>> > I have also triggered a PR build of HBASE-25864 / PR#3243 that uses
>>> this
>>> > RC. The tests are still running.
>>> >
>>> >
>>> https://ci-hadoop.apache.org/blue/organizations/jenkins/HBase%2FHBase-PreCommit-GitHub-PR/detail/PR-3243/5/pipeline/
>>> >
>>> > On Tue, Dec 7, 2021 at 6:02 PM 张铎(Duo Zhang) 
>>> > wrote:
>>> >
>>> > > Ah, Thanks Nick for explaining and thanks Andrew for testing.
>>> > >
>>> > > We still need one more +1 to close this vote.
>>> > >
>>> > > Andrew Purtell  于2021年12月7日周二 05:50写道:
>>> > >
>>> > > > Ok, change my vote to +1 (binding). The hbase-thirdparty build and
>>> > > > artifacts are good.
>>> > > >
>>> > > > > On Dec 6, 2021, at 1:18 PM, Nick Dimiduk 
>>> > wrote:
>>> > > > >
>>> > > > > On Mon, Dec 6, 2021 at 11:49 AM Andrew Purtell <
>>> apurt...@apache.org
>>> > >
>>> > > > wrote:
>>> > > > >
>>> > > > >> -1 (binding)
>>> > > > >>
>>> > > > >> Checked sums and signature, ok
>>> > > > >> RAT check passed, ok
>>> > > > >> Built from source, ok
>>> > > > >> Built HEAD of master (d9315fa043) with
>>> > > -Dhbase-thirdparty.version=4.0.0,
>>> > > > >> hbase-http module tests fail
>>> > > > >>
>>> > > > >
>>> > > > > Adoption of this dependency will require changes to master. I had
>>> > > posted
>>> > > > > necessary changes on https://github.com/apache/hbase/pull/3243
>>> and
>>> > Duo
>>> > > > did
>>> > > > > his own on https://github.com/apache/hbase/pull/3910.
>>> > > > >
>>> > > > > [ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 2, Time
>>> > > elapsed:
>>> > > > >> 2.29 s <<< FAILURE! - in
>>> org.apache.hadoop.hbase.http.TestHttpServer
>>> > > > >> [ERROR] org.apache.hadoop.hbase.http.TestHttpServer.testJersey
>>> Time
>>> > > > >> elapsed: 0.123 s  <<< ERROR!
>>> > > > >> java.io.FileNotFoundException:
>>> > > http://localhost:55106/jersey/foo?op=bar
>>> > > > >> at
>>> > > > >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1898)
>>> > > > >> at
>>> > > > >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:92)
>>> > > > >> at
>>> > > > >>
>>> > > > >>
>>> > > >
>>> > >
>>> >

[jira] [Resolved] (HBASE-26542) Apply a `package` to test protobuf files

2021-12-16 Thread Nick Dimiduk (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-26542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk resolved HBASE-26542.
--
Resolution: Fixed

> Apply a `package` to test protobuf files
> 
>
> Key: HBASE-26542
> URL: https://issues.apache.org/jira/browse/HBASE-26542
> Project: HBase
>  Issue Type: Test
>  Components: Protobufs, test
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 2.5.0, 2.4.9, 2.6.0, 3.0.0-alpha-3
>
>
> This is needed in a couple places in order to test that traces over the IPC 
> layer carry correct span names, and it's good hygiene anyway.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


Re: [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2

2021-12-16 Thread Ankit Singhal
+1

On Thu, Dec 16, 2021 at 1:53 PM Andor Molnar  wrote:

> +1 (non-binding)
>
> Andor
>
>
>
> > On 2021. Dec 16., at 22:30, Josh Elser  wrote:
> >
> > Hi!
> >
> > I'm extremely pleased to send this official vote to merge the feature
> branch for HBASE-26067[1] into master and backport into branch-2 (2.x, not
> 2.5.x). This feature branch introduces the pluggable StoreFileTracker
> interface.
> >
> > The StoreFileTracker allows the StoreFileEngine to be decoupled from
> where the HFiles can be found. The DEFAULT implementation of this
> StoreFileTracker (e.g. files in a family's directory) is still the default
> implementation. This merge would introduce a FILE implementation which uses
> a flat-file in each column family to track the files which make up this
> Store. This feature is notable for HBase as it invalidate the need for
> HBOSS (a distributed locking layer in hbase-filesystem) when Amazon S3 is
> used for HBase data.
> >
> > We had a DISCUSS thread [2] in which the overall sentiment was positive
> to merge.
> >
> > Covering some high-level details/FAQ on this work:
> > * Wellington and Szabolcs have successfully run ITBLL with Chaos Monkies
> using this feature.
> > * YCSB (load) indicates a slight performance improvement when using S3
> as the storage system for HBase as compared to using HBOSS [3]
> > * A new section was added to the HBase book which covers the feature and
> how to use it.
> > * There is some follow-on work expected, tracked in HBASE-26584 [4],
> which includes things like making user consumption easier and additional
> metrics to measure effectiveness of the feature.
> >
> > As is customary, this vote will be open for at least 3 days (2021/12/19
> 2130 GMT). We'll follow the standard ASF lazy-consensus rules for code
> modification (though I do not expect to need the lazy-consensus caveat).
> Please vote:
> >
> > +1: Merge the changes from HBASE-26067 to master and branch-2
> > -1: Do not merge these changes because ...
> >
> > Big thank you to all of the super hard work that Duo, Wellington, and
> Szabolcs have put into this feature.
> >
> > - Josh
> >
> > [1] https://issues.apache.org/jira/browse/HBASE-26067
> > [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1
> > [3]
> https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499
> > [4] https://issues.apache.org/jira/browse/HBASE-26584
>
>


Re: [VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2

2021-12-16 Thread Andor Molnar
+1 (non-binding)

Andor



> On 2021. Dec 16., at 22:30, Josh Elser  wrote:
> 
> Hi!
> 
> I'm extremely pleased to send this official vote to merge the feature branch 
> for HBASE-26067[1] into master and backport into branch-2 (2.x, not 2.5.x). 
> This feature branch introduces the pluggable StoreFileTracker interface.
> 
> The StoreFileTracker allows the StoreFileEngine to be decoupled from where 
> the HFiles can be found. The DEFAULT implementation of this StoreFileTracker 
> (e.g. files in a family's directory) is still the default implementation. 
> This merge would introduce a FILE implementation which uses a flat-file in 
> each column family to track the files which make up this Store. This feature 
> is notable for HBase as it invalidate the need for HBOSS (a distributed 
> locking layer in hbase-filesystem) when Amazon S3 is used for HBase data.
> 
> We had a DISCUSS thread [2] in which the overall sentiment was positive to 
> merge.
> 
> Covering some high-level details/FAQ on this work:
> * Wellington and Szabolcs have successfully run ITBLL with Chaos Monkies 
> using this feature.
> * YCSB (load) indicates a slight performance improvement when using S3 as the 
> storage system for HBase as compared to using HBOSS [3]
> * A new section was added to the HBase book which covers the feature and how 
> to use it.
> * There is some follow-on work expected, tracked in HBASE-26584 [4], which 
> includes things like making user consumption easier and additional metrics to 
> measure effectiveness of the feature.
> 
> As is customary, this vote will be open for at least 3 days (2021/12/19 2130 
> GMT). We'll follow the standard ASF lazy-consensus rules for code 
> modification (though I do not expect to need the lazy-consensus caveat). 
> Please vote:
> 
> +1: Merge the changes from HBASE-26067 to master and branch-2
> -1: Do not merge these changes because ...
> 
> Big thank you to all of the super hard work that Duo, Wellington, and 
> Szabolcs have put into this feature.
> 
> - Josh
> 
> [1] https://issues.apache.org/jira/browse/HBASE-26067
> [2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1
> [3] 
> https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499
> [4] https://issues.apache.org/jira/browse/HBASE-26584



[VOTE] Merge HBASE-26067 (storefile tracking) into master and branch-2

2021-12-16 Thread Josh Elser

Hi!

I'm extremely pleased to send this official vote to merge the feature 
branch for HBASE-26067[1] into master and backport into branch-2 (2.x, 
not 2.5.x). This feature branch introduces the pluggable 
StoreFileTracker interface.


The StoreFileTracker allows the StoreFileEngine to be decoupled from 
where the HFiles can be found. The DEFAULT implementation of this 
StoreFileTracker (e.g. files in a family's directory) is still the 
default implementation. This merge would introduce a FILE implementation 
which uses a flat-file in each column family to track the files which 
make up this Store. This feature is notable for HBase as it invalidate 
the need for HBOSS (a distributed locking layer in hbase-filesystem) 
when Amazon S3 is used for HBase data.


We had a DISCUSS thread [2] in which the overall sentiment was positive 
to merge.


Covering some high-level details/FAQ on this work:
* Wellington and Szabolcs have successfully run ITBLL with Chaos Monkies 
using this feature.
* YCSB (load) indicates a slight performance improvement when using S3 
as the storage system for HBase as compared to using HBOSS [3]
* A new section was added to the HBase book which covers the feature and 
how to use it.
* There is some follow-on work expected, tracked in HBASE-26584 [4], 
which includes things like making user consumption easier and additional 
metrics to measure effectiveness of the feature.


As is customary, this vote will be open for at least 3 days (2021/12/19 
2130 GMT). We'll follow the standard ASF lazy-consensus rules for code 
modification (though I do not expect to need the lazy-consensus caveat). 
Please vote:


+1: Merge the changes from HBASE-26067 to master and branch-2
-1: Do not merge these changes because ...

Big thank you to all of the super hard work that Duo, Wellington, and 
Szabolcs have put into this feature.


- Josh

[1] https://issues.apache.org/jira/browse/HBASE-26067
[2] https://lists.apache.org/thread/6dblom3tc2oz05d263pvmrywlthqq1c1
[3] 
https://issues.apache.org/jira/browse/HBASE-26067?focusedCommentId=17448499=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17448499

[4] https://issues.apache.org/jira/browse/HBASE-26584


[jira] [Resolved] (HBASE-26265) Update ref guide to mention the new store file tracker implementations

2021-12-16 Thread Josh Elser (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-26265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser resolved HBASE-26265.

Resolution: Fixed

> Update ref guide to mention the new store file tracker implementations
> --
>
> Key: HBASE-26265
> URL: https://issues.apache.org/jira/browse/HBASE-26265
> Project: HBase
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Duo Zhang
>Assignee: Wellington Chevreuil
>Priority: Major
> Fix For: HBASE-26067
>
>
> For example, when to use these store file trackers.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Created] (HBASE-26590) Hbase-client Meta lookup performance regression between hbase-1 and hbase-2

2021-12-16 Thread Huaxiang Sun (Jira)
Huaxiang Sun created HBASE-26590:


 Summary: Hbase-client Meta lookup performance regression between 
hbase-1 and hbase-2
 Key: HBASE-26590
 URL: https://issues.apache.org/jira/browse/HBASE-26590
 Project: HBase
  Issue Type: Improvement
  Components: meta
Affects Versions: 2.3.7, 3.0.0-alpha-1
 Environment: ||Version ||Meta Replica Load Balance Enabled||Time       
        ||
||2.4.5-with-fixed||Yes||336458ms||
||2.4.5-with-fixed||No||333253ms||
||2.4.5||Yes||469980ms||
||2.4.5||No||470515ms||
|      *cdh-5.16.2*|                                *No* |  *323412ms*|
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


One of our users complained higher latency after application upgrades from 
hbase-1.2 client (CDH-5.16.2) to hbase-2.4.5 client with meta replica Load 
Balance mode during app restart. I reproduced the regression by a test for meta 
lookup. 

At my test cluster, there are 160k regions for the test table, so there are 
160k entries in meta region. Used one thread to do 1 million meta lookup 
against the meta region server.

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Resolved] (HBASE-26472) Adhere to semantic conventions regarding table data operations

2021-12-16 Thread Nick Dimiduk (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-26472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk resolved HBASE-26472.
--
Resolution: Fixed

The problem is actually HBASE-26542.

> Adhere to semantic conventions regarding table data operations
> --
>
> Key: HBASE-26472
> URL: https://issues.apache.org/jira/browse/HBASE-26472
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> Follow the guidance outlined in 
> https://github.com/open-telemetry/opentelemetry-specification/blob/3e380e2/specification/trace/semantic_conventions/database.dm



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Reopened] (HBASE-26542) Apply a `package` to test protobuf files

2021-12-16 Thread Nick Dimiduk (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-26542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk reopened HBASE-26542:
--

This breaks an RPC test. Reopening for addendum.

> Apply a `package` to test protobuf files
> 
>
> Key: HBASE-26542
> URL: https://issues.apache.org/jira/browse/HBASE-26542
> Project: HBase
>  Issue Type: Test
>  Components: Protobufs, test
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 2.5.0, 2.4.9, 2.6.0, 3.0.0-alpha-3
>
>
> This is needed in a couple places in order to test that traces over the IPC 
> layer carry correct span names, and it's good hygiene anyway.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Created] (HBASE-26589) Protobuf Services should be resolved by their full name

2021-12-16 Thread Nick Dimiduk (Jira)
Nick Dimiduk created HBASE-26589:


 Summary: Protobuf Services should be resolved by their full name
 Key: HBASE-26589
 URL: https://issues.apache.org/jira/browse/HBASE-26589
 Project: HBase
  Issue Type: Task
  Components: IPC/RPC
Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.6.0
Reporter: Nick Dimiduk


I noticed this while investigating test failures resulting from HBASE-26472. 
The test in question ({{TestRpcServerSlowConnectionSetup}}) builds a connection 
header by populating the {{serviceName}} field with the service descriptor's 
{{getFullName()}} method. The value returned by {{getFullName()}} is the 
protobuf service's package and name. However, down in {{RpcServer}}, we resolve 
the service named by the connection header using the service descriptor's 
{{getName()}} method. {{getName()}} returns the protobuf service's name, 
without the package.

It is my opinion that this is a bug in the {{RpcServer}}, it should resolve 
services using their fully qualified name.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Reopened] (HBASE-26472) Adhere to semantic conventions regarding table data operations

2021-12-16 Thread Nick Dimiduk (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-26472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk reopened HBASE-26472:
--

Broken test on branch-2.5 looks related. Investigating.

> Adhere to semantic conventions regarding table data operations
> --
>
> Key: HBASE-26472
> URL: https://issues.apache.org/jira/browse/HBASE-26472
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> Follow the guidance outlined in 
> https://github.com/open-telemetry/opentelemetry-specification/blob/3e380e2/specification/trace/semantic_conventions/database.dm



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Created] (HBASE-26588) Implement a migration tool to help users migrate SFT implementation for a large set of tables

2021-12-16 Thread Duo Zhang (Jira)
Duo Zhang created HBASE-26588:
-

 Summary: Implement a migration tool to help users migrate SFT 
implementation for a large set of tables
 Key: HBASE-26588
 URL: https://issues.apache.org/jira/browse/HBASE-26588
 Project: HBase
  Issue Type: Sub-task
  Components: tooling
Reporter: Duo Zhang


It will be very useful for our users who deploy HBase on S3 like systems.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Created] (HBASE-26587) Introduce a new Admin API to change SFT implementation

2021-12-16 Thread Duo Zhang (Jira)
Duo Zhang created HBASE-26587:
-

 Summary: Introduce a new Admin API to change SFT implementation
 Key: HBASE-26587
 URL: https://issues.apache.org/jira/browse/HBASE-26587
 Project: HBase
  Issue Type: Sub-task
  Components: Admin, API
Reporter: Duo Zhang


Now changing SFT implementation requires two MTP, and there are also lots of 
rules on how to correctly set the MIGRATION SFT, I think we'd better introduce 
a special method to implement the complicated logic by code,to make our users 
life easier.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Created] (HBASE-26586) The MigrateStoreFileTrackerProcedure should not rely on the global config when setting SFT implementation for a table

2021-12-16 Thread Duo Zhang (Jira)
Duo Zhang created HBASE-26586:
-

 Summary: The MigrateStoreFileTrackerProcedure should not rely on 
the global config when setting SFT implementation for a table
 Key: HBASE-26586
 URL: https://issues.apache.org/jira/browse/HBASE-26586
 Project: HBase
  Issue Type: Sub-task
  Components: HFile
Reporter: Duo Zhang


If a table does not have a SFT implementation in its table descriptor, it means 
we are upgrading from an older version where SFT is not implemented, so the SFT 
implementation for the table should always be DEFAULT.

A misconfigure here could cause data loss.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Created] (HBASE-26585) Add SFT configuration to META table descriptor when creating META

2021-12-16 Thread Wellington Chevreuil (Jira)
Wellington Chevreuil created HBASE-26585:


 Summary: Add SFT configuration to META table descriptor when 
creating META
 Key: HBASE-26585
 URL: https://issues.apache.org/jira/browse/HBASE-26585
 Project: HBase
  Issue Type: Sub-task
Reporter: Wellington Chevreuil
Assignee: Wellington Chevreuil


META doesn't get created via CreateTableProcedure, but rather during master 
initialisation on cluster's first run, so changes implemented on HBASE-26246 
don't take effect for meta table.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)