??????[ANNOUNCE] New HBase committer Baiqiang Zhao

[zheng wang]

Congratulations~




--  --
??: 
   "user"   
 

??????[ANNOUNCE] New HBase Committer Xiaolin Ha(??????)

[zheng wang]

Congratulations~


--  --
??: 
   "user"   
 

??????[ANNOUNCE] New HBase committer Geoffrey Jacoby

[zheng wang]

  Congratulations!




--  --
??: 
   "user"   
 

??????[ANNOUNCE] New HBase PMC Huaxiang Sun

[zheng wang]

Congratulations!




--  --
??: 
   "user"   
 

回复： EOL branch-1 and all 1.x ?

[zheng wang]

+1 on EOL.




-- 原始邮件 --
发件人:
"user"  
  

?????? [ANNOUNCE] New HBase committer Yulin Niu

[zheng wang]

Congratulations!




--  --
??: 
   "user"   
 

?????? [ANNOUNCE] New HBase committer Xin Sun

[zheng wang]

Congratulations!




--  --
??: 
   "user"   
 

Re: [ANNOUNCE] Please welcome Viraj Jasani to the Apache HBase PMC

[zheng wang]

Congratulations Viraj!




---Original---
From: "Andrew Purtell"

??????[ANNOUNCE] New HBase Committer Zheng Wang(????)

[zheng wang]

Thanks all. Will continue to contribute~




--  --
??: 
   "user"   
 

?????? HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy

[zheng wang]

And getMethod does.




--  --
??: 
   "user"   
 
https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533&gt
> ;.
> I am using hadoop 3.0.0 and in FilterFileSystem (which LocalFileSystem
> extends from) I do see the method  setStoragePolicy
> <
> 
https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637&gt
> ;
> .
>
> Can someone explain how is this NoSuchMethodException is being thrown or I
> am looking at the wrong code path for LocalFileSystem?
>
> On Tue, Jul 21, 2020 at 7:04 PM Sean Busbey https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt
> &gt
> 
<https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt&gt>;;
> &gt; &gt; ;
> &gt; &gt; &gt; code it appears even if no storage policy is 
specified it
> will take
> &gt; HOT.
> &gt; &gt; &gt;
> &gt; &gt; &gt; Can you explain this a bit more how can I get 
around this
> error or in a
> &gt; &gt; &gt; single node hbase cluster I should be ignoring 
this?
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt; On Tue, Jul 21, 2020 at 3:03 PM zheng wang 
<18031...@qq.com&amp;gt;
> wrote:
> &gt; &gt; &gt;
> &gt; &gt; &gt; &amp;gt; LocalFileSystem? 
The&amp;amp;nbsp;setStoragePolicy
> could only be used
> &gt; in
> &gt; &gt; &gt; &amp;gt; distributed hdfs.
> &gt; &gt; &gt; &amp;gt; &amp;amp;nbsp;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> 
--&amp;amp;nbsp;&amp;amp;nbsp;--
> &gt; &gt; &gt; &amp;gt; ??:
> &gt; &gt; &gt;
> &gt;
> 
&amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> &gt; &gt; &gt; "user"
> &gt; &gt; &gt;
> &gt;
> 
&amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> &gt; &gt; &gt; <
> &gt; &gt; &gt; &amp;gt; 
subharaj.ma...@gmail.com&amp;amp;gt;;
> &gt; &gt; &gt; &amp;gt; 
:&amp;amp;nbsp;2020??7??21??(??) 5:58
> &gt; &gt; &gt; &amp;gt; 
??:&amp;amp;nbsp;"Hbase-User"<
> user@hbase.apache.org&amp;amp;gt;;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; :&amp;amp;nbsp;HBase 2.1.0 
-
> NoSuchMethodException
> &gt; &gt; &gt; &amp;gt;
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &a

?????? HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy

[zheng wang]

Seems getDeclaredMethod does not include the methods that extends from 
parent.




--  --
??: 
   "user"   
 
https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533&gt
> ;.
> I am using hadoop 3.0.0 and in FilterFileSystem (which LocalFileSystem
> extends from) I do see the method  setStoragePolicy
> <
> 
https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637&gt
> ;
> .
>
> Can someone explain how is this NoSuchMethodException is being thrown or I
> am looking at the wrong code path for LocalFileSystem?
>
> On Tue, Jul 21, 2020 at 7:04 PM Sean Busbey https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt
> &gt
> 
<https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt&gt>;;
> &gt; &gt; ;
> &gt; &gt; &gt; code it appears even if no storage policy is 
specified it
> will take
> &gt; HOT.
> &gt; &gt; &gt;
> &gt; &gt; &gt; Can you explain this a bit more how can I get 
around this
> error or in a
> &gt; &gt; &gt; single node hbase cluster I should be ignoring 
this?
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt; On Tue, Jul 21, 2020 at 3:03 PM zheng wang 
<18031...@qq.com&amp;gt;
> wrote:
> &gt; &gt; &gt;
> &gt; &gt; &gt; &amp;gt; LocalFileSystem? 
The&amp;amp;nbsp;setStoragePolicy
> could only be used
> &gt; in
> &gt; &gt; &gt; &amp;gt; distributed hdfs.
> &gt; &gt; &gt; &amp;gt; &amp;amp;nbsp;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> 
--&amp;amp;nbsp;&amp;amp;nbsp;--
> &gt; &gt; &gt; &amp;gt; ??:
> &gt; &gt; &gt;
> &gt;
> 
&amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> &gt; &gt; &gt; "user"
> &gt; &gt; &gt;
> &gt;
> 
&amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> &gt; &gt; &gt; <
> &gt; &gt; &gt; &amp;gt; 
subharaj.ma...@gmail.com&amp;amp;gt;;
> &gt; &gt; &gt; &amp;gt; 
:&amp;amp;nbsp;2020??7??21??(??) 5:58
> &gt; &gt; &gt; &amp;gt; 
??:&amp;amp;nbsp;"Hbase-User"<
> user@hbase.apache.org&amp;amp;gt;;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; :&amp;amp;nbsp;HBase 2.1.0 
-
> NoSuchMethodException
> &gt; &gt; &gt; &amp;gt;
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &

?????? HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy

[zheng wang]

Are you sure you are using hadoop3.0.0?




--  --
??: 
   "user"   
 
https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533>;.
I am using hadoop 3.0.0 and in FilterFileSystem (which LocalFileSystem
extends from) I do see the method  setStoragePolicy
<https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637>;
.

Can someone explain how is this NoSuchMethodException is being thrown or I
am looking at the wrong code path for LocalFileSystem?

On Tue, Jul 21, 2020 at 7:04 PM Sean Busbey https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&gt
> > > ;
> > > code it appears even if no storage policy is specified it will 
take
> HOT.
> > >
> > > Can you explain this a bit more how can I get around this error 
or in a
> > > single node hbase cluster I should be ignoring this?
> > >
> > >
> > > On Tue, Jul 21, 2020 at 3:03 PM zheng wang 
<18031...@qq.com&gt; wrote:
> > >
> > > &gt; LocalFileSystem? The&amp;nbsp;setStoragePolicy 
could only be used
> in
> > > &gt; distributed hdfs.
> > > &gt; &amp;nbsp;
> > > &gt;
> > > &gt;
> > > &gt; 
--&amp;nbsp;&amp;nbsp;--
> > > &gt; ??:
> > >
> 
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > > "user"
> > >
> 
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > > <
> > > &gt; subharaj.ma...@gmail.com&amp;gt;;
> > > &gt; :&amp;nbsp;2020??7??21??(??) 5:58
> > > &gt; 
??:&amp;nbsp;"Hbase-User"

?????? HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy

[zheng wang]

This log info just as a warning that cant make it disappear for now, but will 
not impact anything, so you can just ignore it in local mode.


--  --
??: 
   "user"   
 
https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274>;
code it appears even if no storage policy is specified it will take HOT.

Can you explain this a bit more how can I get around this error or in a
single node hbase cluster I should be ignoring this?


On Tue, Jul 21, 2020 at 3:03 PM zheng wang <18031...@qq.com> wrote:

> LocalFileSystem? The&nbsp;setStoragePolicy could only be used in
> distributed hdfs.
> &nbsp;
>
>
> --&nbsp;&nbsp;--
> ??:
>  
 "user"
>
 <
> subharaj.ma...@gmail.com&gt;;
> :&nbsp;2020??7??21??(??) 5:58
> ??:&nbsp;"Hbase-User"

??????HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy

[zheng wang]

LocalFileSystem? The setStoragePolicy could only be used in distributed 
hdfs.
 


--  --
??: 
   "user"   
 

??????Could not iterate StoreFileScanner - during compaction

[zheng wang]

Hi,
"cur=10259783_10101578851/hb:B/1490097148981/Put/vlen=16591695"
"Invalid onDisksize=-969694035: expected to be at least 33 and at most 
2147483647, or -1"


I guess there is a very big cell cause the block size exceed 
the Integer.MAX_VALUE, and lead to overflow error.





--  --
??: "Mohamed Meeran"

?????? [DISCUSS] Removing problematic terms from our project

[zheng wang]

I like the controller.


Coordinator is a bit long for me to write and speak.
Manager and Admin is used somewhere yet in HBase.




--  --
??: "Andrew Purtell"https://www.merriam-webster.com/dictionary/master
> > > > > > > > > > 
 >;
> > > for
> > > > > > > > > > examples. In particular, 
the progression of an artisan
> > was
> > > > > from
> > > > > > > > > > "apprentice" to 
"journeyman" to "master". A master
> > smith,
> > > > > > > carpenter,
> > > > > > > > > or
> > > > > > > > > > artist would run a shop 
managing lots of workers and
> > > > > apprentices
> > > > > > > who
> > > > > > > > > would
> > > > > > > > > > hope to become masters of 
their own someday. So
> "master"
> > > and
> > > > > > > > "worker"
> > > > > > > > > can
> > > > > > > > > > still go together.
> > > > > > > > > >
> > > > > > > > > > Since it's the least 
problematic term, and by far the
> > > > hardest
> > > > > > term
> > > > > > > > to
> > > > > > > > > > change (both within HBase 
and with effects on
> downstream
> > > > > > projects
> > > > > > > > > such as
> > > > > > > > > > Ambari), I'm -0 
(nonbinding) on changing "master".
> > > > > > > > > >
> > > > > > > > > > Geoffrey
> > > > > > > > > >
> > > > > > > > > > On Mon, Jun 22, 2020 at 
1:32 PM Rushabh Shah
> > > > > > > > > > 
https://issues.apache.org/jira/browse/HBASE-12677
> > > > > > > > > > > >:
> > > > > > > > > > > > 
>>     Update
> > > > replication
> > > > > > docs
> > > > > > > > to
> > > > > > > > > clarify terminology
> > > > > > > > > > > > 
>>     -
> > HBASE-13852 <
> > > > > > > > > > 
https://issues.apache.org/jira/browse/HBASE-13852
> > > > > > > > > > > >:
> > > > > > > > > > > > 
>>     Replace
> > > > > master-slave
> > > > > > > > > terminology in book, site, and 
javadoc
> > > > > > > > > > with a
> > > > > > > > > > > > more
> > > > > > > > > > > > 
>>     modern
> > > vocabulary
> > > > > > > > > > > > 
>>     -
> > HBASE-24576 <
> > > > > > > > > > 
https://issues.apache.org/jira/browse/HBASE-24576
> > > > > > > > > > > >:
> > > > > > > > > > > > 
>>     Changing
> > > > > "whitelist"
> > > > > > > and
> > > > > > > > > "blacklist" in our docs and project
> > > > > > > > > > > > 
>>
> > > > > > > > > > > > 
>> In response to this proposal, a
> > member
> > > of
> > > > > the
> > > > > > > PMC
> > > > > > > > > asked if the term
> > > > > > > > > > > > 
>> 'master' used by itself would be
> > fine,
> > > > > > because
> > > > > > > we
> > > > > > > > > only have use of
> > > > > > > > > > > > 'slave'
> > > > > > > > > > > > 
>> in replication documentation and
> that
> > > is
> > > > > > easily
> > > > > > > > > addressed. In
> > > > > > > > > > response
> > > > > > > > > > > > to
> > > > > > > > > > > > 
>> this question, others on the PMC
> > > > suggested
> > > > > > that
> > > > > > > > > even if only
> > > > > > > > > > 'master'
> > > > > > > > > > > is
> > > > > > > > > > > > 
>> used, in this context it is still a
> > > > > problem.
> > > > > > > > > > > > 
>>
> > > > > > > > > > > > 
>> For folks who are surprised or
> > lacking
> > > > > > context
> > > > > > > on
> > > > > > > > > the details of
> > > > > > > > > > this
> > > > > > > > > > > > 
>> discussion, one PMC member offered
> a
> > > link
> > > > > to
> > > > > > > this
> > > > > > > > > draft RFC as
> > > > > > > > > > > > 
background:
> > > > > > > > > > > > 
>>
> > > > > > > > > 
https://tools.ietf.org/id/draft-knodel-terminology-00.html
> > > > > > > > > > > > 
>>
> > > > > > > > > > > > 
>> There was general support for
> > removing
> > > > the
> > > > > > term
> > > > > > > > > "master" / "hmaster"
> > > > > > > > > > > > from
> > > > > > > > > > > > 
>> our code base and using the terms
> > > > > > "coordinator"
> > > > > > > > or
> > > > > > > > > "leader" instead.
> > > > > > > > > > > In
> > > > > > > > > > > > the
> > > > > > > > > > > > 
>> context of replication, "worker"
> > makes
> > > > less
> > > > > > > sense
> > > > > > > > > and perhaps
> > > > > > > > > > > > 
"destination"
> > > > > > > > > > > > 
>> or "follower" would be more
> > appropriate
> > > > > > terms.
> > > > > > > > > > > > 
>>
> > > > > > > > > > > > 
>> One PMC member's thoughts on
> language
> > > and
> > > > > > > > > non-native English
> > > > > > > > > > speakers
> >

?????? [DISCUSS] Removing problematic terms from our project

[zheng wang]

IMO, master is ok if not used with slave together.


-1/+1/+1/+1


--  --
??: "Andrew Purtell"https://www.merriam-webster.com/dictionary/master
> for
> examples. In particular, the progression of an artisan was from
> "apprentice" to "journeyman" to "master". A master smith, carpenter, or
> artist would run a shop managing lots of workers and apprentices who would
> hope to become masters of their own someday. So "master" and "worker" can
> still go together.
>
> Since it's the least problematic term, and by far the hardest term to
> change (both within HBase and with effects on downstream projects such as
> Ambari), I'm -0 (nonbinding) on changing "master".
>
> Geoffrey
>
> On Mon, Jun 22, 2020 at 1:32 PM Rushabh Shah
> https://issues.apache.org/jira/browse/HBASE-12677
> > >:
> > > >> Update replication docs to 
clarify terminology
> > > >> - HBASE-13852 <
> https://issues.apache.org/jira/browse/HBASE-13852
> > >:
> > > >> Replace master-slave 
terminology in book, site, and javadoc
> with a
> > > more
> > > >> modern vocabulary
> > > >> - HBASE-24576 <
> https://issues.apache.org/jira/browse/HBASE-24576
> > >:
> > > >> Changing "whitelist" and 
"blacklist" in our docs and project
> > > >>
> > > >> In response to this proposal, a member of the PMC asked 
if the term
> > > >> 'master' used by itself would be fine, because we only 
have use of
> > > 'slave'
> > > >> in replication documentation and that is easily 
addressed. In
> response
> > > to
> > > >> this question, others on the PMC suggested that even if 
only
> 'master'
> > is
> > > >> used, in this context it is still a problem.
> > > >>
> > > >> For folks who are surprised or lacking context on the 
details of
> this
> > > >> discussion, one PMC member offered a link to this draft 
RFC as
> > > background:
> > > >> 
https://tools.ietf.org/id/draft-knodel-terminology-00.html
> > > >>
> > > >> There was general support for removing the term 
"master" / "hmaster"
> > > from
> > > >> our code base and using the terms "coordinator" or 
"leader" instead.
> > In
> > > the
> > > >> context of replication, "worker" makes less sense and 
perhaps
> > > "destination"
> > > >> or "follower" would be more appropriate terms.
> > > >>
> > > >> One PMC member's thoughts on language and non-native 
English
> speakers
> > is
> > > >> worth including in its entirety:
> > > >>
> > > >> While words like blacklist/whitelist/slave clearly have 
those
> negative
> > > >> references, word master might not have the same impact 
for non
> native
> > > >> English speakers like myself where the literal 
translation to my
> > mother
> > > >> tongue does not have this same bad connotation. 
Replacing all
> > references
> > > >> for word *master *on our docs/codebase is a huge 
effort, I guess
> such
> > a
> > > >> decision would be more suitable for native English 
speakers folks,
> and
> > > >> maybe we should consider the opinion of contributors 
from that
> ethinic
> > > >> minority as well?
> > > >>
> > > >> These are good questions for public discussion.
> > > >>
> > > >> We have a consensus in the PMC, at this time, that is 
supportive of
> > > making
> > > >> the above discussed terminology changes. However, we 
also have
> > concerns
> > > >> about what it would take to accomplish meaningful 
changes. Several
> on
> > > the
> > > >> PMC offered support in the form of cycles to review 
pull requests
> and
> > > >> patches, and two PMC members offered  personal 
bandwidth for
> creating
> > > and
> > > >> releasing new code lines as needed to complete a 
deprecation cycle.
> > > >>
> > > >> Unfortunately, the terms "master" and "hmaster" appear 
throughout
> our
> > > code
> > > >> base in class names, user facing API subject to our 
project
> > > compatibility
> > > >> guidelines, and configuration variable names, which are 
also
> > implicated
> > > by
> > > >> compatibility guidelines given the impact of changes to 
operators
> and
> > > >> operations. The changes being discussed are not 
backwards compatible
> > > >> changes and cannot be executed with swiftness while 
simultaneously
> > > >> preserving compatibility. There must be a deprecation 
cycle. First,
> we
> > > must
> > > >> tag all implicated public API and configuration 
variables as
> > deprecated,
> > > >> and release HBase 3 with these deprecations in place. 
Then, we must
> > > >> undertake rename and removal as appropriate, and 
release the result
> as
> > > >> HBase 4.
> > > >>
> > > >> One PMC member raised a question in this context 
included here in
> > > entirety:
> > > >>
> > > >> Are we willing to commit to rolling through the major 
versions at a
> > pace
> > > >> that's necessary to make this transition as swift as
> > > >> reasonably possible?
> > > >>
> > > >> This is a question for all of us. For the PMC, who 
would supervise
> the
> > > >> effort, perhaps contribute to it, and certainly vote on 
the release
> > > >> candid

??????[ANNOUNCE] Please welcome Lijin Bin to the HBase PMC

[zheng wang]

Congratulations~




--  --
??: "Guanghao Zhang"

??????how to scan for all values which don't have given timestamps?

[zheng wang]

May be you can split it to two scans.




--  --
??: "Vitaliy Semochkin"https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/filter/TimestampsFilter.html>;;
Which filter should I use?

Regards,
Vitaliy

?????? How to delete row with Long.MAX_VALUE timestamp

[zheng wang]

It seems like the Long.MAX_VALUE is a special value, if set it as the 
timestamp , will be changed to current time.






--  --
??: "Wellington Chevreuil"https://hbase.apache.org/book.html#version.delete
[2]
https://github.com/apache/hbase/blob/branch-1.4/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java#L98

Em qua., 29 de abr. de 2020 ??s 08:57, Alexander Batyrshin 
<0x62...@gmail.com>
escreveu:

>  Hello all,
> We had faced with strange situation: table has rows with Long.MAX_VALUE
> timestamp.
> These rows impossible to delete, because DELETE mutation uses
> System.currentTimeMillis() timestamp.
> Is there any way to delete these rows?
> We use HBase-1.4.10
>
> Example:
>
> hbase(main):037:0> scan 'TRACET', { ROWPREFIXFILTER => 
"\x0439d58wj434dd",
> RAW=>true, VERSIONS=>10}
> 
ROW 
 COLUMN+CELL
>  
\x0439d58wj434dd  
 column=d:_0,
> timestamp=9223372036854775807, value=x
>
>
> hbase(main):045:0* delete 'TRACET', "\x0439d58wj434dd", "d:_0"
> 0 row(s) in 0.0120 seconds
>
> hbase(main):046:0> scan 'TRACET', { ROWPREFIXFILTER => 
"\x0439d58wj434dd",
> RAW=>true, VERSIONS=>10}
> 
ROW 
 COLUMN+CELL
>  
\x0439d58wj434dd  
 column=d:_0,
> timestamp=9223372036854775807, value=x
>  
\x0439d58wj434dd  
 column=d:_0,
> timestamp=1588146570005, type=Delete
>
>
> hbase(main):047:0> delete 'TRACET', "\x0439d58wj434dd", "d:_0",
> 9223372036854775807
> 0 row(s) in 0.0110 seconds
>
> hbase(main):048:0> scan 'TRACET', { ROWPREFIXFILTER => 
"\x0439d58wj434dd",
> RAW=>true, VERSIONS=>10}
> 
ROW 
 COLUMN+CELL
>  
\x0439d58wj434dd  
 column=d:_0,
> timestamp=9223372036854775807, value=x
>  
\x0439d58wj434dd  
 column=d:_0,
> timestamp=1588146678086, type=Delete
>  
\x0439d58wj434dd  
 column=d:_0,
> timestamp=1588146570005, type=Delete
>
>
>
>

Re:Why HBase Delete Reverts Back to Previous Value instead of Totally Deleting it

[zheng wang]

Maybe you used uncorrect method of Delete,the addColumn only can delete one 
version,and the addColumns can delete all versions.



-- Original --
From: "Trixia Belleza"

?????? a problem of long STW because of GC ref-proc

[zheng wang]

Even if set to 64KB,it also has more than 100w softRef ,and will cost too long 
still.


this "GC ref-proc" process 50w softRef and cost 700ms:


2019-09-18T03:16:42.088+0800: 125161.477: 
[GC remark 
2019-09-18T03:16:42.088+0800: 125161.477: 
[Finalize Marking, 0.0018076 secs] 
2019-09-18T03:16:42.089+0800: 125161.479: 
[GC ref-proc
2019-09-18T03:16:42.089+0800: 125161.479: [SoftReference, 
499278 refs, 0.1382086 secs]
2019-09-18T03:16:42.228+0800: 125161.617: [WeakReference, 3750 
refs, 0.0049171 secs]
2019-09-18T03:16:42.233+0800: 125161.622: [FinalReference, 1040 
refs, 0.0009375 secs]
2019-09-18T03:16:42.234+0800: 125161.623: [PhantomReference, 0 
refs, 21921 refs, 0.0058014 secs]
2019-09-18T03:16:42.239+0800: 125161.629: [JNI Weak Reference, 
0.0001070 secs]
, 0.6667733 secs] 
2019-09-18T03:16:42.756+0800: 125162.146: 
[Unloading, 0.0224078 secs]
, 0.6987032 secs] 


--  --
??: "OpenInx";
: 2019??9??30??(??) 10:27
??: "Hbase-User";

: Re: a problem of long STW because of GC ref-proc



100% get is not the right reason for choosing 16KB I think, because  if you
read a block, there's larger possibility that we
will read the adjacent cells in the same block... I think caching a 16KB
block or caching a 64KB block in BucketCache won't
make a big difference ?  (but if you cell byte size is quite small,  then
it will have so many cells encoded in a 64KB block,
then block with smaller size will be better because we search the cells in
a block one by one , means O(N) complexity).


On Mon, Sep 30, 2019 at 10:08 AM zheng wang <18031...@qq.com> wrote:

> Yes,it will be remission by your advise,but there only get request in our
> business,so 16KB is better.
> IMO,the locks of offset will always be used,so is the strong reference a
> better choice?
>
>
>
>
> --  --
> ??: "OpenInx";
> : 2019??9??30??(??) 9:46
> ??: "Hbase-User";
>
> : Re: a problem of long STW because of GC ref-proc
>
>
>
> Seems your block size is very small (16KB), so there will be
> 70*1024*1024/16=4587520 block (at most) in your BucketCache.
> For each block, the RS will maintain a soft reference idLock and a
> BucketEntry in its bucket cache.  So maybe you can try to
> enlarge the block size ?
>
> On Sun, Sep 29, 2019 at 10:14 PM zheng wang <18031...@qq.com> wrote:
>
> > Hi~
> >
> >
> > My live cluster env config below:
> > hbase version:cdh6.0.1(apache hbase2.0.0)
> > hbase config: bucketCache(70g),blocksize(16k)
> >
> >
> > java version:1.8.0_51
> > javaconfig:heap(32g),-XX:+UseG1GC  -XX:MaxGCPauseMillis=100
> > -XX:+ParallelRefProcEnabled
> >
> >
> > About 1-2days ,regionServer would occur a old gen gc that cost 1~2s in
> > remark phase:
> >
> >
> > 2019-09-29T01:55:45.186+0800: 365222.053:
> > [GC remark
> > 2019-09-29T01:55:45.186+0800: 365222.053:
> > [Finalize Marking, 0.0016327 secs]
> > 2019-09-29T01:55:45.188+0800: 365222.054:
> > [GC ref-proc
> > 2019-09-29T01:55:45.188+0800: 365222.054: [SoftReference,
> > 1264586 refs, 0.3151392 secs]
> > 2019-09-29T01:55:45.503+0800: 365222.370: [WeakReference,
> > 4317 refs, 0.0024381 secs]
> > 2019-09-29T01:55:45.505+0800: 365222.372:
> [FinalReference,
> > 9791 refs, 0.0037445 secs]
> > 2019-09-29T01:55:45.509+0800: 365222.376:
> > [PhantomReference, 0 refs, 1963 refs, 0.0018941 secs]
> > 2019-09-29T01:55:45.511+0800: 365222.378: [JNI Weak
> > Reference, 0.0001156 secs]
> > , 1.4554361 secs]
> > 2019-09-29T01:55:46.643+0800: 365223.510:
> > [Unloading, 0.0211370 secs]
> > , 1.4851728 secs]
> >
> > The SoftReference seems used by offsetLock in BucketCache, there is two
> > questions :
> > 1:SoftReference proc cost 0.31s,but why GC ref-proc cost 1.45s at all?
> > 2:Is this a good choice to use SoftReference here?

?????? a problem of long STW because of GC ref-proc

[zheng wang]

Yes,it will be remission by your advise,but there only get request in our 
business,so 16KB is better.
IMO,the locks of offset will always be used,so is the strong reference a better 
choice?




--  --
??: "OpenInx";
: 2019??9??30??(??) 9:46
??: "Hbase-User";

: Re: a problem of long STW because of GC ref-proc



Seems your block size is very small (16KB), so there will be
70*1024*1024/16=4587520 block (at most) in your BucketCache.
For each block, the RS will maintain a soft reference idLock and a
BucketEntry in its bucket cache.  So maybe you can try to
enlarge the block size ?

On Sun, Sep 29, 2019 at 10:14 PM zheng wang <18031...@qq.com> wrote:

> Hi~
>
>
> My live cluster env config below:
> hbase version:cdh6.0.1(apache hbase2.0.0)
> hbase config: bucketCache(70g),blocksize(16k)
>
>
> java version:1.8.0_51
> javaconfig:heap(32g),-XX:+UseG1GC  -XX:MaxGCPauseMillis=100
> -XX:+ParallelRefProcEnabled
>
>
> About 1-2days ,regionServer would occur a old gen gc that cost 1~2s in
> remark phase:
>
>
> 2019-09-29T01:55:45.186+0800: 365222.053:
> [GC remark
> 2019-09-29T01:55:45.186+0800: 365222.053:
> [Finalize Marking, 0.0016327 secs]
> 2019-09-29T01:55:45.188+0800: 365222.054:
> [GC ref-proc
> 2019-09-29T01:55:45.188+0800: 365222.054: [SoftReference,
> 1264586 refs, 0.3151392 secs]
> 2019-09-29T01:55:45.503+0800: 365222.370: [WeakReference,
> 4317 refs, 0.0024381 secs]
> 2019-09-29T01:55:45.505+0800: 365222.372: [FinalReference,
> 9791 refs, 0.0037445 secs]
> 2019-09-29T01:55:45.509+0800: 365222.376:
> [PhantomReference, 0 refs, 1963 refs, 0.0018941 secs]
> 2019-09-29T01:55:45.511+0800: 365222.378: [JNI Weak
> Reference, 0.0001156 secs]
> , 1.4554361 secs]
> 2019-09-29T01:55:46.643+0800: 365223.510:
> [Unloading, 0.0211370 secs]
> , 1.4851728 secs]
>
> The SoftReference seems used by offsetLock in BucketCache, there is two
> questions :
> 1:SoftReference proc cost 0.31s,but why GC ref-proc cost 1.45s at all?
> 2:Is this a good choice to use SoftReference here?

a problem of long STW because of GC ref-proc

[zheng wang]

Hi~


My live cluster env config below:
hbase version:cdh6.0.1(apache hbase2.0.0)
hbase config: bucketCache(70g),blocksize(16k)


java version:1.8.0_51
javaconfig:heap(32g),-XX:+UseG1GC  -XX:MaxGCPauseMillis=100 
-XX:+ParallelRefProcEnabled


About 1-2days ,regionServer would occur a old gen gc that cost 1~2s in remark 
phase:


2019-09-29T01:55:45.186+0800: 365222.053: 
[GC remark 
2019-09-29T01:55:45.186+0800: 365222.053: 
[Finalize Marking, 0.0016327 secs] 
2019-09-29T01:55:45.188+0800: 365222.054: 
[GC ref-proc
2019-09-29T01:55:45.188+0800: 365222.054: [SoftReference, 
1264586 refs, 0.3151392 secs]
2019-09-29T01:55:45.503+0800: 365222.370: [WeakReference, 4317 
refs, 0.0024381 secs]
2019-09-29T01:55:45.505+0800: 365222.372: [FinalReference, 9791 
refs, 0.0037445 secs]
2019-09-29T01:55:45.509+0800: 365222.376: [PhantomReference, 0 
refs, 1963 refs, 0.0018941 secs]
2019-09-29T01:55:45.511+0800: 365222.378: [JNI Weak Reference, 
0.0001156 secs]
, 1.4554361 secs] 
2019-09-29T01:55:46.643+0800: 365223.510: 
[Unloading, 0.0211370 secs]
, 1.4851728 secs]

The SoftReference seems used by offsetLock in BucketCache, there is two 
questions :
1:SoftReference proc cost 0.31s,but why GC ref-proc cost 1.45s at all?
2:Is this a good choice to use SoftReference here?