[jira] [Commented] (HBASE-24781) [Replication] When execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
[
https://issues.apache.org/jira/browse/HBASE-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166934#comment-17166934
]
leizhang commented on HBASE-24781:
--
h1. ReplicationSource do not update metrics after refresh
> [Replication] When execute shell cmd "disable_peer peerId"，the master web UI
> show a wrong number of SizeOfLogQueue
> ---
>
> Key: HBASE-24781
> URL: https://issues.apache.org/jira/browse/HBASE-24781
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.2.5
>Reporter: leizhang
>Priority: Major
>
> Supposed that we have an peer with id 1, when execute shell cmd
> disable_peer '1' , then i can see the SizeOfLogQueue metric of all
> regionservers +1 , after 10 times disable_peer ops , it will increase to
> 11, and it will never decrease to 1 in fulture .
> I can see the function ReplicationSourceManager.refreshSources(peerId)
> called , it will terminate the previous replication source and create a new
> one. and i found the note //Do not clear metrics in the bellow code block:
> {code:java}
> ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
> if (toRemove != null) {
> LOG.info("Terminate replication source for " + toRemove.getPeerId());
> // Do not clear metrics
> toRemove.terminate(terminateMessage, null, false);
> }
> {code}
> this cause the wrong number of sizeOfLogQueue, maybe we should set true when
> execute terminate() ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24781) [Replication] When execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
[
https://issues.apache.org/jira/browse/HBASE-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
leizhang updated HBASE-24781:
-
Description:
Supposed that we have an peer with id 1, when execute shell cmd disable_peer
'1' , then i can see the SizeOfLogQueue metric of all regionservers +1 ,
after 10 times disable_peer ops , it will increase to 11, and it will never
decrease to 1 in fulture .
I can see the function ReplicationSourceManager.refreshSources(peerId) called
, it will terminate the previous replication source and create a new one. and
i found the note //Do not clear metrics in the bellow code block:
{code:java}
ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
if (toRemove != null) {
LOG.info("Terminate replication source for " + toRemove.getPeerId());
// Do not clear metrics
toRemove.terminate(terminateMessage, null, false);
}
{code}
this cause the wrong number of sizeOfLogQueue, mabe we should set true when
execute terminate() ?
was:
Supposed that we have an peer with id 1, when execute shell cmd disable_peer
'1' , then i can see the SizeOfLogQueue metric of all regionservers +1 ,
after 10 times disable_peer ops , it will increase to 11, and it will never
decrease to 1 in fulture .
I can see the function ReplicationSourceManager.refreshSources(peerId) called
, it will terminate the previous replication source and create a new one. and
i found the note //Do not clear metrics in the bellow code block:
{code:java}
ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
if (toRemove != null) {
LOG.info("Terminate replication source for " + toRemove.getPeerId());
// Do not clear metrics
toRemove.terminate(terminateMessage, null, false);
}
{code}
this cause the wrong number of sizeOfLogQueue, maby we should set true when
execute terminate() ?
> [Replication] When execute shell cmd "disable_peer peerId"，the master web UI
> show a wrong number of SizeOfLogQueue
> ---
>
> Key: HBASE-24781
> URL: https://issues.apache.org/jira/browse/HBASE-24781
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.2.5
>Reporter: leizhang
>Priority: Major
>
> Supposed that we have an peer with id 1, when execute shell cmd
> disable_peer '1' , then i can see the SizeOfLogQueue metric of all
> regionservers +1 , after 10 times disable_peer ops , it will increase to
> 11, and it will never decrease to 1 in fulture .
> I can see the function ReplicationSourceManager.refreshSources(peerId)
> called , it will terminate the previous replication source and create a new
> one. and i found the note //Do not clear metrics in the bellow code block:
> {code:java}
> ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
> if (toRemove != null) {
> LOG.info("Terminate replication source for " + toRemove.getPeerId());
> // Do not clear metrics
> toRemove.terminate(terminateMessage, null, false);
> }
> {code}
> this cause the wrong number of sizeOfLogQueue, mabe we should set true when
> execute terminate() ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24781) [Replication] When execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
[
https://issues.apache.org/jira/browse/HBASE-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
leizhang updated HBASE-24781:
-
Description:
Supposed that we have an peer with id 1, when execute shell cmd disable_peer
'1' , then i can see the SizeOfLogQueue metric of all regionservers +1 ,
after 10 times disable_peer ops , it will increase to 11, and it will never
decrease to 1 in fulture .
I can see the function ReplicationSourceManager.refreshSources(peerId) called
, it will terminate the previous replication source and create a new one. and
i found the note //Do not clear metrics in the bellow code block
{code:java}
ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
if (toRemove != null) {
LOG.info("Terminate replication source for " + toRemove.getPeerId());
// Do not clear metrics
toRemove.terminate(terminateMessage, null, false);
}
{code}
this cause the wrong number of sizeOfLogQueue, maby we should set true when
execute terminate() ?
was:
Supposed that we have an peer with id 1, when execute shell cmd disable_peer
'1' , then i can see the SizeOfLogQueue metric of all regionservers +1 ,
after 10 times disable_peer ops , it will increase to 11, and it will never
decrease to 1 in fulture .
I can see the function ReplicationSourceManager.refreshSources(peerId) called
, it will terminate the previous replication source and create a new one. and
i found the note //Do not clear metrics
{code:java}
ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
if (toRemove != null) {
LOG.info("Terminate replication source for " + toRemove.getPeerId());
// Do not clear metrics
toRemove.terminate(terminateMessage, null, false);
}
{code}
this cause the wrong number of sizeOfLogQueue, maby we should set true when
execute terminate() ?
> [Replication] When execute shell cmd "disable_peer peerId"，the master web UI
> show a wrong number of SizeOfLogQueue
> ---
>
> Key: HBASE-24781
> URL: https://issues.apache.org/jira/browse/HBASE-24781
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.2.5
>Reporter: leizhang
>Priority: Major
>
> Supposed that we have an peer with id 1, when execute shell cmd
> disable_peer '1' , then i can see the SizeOfLogQueue metric of all
> regionservers +1 , after 10 times disable_peer ops , it will increase to
> 11, and it will never decrease to 1 in fulture .
> I can see the function ReplicationSourceManager.refreshSources(peerId)
> called , it will terminate the previous replication source and create a new
> one. and i found the note //Do not clear metrics in the bellow code block
> {code:java}
> ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
> if (toRemove != null) {
> LOG.info("Terminate replication source for " + toRemove.getPeerId());
> // Do not clear metrics
> toRemove.terminate(terminateMessage, null, false);
> }
> {code}
> this cause the wrong number of sizeOfLogQueue, maby we should set true when
> execute terminate() ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24781) [Replication] When execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
[
https://issues.apache.org/jira/browse/HBASE-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
leizhang updated HBASE-24781:
-
Description:
Supposed that we have an peer with id 1, when execute shell cmd disable_peer
'1' , then i can see the SizeOfLogQueue metric of all regionservers +1 ,
after 10 times disable_peer ops , it will increase to 11, and it will never
decrease to 1 in fulture .
I can see the function ReplicationSourceManager.refreshSources(peerId) called
, it will terminate the previous replication source and create a new one. and
i found the note //Do not clear metrics in the bellow code block:
{code:java}
ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
if (toRemove != null) {
LOG.info("Terminate replication source for " + toRemove.getPeerId());
// Do not clear metrics
toRemove.terminate(terminateMessage, null, false);
}
{code}
this cause the wrong number of sizeOfLogQueue, maybe we should set true when
execute terminate() ?
was:
Supposed that we have an peer with id 1, when execute shell cmd disable_peer
'1' , then i can see the SizeOfLogQueue metric of all regionservers +1 ,
after 10 times disable_peer ops , it will increase to 11, and it will never
decrease to 1 in fulture .
I can see the function ReplicationSourceManager.refreshSources(peerId) called
, it will terminate the previous replication source and create a new one. and
i found the note //Do not clear metrics in the bellow code block:
{code:java}
ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
if (toRemove != null) {
LOG.info("Terminate replication source for " + toRemove.getPeerId());
// Do not clear metrics
toRemove.terminate(terminateMessage, null, false);
}
{code}
this cause the wrong number of sizeOfLogQueue, mabe we should set true when
execute terminate() ?
> [Replication] When execute shell cmd "disable_peer peerId"，the master web UI
> show a wrong number of SizeOfLogQueue
> ---
>
> Key: HBASE-24781
> URL: https://issues.apache.org/jira/browse/HBASE-24781
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.2.5
>Reporter: leizhang
>Priority: Major
>
> Supposed that we have an peer with id 1, when execute shell cmd
> disable_peer '1' , then i can see the SizeOfLogQueue metric of all
> regionservers +1 , after 10 times disable_peer ops , it will increase to
> 11, and it will never decrease to 1 in fulture .
> I can see the function ReplicationSourceManager.refreshSources(peerId)
> called , it will terminate the previous replication source and create a new
> one. and i found the note //Do not clear metrics in the bellow code block:
> {code:java}
> ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
> if (toRemove != null) {
> LOG.info("Terminate replication source for " + toRemove.getPeerId());
> // Do not clear metrics
> toRemove.terminate(terminateMessage, null, false);
> }
> {code}
> this cause the wrong number of sizeOfLogQueue, maybe we should set true when
> execute terminate() ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24781) [Replication] When execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
[
https://issues.apache.org/jira/browse/HBASE-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
leizhang updated HBASE-24781:
-
Description:
Supposed that we have an peer with id 1, when execute shell cmd disable_peer
'1' , then i can see the SizeOfLogQueue metric of all regionservers +1 ,
after 10 times disable_peer ops , it will increase to 11, and it will never
decrease to 1 in fulture .
I can see the function ReplicationSourceManager.refreshSources(peerId) called
, it will terminate the previous replication source and create a new one. and
i found the note //Do not clear metrics
{code:java}
ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
if (toRemove != null) {
LOG.info("Terminate replication source for " + toRemove.getPeerId());
// Do not clear metrics
toRemove.terminate(terminateMessage, null, false);
}
{code}
this cause the wrong number of sizeOfLogQueue, maby we should set true when
execute terminate() ?
was:
Supposed that we have an peer with id 1, when execute shell cmd disable_peer
'1' , then i can see the SizeOfLogQueue metric of all regionservers +1 ,
after 10 times disable_peer ops , it will increase to 11, and it will never
decrease to 1 in fulture .
I can see the function ReplicationSourceManager.refreshSources(peerId) called
, it will terminate the previous replication source and create a new one. and
i found the note
{code:java}
ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
if (toRemove != null) {
LOG.info("Terminate replication source for " + toRemove.getPeerId());
// Do not clear metrics
toRemove.terminate(terminateMessage, null, false);
}
{code}
this cause the wrong number of sizeOfLogQueue, maby we should set true when
execute terminate() ?
> [Replication] When execute shell cmd "disable_peer peerId"，the master web UI
> show a wrong number of SizeOfLogQueue
> ---
>
> Key: HBASE-24781
> URL: https://issues.apache.org/jira/browse/HBASE-24781
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.2.5
>Reporter: leizhang
>Priority: Major
>
> Supposed that we have an peer with id 1, when execute shell cmd
> disable_peer '1' , then i can see the SizeOfLogQueue metric of all
> regionservers +1 , after 10 times disable_peer ops , it will increase to
> 11, and it will never decrease to 1 in fulture .
> I can see the function ReplicationSourceManager.refreshSources(peerId)
> called , it will terminate the previous replication source and create a new
> one. and i found the note //Do not clear metrics
> {code:java}
> ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
> if (toRemove != null) {
> LOG.info("Terminate replication source for " + toRemove.getPeerId());
> // Do not clear metrics
> toRemove.terminate(terminateMessage, null, false);
> }
> {code}
> this cause the wrong number of sizeOfLogQueue, maby we should set true when
> execute terminate() ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24781) [Replication] When execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
[
https://issues.apache.org/jira/browse/HBASE-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
leizhang updated HBASE-24781:
-
Description:
Supposed that we have an peer with id 1, when execute shell cmd disable_peer
'1' , then i can see the SizeOfLogQueue metric of all regionservers +1 ,
after 10 times disable_peer ops , it will increase to 11, and it will never
decrease to 1 in fulture .
I can see the function ReplicationSourceManager.refreshSources(peerId) called
, it will terminate the previous replication source and create a new one. and
i found the note //Do not clear metrics in the bellow code block:
{code:java}
ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
if (toRemove != null) {
LOG.info("Terminate replication source for " + toRemove.getPeerId());
// Do not clear metrics
toRemove.terminate(terminateMessage, null, false);
}
{code}
this cause the wrong number of sizeOfLogQueue, maby we should set true when
execute terminate() ?
was:
Supposed that we have an peer with id 1, when execute shell cmd disable_peer
'1' , then i can see the SizeOfLogQueue metric of all regionservers +1 ,
after 10 times disable_peer ops , it will increase to 11, and it will never
decrease to 1 in fulture .
I can see the function ReplicationSourceManager.refreshSources(peerId) called
, it will terminate the previous replication source and create a new one. and
i found the note //Do not clear metrics in the bellow code block
{code:java}
ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
if (toRemove != null) {
LOG.info("Terminate replication source for " + toRemove.getPeerId());
// Do not clear metrics
toRemove.terminate(terminateMessage, null, false);
}
{code}
this cause the wrong number of sizeOfLogQueue, maby we should set true when
execute terminate() ?
> [Replication] When execute shell cmd "disable_peer peerId"，the master web UI
> show a wrong number of SizeOfLogQueue
> ---
>
> Key: HBASE-24781
> URL: https://issues.apache.org/jira/browse/HBASE-24781
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.2.5
>Reporter: leizhang
>Priority: Major
>
> Supposed that we have an peer with id 1, when execute shell cmd
> disable_peer '1' , then i can see the SizeOfLogQueue metric of all
> regionservers +1 , after 10 times disable_peer ops , it will increase to
> 11, and it will never decrease to 1 in fulture .
> I can see the function ReplicationSourceManager.refreshSources(peerId)
> called , it will terminate the previous replication source and create a new
> one. and i found the note //Do not clear metrics in the bellow code block:
> {code:java}
> ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
> if (toRemove != null) {
> LOG.info("Terminate replication source for " + toRemove.getPeerId());
> // Do not clear metrics
> toRemove.terminate(terminateMessage, null, false);
> }
> {code}
> this cause the wrong number of sizeOfLogQueue, maby we should set true when
> execute terminate() ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24769) Auto scale RSGroup
[ https://issues.apache.org/jira/browse/HBASE-24769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166924#comment-17166924 ] Sun Xin commented on HBASE-24769: - [https://github.com/apache/hbase/pull/2158] > Auto scale RSGroup > -- > > Key: HBASE-24769 > URL: https://issues.apache.org/jira/browse/HBASE-24769 > Project: HBase > Issue Type: New Feature > Components: rsgroup >Affects Versions: 3.0.0-alpha-1 >Reporter: Sun Xin >Assignee: Sun Xin >Priority: Major > Fix For: 3.0.0-alpha-1 > > > In current use, if RSs go offline or online, we must manually move RSs in or > out RSGroups. > Now we can configure how many servers rsgroups need base on HBASE-24431 , and > then add an AutoScaleChore to periodically check and move servers. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24781) [Replication] When execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
[
https://issues.apache.org/jira/browse/HBASE-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
leizhang updated HBASE-24781:
-
Description:
Supposed that we have an peer with id 1, when execute shell cmd disable_peer
'1' , then i can see the SizeOfLogQueue metric of all regionservers +1 ,
after 10 times disable_peer ops , it will increase to 11, and it will never
decrease to 1 in fulture .
I can see the function ReplicationSourceManager.refreshSources(peerId) called
, it will terminate the previous replication source and create a new one. and
i found the note
{code:java}
ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
if (toRemove != null) {
LOG.info("Terminate replication source for " + toRemove.getPeerId());
// Do not clear metrics
toRemove.terminate(terminateMessage, null, false);
}
{code}
this cause the wrong number of sizeOfLogQueue, maby we should set true when
execute terminate() ?
was:
Supposed that we have an peer with id 1, when execute shell cmd disable_peer
'1' , then i can see the SizeOfLogQueue metric of all regionservers +1 ,
after 10 times disable_peer ops , it will increase to 11, and it will never
decrease to 1 in fulture .
I can see the function ReplicationSourceManager.refreshSources(peerId) called
, it will enqueue the current wals to the source , maybe when the current wal
is already in the replication queue , we try to add a duplicated wal to the
source ,which cause the same wal increase the SizeOfLogQueue metric twice ?
thx
> [Replication] When execute shell cmd "disable_peer peerId"，the master web UI
> show a wrong number of SizeOfLogQueue
> ---
>
> Key: HBASE-24781
> URL: https://issues.apache.org/jira/browse/HBASE-24781
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.2.5
>Reporter: leizhang
>Priority: Major
>
> Supposed that we have an peer with id 1, when execute shell cmd
> disable_peer '1' , then i can see the SizeOfLogQueue metric of all
> regionservers +1 , after 10 times disable_peer ops , it will increase to
> 11, and it will never decrease to 1 in fulture .
> I can see the function ReplicationSourceManager.refreshSources(peerId)
> called , it will terminate the previous replication source and create a new
> one. and i found the note
> {code:java}
> ReplicationSourceInterface toRemove = this.sources.put(peerId, src);
> if (toRemove != null) {
> LOG.info("Terminate replication source for " + toRemove.getPeerId());
> // Do not clear metrics
> toRemove.terminate(terminateMessage, null, false);
> }
> {code}
> this cause the wrong number of sizeOfLogQueue, maby we should set true when
> execute terminate() ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24791) Improve HFileOutputFormat2 to avoid always call getTableRelativePath method
[ https://issues.apache.org/jira/browse/HBASE-24791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-24791: Summary: Improve HFileOutputFormat2 to avoid always call getTableRelativePath method (was: Improve HFileOutputFormat2 to avoid alway call getTableRelativePath) > Improve HFileOutputFormat2 to avoid always call getTableRelativePath method > --- > > Key: HBASE-24791 > URL: https://issues.apache.org/jira/browse/HBASE-24791 > Project: HBase > Issue Type: Improvement >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Major > > Bulkload use HFileOutputFormat2 to write HFile > In the HFileOutputFormat2.RecordWriter > in the write method always called the getTableRelativePath method each time > This is unnecessary -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24791) Improve HFileOutputFormat2 to avoid alway call getTableRelativePath
Yechao Chen created HBASE-24791: --- Summary: Improve HFileOutputFormat2 to avoid alway call getTableRelativePath Key: HBASE-24791 URL: https://issues.apache.org/jira/browse/HBASE-24791 Project: HBase Issue Type: Improvement Reporter: Yechao Chen Assignee: Yechao Chen Bulkload use HFileOutputFormat2 to write HFile In the HFileOutputFormat2.RecordWriter in the write method always called the getTableRelativePath method each time This is unnecessary -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (HBASE-24790) Remove unused counter from SplitLogCounters
[ https://issues.apache.org/jira/browse/HBASE-24790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-24790: Comment: was deleted (was: [ GitHub Pull Request #2147|https://github.com/apache/hbase/pull/2164]) > Remove unused counter from SplitLogCounters > --- > > Key: HBASE-24790 > URL: https://issues.apache.org/jira/browse/HBASE-24790 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Minor > > remove unused counter from SplitLogCounters -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24790) Remove unused counter from SplitLogCounters
[ https://issues.apache.org/jira/browse/HBASE-24790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166910#comment-17166910 ] Yechao Chen commented on HBASE-24790: - [ GitHub Pull Request #2147|https://github.com/apache/hbase/pull/2164] > Remove unused counter from SplitLogCounters > --- > > Key: HBASE-24790 > URL: https://issues.apache.org/jira/browse/HBASE-24790 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Minor > > remove unused counter from SplitLogCounters -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-20717) Implement CCSMap - a better concurrent map with compacted data structure
[ https://issues.apache.org/jira/browse/HBASE-20717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166891#comment-17166891 ] Reid Chan commented on HBASE-20717: --- I'm going to further breakdown the HBASE-20717 into several tasks, since even CCSMap data structure part is huge. This is based on https://issues.apache.org/jira/secure/attachment/12920186/HBASE-20312-master.v9.patch PR: https://github.com/apache/hbase/pull/2165 The first one will be this: Chunk and ChunkPool structure. I did some refactoring and removed some fancy functions, for readability and quickly making it runnable. After then we can decorate it with the fancy parts. > Implement CCSMap - a better concurrent map with compacted data structure > > > Key: HBASE-20717 > URL: https://issues.apache.org/jira/browse/HBASE-20717 > Project: HBase > Issue Type: Sub-task >Reporter: Yu Li >Assignee: Chance Li >Priority: Major > > As described in HBASE-20312, we will implement the base data structure of > CCSMap and relative unit tests in this task. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-20717) Implement CCSMap - a better concurrent map with compacted data structure
[ https://issues.apache.org/jira/browse/HBASE-20717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166891#comment-17166891 ] Reid Chan edited comment on HBASE-20717 at 7/29/20, 5:51 AM: - I'm going to further breakdown the HBASE-20717 into several tasks, since even CCSMap data structure part is huge. The current PR and all upcoming is/will be based on https://issues.apache.org/jira/secure/attachment/12920186/HBASE-20312-master.v9.patch PR: https://github.com/apache/hbase/pull/2165 The first one will be this: Chunk and ChunkPool structure. I did some refactoring and removed some fancy functions, for readability and quickly making it runnable. After then we can decorate it with the fancy parts. was (Author: reidchan): I'm going to further breakdown the HBASE-20717 into several tasks, since even CCSMap data structure part is huge. This is based on https://issues.apache.org/jira/secure/attachment/12920186/HBASE-20312-master.v9.patch PR: https://github.com/apache/hbase/pull/2165 The first one will be this: Chunk and ChunkPool structure. I did some refactoring and removed some fancy functions, for readability and quickly making it runnable. After then we can decorate it with the fancy parts. > Implement CCSMap - a better concurrent map with compacted data structure > > > Key: HBASE-20717 > URL: https://issues.apache.org/jira/browse/HBASE-20717 > Project: HBase > Issue Type: Sub-task >Reporter: Yu Li >Assignee: Chance Li >Priority: Major > > As described in HBASE-20312, we will implement the base data structure of > CCSMap and relative unit tests in this task. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24781) [Replication] When execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
[ https://issues.apache.org/jira/browse/HBASE-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leizhang updated HBASE-24781: - Description: Supposed that we have an peer with id 1, when execute shell cmd disable_peer '1' , then i can see the SizeOfLogQueue metric of all regionservers +1 , after 10 times disable_peer ops , it will increase to 11, and it will never decrease to 1 in fulture . I can see the function ReplicationSourceManager.refreshSources(peerId) called , it will enqueue the current wals to the source , maybe when the current wal is already in the replication queue , we try to add a duplicated wal to the source ,which cause the same wal increase the SizeOfLogQueue metric twice ? thx was: Supposed that we have an peer with id 1, when execute shell cmd disable_peer '1' , then i can see the SizeOfLogQueue metric of all regionservers +1 , after 10 times disable_peer ops , it will increase to 11, and it will never decrease to 1 in fulture . I can see the function ReplicationSourceManager.refreshSources(peerId) called , it will enqueue the current wals to the source , maybe when the current wal is already in the replication queue , we try to add a duplicated wal to the source ,which cause the same wal increase the SizeOfLogQueue twice ? thx > [Replication] When execute shell cmd "disable_peer peerId"，the master web UI > show a wrong number of SizeOfLogQueue > --- > > Key: HBASE-24781 > URL: https://issues.apache.org/jira/browse/HBASE-24781 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.2.5 >Reporter: leizhang >Priority: Major > > Supposed that we have an peer with id 1, when execute shell cmd > disable_peer '1' , then i can see the SizeOfLogQueue metric of all > regionservers +1 , after 10 times disable_peer ops , it will increase to > 11, and it will never decrease to 1 in fulture . > I can see the function ReplicationSourceManager.refreshSources(peerId) > called , it will enqueue the current wals to the source , maybe when the > current wal is already in the replication queue , we try to add a duplicated > wal to the source ,which cause the same wal increase the SizeOfLogQueue > metric twice ? thx > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24665) MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166874#comment-17166874 ] Anoop Sam John commented on HBASE-24665: [~wenfeiyi666], There is an issue with walRollFinished() API which we discussed in the PR review. Can u pls raise a jira to track that too? > MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll > -- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.3.0, master, 2.1.10, 1.4.14, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 2.2.7 > > > when use multiwal, any a wal request roll, all wal will be together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24738) [Shell] processlist command fails with ERROR: Unexpected end of file from server when SSL enabled
[
https://issues.apache.org/jira/browse/HBASE-24738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166866#comment-17166866
]
Hudson commented on HBASE-24738:
Results for branch branch-2
[build #2761 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2761/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2761/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2756/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2742/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2757/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Shell] processlist command fails with ERROR: Unexpected end of file from
> server when SSL enabled
> -
>
> Key: HBASE-24738
> URL: https://issues.apache.org/jira/browse/HBASE-24738
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> HBase Shell command "processlist" fails with ERROR: Unexpected end of file
> from server when HBase SSL enabled.
>
> Below code is commented since beginning, see HBASE-4368,
> [https://github.com/apache/hbase/blob/8076eafb187ce32d4a78aef482b6218d85a985ac/hbase-shell/src/main/ruby/hbase/taskmonitor.rb#L85]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-11676) Scan FORMATTER is not applied for columns using non-printable name in shell
[ https://issues.apache.org/jira/browse/HBASE-11676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-11676. --- Fix Version/s: 2.4.0 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed Merged to branch-2 and master. Thanks for the fix [~bitoffdev] > Scan FORMATTER is not applied for columns using non-printable name in shell > > > Key: HBASE-11676 > URL: https://issues.apache.org/jira/browse/HBASE-11676 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.2.0, 2.3.0 > Environment: 0.98 + hadoop2.2.0 >Reporter: t samkawa >Assignee: Elliot Miller >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > > The "FORMATTER" does not work if the target cell`s column uses binary name. > hbase> create "test1", "f" > hbase> incr "test1", "row1" , "f:a", 1 > hbase> incr "test1", "row1" , "f:\x11", 1 > hbase> scan "test1", COLUMNS=>["f:\x11:toLong","f:a:toLong"] > ROW COLUMN+CELL > row1column=f:\x11, ..., value=\x00\x00\x00\x00\x00\x00\x00\x01 > row1column=f:a, ..., value=1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24790) Remove unused counter from SplitLogCounters
Yechao Chen created HBASE-24790: --- Summary: Remove unused counter from SplitLogCounters Key: HBASE-24790 URL: https://issues.apache.org/jira/browse/HBASE-24790 Project: HBase Issue Type: Improvement Components: wal Reporter: Yechao Chen Assignee: Yechao Chen remove unused counter from SplitLogCounters -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24659) Calculate FIXED_OVERHEAD automatically
[ https://issues.apache.org/jira/browse/HBASE-24659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166826#comment-17166826 ] niuyulin commented on HBASE-24659: -- [~zhangduo], I do performance test via YCSB, the version of client and server are both 3.0.0-SNAPSHOT. the different between w/ and w/o this pr is mostly within 10% > Calculate FIXED_OVERHEAD automatically > -- > > Key: HBASE-24659 > URL: https://issues.apache.org/jira/browse/HBASE-24659 > Project: HBase > Issue Type: Improvement >Reporter: Duo Zhang >Assignee: niuyulin >Priority: Major > Attachments: 50M-performance.pdf > > > Now the FIXED_OVERHEAD in some classes are maintained manually, an we have a > method to TestHeapSizes to confirm that the value is correct. > But it is really hard for developers to count the fields in a complicated > class like HRegion. Since we have the ability to calcuate the accurate size > in UT, I think we it is also possible to calcuate it when loading the class, > which is a one time operation so should not effect the performance too much. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24659) Calculate FIXED_OVERHEAD automatically
[ https://issues.apache.org/jira/browse/HBASE-24659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] niuyulin updated HBASE-24659: - Attachment: 50M-performance.pdf > Calculate FIXED_OVERHEAD automatically > -- > > Key: HBASE-24659 > URL: https://issues.apache.org/jira/browse/HBASE-24659 > Project: HBase > Issue Type: Improvement >Reporter: Duo Zhang >Assignee: niuyulin >Priority: Major > Attachments: 50M-performance.pdf > > > Now the FIXED_OVERHEAD in some classes are maintained manually, an we have a > method to TestHeapSizes to confirm that the value is correct. > But it is really hard for developers to count the fields in a complicated > class like HRegion. Since we have the ability to calcuate the accurate size > in UT, I think we it is also possible to calcuate it when loading the class, > which is a one time operation so should not effect the performance too much. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24781) [Replication] When execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
[ https://issues.apache.org/jira/browse/HBASE-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leizhang updated HBASE-24781: - Summary: [Replication] When execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue (was: when execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue) > [Replication] When execute shell cmd "disable_peer peerId"，the master web UI > show a wrong number of SizeOfLogQueue > --- > > Key: HBASE-24781 > URL: https://issues.apache.org/jira/browse/HBASE-24781 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.2.5 >Reporter: leizhang >Priority: Major > > Supposed that we have an peer with id 1, when execute shell cmd > disable_peer '1' , then i can see the SizeOfLogQueue metric of all > regionservers +1 , after 10 times disable_peer ops , it will increase to > 11, and it will never decrease to 1 in fulture . > I can see the function ReplicationSourceManager.refreshSources(peerId) > called , it will enqueue the current wals to the source , maybe when the > current wal is already in the replication queue , we try to add a duplicated > wal to the source ,which cause the same wal increase the SizeOfLogQueue > twice ? thx > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-24786) [hbase-thirdarty] Put up 3.4.0RC0
[ https://issues.apache.org/jira/browse/HBASE-24786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-24786 started by Duo Zhang. - > [hbase-thirdarty] Put up 3.4.0RC0 > - > > Key: HBASE-24786 > URL: https://issues.apache.org/jira/browse/HBASE-24786 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-24786) [hbase-thirdarty] Put up 3.4.0RC0
[ https://issues.apache.org/jira/browse/HBASE-24786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-24786: - Assignee: Duo Zhang > [hbase-thirdarty] Put up 3.4.0RC0 > - > > Key: HBASE-24786 > URL: https://issues.apache.org/jira/browse/HBASE-24786 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-24788) Zookeeper connection leakage on hbase mapreduce jobs
[ https://issues.apache.org/jira/browse/HBASE-24788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-24788 started by Sandeep Pal. --- > Zookeeper connection leakage on hbase mapreduce jobs > > > Key: HBASE-24788 > URL: https://issues.apache.org/jira/browse/HBASE-24788 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 1.6.0, master >Reporter: Sandeep Pal >Assignee: Sandeep Pal >Priority: Major > > Observed the significant increase in ZK connection on performance testing on > map reduce jobs. Turns out the > [TableOutputFormat.checkOutputSpecs()|https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java#L182] > is not closing the connection it uses to get the hbase admin. It closes the > hbase admin but never close the connection to get the admin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24785) [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 3.4.0
[ https://issues.apache.org/jira/browse/HBASE-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-24785. --- Hadoop Flags: Reviewed Resolution: Fixed Merge to master. Thanks [~busbey] for reviewing. > [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 3.4.0 > > > Key: HBASE-24785 > URL: https://issues.apache.org/jira/browse/HBASE-24785 > Project: HBase > Issue Type: Sub-task > Components: thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24784) [hbase-thirdparty] Set version as 3.4.0 in prep for first RC
[ https://issues.apache.org/jira/browse/HBASE-24784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-24784. --- Hadoop Flags: Reviewed Resolution: Fixed Merge to master. Thanks [~busbey] for reviewing. > [hbase-thirdparty] Set version as 3.4.0 in prep for first RC > > > Key: HBASE-24784 > URL: https://issues.apache.org/jira/browse/HBASE-24784 > Project: HBase > Issue Type: Sub-task > Components: thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24724) Parallelize make with Yetus
[
https://issues.apache.org/jira/browse/HBASE-24724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharath Vissapragada updated HBASE-24724:
-
Status: Patch Available (was: Open)
> Parallelize make with Yetus
>
>
> Key: HBASE-24724
> URL: https://issues.apache.org/jira/browse/HBASE-24724
> Project: HBase
> Issue Type: Sub-task
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
>
> Ytetus by default runs a single process make, which is very very slow. Figure
> out a way to pass {{-j$(nproc)}} to parallelize it. This runs multiple
> targets in parallel to speed it up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24775) [hbtop] StoreFile size should be rounded off
[
https://issues.apache.org/jira/browse/HBASE-24775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166771#comment-17166771
]
Hudson commented on HBASE-24775:
Results for branch branch-2.2
[build #920 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/920/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/920//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/917//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/920//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [hbtop] StoreFile size should be rounded off
>
>
> Key: HBASE-24775
> URL: https://issues.apache.org/jira/browse/HBASE-24775
> Project: HBase
> Issue Type: Bug
> Components: hbtop
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.2.7
>
> Attachments: Screen Shot.png, Screen Shot2.png
>
>
> Currently, hbtop doesn't show StoreFile size rounded off, so when it's
> reached 1GB it will be very ugly as follows:
> !Screen Shot.png!
> We should round off StoreFile size.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24768) Clear cached service kerberos ticket in case of SASL failures thrown from server side
[ https://issues.apache.org/jira/browse/HBASE-24768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166687#comment-17166687 ] Sandeep Guggilam commented on HBASE-24768: -- Sure [~apurtell] > Clear cached service kerberos ticket in case of SASL failures thrown from > server side > - > > Key: HBASE-24768 > URL: https://issues.apache.org/jira/browse/HBASE-24768 > Project: HBase > Issue Type: Bug >Reporter: Sandeep Guggilam >Assignee: Sandeep Guggilam >Priority: Major > > We setup a SASL connection using different mechanisms like Digest, Kerberos > from master to RS for various activities like region assignment etc. In case > of SASL connect failures, we try to dispose of the SaslRpcClient and try to > relogin from the keytab on the client side. However the relogin from keytab > method doesn't clear off the service ticket cached in memory unless TGT is > about to expire within a timeframe. > This actually causes an issue where there is a keytab refresh that happens > because of expiry on the RS server and throws a SASL connect error when > Master reaches out to the RS server with the cached service ticket that no > longer works with the new refreshed keytab. We might need to clear off the > service ticket cached as there could be a credential refresh on the RS server > side when handling connect failures -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-24789) When table level quota is set and violated, pre existing ns quota policy not imposed on table.
[ https://issues.apache.org/jira/browse/HBASE-24789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjeet Nishad reassigned HBASE-24789: -- Assignee: Sanjeet Nishad > When table level quota is set and violated, pre existing ns quota policy not > imposed on table. > -- > > Key: HBASE-24789 > URL: https://issues.apache.org/jira/browse/HBASE-24789 > Project: HBase > Issue Type: Bug > Components: Quotas >Reporter: Sanjeet Nishad >Assignee: Sanjeet Nishad >Priority: Major > > When table level quota is set and violated, pre existing namespace level > quota policy not imposed on table. > > While removing Quota for a table, in addition to deleting from hbase:quota > table, it should be removed from current state in QuotaObserverChore as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24789) When table level quota is set and violated, pre existing ns quota policy not imposed on table.
Sanjeet Nishad created HBASE-24789: -- Summary: When table level quota is set and violated, pre existing ns quota policy not imposed on table. Key: HBASE-24789 URL: https://issues.apache.org/jira/browse/HBASE-24789 Project: HBase Issue Type: Bug Components: Quotas Reporter: Sanjeet Nishad When table level quota is set and violated, pre existing namespace level quota policy not imposed on table. While removing Quota for a table, in addition to deleting from hbase:quota table, it should be removed from current state in QuotaObserverChore as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24788) Zookeeper connection leakage on hbase mapreduce jobs
[ https://issues.apache.org/jira/browse/HBASE-24788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Pal updated HBASE-24788: Description: Observed the significant increase in ZK connection on performance testing on map reduce jobs. Turns out the [TableOutputFormat.checkOutputSpecs()|https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java#L182] is not closing the connection it uses to get the hbase admin. It closes the hbase admin but never close the connection to get the admin.(was: Observed the significant increase in ZK connection on performance testing on map reduce jobs. Turns out the [TableOutputFormat|https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java#L182] is not not closing the connection it uses to get the hbase admin. It closes the hbase admin but never close the connection to get the admin. ) > Zookeeper connection leakage on hbase mapreduce jobs > > > Key: HBASE-24788 > URL: https://issues.apache.org/jira/browse/HBASE-24788 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 1.6.0, master >Reporter: Sandeep Pal >Assignee: Sandeep Pal >Priority: Major > > Observed the significant increase in ZK connection on performance testing on > map reduce jobs. Turns out the > [TableOutputFormat.checkOutputSpecs()|https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java#L182] > is not closing the connection it uses to get the hbase admin. It closes the > hbase admin but never close the connection to get the admin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24788) Zookeeper connection leakage on hbase mapreduce jobs
[ https://issues.apache.org/jira/browse/HBASE-24788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Pal updated HBASE-24788: Description: Observed the significant increase in ZK connection on performance testing on map reduce jobs. Turns out the [TableOutputFormat|https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java#L182] is not not closing the connection it uses to get the hbase admin. It closes the hbase admin but never close the connection to get the admin.(was: Observed the significant increase in ZK connection on performance testing on map reduce jobs. Turns out the TableOutputFormat is not not closing the connection it uses to get the hbase admin. It closes the hbase admin but never close the connection to get the admin. ) > Zookeeper connection leakage on hbase mapreduce jobs > > > Key: HBASE-24788 > URL: https://issues.apache.org/jira/browse/HBASE-24788 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 1.6.0, master >Reporter: Sandeep Pal >Assignee: Sandeep Pal >Priority: Major > > Observed the significant increase in ZK connection on performance testing on > map reduce jobs. Turns out the > [TableOutputFormat|https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java#L182] > is not not closing the connection it uses to get the hbase admin. It closes > the hbase admin but never close the connection to get the admin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24768) Clear cached service kerberos ticket in case of SASL failures thrown from server side
[ https://issues.apache.org/jira/browse/HBASE-24768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166657#comment-17166657 ] Andrew Kyle Purtell commented on HBASE-24768: - If this is going to depend on HADOOP-17159, please be sure to invoke the new UGI method on the HBase side by reflection so the Hadoop side change doesn't introduce a hard dependency on a new method signature. That would violate our runtime/dependency compatibility guidelines outside of a release that explicitly drops support for Hadoop versions (assuming Hadoop releases versions that include HADOOP-17159). If the new method is missing you won't be able to force the relogin. Because you can't depend in this method being there, you'll need a fallback, even if the fallback is only to print a complaint at ERROR or WARN log level. > Clear cached service kerberos ticket in case of SASL failures thrown from > server side > - > > Key: HBASE-24768 > URL: https://issues.apache.org/jira/browse/HBASE-24768 > Project: HBase > Issue Type: Bug >Reporter: Sandeep Guggilam >Assignee: Sandeep Guggilam >Priority: Major > > We setup a SASL connection using different mechanisms like Digest, Kerberos > from master to RS for various activities like region assignment etc. In case > of SASL connect failures, we try to dispose of the SaslRpcClient and try to > relogin from the keytab on the client side. However the relogin from keytab > method doesn't clear off the service ticket cached in memory unless TGT is > about to expire within a timeframe. > This actually causes an issue where there is a keytab refresh that happens > because of expiry on the RS server and throws a SASL connect error when > Master reaches out to the RS server with the cached service ticket that no > longer works with the new refreshed keytab. We might need to clear off the > service ticket cached as there could be a credential refresh on the RS server > side when handling connect failures -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24788) Zookeeper connection leakage on hbase mapreduce jobs
Sandeep Pal created HBASE-24788: --- Summary: Zookeeper connection leakage on hbase mapreduce jobs Key: HBASE-24788 URL: https://issues.apache.org/jira/browse/HBASE-24788 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 1.6.0, master Reporter: Sandeep Pal Assignee: Sandeep Pal Observed the significant increase in ZK connection on performance testing on map reduce jobs. Turns out the TableOutputFormat is not not closing the connection it uses to get the hbase admin. It closes the hbase admin but never close the connection to get the admin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-11676) Scan FORMATTER is not applied for columns using non-printable name in shell
[ https://issues.apache.org/jira/browse/HBASE-11676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166650#comment-17166650 ] Elliot Miller commented on HBASE-11676: --- * Patch is up for master: [https://github.com/apache/hbase/pull/2161] * I'll put up a backport for branch-2 once it is approved on master. > Scan FORMATTER is not applied for columns using non-printable name in shell > > > Key: HBASE-11676 > URL: https://issues.apache.org/jira/browse/HBASE-11676 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.2.0, 2.3.0 > Environment: 0.98 + hadoop2.2.0 >Reporter: t samkawa >Assignee: Elliot Miller >Priority: Minor > > The "FORMATTER" does not work if the target cell`s column uses binary name. > hbase> create "test1", "f" > hbase> incr "test1", "row1" , "f:a", 1 > hbase> incr "test1", "row1" , "f:\x11", 1 > hbase> scan "test1", COLUMNS=>["f:\x11:toLong","f:a:toLong"] > ROW COLUMN+CELL > row1column=f:\x11, ..., value=\x00\x00\x00\x00\x00\x00\x00\x01 > row1column=f:a, ..., value=1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24665) MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
[
https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166637#comment-17166637
]
Hudson commented on HBASE-24665:
Results for branch branch-2.3
[build #193 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/193/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/193/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/193/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/193/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/193/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
> --
>
> Key: HBASE-24665
> URL: https://issues.apache.org/jira/browse/HBASE-24665
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.3.0, master, 2.1.10, 1.4.14, 2.2.6
>Reporter: wenfeiyi666
>Assignee: wenfeiyi666
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 2.2.7
>
>
> when use multiwal, any a wal request roll, all wal will be together roll.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-20226) Performance Improvement Taking Large Snapshots In Remote Filesystems
[ https://issues.apache.org/jira/browse/HBASE-20226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166636#comment-17166636 ] Bharath Vissapragada commented on HBASE-20226: -- [~zyork] Thanks for the quick clarification. We use a "fast" FS (non-S3) but the delete is still choked for some HDFS side reason and we clearly aren't saturating the handlers and CPU on the namenode side. That is very much visible in these deletes that are sequential. So the attempt here is to parallelize that. I put up the patch [here|https://github.com/apache/hbase/pull/2159], you have a few cycles to take look? > Performance Improvement Taking Large Snapshots In Remote Filesystems > > > Key: HBASE-20226 > URL: https://issues.apache.org/jira/browse/HBASE-20226 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 3.0.0-alpha-1, 2.3.0, 1.7.0 > Environment: HBase 1.4.0 running on an AWS EMR cluster with the > hbase.rootdir set to point to a folder in S3 >Reporter: Saad Mufti >Assignee: Bharath Vissapragada >Priority: Minor > Labels: perfomance > Attachments: HBASE-20226..01.patch > > > When taking a snapshot of any table, one of the last steps is to delete the > region manifests, which have already been rolled up into a larger overall > manifest and thus have redundant information. > This proposal is to do the deletion in a thread pool bounded by > hbase.snapshot.thread.pool.max . For large tables with a lot of regions, the > current single threaded deletion is taking longer than all the rest of the > snapshot tasks when the Hbase data and the snapshot folder are both in a > remote filesystem like S3. > I have a patch for this proposal almost ready and will submit it tomorrow for > feedback, although I haven't had a chance to write any tests yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-24756) Backport HBASE-24336 to branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-24756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-24756 started by Pankaj Kumar. > Backport HBASE-24336 to branch-2.2 > -- > > Key: HBASE-24756 > URL: https://issues.apache.org/jira/browse/HBASE-24756 > Project: HBase > Issue Type: Sub-task > Components: metrics >Affects Versions: 2.2.5 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Minor > > HBASE-24336 is applicable to branch-2.2 also. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24756) Backport HBASE-24336 to branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-24756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166631#comment-17166631 ] Pankaj Kumar commented on HBASE-24756: -- Raised PR, [https://github.com/apache/hbase/pull/2160] > Backport HBASE-24336 to branch-2.2 > -- > > Key: HBASE-24756 > URL: https://issues.apache.org/jira/browse/HBASE-24756 > Project: HBase > Issue Type: Sub-task > Components: metrics >Affects Versions: 2.2.5 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Minor > > HBASE-24336 is applicable to branch-2.2 also. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-20226) Performance Improvement Taking Large Snapshots In Remote Filesystems
[ https://issues.apache.org/jira/browse/HBASE-20226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166618#comment-17166618 ] Zach York commented on HBASE-20226: --- [~bharathv] This improvement can still be valid if you are using a slow filesystem. The linked JIRA solves this issue by allowing the user to move to a faster filesystem for temporary storage, but for people who don't want that/can't do that, maybe this improvement still makes sense > Performance Improvement Taking Large Snapshots In Remote Filesystems > > > Key: HBASE-20226 > URL: https://issues.apache.org/jira/browse/HBASE-20226 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 3.0.0-alpha-1, 2.3.0, 1.7.0 > Environment: HBase 1.4.0 running on an AWS EMR cluster with the > hbase.rootdir set to point to a folder in S3 >Reporter: Saad Mufti >Assignee: Bharath Vissapragada >Priority: Minor > Labels: perfomance > Attachments: HBASE-20226..01.patch > > > When taking a snapshot of any table, one of the last steps is to delete the > region manifests, which have already been rolled up into a larger overall > manifest and thus have redundant information. > This proposal is to do the deletion in a thread pool bounded by > hbase.snapshot.thread.pool.max . For large tables with a lot of regions, the > current single threaded deletion is taking longer than all the rest of the > snapshot tasks when the Hbase data and the snapshot folder are both in a > remote filesystem like S3. > I have a patch for this proposal almost ready and will submit it tomorrow for > feedback, although I haven't had a chance to write any tests yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-11062) hbtop
[ https://issues.apache.org/jira/browse/HBASE-11062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166610#comment-17166610 ] Michael Stack commented on HBASE-11062: --- [~brfrn169] just some comments on this utility after using it some. * First, it is very nice. Thanks. * I was looking at why so many hits to hbase:meta and using the 'region' mode I could see that actually the hits to meta were generally low; it was just on occasion that hbase:meta was hit hard. In the UI, the server carrying meta can have high hits because of meta+neighbors and it is easy to accredit them all to meta when it is not always the case. This by-Region view helps. * I switched to Client mode and I could see the request by client and what was interesting was that the hits to meta came from CM or the Master node doing their Scans but when these were happening the hits on meta spiked up. Obvious I suppose but w/o knowing where the Scans were coming from, its easy to think clients are running wild doing too many meta lookups. * It looks like this 'client' mode could help identify 'bad actors' since it uses src host ip rather than 'user' which is pretty useless if all clients identify w/ same 'user' name. What is an FREAD/S? What is SF and #SF? Thanks. > hbtop > - > > Key: HBASE-11062 > URL: https://issues.apache.org/jira/browse/HBASE-11062 > Project: HBase > Issue Type: New Feature > Components: hbtop >Reporter: Andrew Kyle Purtell >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0, 2.1.7, 2.2.2 > > Attachments: HBASE-11062-master-addendum-v1.patch > > > A top-like monitor could be useful for testing, debugging, operations of > clusters of moderate size, and possibly for diagnosing issues in large > clusters. > Consider a curses interface like the one presented by atop > (http://www.atoptool.nl/images/screenshots/genericw.png) - with aggregate > metrics collected over a monitoring interval in the upper portion of the > pane, and a listing of discrete measurements sorted and filtered by various > criteria in the bottom part of the pane. One might imagine a cluster overview > with cluster aggregate metrics above and a list of regionservers sorted by > utilization below; and a regionserver view with process metrics above and a > list of metrics by operation type below, or a list of client connections, or > a list of threads, sorted by utilization, throughput, or latency. > Generically 'htop' is taken but would be distinctive in the HBase context, a > utility org.apache.hadoop.hbase.HTop > No need necessarily for a curses interface. Could be an external monitor with > a web front end as has been discussed before. I do like the idea of a process > that runs in a terminal because I interact with dev and test HBase clusters > exclusively by SSH. > UPDATE: > The tool name is changed from htop to hbtop. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-20226) Performance Improvement Taking Large Snapshots In Remote Filesystems
[ https://issues.apache.org/jira/browse/HBASE-20226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-20226: - Labels: perfomance (was: ) > Performance Improvement Taking Large Snapshots In Remote Filesystems > > > Key: HBASE-20226 > URL: https://issues.apache.org/jira/browse/HBASE-20226 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 3.0.0-alpha-1, 2.3.0, 1.7.0 > Environment: HBase 1.4.0 running on an AWS EMR cluster with the > hbase.rootdir set to point to a folder in S3 >Reporter: Saad Mufti >Priority: Minor > Labels: perfomance > Attachments: HBASE-20226..01.patch > > > When taking a snapshot of any table, one of the last steps is to delete the > region manifests, which have already been rolled up into a larger overall > manifest and thus have redundant information. > This proposal is to do the deletion in a thread pool bounded by > hbase.snapshot.thread.pool.max . For large tables with a lot of regions, the > current single threaded deletion is taking longer than all the rest of the > snapshot tasks when the Hbase data and the snapshot folder are both in a > remote filesystem like S3. > I have a patch for this proposal almost ready and will submit it tomorrow for > feedback, although I haven't had a chance to write any tests yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-20226) Performance Improvement Taking Large Snapshots In Remote Filesystems
[ https://issues.apache.org/jira/browse/HBASE-20226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada reassigned HBASE-20226: Assignee: Bharath Vissapragada > Performance Improvement Taking Large Snapshots In Remote Filesystems > > > Key: HBASE-20226 > URL: https://issues.apache.org/jira/browse/HBASE-20226 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 3.0.0-alpha-1, 2.3.0, 1.7.0 > Environment: HBase 1.4.0 running on an AWS EMR cluster with the > hbase.rootdir set to point to a folder in S3 >Reporter: Saad Mufti >Assignee: Bharath Vissapragada >Priority: Minor > Labels: perfomance > Attachments: HBASE-20226..01.patch > > > When taking a snapshot of any table, one of the last steps is to delete the > region manifests, which have already been rolled up into a larger overall > manifest and thus have redundant information. > This proposal is to do the deletion in a thread pool bounded by > hbase.snapshot.thread.pool.max . For large tables with a lot of regions, the > current single threaded deletion is taking longer than all the rest of the > snapshot tasks when the Hbase data and the snapshot folder are both in a > remote filesystem like S3. > I have a patch for this proposal almost ready and will submit it tomorrow for > feedback, although I haven't had a chance to write any tests yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-20226) Performance Improvement Taking Large Snapshots In Remote Filesystems
[ https://issues.apache.org/jira/browse/HBASE-20226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-20226: - Affects Version/s: (was: 1.4.0) 1.7.0 3.0.0-alpha-1 2.3.0 > Performance Improvement Taking Large Snapshots In Remote Filesystems > > > Key: HBASE-20226 > URL: https://issues.apache.org/jira/browse/HBASE-20226 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 3.0.0-alpha-1, 2.3.0, 1.7.0 > Environment: HBase 1.4.0 running on an AWS EMR cluster with the > hbase.rootdir set to point to a folder in S3 >Reporter: Saad Mufti >Priority: Minor > Attachments: HBASE-20226..01.patch > > > When taking a snapshot of any table, one of the last steps is to delete the > region manifests, which have already been rolled up into a larger overall > manifest and thus have redundant information. > This proposal is to do the deletion in a thread pool bounded by > hbase.snapshot.thread.pool.max . For large tables with a lot of regions, the > current single threaded deletion is taking longer than all the rest of the > snapshot tasks when the Hbase data and the snapshot folder are both in a > remote filesystem like S3. > I have a patch for this proposal almost ready and will submit it tomorrow for > feedback, although I haven't had a chance to write any tests yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-20226) Performance Improvement Taking Large Snapshots In Remote Filesystems
[ https://issues.apache.org/jira/browse/HBASE-20226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166548#comment-17166548 ] Bharath Vissapragada commented on HBASE-20226: -- Thanks for taking a look [~yuzhih...@gmail.com]. I've [cleaned up|https://github.com/apache/hbase/pull/2159] the patch a bit.. > Performance Improvement Taking Large Snapshots In Remote Filesystems > > > Key: HBASE-20226 > URL: https://issues.apache.org/jira/browse/HBASE-20226 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 1.4.0 > Environment: HBase 1.4.0 running on an AWS EMR cluster with the > hbase.rootdir set to point to a folder in S3 >Reporter: Saad Mufti >Priority: Minor > Attachments: HBASE-20226..01.patch > > > When taking a snapshot of any table, one of the last steps is to delete the > region manifests, which have already been rolled up into a larger overall > manifest and thus have redundant information. > This proposal is to do the deletion in a thread pool bounded by > hbase.snapshot.thread.pool.max . For large tables with a lot of regions, the > current single threaded deletion is taking longer than all the rest of the > snapshot tasks when the Hbase data and the snapshot folder are both in a > remote filesystem like S3. > I have a patch for this proposal almost ready and will submit it tomorrow for > feedback, although I haven't had a chance to write any tests yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24738) [Shell] processlist command fails with ERROR: Unexpected end of file from server when SSL enabled
[ https://issues.apache.org/jira/browse/HBASE-24738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-24738. --- Fix Version/s: 2.4.0 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed Merged to branch-2 and master. Thanks for patch [~pankajkumar] . Thanks for review [~bitoffdev] > [Shell] processlist command fails with ERROR: Unexpected end of file from > server when SSL enabled > - > > Key: HBASE-24738 > URL: https://issues.apache.org/jira/browse/HBASE-24738 > Project: HBase > Issue Type: Bug > Components: shell >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > HBase Shell command "processlist" fails with ERROR: Unexpected end of file > from server when HBase SSL enabled. > > Below code is commented since beginning, see HBASE-4368, > [https://github.com/apache/hbase/blob/8076eafb187ce32d4a78aef482b6218d85a985ac/hbase-shell/src/main/ruby/hbase/taskmonitor.rb#L85] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24738) [Shell] processlist command fails with ERROR: Unexpected end of file from server when SSL enabled
[
https://issues.apache.org/jira/browse/HBASE-24738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166512#comment-17166512
]
Elliot Miller commented on HBASE-24738:
---
{quote}Will update the PR with above approach.
{quote}
Awesome; thanks!
> [Shell] processlist command fails with ERROR: Unexpected end of file from
> server when SSL enabled
> -
>
> Key: HBASE-24738
> URL: https://issues.apache.org/jira/browse/HBASE-24738
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
>
> HBase Shell command "processlist" fails with ERROR: Unexpected end of file
> from server when HBase SSL enabled.
>
> Below code is commented since beginning, see HBASE-4368,
> [https://github.com/apache/hbase/blob/8076eafb187ce32d4a78aef482b6218d85a985ac/hbase-shell/src/main/ruby/hbase/taskmonitor.rb#L85]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-11676) Scan FORMATTER is not applied for columns using non-printable name in shell
[ https://issues.apache.org/jira/browse/HBASE-11676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliot Miller updated HBASE-11676: -- Affects Version/s: 2.3.0 > Scan FORMATTER is not applied for columns using non-printable name in shell > > > Key: HBASE-11676 > URL: https://issues.apache.org/jira/browse/HBASE-11676 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.3.0 > Environment: 0.98 + hadoop2.2.0 >Reporter: t samkawa >Assignee: Elliot Miller >Priority: Minor > > The "FORMATTER" does not work if the target cell`s column uses binary name. > hbase> create "test1", "f" > hbase> incr "test1", "row1" , "f:a", 1 > hbase> incr "test1", "row1" , "f:\x11", 1 > hbase> scan "test1", COLUMNS=>["f:\x11:toLong","f:a:toLong"] > ROW COLUMN+CELL > row1column=f:\x11, ..., value=\x00\x00\x00\x00\x00\x00\x00\x01 > row1column=f:a, ..., value=1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-11676) Scan FORMATTER is not applied for columns using non-printable name in shell
[ https://issues.apache.org/jira/browse/HBASE-11676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliot Miller updated HBASE-11676: -- Affects Version/s: 2.2.0 > Scan FORMATTER is not applied for columns using non-printable name in shell > > > Key: HBASE-11676 > URL: https://issues.apache.org/jira/browse/HBASE-11676 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.2.0, 2.3.0 > Environment: 0.98 + hadoop2.2.0 >Reporter: t samkawa >Assignee: Elliot Miller >Priority: Minor > > The "FORMATTER" does not work if the target cell`s column uses binary name. > hbase> create "test1", "f" > hbase> incr "test1", "row1" , "f:a", 1 > hbase> incr "test1", "row1" , "f:\x11", 1 > hbase> scan "test1", COLUMNS=>["f:\x11:toLong","f:a:toLong"] > ROW COLUMN+CELL > row1column=f:\x11, ..., value=\x00\x00\x00\x00\x00\x00\x00\x01 > row1column=f:a, ..., value=1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-11676) Scan FORMATTER is not applied for columns using non-printable name in shell
[ https://issues.apache.org/jira/browse/HBASE-11676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliot Miller updated HBASE-11676: -- Affects Version/s: 3.0.0-alpha-1 > Scan FORMATTER is not applied for columns using non-printable name in shell > > > Key: HBASE-11676 > URL: https://issues.apache.org/jira/browse/HBASE-11676 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.3.0 > Environment: 0.98 + hadoop2.2.0 >Reporter: t samkawa >Assignee: Elliot Miller >Priority: Minor > > The "FORMATTER" does not work if the target cell`s column uses binary name. > hbase> create "test1", "f" > hbase> incr "test1", "row1" , "f:a", 1 > hbase> incr "test1", "row1" , "f:\x11", 1 > hbase> scan "test1", COLUMNS=>["f:\x11:toLong","f:a:toLong"] > ROW COLUMN+CELL > row1column=f:\x11, ..., value=\x00\x00\x00\x00\x00\x00\x00\x01 > row1column=f:a, ..., value=1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-11676) Scan FORMATTER is not applied for columns using non-printable name in shell
[
https://issues.apache.org/jira/browse/HBASE-11676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166481#comment-17166481
]
Elliot Miller commented on HBASE-11676:
---
The root cause of this issue is how we map from column names (ie.
{{"family:qualifier"}}) to converter (ie. {{"toLong"}}). We have been using
{{String.from_java_bytes}} to format the column name when storing a converter
and the *formatted* column name (by default this uses
{{org.apache.hadoop.hbase.util.Bytes.toStringBinary}}) when retrieving a
converter from the mapping.
Incidentally, this means that _you can work around the issue_ by setting the
column name formatter to {{toString}} rather than {{toStringBinary}}. Note that
the value is correctly printed as a long. The caveat is that the column
qualifier will not be visible (since toString drops non-printable characters).
Example:
{code:ruby}
hbase(main):007:0> create 'test1', 'f'
Created table test1
Took 0.7032 seconds
=> Hbase::Table - test1
hbase(main):008:0> incr 'test1', 'row1', "f:\x11", 1
COUNTER VALUE = 1
Took 0.0418 seconds
hbase(main):009:0> scan 'test1', FORMATTER => 'toString', COLUMNS =>
["f:\x11:toLong"]
ROWCOLUMN+CELL
row1 column=f:,
timestamp=2020-07-28T14:46:39.052Z, value=1
1 row(s)
Took 0.0040 seconds
{code}
This issue is also present in the {{get}} command, so I will fix that as well.
Example:
{code:ruby}
hbase(main):018:0> get 'test1', 'row1', COLUMNS => "f:\x11:toLong"
COLUMN CELL
f:\x11timestamp=2020-07-28T14:46:39.052Z,
value=\x00\x00\x00\x00\x00\x00\x00\x01
1 row(s)
Took 0.0036 seconds
{code}
The fix is to use the same method for retrieval that is used when storing the
mapping from column name to converter (ie. {{String.from_java_bytes}}).
*I will be putting up a patch for this shortly.*
> Scan FORMATTER is not applied for columns using non-printable name in shell
>
>
> Key: HBASE-11676
> URL: https://issues.apache.org/jira/browse/HBASE-11676
> Project: HBase
> Issue Type: Bug
> Components: shell
> Environment: 0.98 + hadoop2.2.0
>Reporter: t samkawa
>Assignee: Elliot Miller
>Priority: Minor
>
> The "FORMATTER" does not work if the target cell`s column uses binary name.
> hbase> create "test1", "f"
> hbase> incr "test1", "row1" , "f:a", 1
> hbase> incr "test1", "row1" , "f:\x11", 1
> hbase> scan "test1", COLUMNS=>["f:\x11:toLong","f:a:toLong"]
> ROW COLUMN+CELL
> row1column=f:\x11, ..., value=\x00\x00\x00\x00\x00\x00\x00\x01
> row1column=f:a, ..., value=1
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-24785) [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 3.4.0
[ https://issues.apache.org/jira/browse/HBASE-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-24785. --- Resolution: Fixed > [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 3.4.0 > > > Key: HBASE-24785 > URL: https://issues.apache.org/jira/browse/HBASE-24785 > Project: HBase > Issue Type: Sub-task > Components: thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-24784) [hbase-thirdparty] Set version as 3.4.0 in prep for first RC
[ https://issues.apache.org/jira/browse/HBASE-24784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-24784: --- > [hbase-thirdparty] Set version as 3.4.0 in prep for first RC > > > Key: HBASE-24784 > URL: https://issues.apache.org/jira/browse/HBASE-24784 > Project: HBase > Issue Type: Sub-task > Components: thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24784) [hbase-thirdparty] Set version as 3.4.0 in prep for first RC
[ https://issues.apache.org/jira/browse/HBASE-24784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-24784. --- Resolution: Fixed > [hbase-thirdparty] Set version as 3.4.0 in prep for first RC > > > Key: HBASE-24784 > URL: https://issues.apache.org/jira/browse/HBASE-24784 > Project: HBase > Issue Type: Sub-task > Components: thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-24785) [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 3.4.0
[ https://issues.apache.org/jira/browse/HBASE-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-24785: --- > [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 3.4.0 > > > Key: HBASE-24785 > URL: https://issues.apache.org/jira/browse/HBASE-24785 > Project: HBase > Issue Type: Sub-task > Components: thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24787) [hbase-thirdparty] Set version as 3.4.1-SNAPSHOT
Duo Zhang created HBASE-24787: - Summary: [hbase-thirdparty] Set version as 3.4.1-SNAPSHOT Key: HBASE-24787 URL: https://issues.apache.org/jira/browse/HBASE-24787 Project: HBase Issue Type: Sub-task Components: thirdparty Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24786) [hbase-thirdarty] Put up 3.4.0RC0
Duo Zhang created HBASE-24786: - Summary: [hbase-thirdarty] Put up 3.4.0RC0 Key: HBASE-24786 URL: https://issues.apache.org/jira/browse/HBASE-24786 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24785) [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 3.4.0
Duo Zhang created HBASE-24785: - Summary: [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 3.4.0 Key: HBASE-24785 URL: https://issues.apache.org/jira/browse/HBASE-24785 Project: HBase Issue Type: Sub-task Components: thirdparty Reporter: Duo Zhang Assignee: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24785) [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 3.4.0
[ https://issues.apache.org/jira/browse/HBASE-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-24785: -- Fix Version/s: thirdparty-3.4.0 > [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 3.4.0 > > > Key: HBASE-24785 > URL: https://issues.apache.org/jira/browse/HBASE-24785 > Project: HBase > Issue Type: Sub-task > Components: thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24784) [hbase-thirdparty] Set version as 3.4.0 in prep for first RC
Duo Zhang created HBASE-24784: - Summary: [hbase-thirdparty] Set version as 3.4.0 in prep for first RC Key: HBASE-24784 URL: https://issues.apache.org/jira/browse/HBASE-24784 Project: HBase Issue Type: Sub-task Components: thirdparty Reporter: Duo Zhang Assignee: Duo Zhang Fix For: thirdparty-3.4.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24783) [hbase-thirdparty] Release 3.4.0
Duo Zhang created HBASE-24783: - Summary: [hbase-thirdparty] Release 3.4.0 Key: HBASE-24783 URL: https://issues.apache.org/jira/browse/HBASE-24783 Project: HBase Issue Type: Umbrella Reporter: Duo Zhang Assignee: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24782) [hbase-thirdparty] Bump dependencis in hbase-thirdparty
[ https://issues.apache.org/jira/browse/HBASE-24782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-24782. --- Hadoop Flags: Reviewed Resolution: Fixed Merged to master. Thanks [~busbey] for reviewing. > [hbase-thirdparty] Bump dependencis in hbase-thirdparty > --- > > Key: HBASE-24782 > URL: https://issues.apache.org/jira/browse/HBASE-24782 > Project: HBase > Issue Type: Task > Components: dependencies, thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24752) NPE/500 accessing webui on master startup
[
https://issues.apache.org/jira/browse/HBASE-24752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166422#comment-17166422
]
wenfeiyi666 commented on HBASE-24752:
-
ping [~ndimiduk]
> NPE/500 accessing webui on master startup
> -
>
> Key: HBASE-24752
> URL: https://issues.apache.org/jira/browse/HBASE-24752
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.3.0
>Reporter: Nick Dimiduk
>Assignee: wenfeiyi666
>Priority: Minor
>
> I got a 500 error while accessing the master web ui (non-backup master)
> during startup.
> {noformat}
> 2020-07-21 18:56:39,762 WARN org.eclipse.jetty.servlet.ServletHandler:
> /master-status
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:339)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:397)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:388)
> at
> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:79)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> ...
> {noformat}
> {{MasterStatusTmplImpl.java:339}} appears to align with
> {{MasterStatusTmpl.jamon:227}}:
> {noformat}
> <%if
> master.getMasterCoprocessorHost().findCoprocessor("RSGroupAdminEndpoint") !=
> null &&
> serverManager.getOnlineServersList().size() > 0 %>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24782) [hbase-thirdparty] Bump dependencis in hbase-thirdparty
[ https://issues.apache.org/jira/browse/HBASE-24782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166405#comment-17166405 ] Duo Zhang commented on HBASE-24782: --- Periodical dependencies bump before releasing. Will not upgrade gson as branch-1 still needs the java7 support. > [hbase-thirdparty] Bump dependencis in hbase-thirdparty > --- > > Key: HBASE-24782 > URL: https://issues.apache.org/jira/browse/HBASE-24782 > Project: HBase > Issue Type: Task > Components: dependencies, thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24738) [Shell] processlist command fails with ERROR: Unexpected end of file from server when SSL enabled
[
https://issues.apache.org/jira/browse/HBASE-24738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166388#comment-17166388
]
Pankaj Kumar commented on HBASE-24738:
--
{quote} client configurations have been pretty simple and just need either a
Zookeeper-based or master-based connection registry
{quote}
Make sense.
{quote}We could have the client attempt an HTTP connection and try HTTPS if the
first connection gets dropped.
{quote}
Will update the PR with above approach.
> [Shell] processlist command fails with ERROR: Unexpected end of file from
> server when SSL enabled
> -
>
> Key: HBASE-24738
> URL: https://issues.apache.org/jira/browse/HBASE-24738
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
>
> HBase Shell command "processlist" fails with ERROR: Unexpected end of file
> from server when HBase SSL enabled.
>
> Below code is commented since beginning, see HBASE-4368,
> [https://github.com/apache/hbase/blob/8076eafb187ce32d4a78aef482b6218d85a985ac/hbase-shell/src/main/ruby/hbase/taskmonitor.rb#L85]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24782) [hbase-thirdparty] Bump dependencis in hbase-thirdparty
[ https://issues.apache.org/jira/browse/HBASE-24782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-24782: -- Release Note: netty 4.1.18.Final > 4.1.51.Final protobuf 3.11.4 -> 3.12.2 > [hbase-thirdparty] Bump dependencis in hbase-thirdparty > --- > > Key: HBASE-24782 > URL: https://issues.apache.org/jira/browse/HBASE-24782 > Project: HBase > Issue Type: Task > Components: dependencies, thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24782) [hbase-thirdparty] Bump dependencis in hbase-thirdparty
Duo Zhang created HBASE-24782: - Summary: [hbase-thirdparty] Bump dependencis in hbase-thirdparty Key: HBASE-24782 URL: https://issues.apache.org/jira/browse/HBASE-24782 Project: HBase Issue Type: Task Components: dependencies, thirdparty Reporter: Duo Zhang Assignee: Duo Zhang Fix For: thirdparty-3.4.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22138) [hbase-thirdparty] Undo our direct dependence on protos in google.protobuf.Any in Procedure.proto
[ https://issues.apache.org/jira/browse/HBASE-22138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22138: -- Fix Version/s: (was: thirdparty-3.4.0) > [hbase-thirdparty] Undo our direct dependence on protos in > google.protobuf.Any in Procedure.proto > - > > Key: HBASE-22138 > URL: https://issues.apache.org/jira/browse/HBASE-22138 > Project: HBase > Issue Type: Sub-task > Components: thirdparty >Reporter: Michael Stack >Priority: Major > > in our shaded jar, we've bundled a few unshaded google protos. We make use of > these protos in some of our core classes. What is needed is a bit of careful > work undoing our dependence on these types being careful to unsure we don't > break compatibility (it should be fine but needs some careful operation). > I've targeted this at the next version of hbase-thirdparty. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23811) [OpenTracing] Add shaded JaegerTracing tracer to hbase-thirdparty
[ https://issues.apache.org/jira/browse/HBASE-23811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-23811: -- Fix Version/s: (was: thirdparty-3.4.0) > [OpenTracing] Add shaded JaegerTracing tracer to hbase-thirdparty > - > > Key: HBASE-23811 > URL: https://issues.apache.org/jira/browse/HBASE-23811 > Project: HBase > Issue Type: Sub-task > Components: thirdparty >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > > JaegerTracing pulls in lots of dependencies. Some, like libthrift (0.13.0) > conflicts the one ships in HBase (0.12.0). > Additionally, not everyone may want to use Jaeger. > I propose to shade JaegerTracing and its dependencies into an uber jar, place > it as a hbase-thirdparty artifact. As an added benefit, this makes the > management of tracers in the HBase's dependency tree much easier. Finally, we > can follow the same suit and provide Zipkin tracer support in the future. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-19256) [hbase-thirdparty] shade jetty
[ https://issues.apache.org/jira/browse/HBASE-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19256: -- Fix Version/s: thirdparty-3.4.0 > [hbase-thirdparty] shade jetty > -- > > Key: HBASE-19256 > URL: https://issues.apache.org/jira/browse/HBASE-19256 > Project: HBase > Issue Type: Task > Components: dependencies, thirdparty >Reporter: Mike Drob >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-19256) [hbase-thirdparty] shade jetty
[ https://issues.apache.org/jira/browse/HBASE-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-19256. --- Hadoop Flags: Reviewed Release Note: Introduce two new modules for hbase-thirdparty, one is hbase-shaded-jetty, for shading jetty related jars, and the other is hbase-shaded-jersey, for shading jersey related jars. Resolution: Fixed > [hbase-thirdparty] shade jetty > -- > > Key: HBASE-19256 > URL: https://issues.apache.org/jira/browse/HBASE-19256 > Project: HBase > Issue Type: Task > Components: dependencies, thirdparty >Reporter: Mike Drob >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23810) License checker for hbase-thirdparty
[ https://issues.apache.org/jira/browse/HBASE-23810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-23810: -- Fix Version/s: (was: thirdparty-3.4.0) > License checker for hbase-thirdparty > > > Key: HBASE-23810 > URL: https://issues.apache.org/jira/browse/HBASE-23810 > Project: HBase > Issue Type: Wish > Components: thirdparty >Reporter: Wei-Chiu Chuang >Priority: Minor > > I'm playing with hbase-thirdparty artifacts, and realized it doesn't perform > license check like the core hbase do. > This is a wish list to add that support some day. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24753) HA masters based on raft
[
https://issues.apache.org/jira/browse/HBASE-24753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166371#comment-17166371
]
Duo Zhang commented on HBASE-24753:
---
{quote}
There is one interesting usecase by Cloud to drop a cluster and recreate it
later on existing data. This was/is possible because we never store any
persisting data locally but always on FS. I would say lets not break that. I
read in another jira also says that the Root table data can be stored locally
(RAFT will be in place) not on FS. I would say lets not do that. Let us
continue to have the storage isolation.
{quote}
I do not think the problem here is local storage, it is HDFS, and also
ZooKeeper. We could use EBS as local storage, and it also supports snapshot so
recreate a cluster is easy.
> HA masters based on raft
>
>
> Key: HBASE-24753
> URL: https://issues.apache.org/jira/browse/HBASE-24753
> Project: HBase
> Issue Type: New Feature
> Components: master
>Reporter: Duo Zhang
>Priority: Major
>
> For better availability, for moving bootstrap information from zookeeper to
> our own service so finally we could remove the dependency on zookeeper
> completely.
> This has been in my mind for a long time, and since the there is a dicussion
> in HBASE-11288 about how to storing root table, and also in HBASE-24749, we
> want to have better performance on a filesystem can not support list and
> rename well, where requires a storage engine at the bottom to store the
> storefiles information for meta table, I think it is the time to throw this
> idea out.
> The basic solution is to build a raft group to store the bootstrap
> information, for now it is cluster id(it is on the file system already?) and
> the root table. For region servers they will always go to the leader to ask
> for the information so they can always see the newest data, and for client,
> we enable 'follower read', to reduce the load of the leader(and there are
> some solutions to even let 'follower read' to always get the newest data in
> raft).
> With this solution in place, as long as root table will not be in a format of
> region(we could just use rocksdb to store it locally), the cyclic dependency
> in HBASE-24749 has also been solved, as we do not need to find a place to
> store the storefiles information for root table any more.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24775) [hbtop] StoreFile size should be rounded off
[
https://issues.apache.org/jira/browse/HBASE-24775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166364#comment-17166364
]
Hudson commented on HBASE-24775:
Results for branch branch-2.3
[build #192 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/192/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/192/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/192/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/192/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/192/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [hbtop] StoreFile size should be rounded off
>
>
> Key: HBASE-24775
> URL: https://issues.apache.org/jira/browse/HBASE-24775
> Project: HBase
> Issue Type: Bug
> Components: hbtop
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.2.7
>
> Attachments: Screen Shot.png, Screen Shot2.png
>
>
> Currently, hbtop doesn't show StoreFile size rounded off, so when it's
> reached 1GB it will be very ugly as follows:
> !Screen Shot.png!
> We should round off StoreFile size.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24777) InfoServer support ipv6 host and port
[
https://issues.apache.org/jira/browse/HBASE-24777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166363#comment-17166363
]
Hudson commented on HBASE-24777:
Results for branch branch-2.3
[build #192 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/192/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/192/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/192/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/192/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/192/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> InfoServer support ipv6 host and port
> -
>
> Key: HBASE-24777
> URL: https://issues.apache.org/jira/browse/HBASE-24777
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Affects Versions: 2.3.0, 2.1.9, 2.2.6
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 2.2.7
>
>
> ipv6 host and port is
> [ a:::b:c:d]:16010
> ipv4 host and port is
> a.b.c.d:16010
> we can use HostAndPort to handle it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24775) [hbtop] StoreFile size should be rounded off
[
https://issues.apache.org/jira/browse/HBASE-24775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166335#comment-17166335
]
Hudson commented on HBASE-24775:
Results for branch branch-1
[build #1333 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1333/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1333//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1333//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1333//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> [hbtop] StoreFile size should be rounded off
>
>
> Key: HBASE-24775
> URL: https://issues.apache.org/jira/browse/HBASE-24775
> Project: HBase
> Issue Type: Bug
> Components: hbtop
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.2.7
>
> Attachments: Screen Shot.png, Screen Shot2.png
>
>
> Currently, hbtop doesn't show StoreFile size rounded off, so when it's
> reached 1GB it will be very ugly as follows:
> !Screen Shot.png!
> We should round off StoreFile size.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24757) ReplicationSink should limit the batch rowcount for batch mutations based on hbase.rpc.rows.warning.threshold
[
https://issues.apache.org/jira/browse/HBASE-24757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166334#comment-17166334
]
Hudson commented on HBASE-24757:
Results for branch branch-1
[build #1333 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1333/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1333//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1333//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1333//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> ReplicationSink should limit the batch rowcount for batch mutations based on
> hbase.rpc.rows.warning.threshold
> -
>
> Key: HBASE-24757
> URL: https://issues.apache.org/jira/browse/HBASE-24757
> Project: HBase
> Issue Type: Improvement
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 2.2.7
>
>
> At times there are quite a large no of WAL Edits to ship as part of
> Replication and sometimes replication queues accumulate huge list of Edits to
> process. ReplicationSink at the sink server usually goes through all Edits
> and creates map of table -> list of rows grouped by clusterIds, and performs
> batch mutation of all rows per table level. However, there is no limit to no
> of Rows that are sent as part of batch mutate call. If no of rows > limit
> threshold defined by hbase.rpc.rows.warning.threshold, we usually get warn
> "Large batch operation detected". If hbase.rpc.rows.size.threshold.reject is
> turned on, RS will reject the whole batch without processing.
> We should let Replication Sink honour this threshold value and accordingly
> keep the size lower per batch mutation call.
> Replication triggered batch mutations should always be consumed but keeping
> limit of mutation low enough will let the system function at the same pace
> and without triggering redundant warnings. This will also restrict
> exploitation of cpu cycles at the destination server.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24777) InfoServer support ipv6 host and port
[
https://issues.apache.org/jira/browse/HBASE-24777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166333#comment-17166333
]
Hudson commented on HBASE-24777:
Results for branch branch-1
[build #1333 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1333/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1333//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1333//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1333//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> InfoServer support ipv6 host and port
> -
>
> Key: HBASE-24777
> URL: https://issues.apache.org/jira/browse/HBASE-24777
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Affects Versions: 2.3.0, 2.1.9, 2.2.6
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 2.2.7
>
>
> ipv6 host and port is
> [ a:::b:c:d]:16010
> ipv4 host and port is
> a.b.c.d:16010
> we can use HostAndPort to handle it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24777) InfoServer support ipv6 host and port
[ https://issues.apache.org/jira/browse/HBASE-24777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-24777: Affects Version/s: (was: 2.2.5) 2.2.6 > InfoServer support ipv6 host and port > - > > Key: HBASE-24777 > URL: https://issues.apache.org/jira/browse/HBASE-24777 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 2.3.0, 2.1.9, 2.2.6 >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 2.2.7 > > > ipv6 host and port is > [ a:::b:c:d]:16010 > ipv4 host and port is > a.b.c.d:16010 > we can use HostAndPort to handle it -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24753) HA masters based on raft
[ https://issues.apache.org/jira/browse/HBASE-24753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166294#comment-17166294 ] Tamas Penzes commented on HBASE-24753: -- [~zhangduo] Ozone uses Ratis 1.0.0 according to their pom.xml [https://github.com/apache/hadoop-ozone/blob/master/pom.xml#L82] Looks like it's stable enough for them. > HA masters based on raft > > > Key: HBASE-24753 > URL: https://issues.apache.org/jira/browse/HBASE-24753 > Project: HBase > Issue Type: New Feature > Components: master >Reporter: Duo Zhang >Priority: Major > > For better availability, for moving bootstrap information from zookeeper to > our own service so finally we could remove the dependency on zookeeper > completely. > This has been in my mind for a long time, and since the there is a dicussion > in HBASE-11288 about how to storing root table, and also in HBASE-24749, we > want to have better performance on a filesystem can not support list and > rename well, where requires a storage engine at the bottom to store the > storefiles information for meta table, I think it is the time to throw this > idea out. > The basic solution is to build a raft group to store the bootstrap > information, for now it is cluster id(it is on the file system already?) and > the root table. For region servers they will always go to the leader to ask > for the information so they can always see the newest data, and for client, > we enable 'follower read', to reduce the load of the leader(and there are > some solutions to even let 'follower read' to always get the newest data in > raft). > With this solution in place, as long as root table will not be in a format of > region(we could just use rocksdb to store it locally), the cyclic dependency > in HBASE-24749 has also been solved, as we do not need to find a place to > store the storefiles information for root table any more. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24775) [hbtop] StoreFile size should be rounded off
[
https://issues.apache.org/jira/browse/HBASE-24775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166284#comment-17166284
]
Hudson commented on HBASE-24775:
Results for branch branch-2
[build #2760 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2760/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2760/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2756/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2756/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2756/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [hbtop] StoreFile size should be rounded off
>
>
> Key: HBASE-24775
> URL: https://issues.apache.org/jira/browse/HBASE-24775
> Project: HBase
> Issue Type: Bug
> Components: hbtop
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.2.7
>
> Attachments: Screen Shot.png, Screen Shot2.png
>
>
> Currently, hbtop doesn't show StoreFile size rounded off, so when it's
> reached 1GB it will be very ugly as follows:
> !Screen Shot.png!
> We should round off StoreFile size.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24777) InfoServer support ipv6 host and port
[
https://issues.apache.org/jira/browse/HBASE-24777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166282#comment-17166282
]
Hudson commented on HBASE-24777:
Results for branch branch-2
[build #2760 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2760/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2760/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2756/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2756/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2756/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> InfoServer support ipv6 host and port
> -
>
> Key: HBASE-24777
> URL: https://issues.apache.org/jira/browse/HBASE-24777
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Affects Versions: 2.3.0, 2.1.9, 2.2.5
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 2.2.7
>
>
> ipv6 host and port is
> [ a:::b:c:d]:16010
> ipv4 host and port is
> a.b.c.d:16010
> we can use HostAndPort to handle it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24632) Enable procedure-based log splitting as default in hbase3
[
https://issues.apache.org/jira/browse/HBASE-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166283#comment-17166283
]
Hudson commented on HBASE-24632:
Results for branch branch-2
[build #2760 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2760/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2760/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2756/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2756/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2756/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Enable procedure-based log splitting as default in hbase3
> -
>
> Key: HBASE-24632
> URL: https://issues.apache.org/jira/browse/HBASE-24632
> Project: HBase
> Issue Type: Sub-task
> Components: wal
>Reporter: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Means changing this value in HConstants to false:
>public static final boolean DEFAULT_HBASE_SPLIT_COORDINATED_BY_ZK = true;
> Should probably also deprecate the current zk distributed split too so we can
> clear out those classes to.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24781) when execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
[ https://issues.apache.org/jira/browse/HBASE-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leizhang updated HBASE-24781: - Description: Supposed that we have an peer with id 1, when execute shell cmd disable_peer '1' , then i can see the SizeOfLogQueue metric of all regionservers +1 , after 10 times disable_peer ops , it will increase to 11, and it will never decrease to 1 in fulture . I can see the function ReplicationSourceManager.refreshSources(peerId) called , it will enqueue the current wals to the source , maybe when the current wal is already in the replication queue , we try to add a duplicated wal to the source ,which cause the same wal increase the SizeOfLogQueue twice ? thx was: Supposed that we have an peer with id 1, when execute shell cmd disable_peer '1' , then i can see the SizeOfLogQueue metric of all regionservers +1 , after 10 times disable_peer ops , it will increase to 11, and it will never decrease to 1 in fulture . I can see the function ReplicationSourceManager.refreshSources(peerId) called , it will enqueue the current wals to the source , maybe when the current wal is already in the replication queue , we try to and a duplicated wal to the source ,which cause the same wal increase the SizeOfLogQueue twice ? thx > when execute shell cmd "disable_peer peerId"，the master web UI show a wrong > number of SizeOfLogQueue > - > > Key: HBASE-24781 > URL: https://issues.apache.org/jira/browse/HBASE-24781 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.2.5 >Reporter: leizhang >Priority: Major > > Supposed that we have an peer with id 1, when execute shell cmd > disable_peer '1' , then i can see the SizeOfLogQueue metric of all > regionservers +1 , after 10 times disable_peer ops , it will increase to > 11, and it will never decrease to 1 in fulture . > I can see the function ReplicationSourceManager.refreshSources(peerId) > called , it will enqueue the current wals to the source , maybe when the > current wal is already in the replication queue , we try to add a duplicated > wal to the source ,which cause the same wal increase the SizeOfLogQueue > twice ? thx > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24781) when execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
[ https://issues.apache.org/jira/browse/HBASE-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leizhang updated HBASE-24781: - Description: Supposed that we have an peer with id 1, when execute shell cmd disable_peer '1' , then i can see the SizeOfLogQueue metric of all regionservers +1 , after 10 times disable_peer ops , it will increase to 11, and it will never decrease to 1 in fulture . I can see the function ReplicationSourceManager.refreshSources(peerId) called , it will enqueue the current wals to the source , maybe when the current wal is already in the replication queue , we try to and a duplicated wal to the source ,which cause the same wal increase the SizeOfLogQueue twice ? thx was: Supposed that we have an source peer with id 1, when execute shell cmd disable_peer '1' , then i can see the SizeOfLogQueue metric of all regionservers +1 , after 10 times disable_peer ops , it will increase to 11, and it will never decrease to 1 in fulture . I can see the function ReplicationSourceManager.refreshSources(peerId) called , it will enqueue the current wals to the source , maybe when the current wal is already in the replication queue , we try to and a duplicated wal to the source ,which cause the same wal increase the SizeOfLogQueue twice ? thx > when execute shell cmd "disable_peer peerId"，the master web UI show a wrong > number of SizeOfLogQueue > - > > Key: HBASE-24781 > URL: https://issues.apache.org/jira/browse/HBASE-24781 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.2.5 >Reporter: leizhang >Priority: Major > > Supposed that we have an peer with id 1, when execute shell cmd > disable_peer '1' , then i can see the SizeOfLogQueue metric of all > regionservers +1 , after 10 times disable_peer ops , it will increase to > 11, and it will never decrease to 1 in fulture . > I can see the function ReplicationSourceManager.refreshSources(peerId) > called , it will enqueue the current wals to the source , maybe when the > current wal is already in the replication queue , we try to and a duplicated > wal to the source ,which cause the same wal increase the SizeOfLogQueue > twice ? thx > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24781) when execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
[ https://issues.apache.org/jira/browse/HBASE-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leizhang updated HBASE-24781: - Description: Supposed that we have an source peer with id 1, when execute shell cmd disable_peer '1' , then i can see the SizeOfLogQueue metric of all regionservers +1 , after 10 times disable_peer ops , it will increase to 11, and it will never decrease to 1 in fulture . I can see the function ReplicationSourceManager.refreshSources(peerId) called , it will enqueue the current wals to the source , maybe when the current wal is already in the replication queue , we try to and a duplicated wal to the source ,which cause the same wal increase the SizeOfLogQueue twice ? thx was: Supposed that we have an source peer with id 1, when execute shell cmd disable_peer '1' , then i can see the SizeOfLogQueue metric of all regionservers +1 , after 10 times disable_peer ops , it will increase to 11, and it will never decrease to 1 in fulture . I can see the function ReplicationSourceManager.refreshSources(peerId) called , it will enqueue the current wals to the source , maybe when the current wal is already in the replication queue , we try to and a duplicated wal to the source ,and cause the same wal increase the SizeOfLogQueue twice ? thx > when execute shell cmd "disable_peer peerId"，the master web UI show a wrong > number of SizeOfLogQueue > - > > Key: HBASE-24781 > URL: https://issues.apache.org/jira/browse/HBASE-24781 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.2.5 >Reporter: leizhang >Priority: Major > > Supposed that we have an source peer with id 1, when execute shell cmd > disable_peer '1' , then i can see the SizeOfLogQueue metric of all > regionservers +1 , after 10 times disable_peer ops , it will increase to > 11, and it will never decrease to 1 in fulture . > I can see the function ReplicationSourceManager.refreshSources(peerId) > called , it will enqueue the current wals to the source , maybe when the > current wal is already in the replication queue , we try to and a duplicated > wal to the source ,which cause the same wal increase the SizeOfLogQueue > twice ? thx > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24781) when execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue
leizhang created HBASE-24781: Summary: when execute shell cmd "disable_peer peerId"，the master web UI show a wrong number of SizeOfLogQueue Key: HBASE-24781 URL: https://issues.apache.org/jira/browse/HBASE-24781 Project: HBase Issue Type: Bug Components: Replication Affects Versions: 2.2.5 Reporter: leizhang Supposed that we have an source peer with id 1, when execute shell cmd disable_peer '1' , then i can see the SizeOfLogQueue metric of all regionservers +1 , after 10 times disable_peer ops , it will increase to 11, and it will never decrease to 1 in fulture . I can see the function ReplicationSourceManager.refreshSources(peerId) called , it will enqueue the current wals to the source , maybe when the current wal is already in the replication queue , we try to and a duplicated wal to the source ,and cause the same wal increase the SizeOfLogQueue twice ? thx -- This message was sent by Atlassian Jira (v8.3.4#803005)
