[jira] [Commented] (HBASE-15593) Time limit of scanning should be offered by client
[ https://issues.apache.org/jira/browse/HBASE-15593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231702#comment-15231702 ] Phil Yang commented on HBASE-15593: --- Thanks for your reply. I use timeout in my patch and it fixes the bug of scanner time limit. > Time limit of scanning should be offered by client > -- > > Key: HBASE-15593 > URL: https://issues.apache.org/jira/browse/HBASE-15593 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.1.4 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-15593-v1.patch > > > In RSRpcServices.scan, we will set a time limit equaling to > Math.min(scannerLeaseTimeoutPeriod, rpcTimeout) / 2, and will response > heartbeat message if we reach this limit. However, two timeout settings > (hbase.client.scanner.timeout.period and hbase.rpc.timeout) are read from > RS's configure, which may be different from client's. If client's setting is > much less than server's, there may still be timeout at client side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15586) Unify human readable numbers in the web UI
[ https://issues.apache.org/jira/browse/HBASE-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231691#comment-15231691 ] Hudson commented on HBASE-15586: SUCCESS: Integrated in HBase-1.4 #76 (See [https://builds.apache.org/job/HBase-1.4/76/]) HBASE-15586 Unify human readable numbers in the web UI (enis: rev d94c18a0e72443da0857a473f9b1ff9d2bb40854) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RegionListTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon > Unify human readable numbers in the web UI > --- > > Key: HBASE-15586 > URL: https://issues.apache.org/jira/browse/HBASE-15586 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: Screen Shot 2016-04-01 at 3.52.00 PM.png, Screen Shot > 2016-04-01 at 3.52.12 PM.png, Screen Shot 2016-04-01 at 3.52.32 PM.png, > Screen Shot 2016-04-01 at 3.52.38 PM.png, Screen Shot 2016-04-01 at 3.52.49 > PM.png, hbase-15586-v1.patch, hbase-15586-v1.patch, hbase-15586-v2.patch > > > I was looking at something else in the regionserver web ui trying to > understand the WAL size, and we are reporting that as raw bytes, not in MB / > GB range. Did a quick sweep to unify all the human-readable representations > in the UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15537) Make multi WAL work with WALs other than FSHLog
[
https://issues.apache.org/jira/browse/HBASE-15537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231690#comment-15231690
]
Hudson commented on HBASE-15537:
SUCCESS: Integrated in HBase-1.4 #76 (See
[https://builds.apache.org/job/HBase-1.4/76/])
HBASE-15537 Make multi WAL work with WALs other than FSHLog (zhangduo: rev
2b75562701a2c187f30cb50219817984a36bf013)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/multiwal/TestReplicationSyncUpToolWithMultipleWAL.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/multiwal/TestReplicationKillMasterRSCompressedWithMultipleWAL.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestBoundedRegionGroupingStrategy.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALFactory.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RegionGroupingProvider.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/multiwal/TestReplicationEndpointWithMultipleWAL.java
> Make multi WAL work with WALs other than FSHLog
> ---
>
> Key: HBASE-15537
> URL: https://issues.apache.org/jira/browse/HBASE-15537
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15537-branch-1.patch, HBASE-15537-v3.patch,
> HBASE-15537-v4.patch, HBASE-15537-v5.patch, HBASE-15537-v6.patch,
> HBASE-15537.patch, HBASE-15537_v2.patch
>
>
> The multi WAL should not be bound with {{FSHLog}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15593) Time limit of scanning should be offered by client
[ https://issues.apache.org/jira/browse/HBASE-15593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231683#comment-15231683 ] stack commented on HBASE-15593: --- #2 has less environmental dependency. Ok to ignore the trip from client to server IMO. > Time limit of scanning should be offered by client > -- > > Key: HBASE-15593 > URL: https://issues.apache.org/jira/browse/HBASE-15593 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.1.4 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-15593-v1.patch > > > In RSRpcServices.scan, we will set a time limit equaling to > Math.min(scannerLeaseTimeoutPeriod, rpcTimeout) / 2, and will response > heartbeat message if we reach this limit. However, two timeout settings > (hbase.client.scanner.timeout.period and hbase.rpc.timeout) are read from > RS's configure, which may be different from client's. If client's setting is > much less than server's, there may still be timeout at client side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15593) Time limit of scanning should be offered by client
[ https://issues.apache.org/jira/browse/HBASE-15593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231681#comment-15231681 ] stack commented on HBASE-15593: --- I don't believe there is a timeout set by the Client. We need one though as you note. > Time limit of scanning should be offered by client > -- > > Key: HBASE-15593 > URL: https://issues.apache.org/jira/browse/HBASE-15593 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.1.4 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-15593-v1.patch > > > In RSRpcServices.scan, we will set a time limit equaling to > Math.min(scannerLeaseTimeoutPeriod, rpcTimeout) / 2, and will response > heartbeat message if we reach this limit. However, two timeout settings > (hbase.client.scanner.timeout.period and hbase.rpc.timeout) are read from > RS's configure, which may be different from client's. If client's setting is > much less than server's, there may still be timeout at client side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15572) Adding optional timestamp semantics to HBase-Spark
[
https://issues.apache.org/jira/browse/HBASE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231680#comment-15231680
]
Hadoop QA commented on HBASE-15572:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 59s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 44s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 38s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 38s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} scaladoc {color} | {color:red} 0m 7s
{color} | {color:red} root in master failed. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 51s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 7m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 45s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 7m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 41s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 35s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} scaladoc {color} | {color:red} 0m 7s
{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} scaladoc {color} | {color:red} 0m 7s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 16s
{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 17s
{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 17s
{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 15s
{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 144m 18s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
18s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 250m 51s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit
[jira] [Updated] (HBASE-15612) Minor improvements to CellCounter and RowCounter documentation
[ https://issues.apache.org/jira/browse/HBASE-15612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15612: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) Nice one [~esteban] Pushed to master. > Minor improvements to CellCounter and RowCounter documentation > -- > > Key: HBASE-15612 > URL: https://issues.apache.org/jira/browse/HBASE-15612 > Project: HBase > Issue Type: Improvement > Components: documentation, mapreduce >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-15612.patch > > > Both Javadoc and the HBase Book need to reflect that is possible to specify > an optional time range in the command line arguments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-15615:
---
Description:
In RpcRetryingCallerImpl, it get pause time by expectedSleep =
callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries is
0.
RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
was:
In RpcRetryingCallerImpl, it get pause time by expectedSleep =
callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries is
0.
RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}.
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-15615:
---
Description:
In RpcRetryingCallerImpl, it get pause time by expectedSleep =
callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries is
0.
RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}.
was:
In RpcRetryingCallerImpl, it get pause time by expectedSleep =
callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
be bumped up twice.
Now RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}. So
the pasue time is 3 * hbase.client.pause when tries is 0.
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-15615:
---
Description:
In RpcRetryingCallerImpl, it get pause time by expectedSleep =
callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
be bumped up twice.
Now RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}. So
the pasue time is 3 * hbase.client.pause when tries is 0.
was:
In RpcRetryingCallerImpl, it get pause time by expectedSleep =
callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
be bumped up twice.
Now RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}; So
the pasue time is 3 * hbase.client.pause when tries is 0.
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice.
> Now RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}.
> So the pasue time is 3 * hbase.client.pause when tries is 0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
Guanghao Zhang created HBASE-15615:
--
Summary: Wrong sleep time when RegionServerCallable need retry
Key: HBASE-15615
URL: https://issues.apache.org/jira/browse/HBASE-15615
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0
Reporter: Guanghao Zhang
In RpcRetryingCallerImpl, it get pause time by expectedSleep =
callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
be bumped up twice.
Now RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}; So
the pasue time is 3 * hbase.client.pause when tries is 0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15380) Purge rollback support in Store etc.
[ https://issues.apache.org/jira/browse/HBASE-15380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15380: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to master only. The checkstyle complaints are a little silly. Thanks for reviews [~chenheng] and [~anoop.hbase] > Purge rollback support in Store etc. > > > Key: HBASE-15380 > URL: https://issues.apache.org/jira/browse/HBASE-15380 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: 15380.patch, 15380v2.patch, 15380v3.patch > > > Rollback is no longer needed after "HBASE-15158 Change order in which we do > write pipeline operations; do all under row locks". Purge support. Will > simplify this segment work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15507) Online modification of enabled ReplicationPeerConfig
[
https://issues.apache.org/jira/browse/HBASE-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231659#comment-15231659
]
Hadoop QA commented on HBASE-15507:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 4s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 4s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 19s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 5s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 13m
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 34s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 12s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 24s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 54s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 54s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 38s
{color} | {color:red} hbase-client: patch generated 36 new + 99 unchanged - 3
fixed = 135 total (was 102) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 0s
{color} | {color:red} hbase-client: patch generated 36 new + 99 unchanged - 3
fixed = 135 total (was 102) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 26s
{color} | {color:red} hbase-server: patch generated 36 new + 99 unchanged - 3
fixed = 135 total (was 102) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 26s
{color} | {color:red} hbase-server: patch generated 36 new + 99 unchanged - 3
fixed = 135 total (was 102) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 47s
{color} | {color:red} hbase-shell: patch generated 36 new + 99 unchanged - 3
fixed = 135 total (was 102) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 48s
{color} | {color:red} hbase-shell: patch generated 36 new + 99 unchanged - 3
fixed = 135 total (was 102) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
37m 9s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 58s
{color} | {color:red} hbase-client generated 1 new
[jira] [Commented] (HBASE-15335) Add composite key support in row key
[
https://issues.apache.org/jira/browse/HBASE-15335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231654#comment-15231654
]
Hadoop QA commented on HBASE-15335:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-15335 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12797667/HBASE-15335-1.patch |
| JIRA Issue | HBASE-15335 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1340/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
> Add composite key support in row key
>
>
> Key: HBASE-15335
> URL: https://issues.apache.org/jira/browse/HBASE-15335
> Project: HBase
> Issue Type: Sub-task
>Reporter: Zhan Zhang
>Assignee: Zhan Zhang
> Attachments: HBASE-15335-1.patch
>
>
> Add composite key filter support in the connector.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15606) Limit creating zk connection in HBaseAdmin#getCompactionState() only to case when 'hbase:meta' is checked.
[ https://issues.apache.org/jira/browse/HBASE-15606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231650#comment-15231650 ] Samir Ahmic commented on HBASE-15606: - HBaseAdmin#getCompactionState() is changed since branch-1 let me see what can be done there to limit connection creation. > Limit creating zk connection in HBaseAdmin#getCompactionState() only to case > when 'hbase:meta' is checked. > -- > > Key: HBASE-15606 > URL: https://issues.apache.org/jira/browse/HBASE-15606 > Project: HBase > Issue Type: Improvement > Components: Admin >Affects Versions: 2.0.0 >Reporter: Samir Ahmic >Assignee: Samir Ahmic >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15606_v0.patch > > > We are creating new zk connection for every call to this function if we are > checking case of normal compaction. Connection is used to locate 'hbase:meta' > table. > Aim of this jira is to limit connection creation only to case when meta table > is checked. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15335) Add composite key support in row key
[ https://issues.apache.org/jira/browse/HBASE-15335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-15335: --- Status: Patch Available (was: Open) > Add composite key support in row key > > > Key: HBASE-15335 > URL: https://issues.apache.org/jira/browse/HBASE-15335 > Project: HBase > Issue Type: Sub-task >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Attachments: HBASE-15335-1.patch > > > Add composite key filter support in the connector. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15335) Add composite key support in row key
[ https://issues.apache.org/jira/browse/HBASE-15335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-15335: --- Assignee: Zhan Zhang > Add composite key support in row key > > > Key: HBASE-15335 > URL: https://issues.apache.org/jira/browse/HBASE-15335 > Project: HBase > Issue Type: Sub-task >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Attachments: HBASE-15335-1.patch > > > Add composite key filter support in the connector. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15577) there need be a mechanism to enable ZK's ACL check when the authentication strategy is simple
[
https://issues.apache.org/jira/browse/HBASE-15577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenxu updated HBASE-15577:
---
Attachment: HBASE-15577-03.patch
> there need be a mechanism to enable ZK's ACL check when the authentication
> strategy is simple
> -
>
> Key: HBASE-15577
> URL: https://issues.apache.org/jira/browse/HBASE-15577
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.3
>Reporter: chenxu
>Assignee: chenxu
> Attachments: HBASE-15577-02.patch, HBASE-15577-03.patch,
> HBASE-15577.patch, zk-set-acl.patch
>
>
> if the hbase.security.authentication is set to simple, the ZKUtil.createACL
> just return Ids.OPEN_ACL_UNSAFE, means that there is no ACL check on the ZK's
> node.
> we can refactoring this to enables the ACL's check function
> manual steps to verify the patch
> *1.set this property in the hbase-site.xml*
> {quote}
>hbase.security.authentication(simple)
>hbase.zookeeper.acl (digest:admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc=:cdrwa)
>hbase.zookeeper.auth(digest:admin)
> {quote}
> the digest can generate by the
> DigestAuthenticationProvider.generateDigest("admin")
> *2.start the cluster*
> *3.verify the zk's node*
> {quote}
>(1)getAcl /hbase, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
>'world,'anyone: r
>(2)getAcl /hbase/table-lock, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
> {quote}
> if the node is below, all the client can read the node, but only the
> server(Regionserver & Master which has the auth info) can modify it
> {quote}
> /hbase
> /hbase/meta-region-server
> /hbase/master
> /hbase/hbaseid
> /hbase/rs
> /hbase/table
> /hbase/table/$tableName
> {quote}
> otherwise, only the server can read and modify the node, the Client can't see
> them
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15335) Add composite key support in row key
[ https://issues.apache.org/jira/browse/HBASE-15335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-15335: --- Attachment: HBASE-15335-1.patch > Add composite key support in row key > > > Key: HBASE-15335 > URL: https://issues.apache.org/jira/browse/HBASE-15335 > Project: HBase > Issue Type: Sub-task >Reporter: Zhan Zhang > Attachments: HBASE-15335-1.patch > > > Add composite key filter support in the connector. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15577) there need be a mechanism to enable ZK's ACL check when the authentication strategy is simple
[
https://issues.apache.org/jira/browse/HBASE-15577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenxu updated HBASE-15577:
---
Attachment: (was: HBASE-15577-03.patch)
> there need be a mechanism to enable ZK's ACL check when the authentication
> strategy is simple
> -
>
> Key: HBASE-15577
> URL: https://issues.apache.org/jira/browse/HBASE-15577
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.3
>Reporter: chenxu
>Assignee: chenxu
> Attachments: HBASE-15577-02.patch, HBASE-15577.patch, zk-set-acl.patch
>
>
> if the hbase.security.authentication is set to simple, the ZKUtil.createACL
> just return Ids.OPEN_ACL_UNSAFE, means that there is no ACL check on the ZK's
> node.
> we can refactoring this to enables the ACL's check function
> manual steps to verify the patch
> *1.set this property in the hbase-site.xml*
> {quote}
>hbase.security.authentication(simple)
>hbase.zookeeper.acl (digest:admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc=:cdrwa)
>hbase.zookeeper.auth(digest:admin)
> {quote}
> the digest can generate by the
> DigestAuthenticationProvider.generateDigest("admin")
> *2.start the cluster*
> *3.verify the zk's node*
> {quote}
>(1)getAcl /hbase, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
>'world,'anyone: r
>(2)getAcl /hbase/table-lock, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
> {quote}
> if the node is below, all the client can read the node, but only the
> server(Regionserver & Master which has the auth info) can modify it
> {quote}
> /hbase
> /hbase/meta-region-server
> /hbase/master
> /hbase/hbaseid
> /hbase/rs
> /hbase/table
> /hbase/table/$tableName
> {quote}
> otherwise, only the server can read and modify the node, the Client can't see
> them
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15335) Add composite key support in row key
[ https://issues.apache.org/jira/browse/HBASE-15335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-15335: --- Attachment: (was: HBASE-15335-1.patch) > Add composite key support in row key > > > Key: HBASE-15335 > URL: https://issues.apache.org/jira/browse/HBASE-15335 > Project: HBase > Issue Type: Sub-task >Reporter: Zhan Zhang > Attachments: HBASE-15335-1.patch > > > Add composite key filter support in the connector. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15606) Limit creating zk connection in HBaseAdmin#getCompactionState() only to case when 'hbase:meta' is checked.
[ https://issues.apache.org/jira/browse/HBASE-15606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231636#comment-15231636 ] Hudson commented on HBASE-15606: FAILURE: Integrated in HBase-Trunk_matrix #830 (See [https://builds.apache.org/job/HBase-Trunk_matrix/830/]) HBASE-15606 Limit creating zk connection in (stack: rev d393603dea23306cd3f18f6dbd1cf14561d45bd0) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java > Limit creating zk connection in HBaseAdmin#getCompactionState() only to case > when 'hbase:meta' is checked. > -- > > Key: HBASE-15606 > URL: https://issues.apache.org/jira/browse/HBASE-15606 > Project: HBase > Issue Type: Improvement > Components: Admin >Affects Versions: 2.0.0 >Reporter: Samir Ahmic >Assignee: Samir Ahmic >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15606_v0.patch > > > We are creating new zk connection for every call to this function if we are > checking case of normal compaction. Connection is used to locate 'hbase:meta' > table. > Aim of this jira is to limit connection creation only to case when meta table > is checked. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15400) Use DateTieredCompactor for Date Tiered Compaction
[ https://issues.apache.org/jira/browse/HBASE-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231637#comment-15231637 ] Hudson commented on HBASE-15400: FAILURE: Integrated in HBase-Trunk_matrix #830 (See [https://builds.apache.org/job/HBase-Trunk_matrix/830/]) HBASE-15400 Use DateTieredCompactor for Date Tiered Compaction (Clara (tedyu: rev f60fc9d1a0625970aa2fd14d29e4c1266f9571b7) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/FIFOCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/SortedCompactionPolicy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/EverythingPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionPolicy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDateTieredCompactionPolicy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MockStoreFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionConfiguration.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DateTieredStoreEngine.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionRequest.java HBASE-15400 Use DateTieredCompactor for Date Tiered Compaction - drop (tedyu: rev a146a71a332fdd58f9bf4e748861f5b050a5f22f) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDateTieredCompaction.java > Use DateTieredCompactor for Date Tiered Compaction > -- > > Key: HBASE-15400 > URL: https://issues.apache.org/jira/browse/HBASE-15400 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15400-0.98.patch, HBASE-15400-15389-v12.patch, > HBASE-15400-branch-1.patch, HBASE-15400-v1.pa, HBASE-15400-v3-v3.patch, > HBASE-15400-v3-v4.patch, HBASE-15400-v3-v5.patch, HBASE-15400-v3.patch, > HBASE-15400-v6.patch, HBASE-15400-v7.patch, HBASE-15400.patch > > > When we compact, we can output multiple files along the current window > boundaries. There are two use cases: > 1. Major compaction: We want to output date tiered store files with data > older than max age archived in trunks of the window size on the higher tier. > Once a window is old enough, we don't combine the windows to promote to the > next tier any further. So files in these windows retain the same timespan as > they were minor-compacted last time, which is the window size of the highest > tier. Major compaction will touch these files and we want to maintain the > same layout. This way, TTL and archiving will be simpler and more efficient. > 2. Bulk load files and the old file generated by major compaction before > upgrading to DTCP. > Pros: > 1. Restore locality, process versioning, updates and deletes while > maintaining the tiered layout. > 2. The best way to fix a skewed layout. > > This work is based on a prototype of DateTieredCompactor from HBASE-15389 and > focused on the part to meet needs for these two use cases while supporting > others. I have to call out a few design decisions: > 1. We only want to output the files along all windows for major compaction. > And we want to output multiple files older than max age in the trunks of the > maximum tier window size determined by base window size, windows per tier and > max age. > 2. For minor compaction, we don't want to output too many files, which will > remain around because of current restriction of contiguous compaction by seq > id. I will only output two files if all the files in the windows are being > combined, one for the data within window and the other for the out-of-window > tail. If there is any file in the window excluded from compaction, only one > file will be output from compaction. When the windows are promoted, the > situation of out of order data will gradually improve. For the incoming > window, we need to accommodat
[jira] [Commented] (HBASE-15586) Unify human readable numbers in the web UI
[ https://issues.apache.org/jira/browse/HBASE-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231635#comment-15231635 ] Hudson commented on HBASE-15586: FAILURE: Integrated in HBase-Trunk_matrix #830 (See [https://builds.apache.org/job/HBase-Trunk_matrix/830/]) HBASE-15586 Unify human readable numbers in the web UI (enis: rev 2dcd08bc3d8ab2f26da85128102926c68a95186f) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RegionListTmpl.jamon > Unify human readable numbers in the web UI > --- > > Key: HBASE-15586 > URL: https://issues.apache.org/jira/browse/HBASE-15586 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: Screen Shot 2016-04-01 at 3.52.00 PM.png, Screen Shot > 2016-04-01 at 3.52.12 PM.png, Screen Shot 2016-04-01 at 3.52.32 PM.png, > Screen Shot 2016-04-01 at 3.52.38 PM.png, Screen Shot 2016-04-01 at 3.52.49 > PM.png, hbase-15586-v1.patch, hbase-15586-v1.patch, hbase-15586-v2.patch > > > I was looking at something else in the regionserver web ui trying to > understand the WAL size, and we are reporting that as raw bytes, not in MB / > GB range. Did a quick sweep to unify all the human-readable representations > in the UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15335) Add composite key support in row key
[ https://issues.apache.org/jira/browse/HBASE-15335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-15335: --- Attachment: HBASE-15335-1.patch This patch is built on top of HBASE-15333, which delayed for a while. Upload this patch for preview. > Add composite key support in row key > > > Key: HBASE-15335 > URL: https://issues.apache.org/jira/browse/HBASE-15335 > Project: HBase > Issue Type: Sub-task >Reporter: Zhan Zhang > Attachments: HBASE-15335-1.patch > > > Add composite key filter support in the connector. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15473) Documentation for the usage of hbase dataframe user api (JSON, Avro, etc)
[ https://issues.apache.org/jira/browse/HBASE-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231629#comment-15231629 ] Zhan Zhang commented on HBASE-15473: Currently the dataframe support has several places broken, e.g., data loss, data duplication, etc. Will update the document after these issues are fixed. > Documentation for the usage of hbase dataframe user api (JSON, Avro, etc) > - > > Key: HBASE-15473 > URL: https://issues.apache.org/jira/browse/HBASE-15473 > Project: HBase > Issue Type: Sub-task > Components: documentation, spark >Reporter: Zhan Zhang >Assignee: Zhan Zhang >Priority: Blocker > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15473) Documentation for the usage of hbase dataframe user api (JSON, Avro, etc)
[ https://issues.apache.org/jira/browse/HBASE-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-15473: --- Assignee: Zhan Zhang > Documentation for the usage of hbase dataframe user api (JSON, Avro, etc) > - > > Key: HBASE-15473 > URL: https://issues.apache.org/jira/browse/HBASE-15473 > Project: HBase > Issue Type: Sub-task > Components: documentation, spark >Reporter: Zhan Zhang >Assignee: Zhan Zhang >Priority: Blocker > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15550) Backport HBASE-12220 hedgedRead metrics to 1.2+
[ https://issues.apache.org/jira/browse/HBASE-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231621#comment-15231621 ] stack commented on HBASE-15550: --- Doing it. > Backport HBASE-12220 hedgedRead metrics to 1.2+ > --- > > Key: HBASE-15550 > URL: https://issues.apache.org/jira/browse/HBASE-15550 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.0, 1.2.2 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: 15550.branch-1.2.txt > > > hedgedRead metrics are in master branch only. They were put in branch-1 but > then reverted because old versions of hadoop didn't supported hedgedReads. > 1.2+ has minimal hadoop version. Retry backport. > FYI [~phobos182] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15400) Use DateTieredCompactor for Date Tiered Compaction
[ https://issues.apache.org/jira/browse/HBASE-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clara Xiong updated HBASE-15400: Attachment: HBASE-15400-0.98.patch Patch for 0.98. We need to port it to 0.98. > Use DateTieredCompactor for Date Tiered Compaction > -- > > Key: HBASE-15400 > URL: https://issues.apache.org/jira/browse/HBASE-15400 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15400-0.98.patch, HBASE-15400-15389-v12.patch, > HBASE-15400-branch-1.patch, HBASE-15400-v1.pa, HBASE-15400-v3-v3.patch, > HBASE-15400-v3-v4.patch, HBASE-15400-v3-v5.patch, HBASE-15400-v3.patch, > HBASE-15400-v6.patch, HBASE-15400-v7.patch, HBASE-15400.patch > > > When we compact, we can output multiple files along the current window > boundaries. There are two use cases: > 1. Major compaction: We want to output date tiered store files with data > older than max age archived in trunks of the window size on the higher tier. > Once a window is old enough, we don't combine the windows to promote to the > next tier any further. So files in these windows retain the same timespan as > they were minor-compacted last time, which is the window size of the highest > tier. Major compaction will touch these files and we want to maintain the > same layout. This way, TTL and archiving will be simpler and more efficient. > 2. Bulk load files and the old file generated by major compaction before > upgrading to DTCP. > Pros: > 1. Restore locality, process versioning, updates and deletes while > maintaining the tiered layout. > 2. The best way to fix a skewed layout. > > This work is based on a prototype of DateTieredCompactor from HBASE-15389 and > focused on the part to meet needs for these two use cases while supporting > others. I have to call out a few design decisions: > 1. We only want to output the files along all windows for major compaction. > And we want to output multiple files older than max age in the trunks of the > maximum tier window size determined by base window size, windows per tier and > max age. > 2. For minor compaction, we don't want to output too many files, which will > remain around because of current restriction of contiguous compaction by seq > id. I will only output two files if all the files in the windows are being > combined, one for the data within window and the other for the out-of-window > tail. If there is any file in the window excluded from compaction, only one > file will be output from compaction. When the windows are promoted, the > situation of out of order data will gradually improve. For the incoming > window, we need to accommodate the case with user-specified future data. > 3. We have to pass the boundaries with the list of store file as a complete > time snapshot instead of two separate calls because window layout is > determined by the time the computation is called. So we will need new type of > compaction request. > 4. Since we will assign the same seq id for all output files, we need to sort > by maxTimestamp subsequently. Right now all compaction policy gets the files > sorted for StoreFileManager which sorts by seq id and other criteria. I will > use this order for DTCP only, to avoid impacting other compaction policies. > 5. We need some cleanup of current design of StoreEngine and CompactionPolicy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15614) Report metrics from JvmPauseMonitor
[
https://issues.apache.org/jira/browse/HBASE-15614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231606#comment-15231606
]
Heng Chen commented on HBASE-15614:
---
+1 for this. Now we grep this log and post it to OpenTSDB
> Report metrics from JvmPauseMonitor
> ---
>
> Key: HBASE-15614
> URL: https://issues.apache.org/jira/browse/HBASE-15614
> Project: HBase
> Issue Type: Improvement
> Components: metrics, regionserver
>Reporter: Nick Dimiduk
>
> We have {{JvmPauseMonitor}} for detecting JVM pauses; pauses are logged at
> WARN. Would also be good to expose this information on a dashboard via
> metrics system -- make it easier to get this info off the host and into a
> central location for the operator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15577) there need be a mechanism to enable ZK's ACL check when the authentication strategy is simple
[
https://issues.apache.org/jira/browse/HBASE-15577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231607#comment-15231607
]
Yu Li commented on HBASE-15577:
---
The new {{TestSimpleZKACL}} case looks good.
Please use LOG.warn instead of printStackTrace in the below part:
{noformat}
+ } catch (IOException e) {
+e.printStackTrace();
+return Ids.OPEN_ACL_UNSAFE;
+ }
{noformat}
All other parts lgtm. Thanks.
> there need be a mechanism to enable ZK's ACL check when the authentication
> strategy is simple
> -
>
> Key: HBASE-15577
> URL: https://issues.apache.org/jira/browse/HBASE-15577
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.3
>Reporter: chenxu
>Assignee: chenxu
> Attachments: HBASE-15577-02.patch, HBASE-15577-03.patch,
> HBASE-15577.patch, zk-set-acl.patch
>
>
> if the hbase.security.authentication is set to simple, the ZKUtil.createACL
> just return Ids.OPEN_ACL_UNSAFE, means that there is no ACL check on the ZK's
> node.
> we can refactoring this to enables the ACL's check function
> manual steps to verify the patch
> *1.set this property in the hbase-site.xml*
> {quote}
>hbase.security.authentication(simple)
>hbase.zookeeper.acl (digest:admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc=:cdrwa)
>hbase.zookeeper.auth(digest:admin)
> {quote}
> the digest can generate by the
> DigestAuthenticationProvider.generateDigest("admin")
> *2.start the cluster*
> *3.verify the zk's node*
> {quote}
>(1)getAcl /hbase, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
>'world,'anyone: r
>(2)getAcl /hbase/table-lock, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
> {quote}
> if the node is below, all the client can read the node, but only the
> server(Regionserver & Master which has the auth info) can modify it
> {quote}
> /hbase
> /hbase/meta-region-server
> /hbase/master
> /hbase/hbaseid
> /hbase/rs
> /hbase/table
> /hbase/table/$tableName
> {quote}
> otherwise, only the server can read and modify the node, the Client can't see
> them
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15519) Add per-user metrics
[ https://issues.apache.org/jira/browse/HBASE-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-15519: -- Attachment: hbase-15519_v1.patch Reattach for hadoopqa. > Add per-user metrics > - > > Key: HBASE-15519 > URL: https://issues.apache.org/jira/browse/HBASE-15519 > Project: HBase > Issue Type: Sub-task > Components: metrics >Affects Versions: 1.2.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-15519_v0.patch, hbase-15519_v1.patch, > hbase-15519_v1.patch > > > Per-user metrics will be useful in multi-tenant cases where we can emit > number of requests, operations, num RPCs etc at the per-user aggregate level > per regionserver. We currently have throttles per user, but no way to monitor > resource usage per-user. > Looking at these metrics, operators can adjust throttles, do capacity > planning, etc per-user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15614) Report metrics from JvmPauseMonitor
[
https://issues.apache.org/jira/browse/HBASE-15614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231598#comment-15231598
]
stack commented on HBASE-15614:
---
No magic to JvmPauseMonitor. If it is seeing pauses that are not in GC, that is
interesting. On article, how you think blocked i/o would manifest in the GC
log? Wouldn't the timestamps be just off? Just trying to avoid double metric on
same base phenomenon.
You running an hbase cluster now [~ndimiduk] If so, i defer to your operator
self... don't mind my dev BS.
> Report metrics from JvmPauseMonitor
> ---
>
> Key: HBASE-15614
> URL: https://issues.apache.org/jira/browse/HBASE-15614
> Project: HBase
> Issue Type: Improvement
> Components: metrics, regionserver
>Reporter: Nick Dimiduk
>
> We have {{JvmPauseMonitor}} for detecting JVM pauses; pauses are logged at
> WARN. Would also be good to expose this information on a dashboard via
> metrics system -- make it easier to get this info off the host and into a
> central location for the operator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15537) Make multi WAL work with WALs other than FSHLog
[
https://issues.apache.org/jira/browse/HBASE-15537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231594#comment-15231594
]
stack commented on HBASE-15537:
---
Let me try and add some crass perf numbers here in a day or so.
> Make multi WAL work with WALs other than FSHLog
> ---
>
> Key: HBASE-15537
> URL: https://issues.apache.org/jira/browse/HBASE-15537
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15537-branch-1.patch, HBASE-15537-v3.patch,
> HBASE-15537-v4.patch, HBASE-15537-v5.patch, HBASE-15537-v6.patch,
> HBASE-15537.patch, HBASE-15537_v2.patch
>
>
> The multi WAL should not be bound with {{FSHLog}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15605) Remove PB references from HCD and HTD for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231591#comment-15231591 ] Sean Busbey commented on HBASE-15605: - Is there a substantial advantage to moving the methods to ProtobufUtil instead of just marking the methods IA.Private? > Remove PB references from HCD and HTD for 2.0 > - > > Key: HBASE-15605 > URL: https://issues.apache.org/jira/browse/HBASE-15605 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-15605.patch, HBASE-15605_1.patch, > HBASE-15605_2.patch, HBASE-15605_branch-1.patch > > > This task is sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15527) Refactor Compactor related classes
[ https://issues.apache.org/jira/browse/HBASE-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231590#comment-15231590 ] stack commented on HBASE-15527: --- Skimmed. +1 after fixing checkstyle (can do on commit). The test failures seem well-knowns. +1 on backport too. > Refactor Compactor related classes > -- > > Key: HBASE-15527 > URL: https://issues.apache.org/jira/browse/HBASE-15527 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Affects Versions: 2.0.0, 1.3.0, 0.98.18, 1.4.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 0.98.19, 1.4.0 > > Attachments: HBASE-15527.patch > > > DefaultCompactor and MultiOutputCompactor still have a lot of duplicate code > now, need refactoring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15550) Backport HBASE-12220 hedgedRead metrics to 1.2+
[ https://issues.apache.org/jira/browse/HBASE-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231583#comment-15231583 ] Enis Soztutar commented on HBASE-15550: --- Lets do this for 1.3+ at least. > Backport HBASE-12220 hedgedRead metrics to 1.2+ > --- > > Key: HBASE-15550 > URL: https://issues.apache.org/jira/browse/HBASE-15550 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.0, 1.2.2 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: 15550.branch-1.2.txt > > > hedgedRead metrics are in master branch only. They were put in branch-1 but > then reverted because old versions of hadoop didn't supported hedgedReads. > 1.2+ has minimal hadoop version. Retry backport. > FYI [~phobos182] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15600) Add provision for adding mutations to memstore or able to write to same region in batchMutate coprocessor hooks
[
https://issues.apache.org/jira/browse/HBASE-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231581#comment-15231581
]
stack commented on HBASE-15600:
---
What is meant by this?
bq. but writing from batchMutate API is not allowed because of mvcc.
Why is mvcc in your way?
> Add provision for adding mutations to memstore or able to write to same
> region in batchMutate coprocessor hooks
> ---
>
> Key: HBASE-15600
> URL: https://issues.apache.org/jira/browse/HBASE-15600
> Project: HBase
> Issue Type: Improvement
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
> Fix For: 2.0.0, 0.98.19, 1.1.5, 1.2.2, 1.0.5
>
> Attachments: HBASE-15600.patch, HBASE-15600_v1.patch,
> HBASE-15600_v2.patch
>
>
> As part of PHOENIX-1734 we need to write the index updates to same region
> from coprocessors but writing from batchMutate API is not allowed because of
> mvcc.
> Raised PHOENIX-2742 to discuss any alternative way to write to the same
> region directly or not but not having any proper solution there.
> Currently we have provision to write wal edits from coprocessors. We can set
> wal edits in MiniBatchOperationInProgress.
> {noformat}
> /**
>* Sets the walEdit for the operation(Mutation) at the specified position.
>* @param index
>* @param walEdit
>*/
> public void setWalEdit(int index, WALEdit walEdit) {
> this.walEditsFromCoprocessors[getAbsoluteIndex(index)] = walEdit;
> }
> {noformat}
> Similarly we can allow to write mutations from coprocessors to memstore as
> well. Or else we should provide the batch mutation API allow write in batch
> mutate coprocessors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15605) Remove PB references from HCD and HTD for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231578#comment-15231578 ] stack commented on HBASE-15605: --- I'm good w/ rattionale that: + Should have been IA.Private in first place + Deprecate in 1.3 is good enough given above I don't think anyone using this stuff anyway outside of internals. So +1 on gross remove. > Remove PB references from HCD and HTD for 2.0 > - > > Key: HBASE-15605 > URL: https://issues.apache.org/jira/browse/HBASE-15605 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-15605.patch, HBASE-15605_1.patch, > HBASE-15605_2.patch, HBASE-15605_branch-1.patch > > > This task is sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15572) Adding optional timestamp semantics to HBase-Spark
[
https://issues.apache.org/jira/browse/HBASE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231573#comment-15231573
]
Sean Busbey commented on HBASE-15572:
-
{quote}
(1) I will submit v9 which renames ’timestamp’ to be
“spark.hbase.query.timestamp”, as well as the other three parameters in
SparkHBaseConf.
{quote}
Is there a reason this can't be "hbase.spark.query.timestamp" ?
{quote}
Yes, you are right. A code example will make it clear, so I will add more
detailed description in HBASE-15473: Documentation for the usage of hbase
dataframe user api (JSON, Avro, etc).
{quote}
I would like documentation complete as a part of adding this feature rather
than as a follow on. Is there some reason we can't document this addition
unless we also document the use of JSON, Avro, etc?
> Adding optional timestamp semantics to HBase-Spark
> --
>
> Key: HBASE-15572
> URL: https://issues.apache.org/jira/browse/HBASE-15572
> Project: HBase
> Issue Type: New Feature
> Components: spark
>Reporter: Weiqing Yang
>Assignee: Weiqing Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-15572-1.patch, HBASE-15572-2.patch,
> HBASE-15572-3.patch, HBASE-15572-4.patch, HBASE-15572-5.patch,
> HBASE-15572-6.patch, HBASE-15572-7.patch, HBASE-15572-8.patch,
> HBASE-15572-9.patch
>
>
> Right now the timestamp is always latest. With this patch, users can select
> timestamps they want.
> In this patch, 4 parameters, "timestamp", "minTimestamp", "maxiTimestamp" and
> "maxVersions" are added to HBaseSparkConf. Users can select a timestamp, they
> can also select a time range with minimum timestamp and maximum timestamp. A
> new test for selecting records with different timestamps is added.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15400) Use DateTieredCompactor for Date Tiered Compaction
[ https://issues.apache.org/jira/browse/HBASE-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231567#comment-15231567 ] Hudson commented on HBASE-15400: FAILURE: Integrated in HBase-1.4 #75 (See [https://builds.apache.org/job/HBase-1.4/75/]) HBASE-15400 Use DateTieredCompactor for Date Tiered Compaction (Clara (tedyu: rev b17350210b924f6adb4d002d41691fb51369a46c) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDateTieredCompaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactionPolicy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MockStoreFile.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/EverythingPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/SortedCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionConfiguration.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/FIFOCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionRequest.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DateTieredStoreEngine.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDateTieredCompactionPolicy.java > Use DateTieredCompactor for Date Tiered Compaction > -- > > Key: HBASE-15400 > URL: https://issues.apache.org/jira/browse/HBASE-15400 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15400-15389-v12.patch, HBASE-15400-branch-1.patch, > HBASE-15400-v1.pa, HBASE-15400-v3-v3.patch, HBASE-15400-v3-v4.patch, > HBASE-15400-v3-v5.patch, HBASE-15400-v3.patch, HBASE-15400-v6.patch, > HBASE-15400-v7.patch, HBASE-15400.patch > > > When we compact, we can output multiple files along the current window > boundaries. There are two use cases: > 1. Major compaction: We want to output date tiered store files with data > older than max age archived in trunks of the window size on the higher tier. > Once a window is old enough, we don't combine the windows to promote to the > next tier any further. So files in these windows retain the same timespan as > they were minor-compacted last time, which is the window size of the highest > tier. Major compaction will touch these files and we want to maintain the > same layout. This way, TTL and archiving will be simpler and more efficient. > 2. Bulk load files and the old file generated by major compaction before > upgrading to DTCP. > Pros: > 1. Restore locality, process versioning, updates and deletes while > maintaining the tiered layout. > 2. The best way to fix a skewed layout. > > This work is based on a prototype of DateTieredCompactor from HBASE-15389 and > focused on the part to meet needs for these two use cases while supporting > others. I have to call out a few design decisions: > 1. We only want to output the files along all windows for major compaction. > And we want to output multiple files older than max age in the trunks of the > maximum tier window size determined by base window size, windows per tier and > max age. > 2. For minor compaction, we don't want to output too many files, which will > remain around because of current restriction of contiguous compaction by seq > id. I will only output two files if all the files in the windows are being > combined, one for the data within window and the other for the out-of-window > tail. If there is any file in the window excluded from compaction, only one > file will be output from compaction. When the windows are promoted, the > situation of out of order data will gradually improve. For the incoming > window, we need to accommodate the case with user-specified future data. > 3. We have to pass the boundaries with the list of store file as a complete > time snapshot instead of two separate calls
[jira] [Commented] (HBASE-15381) Implement a distributed MOB compaction by procedure
[ https://issues.apache.org/jira/browse/HBASE-15381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231566#comment-15231566 ] Jingcheng Du commented on HBASE-15381: -- And I think if the distributed mob compaction is committed, the sweep tool can be removed from the code. > Implement a distributed MOB compaction by procedure > --- > > Key: HBASE-15381 > URL: https://issues.apache.org/jira/browse/HBASE-15381 > Project: HBase > Issue Type: Improvement > Components: mob >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Attachments: mob distributed compaction design.pdf > > > In MOB, there is a periodical compaction which runs in HMaster (It can be > disabled by configuration), some small mob files are merged into bigger ones. > Now the compaction only runs in HMaster which is not efficient and might > impact the running of HMaster. In this JIRA, a distributed MOB compaction is > introduced, it is triggered by HMaster, but all the compaction jobs are > distributed to HRegionServers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15605) Remove PB references from HCD and HTD for 2.0
[
https://issues.apache.org/jira/browse/HBASE-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231565#comment-15231565
]
Hadoop QA commented on HBASE-15605:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 5s
{color} | {color:red} hbase-client: patch generated 4 new + 98 unchanged - 0
fixed = 102 total (was 98) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 59s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 54s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 48s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12797656/HBASE-15605_branch-1.patch
|
| JIRA Issue | HBASE-15605 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux pietas.apache.org 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT
Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 394b89d |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1
[jira] [Updated] (HBASE-15577) there need be a mechanism to enable ZK's ACL check when the authentication strategy is simple
[
https://issues.apache.org/jira/browse/HBASE-15577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenxu updated HBASE-15577:
---
Attachment: HBASE-15577-03.patch
> there need be a mechanism to enable ZK's ACL check when the authentication
> strategy is simple
> -
>
> Key: HBASE-15577
> URL: https://issues.apache.org/jira/browse/HBASE-15577
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.3
>Reporter: chenxu
>Assignee: chenxu
> Attachments: HBASE-15577-02.patch, HBASE-15577-03.patch,
> HBASE-15577.patch, zk-set-acl.patch
>
>
> if the hbase.security.authentication is set to simple, the ZKUtil.createACL
> just return Ids.OPEN_ACL_UNSAFE, means that there is no ACL check on the ZK's
> node.
> we can refactoring this to enables the ACL's check function
> manual steps to verify the patch
> *1.set this property in the hbase-site.xml*
> {quote}
>hbase.security.authentication(simple)
>hbase.zookeeper.acl (digest:admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc=:cdrwa)
>hbase.zookeeper.auth(digest:admin)
> {quote}
> the digest can generate by the
> DigestAuthenticationProvider.generateDigest("admin")
> *2.start the cluster*
> *3.verify the zk's node*
> {quote}
>(1)getAcl /hbase, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
>'world,'anyone: r
>(2)getAcl /hbase/table-lock, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
> {quote}
> if the node is below, all the client can read the node, but only the
> server(Regionserver & Master which has the auth info) can modify it
> {quote}
> /hbase
> /hbase/meta-region-server
> /hbase/master
> /hbase/hbaseid
> /hbase/rs
> /hbase/table
> /hbase/table/$tableName
> {quote}
> otherwise, only the server can read and modify the node, the Client can't see
> them
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15577) there need be a mechanism to enable ZK's ACL check when the authentication strategy is simple
[
https://issues.apache.org/jira/browse/HBASE-15577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenxu updated HBASE-15577:
---
Attachment: (was: HBASE-15577-03.patch)
> there need be a mechanism to enable ZK's ACL check when the authentication
> strategy is simple
> -
>
> Key: HBASE-15577
> URL: https://issues.apache.org/jira/browse/HBASE-15577
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.3
>Reporter: chenxu
>Assignee: chenxu
> Attachments: HBASE-15577-02.patch, HBASE-15577.patch, zk-set-acl.patch
>
>
> if the hbase.security.authentication is set to simple, the ZKUtil.createACL
> just return Ids.OPEN_ACL_UNSAFE, means that there is no ACL check on the ZK's
> node.
> we can refactoring this to enables the ACL's check function
> manual steps to verify the patch
> *1.set this property in the hbase-site.xml*
> {quote}
>hbase.security.authentication(simple)
>hbase.zookeeper.acl (digest:admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc=:cdrwa)
>hbase.zookeeper.auth(digest:admin)
> {quote}
> the digest can generate by the
> DigestAuthenticationProvider.generateDigest("admin")
> *2.start the cluster*
> *3.verify the zk's node*
> {quote}
>(1)getAcl /hbase, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
>'world,'anyone: r
>(2)getAcl /hbase/table-lock, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
> {quote}
> if the node is below, all the client can read the node, but only the
> server(Regionserver & Master which has the auth info) can modify it
> {quote}
> /hbase
> /hbase/meta-region-server
> /hbase/master
> /hbase/hbaseid
> /hbase/rs
> /hbase/table
> /hbase/table/$tableName
> {quote}
> otherwise, only the server can read and modify the node, the Client can't see
> them
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15600) Add provision for adding mutations to memstore or able to write to same region in batchMutate coprocessor hooks
[
https://issues.apache.org/jira/browse/HBASE-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231558#comment-15231558
]
ramkrishna.s.vasudevan commented on HBASE-15600:
{code}
if (coprocessorHost.preBatchMutate(miniBatchOp)) {
3121 return 0L;3122 return 0L;
3123} else if (miniBatchOp.operationsFromCoprocessors!=null) {
3124WALEdit walEditForCpOperations=
miniBatchOp.getWalEdit(0);
{code}
You are adding this as an else if condition . Better to add this as an 'if'
condition only ?
So if there are some WALEdits added as part of prePut and preDelete, you will
once again add new Cells in preBatchMutate and in the 0th index of the already
added WALEdits, is it fine in doing like that?
bq.WALEdit walEditForCpOperations= miniBatchOp.getWalEdit(0);
Even here you are getting the first WALEdit only.
bq.public void addOperations(T[] newOperations) {
May need proper documentation here and how does it work. Better to rename it to
addOperationsfromCP.
The default status of this operations is going to be NOT_RUN only correct?
Some where I feel that the index that we use to iterate and this new operations
should some how be in sync. May be it is difficult to just add these new
miniBatchOps to the acutal Operations already available and adjust the first
and last index based on that?
> Add provision for adding mutations to memstore or able to write to same
> region in batchMutate coprocessor hooks
> ---
>
> Key: HBASE-15600
> URL: https://issues.apache.org/jira/browse/HBASE-15600
> Project: HBase
> Issue Type: Improvement
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
> Fix For: 2.0.0, 0.98.19, 1.1.5, 1.2.2, 1.0.5
>
> Attachments: HBASE-15600.patch, HBASE-15600_v1.patch,
> HBASE-15600_v2.patch
>
>
> As part of PHOENIX-1734 we need to write the index updates to same region
> from coprocessors but writing from batchMutate API is not allowed because of
> mvcc.
> Raised PHOENIX-2742 to discuss any alternative way to write to the same
> region directly or not but not having any proper solution there.
> Currently we have provision to write wal edits from coprocessors. We can set
> wal edits in MiniBatchOperationInProgress.
> {noformat}
> /**
>* Sets the walEdit for the operation(Mutation) at the specified position.
>* @param index
>* @param walEdit
>*/
> public void setWalEdit(int index, WALEdit walEdit) {
> this.walEditsFromCoprocessors[getAbsoluteIndex(index)] = walEdit;
> }
> {noformat}
> Similarly we can allow to write mutations from coprocessors to memstore as
> well. Or else we should provide the batch mutation API allow write in batch
> mutate coprocessors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15527) Refactor Compactor related classes
[ https://issues.apache.org/jira/browse/HBASE-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231557#comment-15231557 ] Duo Zhang commented on HBASE-15527: --- This is just a refactoring so no new unit tests. [~enis] PTAL. Thanks. > Refactor Compactor related classes > -- > > Key: HBASE-15527 > URL: https://issues.apache.org/jira/browse/HBASE-15527 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Affects Versions: 2.0.0, 1.3.0, 0.98.18, 1.4.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 0.98.19, 1.4.0 > > Attachments: HBASE-15527.patch > > > DefaultCompactor and MultiOutputCompactor still have a lot of duplicate code > now, need refactoring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15605) Remove PB references from HCD and HTD for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231556#comment-15231556 ] Enis Soztutar commented on HBASE-15605: --- +1 to both patches. > Remove PB references from HCD and HTD for 2.0 > - > > Key: HBASE-15605 > URL: https://issues.apache.org/jira/browse/HBASE-15605 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-15605.patch, HBASE-15605_1.patch, > HBASE-15605_2.patch, HBASE-15605_branch-1.patch > > > This task is sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15577) there need be a mechanism to enable ZK's ACL check when the authentication strategy is simple
[
https://issues.apache.org/jira/browse/HBASE-15577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231551#comment-15231551
]
chenxu commented on HBASE-15577:
it's my mistake, the UT case has uploaded, very glad if you have some advise on
it
> there need be a mechanism to enable ZK's ACL check when the authentication
> strategy is simple
> -
>
> Key: HBASE-15577
> URL: https://issues.apache.org/jira/browse/HBASE-15577
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.3
>Reporter: chenxu
>Assignee: chenxu
> Attachments: HBASE-15577-02.patch, HBASE-15577-03.patch,
> HBASE-15577.patch, zk-set-acl.patch
>
>
> if the hbase.security.authentication is set to simple, the ZKUtil.createACL
> just return Ids.OPEN_ACL_UNSAFE, means that there is no ACL check on the ZK's
> node.
> we can refactoring this to enables the ACL's check function
> manual steps to verify the patch
> *1.set this property in the hbase-site.xml*
> {quote}
>hbase.security.authentication(simple)
>hbase.zookeeper.acl (digest:admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc=:cdrwa)
>hbase.zookeeper.auth(digest:admin)
> {quote}
> the digest can generate by the
> DigestAuthenticationProvider.generateDigest("admin")
> *2.start the cluster*
> *3.verify the zk's node*
> {quote}
>(1)getAcl /hbase, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
>'world,'anyone: r
>(2)getAcl /hbase/table-lock, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
> {quote}
> if the node is below, all the client can read the node, but only the
> server(Regionserver & Master which has the auth info) can modify it
> {quote}
> /hbase
> /hbase/meta-region-server
> /hbase/master
> /hbase/hbaseid
> /hbase/rs
> /hbase/table
> /hbase/table/$tableName
> {quote}
> otherwise, only the server can read and modify the node, the Client can't see
> them
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15577) there need be a mechanism to enable ZK's ACL check when the authentication strategy is simple
[
https://issues.apache.org/jira/browse/HBASE-15577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenxu updated HBASE-15577:
---
Attachment: HBASE-15577-03.patch
> there need be a mechanism to enable ZK's ACL check when the authentication
> strategy is simple
> -
>
> Key: HBASE-15577
> URL: https://issues.apache.org/jira/browse/HBASE-15577
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.3
>Reporter: chenxu
>Assignee: chenxu
> Attachments: HBASE-15577-02.patch, HBASE-15577-03.patch,
> HBASE-15577.patch, zk-set-acl.patch
>
>
> if the hbase.security.authentication is set to simple, the ZKUtil.createACL
> just return Ids.OPEN_ACL_UNSAFE, means that there is no ACL check on the ZK's
> node.
> we can refactoring this to enables the ACL's check function
> manual steps to verify the patch
> *1.set this property in the hbase-site.xml*
> {quote}
>hbase.security.authentication(simple)
>hbase.zookeeper.acl (digest:admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc=:cdrwa)
>hbase.zookeeper.auth(digest:admin)
> {quote}
> the digest can generate by the
> DigestAuthenticationProvider.generateDigest("admin")
> *2.start the cluster*
> *3.verify the zk's node*
> {quote}
>(1)getAcl /hbase, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
>'world,'anyone: r
>(2)getAcl /hbase/table-lock, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
> {quote}
> if the node is below, all the client can read the node, but only the
> server(Regionserver & Master which has the auth info) can modify it
> {quote}
> /hbase
> /hbase/meta-region-server
> /hbase/master
> /hbase/hbaseid
> /hbase/rs
> /hbase/table
> /hbase/table/$tableName
> {quote}
> otherwise, only the server can read and modify the node, the Client can't see
> them
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15577) there need be a mechanism to enable ZK's ACL check when the authentication strategy is simple
[
https://issues.apache.org/jira/browse/HBASE-15577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenxu updated HBASE-15577:
---
Attachment: (was: HBASE-15577-03.patch)
> there need be a mechanism to enable ZK's ACL check when the authentication
> strategy is simple
> -
>
> Key: HBASE-15577
> URL: https://issues.apache.org/jira/browse/HBASE-15577
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.3
>Reporter: chenxu
>Assignee: chenxu
> Attachments: HBASE-15577-02.patch, HBASE-15577.patch, zk-set-acl.patch
>
>
> if the hbase.security.authentication is set to simple, the ZKUtil.createACL
> just return Ids.OPEN_ACL_UNSAFE, means that there is no ACL check on the ZK's
> node.
> we can refactoring this to enables the ACL's check function
> manual steps to verify the patch
> *1.set this property in the hbase-site.xml*
> {quote}
>hbase.security.authentication(simple)
>hbase.zookeeper.acl (digest:admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc=:cdrwa)
>hbase.zookeeper.auth(digest:admin)
> {quote}
> the digest can generate by the
> DigestAuthenticationProvider.generateDigest("admin")
> *2.start the cluster*
> *3.verify the zk's node*
> {quote}
>(1)getAcl /hbase, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
>'world,'anyone: r
>(2)getAcl /hbase/table-lock, result is:
>'digest,'admin:0DPiKuNIrrVmD8IUCuw1hQxNqZc= : cdrwa
> {quote}
> if the node is below, all the client can read the node, but only the
> server(Regionserver & Master which has the auth info) can modify it
> {quote}
> /hbase
> /hbase/meta-region-server
> /hbase/master
> /hbase/hbaseid
> /hbase/rs
> /hbase/table
> /hbase/table/$tableName
> {quote}
> otherwise, only the server can read and modify the node, the Client can't see
> them
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15406) Split / merge switch left disabled after early termination of hbck
[
https://issues.apache.org/jira/browse/HBASE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231550#comment-15231550
]
Heng Chen commented on HBASE-15406:
---
Could you help me review the latest patch on review board, thanks! [~mantonov]
> Split / merge switch left disabled after early termination of hbck
> --
>
> Key: HBASE-15406
> URL: https://issues.apache.org/jira/browse/HBASE-15406
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15406.patch, HBASE-15406.v1.patch,
> HBASE-15406_v1.patch, HBASE-15406_v2.patch, test.patch, wip.patch
>
>
> This was what I did on cluster with 1.4.0-SNAPSHOT built Thursday:
> Run 'hbase hbck -disableSplitAndMerge' on gateway node of the cluster
> Terminate hbck early
> Enter hbase shell where I observed:
> {code}
> hbase(main):001:0> splitormerge_enabled 'SPLIT'
> false
> 0 row(s) in 0.3280 seconds
> hbase(main):002:0> splitormerge_enabled 'MERGE'
> false
> 0 row(s) in 0.0070 seconds
> {code}
> Expectation is that the split / merge switches should be restored to default
> value after hbck exits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15381) Implement a distributed MOB compaction by procedure
[ https://issues.apache.org/jira/browse/HBASE-15381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231546#comment-15231546 ] Jingcheng Du commented on HBASE-15381: -- Thanks [~tedyu]. Sweep tool is a tool, and distributed mob compaction is a compaction mechanism that runs periodically in the cluster. Sweep tool uses MapReduce, it is distributed to RSs by mapper and reducer, and this tool scans all the mob table and merges the linked small files. ( cells in HBase -> mob files). Distributed mob compaction uses procedure, and distributed to RSs by procedure too. It directly handles the mob files and merges the small files into bigger ones, at last adds the new reference cells back to hbase by bulk load. (mob files -> cells in HBase). Sweeper is "cells in HBase" -> mob files, and mob compaction is "mob files -> cells in HBase), two different directions. > Implement a distributed MOB compaction by procedure > --- > > Key: HBASE-15381 > URL: https://issues.apache.org/jira/browse/HBASE-15381 > Project: HBase > Issue Type: Improvement > Components: mob >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Attachments: mob distributed compaction design.pdf > > > In MOB, there is a periodical compaction which runs in HMaster (It can be > disabled by configuration), some small mob files are merged into bigger ones. > Now the compaction only runs in HMaster which is not efficient and might > impact the running of HMaster. In this JIRA, a distributed MOB compaction is > introduced, it is triggered by HMaster, but all the compaction jobs are > distributed to HRegionServers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15537) Make multi WAL work with WALs other than FSHLog
[
https://issues.apache.org/jira/browse/HBASE-15537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-15537:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Pushed to branch-1.3+.
Thanks all for reviewing.
> Make multi WAL work with WALs other than FSHLog
> ---
>
> Key: HBASE-15537
> URL: https://issues.apache.org/jira/browse/HBASE-15537
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15537-branch-1.patch, HBASE-15537-v3.patch,
> HBASE-15537-v4.patch, HBASE-15537-v5.patch, HBASE-15537-v6.patch,
> HBASE-15537.patch, HBASE-15537_v2.patch
>
>
> The multi WAL should not be bound with {{FSHLog}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15605) Remove PB references from HCD and HTD for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15605: --- Attachment: HBASE-15605_branch-1.patch Patch for branch-1 that deprecates the convert() method in HCD and HTD. > Remove PB references from HCD and HTD for 2.0 > - > > Key: HBASE-15605 > URL: https://issues.apache.org/jira/browse/HBASE-15605 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-15605.patch, HBASE-15605_1.patch, > HBASE-15605_2.patch, HBASE-15605_branch-1.patch > > > This task is sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15614) Report metrics from JvmPauseMonitor
[
https://issues.apache.org/jira/browse/HBASE-15614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231530#comment-15231530
]
Yu Li commented on HBASE-15614:
---
Also observed non-GC caused pause in our online RS logs, so one more +1 from me
for adding the metrics. Thanks.
> Report metrics from JvmPauseMonitor
> ---
>
> Key: HBASE-15614
> URL: https://issues.apache.org/jira/browse/HBASE-15614
> Project: HBase
> Issue Type: Improvement
> Components: metrics, regionserver
>Reporter: Nick Dimiduk
>
> We have {{JvmPauseMonitor}} for detecting JVM pauses; pauses are logged at
> WARN. Would also be good to expose this information on a dashboard via
> metrics system -- make it easier to get this info off the host and into a
> central location for the operator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15605) Remove PB references from HCD and HTD for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15605: --- Attachment: HBASE-15605_2.patch Removed other unused imports and the ordering of the imports that was not introduced by this patch (but the checkstyle was complaining about it). Test cases seems to pass locally. May be general flakiness. Will try once again. > Remove PB references from HCD and HTD for 2.0 > - > > Key: HBASE-15605 > URL: https://issues.apache.org/jira/browse/HBASE-15605 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-15605.patch, HBASE-15605_1.patch, > HBASE-15605_2.patch > > > This task is sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15605) Remove PB references from HCD and HTD for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15605: --- Status: Patch Available (was: Open) > Remove PB references from HCD and HTD for 2.0 > - > > Key: HBASE-15605 > URL: https://issues.apache.org/jira/browse/HBASE-15605 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-15605.patch, HBASE-15605_1.patch, > HBASE-15605_2.patch > > > This task is sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15605) Remove PB references from HCD and HTD for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15605: --- Status: Open (was: Patch Available) > Remove PB references from HCD and HTD for 2.0 > - > > Key: HBASE-15605 > URL: https://issues.apache.org/jira/browse/HBASE-15605 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-15605.patch, HBASE-15605_1.patch > > > This task is sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15610) Remove deprecated HConnection for 2.0 thus removing all PB references for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231520#comment-15231520 ] ramkrishna.s.vasudevan commented on HBASE-15610: Okie. Let me come with a plan for this. Need to consider the timeline also. But yes it is better to do it now as its been deprecated for a long time. > Remove deprecated HConnection for 2.0 thus removing all PB references for 2.0 > - > > Key: HBASE-15610 > URL: https://issues.apache.org/jira/browse/HBASE-15610 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > > This is sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15605) Remove PB references from HCD and HTD for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231515#comment-15231515 ] ramkrishna.s.vasudevan commented on HBASE-15605: bq.So can we deprecate in say 1.3/1.4 and remove in 2.0 Ya I think only this can be done. We cannot have this in 2.0 anyway because if we really want to shade the PB to have our changes then if we expose these deprecated methods in 2.0 then we cannot have our verion of PB. Will add a patch to deprecate in 1.3 and 1.4. > Remove PB references from HCD and HTD for 2.0 > - > > Key: HBASE-15605 > URL: https://issues.apache.org/jira/browse/HBASE-15605 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-15605.patch, HBASE-15605_1.patch > > > This task is sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15093) Replication can report incorrect size of log queue for the global source when multiwal is enabled
[
https://issues.apache.org/jira/browse/HBASE-15093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231513#comment-15231513
]
Hadoop QA commented on HBASE-15093:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 36s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 24s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 52s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 25s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 2s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 46s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 57s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 15s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 10s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 17s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 17s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 25s
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 25s
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 58s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 36s {color}
| {color:red} hbase-server in the patch
[jira] [Commented] (HBASE-15400) Use DateTieredCompactor for Date Tiered Compaction
[ https://issues.apache.org/jira/browse/HBASE-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231502#comment-15231502 ] Hudson commented on HBASE-15400: SUCCESS: Integrated in HBase-1.3 #638 (See [https://builds.apache.org/job/HBase-1.3/638/]) HBASE-15400 Use DateTieredCompactor for Date Tiered Compaction (Clara (tedyu: rev acc5d262a057427bf5593263825666b6a1a30232) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactionPolicy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MockStoreFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/EverythingPolicy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDateTieredCompaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DateTieredStoreEngine.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/SortedCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionRequest.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/FIFOCompactionPolicy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDateTieredCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionConfiguration.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java > Use DateTieredCompactor for Date Tiered Compaction > -- > > Key: HBASE-15400 > URL: https://issues.apache.org/jira/browse/HBASE-15400 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15400-15389-v12.patch, HBASE-15400-branch-1.patch, > HBASE-15400-v1.pa, HBASE-15400-v3-v3.patch, HBASE-15400-v3-v4.patch, > HBASE-15400-v3-v5.patch, HBASE-15400-v3.patch, HBASE-15400-v6.patch, > HBASE-15400-v7.patch, HBASE-15400.patch > > > When we compact, we can output multiple files along the current window > boundaries. There are two use cases: > 1. Major compaction: We want to output date tiered store files with data > older than max age archived in trunks of the window size on the higher tier. > Once a window is old enough, we don't combine the windows to promote to the > next tier any further. So files in these windows retain the same timespan as > they were minor-compacted last time, which is the window size of the highest > tier. Major compaction will touch these files and we want to maintain the > same layout. This way, TTL and archiving will be simpler and more efficient. > 2. Bulk load files and the old file generated by major compaction before > upgrading to DTCP. > Pros: > 1. Restore locality, process versioning, updates and deletes while > maintaining the tiered layout. > 2. The best way to fix a skewed layout. > > This work is based on a prototype of DateTieredCompactor from HBASE-15389 and > focused on the part to meet needs for these two use cases while supporting > others. I have to call out a few design decisions: > 1. We only want to output the files along all windows for major compaction. > And we want to output multiple files older than max age in the trunks of the > maximum tier window size determined by base window size, windows per tier and > max age. > 2. For minor compaction, we don't want to output too many files, which will > remain around because of current restriction of contiguous compaction by seq > id. I will only output two files if all the files in the windows are being > combined, one for the data within window and the other for the out-of-window > tail. If there is any file in the window excluded from compaction, only one > file will be output from compaction. When the windows are promoted, the > situation of out of order data will gradually improve. For the incoming > window, we need to accommodate the case with user-specified future data. > 3. We have to pass the boundaries with the list of store file as a complete > time snapshot instead of two separate call
[jira] [Commented] (HBASE-15586) Unify human readable numbers in the web UI
[ https://issues.apache.org/jira/browse/HBASE-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231501#comment-15231501 ] Hudson commented on HBASE-15586: SUCCESS: Integrated in HBase-1.3 #638 (See [https://builds.apache.org/job/HBase-1.3/638/]) HBASE-15586 Unify human readable numbers in the web UI (enis: rev 2fc7c8bbc5424c6291dc6ca0eac9d5605f607617) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RegionListTmpl.jamon > Unify human readable numbers in the web UI > --- > > Key: HBASE-15586 > URL: https://issues.apache.org/jira/browse/HBASE-15586 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: Screen Shot 2016-04-01 at 3.52.00 PM.png, Screen Shot > 2016-04-01 at 3.52.12 PM.png, Screen Shot 2016-04-01 at 3.52.32 PM.png, > Screen Shot 2016-04-01 at 3.52.38 PM.png, Screen Shot 2016-04-01 at 3.52.49 > PM.png, hbase-15586-v1.patch, hbase-15586-v1.patch, hbase-15586-v2.patch > > > I was looking at something else in the regionserver web ui trying to > understand the WAL size, and we are reporting that as raw bytes, not in MB / > GB range. Did a quick sweep to unify all the human-readable representations > in the UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15537) Make multi WAL work with WALs other than FSHLog
[
https://issues.apache.org/jira/browse/HBASE-15537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231489#comment-15231489
]
Duo Zhang commented on HBASE-15537:
---
So no more concerns. Let me commit this.
> Make multi WAL work with WALs other than FSHLog
> ---
>
> Key: HBASE-15537
> URL: https://issues.apache.org/jira/browse/HBASE-15537
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15537-branch-1.patch, HBASE-15537-v3.patch,
> HBASE-15537-v4.patch, HBASE-15537-v5.patch, HBASE-15537-v6.patch,
> HBASE-15537.patch, HBASE-15537_v2.patch
>
>
> The multi WAL should not be bound with {{FSHLog}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15537) Make multi WAL work with WALs other than FSHLog
[
https://issues.apache.org/jira/browse/HBASE-15537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231488#comment-15231488
]
Duo Zhang commented on HBASE-15537:
---
Good. I will also take a look.
> Make multi WAL work with WALs other than FSHLog
> ---
>
> Key: HBASE-15537
> URL: https://issues.apache.org/jira/browse/HBASE-15537
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15537-branch-1.patch, HBASE-15537-v3.patch,
> HBASE-15537-v4.patch, HBASE-15537-v5.patch, HBASE-15537-v6.patch,
> HBASE-15537.patch, HBASE-15537_v2.patch
>
>
> The multi WAL should not be bound with {{FSHLog}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15572) Adding optional timestamp semantics to HBase-Spark
[ https://issues.apache.org/jira/browse/HBASE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiqing Yang updated HBASE-15572: - Attachment: HBASE-15572-9.patch > Adding optional timestamp semantics to HBase-Spark > -- > > Key: HBASE-15572 > URL: https://issues.apache.org/jira/browse/HBASE-15572 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Weiqing Yang >Assignee: Weiqing Yang > Fix For: 2.0.0 > > Attachments: HBASE-15572-1.patch, HBASE-15572-2.patch, > HBASE-15572-3.patch, HBASE-15572-4.patch, HBASE-15572-5.patch, > HBASE-15572-6.patch, HBASE-15572-7.patch, HBASE-15572-8.patch, > HBASE-15572-9.patch > > > Right now the timestamp is always latest. With this patch, users can select > timestamps they want. > In this patch, 4 parameters, "timestamp", "minTimestamp", "maxiTimestamp" and > "maxVersions" are added to HBaseSparkConf. Users can select a timestamp, they > can also select a time range with minimum timestamp and maximum timestamp. A > new test for selecting records with different timestamps is added. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15381) Implement a distributed MOB compaction by procedure
[ https://issues.apache.org/jira/browse/HBASE-15381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231457#comment-15231457 ] Ted Yu commented on HBASE-15381: Jingcheng: Can you clarify the relationship between distributed MOB compaction and the current MOB sweeper tool ? Thanks > Implement a distributed MOB compaction by procedure > --- > > Key: HBASE-15381 > URL: https://issues.apache.org/jira/browse/HBASE-15381 > Project: HBase > Issue Type: Improvement > Components: mob >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Attachments: mob distributed compaction design.pdf > > > In MOB, there is a periodical compaction which runs in HMaster (It can be > disabled by configuration), some small mob files are merged into bigger ones. > Now the compaction only runs in HMaster which is not efficient and might > impact the running of HMaster. In this JIRA, a distributed MOB compaction is > introduced, it is triggered by HMaster, but all the compaction jobs are > distributed to HRegionServers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15586) Unify human readable numbers in the web UI
[ https://issues.apache.org/jira/browse/HBASE-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-15586: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. > Unify human readable numbers in the web UI > --- > > Key: HBASE-15586 > URL: https://issues.apache.org/jira/browse/HBASE-15586 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: Screen Shot 2016-04-01 at 3.52.00 PM.png, Screen Shot > 2016-04-01 at 3.52.12 PM.png, Screen Shot 2016-04-01 at 3.52.32 PM.png, > Screen Shot 2016-04-01 at 3.52.38 PM.png, Screen Shot 2016-04-01 at 3.52.49 > PM.png, hbase-15586-v1.patch, hbase-15586-v1.patch, hbase-15586-v2.patch > > > I was looking at something else in the regionserver web ui trying to > understand the WAL size, and we are reporting that as raw bytes, not in MB / > GB range. Did a quick sweep to unify all the human-readable representations > in the UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15572) Adding optional timestamp semantics to HBase-Spark
[ https://issues.apache.org/jira/browse/HBASE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231441#comment-15231441 ] Weiqing Yang commented on HBASE-15572: -- Thanks for the feedback. That’s a good idea. However, it may cause potential troubles. For example, if a virtual column is put into the sql schema, users may have select timestamp from table. It is a valid query in sql, but not valid in Hbase, since this timestamp is implicit in Hbase attached to a row, there is really no such column in the Hbase row. We cannot construct a scan using this. Thus it is an invalid query for HBase. > Adding optional timestamp semantics to HBase-Spark > -- > > Key: HBASE-15572 > URL: https://issues.apache.org/jira/browse/HBASE-15572 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Weiqing Yang >Assignee: Weiqing Yang > Fix For: 2.0.0 > > Attachments: HBASE-15572-1.patch, HBASE-15572-2.patch, > HBASE-15572-3.patch, HBASE-15572-4.patch, HBASE-15572-5.patch, > HBASE-15572-6.patch, HBASE-15572-7.patch, HBASE-15572-8.patch > > > Right now the timestamp is always latest. With this patch, users can select > timestamps they want. > In this patch, 4 parameters, "timestamp", "minTimestamp", "maxiTimestamp" and > "maxVersions" are added to HBaseSparkConf. Users can select a timestamp, they > can also select a time range with minimum timestamp and maximum timestamp. A > new test for selecting records with different timestamps is added. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15572) Adding optional timestamp semantics to HBase-Spark
[ https://issues.apache.org/jira/browse/HBASE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231435#comment-15231435 ] Weiqing Yang commented on HBASE-15572: -- Thanks for the reply. Yes, you are right. A code example will make it clear, so I will add more detailed description in HBASE-15473: Documentation for the usage of hbase dataframe user api (JSON, Avro, etc). (1) I will submit v9 which renames ’timestamp’ to be “spark.hbase.query.timestamp”, as well as the other three parameters in SparkHBaseConf. (2) This only used for SparkSQL. > Adding optional timestamp semantics to HBase-Spark > -- > > Key: HBASE-15572 > URL: https://issues.apache.org/jira/browse/HBASE-15572 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Weiqing Yang >Assignee: Weiqing Yang > Fix For: 2.0.0 > > Attachments: HBASE-15572-1.patch, HBASE-15572-2.patch, > HBASE-15572-3.patch, HBASE-15572-4.patch, HBASE-15572-5.patch, > HBASE-15572-6.patch, HBASE-15572-7.patch, HBASE-15572-8.patch > > > Right now the timestamp is always latest. With this patch, users can select > timestamps they want. > In this patch, 4 parameters, "timestamp", "minTimestamp", "maxiTimestamp" and > "maxVersions" are added to HBaseSparkConf. Users can select a timestamp, they > can also select a time range with minimum timestamp and maximum timestamp. A > new test for selecting records with different timestamps is added. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14983) Create metrics for per block type hit/miss ratios
[ https://issues.apache.org/jira/browse/HBASE-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231398#comment-15231398 ] Mikhail Antonov commented on HBASE-14983: - Yeah. Thanks [~enis]! > Create metrics for per block type hit/miss ratios > - > > Key: HBASE-14983 > URL: https://issues.apache.org/jira/browse/HBASE-14983 > Project: HBase > Issue Type: Improvement > Components: metrics >Affects Versions: 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-14983-branch-1.patch, HBASE-14983-v1.patch, > HBASE-14983-v10.patch, HBASE-14983-v2.patch, HBASE-14983-v3.patch, > HBASE-14983-v4.patch, HBASE-14983-v5.patch, HBASE-14983-v6.patch, > HBASE-14983-v7.patch, HBASE-14983-v8.patch, HBASE-14983-v9.patch, > HBASE-14983.patch, Screen Shot 2015-12-15 at 3.33.09 PM.png > > > Missing a root index block is worse than missing a data block. We should know > the difference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14983) Create metrics for per block type hit/miss ratios
[ https://issues.apache.org/jira/browse/HBASE-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231394#comment-15231394 ] Enis Soztutar commented on HBASE-14983: --- bq. SUCCESS: Integrated in HBase-1.3-IT #590 (See https://builds.apache.org/job/HBase-1.3-IT/590/) Seeing this, I assumed that this has been committed to branch-1.3 as well, but the jenkins job was building from branch-1, instead of branch-1.3. So I fixed it. [~mantonov] FYI. See https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3-IT/. I'll also commit this to branch-1.3 as well. > Create metrics for per block type hit/miss ratios > - > > Key: HBASE-14983 > URL: https://issues.apache.org/jira/browse/HBASE-14983 > Project: HBase > Issue Type: Improvement > Components: metrics >Affects Versions: 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-14983-branch-1.patch, HBASE-14983-v1.patch, > HBASE-14983-v10.patch, HBASE-14983-v2.patch, HBASE-14983-v3.patch, > HBASE-14983-v4.patch, HBASE-14983-v5.patch, HBASE-14983-v6.patch, > HBASE-14983-v7.patch, HBASE-14983-v8.patch, HBASE-14983-v9.patch, > HBASE-14983.patch, Screen Shot 2015-12-15 at 3.33.09 PM.png > > > Missing a root index block is worse than missing a data block. We should know > the difference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15614) Report metrics from JvmPauseMonitor
[
https://issues.apache.org/jira/browse/HBASE-15614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231368#comment-15231368
]
Enis Soztutar commented on HBASE-15614:
---
Yes please. I was thinking about this as well. We need how many times we paused
and for how long according to the JVM pause monitor.
We have seen a couple of cases where the OS pauses the process for ~10-30
seconds, and GC logs or GC metrics do not show any activity. OS can pause the
process for various reasons swap, or bad disk drivers.
> Report metrics from JvmPauseMonitor
> ---
>
> Key: HBASE-15614
> URL: https://issues.apache.org/jira/browse/HBASE-15614
> Project: HBase
> Issue Type: Improvement
> Components: metrics, regionserver
>Reporter: Nick Dimiduk
>
> We have {{JvmPauseMonitor}} for detecting JVM pauses; pauses are logged at
> WARN. Would also be good to expose this information on a dashboard via
> metrics system -- make it easier to get this info off the host and into a
> central location for the operator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15606) Limit creating zk connection in HBaseAdmin#getCompactionState() only to case when 'hbase:meta' is checked.
[ https://issues.apache.org/jira/browse/HBASE-15606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15606: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Applied to master branch. Doesn't seem appropriate to branch-1. If you backport it, I'll commit it [~asamir] (I tried and it doesn't apply). > Limit creating zk connection in HBaseAdmin#getCompactionState() only to case > when 'hbase:meta' is checked. > -- > > Key: HBASE-15606 > URL: https://issues.apache.org/jira/browse/HBASE-15606 > Project: HBase > Issue Type: Improvement > Components: Admin >Affects Versions: 2.0.0 >Reporter: Samir Ahmic >Assignee: Samir Ahmic >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15606_v0.patch > > > We are creating new zk connection for every call to this function if we are > checking case of normal compaction. Connection is used to locate 'hbase:meta' > table. > Aim of this jira is to limit connection creation only to case when meta table > is checked. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14983) Create metrics for per block type hit/miss ratios
[ https://issues.apache.org/jira/browse/HBASE-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-14983: -- Fix Version/s: 1.3.0 > Create metrics for per block type hit/miss ratios > - > > Key: HBASE-14983 > URL: https://issues.apache.org/jira/browse/HBASE-14983 > Project: HBase > Issue Type: Improvement > Components: metrics >Affects Versions: 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-14983-branch-1.patch, HBASE-14983-v1.patch, > HBASE-14983-v10.patch, HBASE-14983-v2.patch, HBASE-14983-v3.patch, > HBASE-14983-v4.patch, HBASE-14983-v5.patch, HBASE-14983-v6.patch, > HBASE-14983-v7.patch, HBASE-14983-v8.patch, HBASE-14983-v9.patch, > HBASE-14983.patch, Screen Shot 2015-12-15 at 3.33.09 PM.png > > > Missing a root index block is worse than missing a data block. We should know > the difference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15586) Unify human readable numbers in the web UI
[ https://issues.apache.org/jira/browse/HBASE-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231352#comment-15231352 ] Enis Soztutar commented on HBASE-15586: --- Thanks Stack. Test failure is due to HBASE-15613. [~busbey], [~ndimiduk] do you want this in 1.1 / 1.2? > Unify human readable numbers in the web UI > --- > > Key: HBASE-15586 > URL: https://issues.apache.org/jira/browse/HBASE-15586 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: Screen Shot 2016-04-01 at 3.52.00 PM.png, Screen Shot > 2016-04-01 at 3.52.12 PM.png, Screen Shot 2016-04-01 at 3.52.32 PM.png, > Screen Shot 2016-04-01 at 3.52.38 PM.png, Screen Shot 2016-04-01 at 3.52.49 > PM.png, hbase-15586-v1.patch, hbase-15586-v1.patch, hbase-15586-v2.patch > > > I was looking at something else in the regionserver web ui trying to > understand the WAL size, and we are reporting that as raw bytes, not in MB / > GB range. Did a quick sweep to unify all the human-readable representations > in the UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15613) TestNamespaceCommand times out
[
https://issues.apache.org/jira/browse/HBASE-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231350#comment-15231350
]
Enis Soztutar commented on HBASE-15613:
---
This is the relevant stack trace.
{code}
"hconnection-0x166b3616-metaLookup-shared--pool118-t1" #647 daemon prio=5
os_prio=31 tid=0x7fd485469000 nid=0x1e327 in Object.wait()
[0x000140e1f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:460)
at
io.netty.util.concurrent.DefaultPromise.await0(DefaultPromise.java:355)
- locked <0x000774a8cca0> (a org.apache.hadoop.hbase.ipc.AsyncCall)
at
io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:266)
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:42)
at
org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:226)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:331)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:35264)
at
org.apache.hadoop.hbase.client.ClientSmallScanner$SmallScannerCallable.call(ClientSmallScanner.java:200)
at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:1)
at
org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithoutRetries(RpcRetryingCallerImpl.java:165)
at
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:360)
at
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:1)
at
org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:99)
at
org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:65)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
"hconnection-0x79d11914-shared-pool115-t1" #644 daemon prio=5 os_prio=31
tid=0x7fd4830c8000 nid=0x2261f in Object.wait() [0x00014066c000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:460)
at
io.netty.util.concurrent.DefaultPromise.await0(DefaultPromise.java:355)
- locked <0x000774a34908> (a org.apache.hadoop.hbase.ipc.AsyncCall)
at
io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:266)
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:42)
at
org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:226)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:331)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.multi(ClientProtos.java:35312)
at
org.apache.hadoop.hbase.client.MultiServerCallable.call(MultiServerCallable.java:125)
at
org.apache.hadoop.hbase.client.MultiServerCallable.call(MultiServerCallable.java:1)
at
org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithoutRetries(RpcRetryingCallerImpl.java:165)
at
org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl$SingleServerRequestRunnable.run(AsyncProcess.java:746)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
"PriorityRpcServer.handler=3,queue=1,port=53673" #161 daemon prio=5 os_prio=31
tid=0x7fd4868af000 nid=0x10203 in Object.wait() [0x0001360de000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:460)
at java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:348)
at
org.apache.hadoop.hbase.client.ResultBoundedCompletionService.poll(ResultBoundedCompletionService.java:159)
- locked <0x0007746290b8> (a
[Lorg.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFut
[jira] [Commented] (HBASE-15614) Report metrics from JvmPauseMonitor
[
https://issues.apache.org/jira/browse/HBASE-15614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231310#comment-15231310
]
Nick Dimiduk commented on HBASE-15614:
--
Maybe? The pause monitor thread is reporting 2-3sec pauses that I'm not seeing
in my GC metrics. Will look a bit deeper and circle back. GC metrics wouldn't
report pauses such as this [ugly
one|http://www.evanjones.ca/jvm-mmap-pause.html]; presumably our
{{JvmPauseMonitor}} would catch this.
> Report metrics from JvmPauseMonitor
> ---
>
> Key: HBASE-15614
> URL: https://issues.apache.org/jira/browse/HBASE-15614
> Project: HBase
> Issue Type: Improvement
> Components: metrics, regionserver
>Reporter: Nick Dimiduk
>
> We have {{JvmPauseMonitor}} for detecting JVM pauses; pauses are logged at
> WARN. Would also be good to expose this information on a dashboard via
> metrics system -- make it easier to get this info off the host and into a
> central location for the operator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15400) Use DateTieredCompactor for Date Tiered Compaction
[ https://issues.apache.org/jira/browse/HBASE-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231298#comment-15231298 ] Hudson commented on HBASE-15400: SUCCESS: Integrated in HBase-1.3-IT #600 (See [https://builds.apache.org/job/HBase-1.3-IT/600/]) HBASE-15400 Use DateTieredCompactor for Date Tiered Compaction (Clara (tedyu: rev b17350210b924f6adb4d002d41691fb51369a46c) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/SortedCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DateTieredStoreEngine.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MockStoreFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionRequest.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/FIFOCompactionPolicy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/EverythingPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionPolicy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDateTieredCompaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionConfiguration.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDateTieredCompactionPolicy.java > Use DateTieredCompactor for Date Tiered Compaction > -- > > Key: HBASE-15400 > URL: https://issues.apache.org/jira/browse/HBASE-15400 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15400-15389-v12.patch, HBASE-15400-branch-1.patch, > HBASE-15400-v1.pa, HBASE-15400-v3-v3.patch, HBASE-15400-v3-v4.patch, > HBASE-15400-v3-v5.patch, HBASE-15400-v3.patch, HBASE-15400-v6.patch, > HBASE-15400-v7.patch, HBASE-15400.patch > > > When we compact, we can output multiple files along the current window > boundaries. There are two use cases: > 1. Major compaction: We want to output date tiered store files with data > older than max age archived in trunks of the window size on the higher tier. > Once a window is old enough, we don't combine the windows to promote to the > next tier any further. So files in these windows retain the same timespan as > they were minor-compacted last time, which is the window size of the highest > tier. Major compaction will touch these files and we want to maintain the > same layout. This way, TTL and archiving will be simpler and more efficient. > 2. Bulk load files and the old file generated by major compaction before > upgrading to DTCP. > Pros: > 1. Restore locality, process versioning, updates and deletes while > maintaining the tiered layout. > 2. The best way to fix a skewed layout. > > This work is based on a prototype of DateTieredCompactor from HBASE-15389 and > focused on the part to meet needs for these two use cases while supporting > others. I have to call out a few design decisions: > 1. We only want to output the files along all windows for major compaction. > And we want to output multiple files older than max age in the trunks of the > maximum tier window size determined by base window size, windows per tier and > max age. > 2. For minor compaction, we don't want to output too many files, which will > remain around because of current restriction of contiguous compaction by seq > id. I will only output two files if all the files in the windows are being > combined, one for the data within window and the other for the out-of-window > tail. If there is any file in the window excluded from compaction, only one > file will be output from compaction. When the windows are promoted, the > situation of out of order data will gradually improve. For the incoming > window, we need to accommodate the case with user-specified future data. > 3. We have to pass the boundaries with the list of store file as a complete > time snapshot instead of two separat
[jira] [Commented] (HBASE-15380) Purge rollback support in Store etc.
[
https://issues.apache.org/jira/browse/HBASE-15380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231285#comment-15231285
]
Hadoop QA commented on HBASE-15380:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 30s
{color} | {color:red} hbase-server: patch generated 3 new + 74 unchanged - 3
fixed = 77 total (was 77) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 7s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 22s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 145m 32s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12797595/15380v3.patch |
| JIRA Issue | HBASE-15380 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revisi
[jira] [Commented] (HBASE-15518) Add Per-Table metrics back
[
https://issues.apache.org/jira/browse/HBASE-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231293#comment-15231293
]
Hadoop QA commented on HBASE-15518:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 46s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 45s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
6s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 14s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 51s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 35s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 44s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 44s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s {color} | {color:green} hbase-hadoop-compat: patch generated 0 new + 0
unchanged - 3 fixed = 0 total (was 3) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} hbase-hadoop-compat: patch generated 0 new + 0
unchanged - 3 fixed = 0 total (was 3) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} hbase-hadoop2-compat: patch generated 0 new + 0
unchanged - 3 fixed = 0 total (was 3) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} hbase-hadoop2-compat: patch generated 0 new + 0
unchanged - 3 fixed = 0 total (was 3) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} hbase-server: patch generated 0 new + 0 unchanged -
3 fixed = 0 total (was 3) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} hbase-server: patch generated 0 new + 0 unchanged -
3 fixed = 0 total (was 3) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 21s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 50
[jira] [Commented] (HBASE-15507) Online modification of enabled ReplicationPeerConfig
[ https://issues.apache.org/jira/browse/HBASE-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231244#comment-15231244 ] Ted Yu commented on HBASE-15507: Gone over patch v2. lgtm Let's see what Hadoop QA says. > Online modification of enabled ReplicationPeerConfig > > > Key: HBASE-15507 > URL: https://issues.apache.org/jira/browse/HBASE-15507 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby > Labels: Replication, shell > Fix For: 2.0.0 > > Attachments: HBASE-15507-v1.patch, HBASE-15507-v2.patch, > HBASE-15507.patch > > > It's currently possible to update the table CFs for a replication peer while > it's running, but not the peer configuration or data maps introduced as part > of custom replication endpoints in HBASE-11367. > This means that if you have a custom endpoint that depends on some > configuration parameter, you have to remove the peer and recreate it in order > to change the param. In some use cases that's not possible without risking > data loss. > HBASE-11393, which will consolidate tableCFs in the same znode as the rest of > ReplicationPeerConfig, may help here, but with or without it it still seems > like there needs to be further work to add update config/data API support to > ReplicationAdmin and a callback mechanism to notify the endpoints of config > parameter changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15481) Add pre/post roll to WALObserver
[ https://issues.apache.org/jira/browse/HBASE-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231241#comment-15231241 ] Sean Busbey commented on HBASE-15481: - [~Apache9] does Matteo's reasoning above for the current state of the world make sense / address your concerns? I suspect if we want to fix this properly (for e.g. 2.y or later) we'll need to take a look at the combination of our WAL interfaces and the stuff that's coincidentally built on them (namely replication). Given that, I'm fine with this as a stop-gap measure. > Add pre/post roll to WALObserver > > > Key: HBASE-15481 > URL: https://issues.apache.org/jira/browse/HBASE-15481 > Project: HBase > Issue Type: New Feature >Affects Versions: 1.3.0 >Reporter: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15481-v0.patch, HBASE-15481-v1.patch, > HBASE-15481-v1.patch, HBASE-15481-v2.patch > > > currently the WALObserver has only a pre/post Write. It will be useful to > have a pre/post Roll too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15537) Make multi WAL work with WALs other than FSHLog
[
https://issues.apache.org/jira/browse/HBASE-15537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231234#comment-15231234
]
stack commented on HBASE-15537:
---
I need to fix it. Filed HBASE-15307 a while ago.
> Make multi WAL work with WALs other than FSHLog
> ---
>
> Key: HBASE-15537
> URL: https://issues.apache.org/jira/browse/HBASE-15537
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15537-branch-1.patch, HBASE-15537-v3.patch,
> HBASE-15537-v4.patch, HBASE-15537-v5.patch, HBASE-15537-v6.patch,
> HBASE-15537.patch, HBASE-15537_v2.patch
>
>
> The multi WAL should not be bound with {{FSHLog}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15307) TestFailedAppendAndSync.testLockupAroundBadAssignSync is flakey
[
https://issues.apache.org/jira/browse/HBASE-15307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-15307:
--
Priority: Critical (was: Major)
> TestFailedAppendAndSync.testLockupAroundBadAssignSync is flakey
> ---
>
> Key: HBASE-15307
> URL: https://issues.apache.org/jira/browse/HBASE-15307
> Project: HBase
> Issue Type: Sub-task
> Components: flakey, test
>Reporter: stack
>Assignee: stack
>Priority: Critical
>
> {code}
> Error Message
> test timed out after 30 milliseconds
> Stacktrace
> org.junit.runners.model.TestTimedOutException: test timed out after 30
> milliseconds
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:148)
> at
> org.apache.hadoop.hbase.regionserver.TestFailedAppendAndSync.testLockupAroundBadAssignSync(TestFailedAppendAndSync.java:242)
> Standard Output
> 2016-02-22 18:14:09,154 INFO [main] hbase.ResourceChecker(148): before:
> regionserver.TestFailedAppendAndSync#testLockupAroundBadAssignSync Thread=4,
> OpenFileDescriptor=206, MaxFileDescriptor=6, SystemLoadAverage=1043,
> ProcessCount=300, AvailableMemoryMB=1709
> 2016-02-22 18:14:09,809 DEBUG [main] hbase.HBaseTestingUtility(338): Setting
> hbase.rootdir to
> /home/jenkins/jenkins-slave/workspace/HBase-Trunk_matrix/jdk/latest1.8/label/yahoo-not-h2/hbase-server/target/test-data/892bb23a-6932-4631-b4e0-a6384423d533
> 2016-02-22 18:14:10,167 WARN [Time-limited test] util.NativeCodeLoader(62):
> Unable to load native-hadoop library for your platform... using builtin-java
> classes where applicable
> 2016-02-22 18:14:10,452 INFO [Time-limited test] wal.FSHLog(527): WAL
> configuration: blocksize=32 MB, rollsize=30.40 MB, prefix=wal, suffix=,
> logDir=/home/jenkins/jenkins-slave/workspace/HBase-Trunk_matrix/jdk/latest1.8/label/yahoo-not-h2/hbase-server/target/test-data/892bb23a-6932-4631-b4e0-a6384423d533/TestHRegiontestLockupAroundBadAssignSync/testLockupAroundBadAssignSync,
>
> archiveDir=/home/jenkins/jenkins-slave/workspace/HBase-Trunk_matrix/jdk/latest1.8/label/yahoo-not-h2/hbase-server/target/test-data/892bb23a-6932-4631-b4e0-a6384423d533/TestHRegiontestLockupAroundBadAssignSync/oldWALs
> 2016-02-22 18:14:10,534 INFO [Time-limited test] wal.FSHLog(874): New WAL
> /home/jenkins/jenkins-slave/workspace/HBase-Trunk_matrix/jdk/latest1.8/label/yahoo-not-h2/hbase-server/target/test-data/892bb23a-6932-4631-b4e0-a6384423d533/TestHRegiontestLockupAroundBadAssignSync/testLockupAroundBadAssignSync/wal.1456164850475
> 2016-02-22 18:14:10,673 INFO [Time-limited test] regionserver.HRegion(6118):
> creating HRegion testLockupAroundBadAssignSync HTD ==
> 'testLockupAroundBadAssignSync', {TABLE_ATTRIBUTES => {DURABILITY =>
> 'SYNC_WAL', READONLY => 'false'}, {NAME => 'MyCF', BLOOMFILTER => 'ROW',
> VERSIONS => '2147483647', IN_MEMORY => 'false', KEEP_DELETED_CELLS =>
> 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION =>
> 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536',
> REPLICATION_SCOPE => '0'} RootDir =
> /home/jenkins/jenkins-slave/workspace/HBase-Trunk_matrix/jdk/latest1.8/label/yahoo-not-h2/hbase-server/target/test-data/892bb23a-6932-4631-b4e0-a6384423d533
> Table name == testLockupAroundBadAssignSync
> 2016-02-22 18:14:10,950 DEBUG [Time-limited test] regionserver.HRegion(718):
> Instantiated
> testLockupAroundBadAssignSync,,1456164850613.7eee51a40e497870a49a2965af62a054.
> 2016-02-22 18:14:11,253 INFO
> [StoreOpener-7eee51a40e497870a49a2965af62a054-1] hfile.CacheConfig(292):
> CacheConfig:disabled
> 2016-02-22 18:14:11,265 INFO
> [StoreOpener-7eee51a40e497870a49a2965af62a054-1]
> compactions.CompactionConfiguration(107): size [134217728,
> 9223372036854775807, 9223372036854775807); files [3, 10); ratio 1.20;
> off-peak ratio 5.00; throttle point 2684354560; major period 60480,
> major jitter 0.50, min locality to compact 0.00
> 2016-02-22 18:14:11,269 DEBUG
> [StoreOpener-7eee51a40e497870a49a2965af62a054-1]
> regionserver.HRegionFileSystem(202): No StoreFiles for:
> /home/jenkins/jenkins-slave/workspace/HBase-Trunk_matrix/jdk/latest1.8/label/yahoo-not-h2/hbase-server/target/test-data/892bb23a-6932-4631-b4e0-a6384423d533/data/default/testLockupAroundBadAssignSync/7eee51a40e497870a49a2965af62a054/MyCF
> 2016-02-22 18:14:11,303 DEBUG [Time-limited test] regionserver.HRegion(3838):
> Found 0 recovered edits file(s) under
> /home/jenkins/jenkins-slave/workspace/HBase-Trunk_matrix/jdk/latest1.8/label/yahoo-not-h2/hbase-server/target/test-data/892bb23a-6932-4631-b4e0-a6384423d533/data/default/testLockupAroundBadAssignSync/7eee51a40e497870a49a2965af62a054
> 2016-02-22 18:14:11,337 DEBUG [Time-limited test] wal.WALSplitter(729
[jira] [Commented] (HBASE-15586) Unify human readable numbers in the web UI
[
https://issues.apache.org/jira/browse/HBASE-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231230#comment-15231230
]
Hadoop QA commented on HBASE-15586:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 29s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 98m 34s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 135m 47s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.security.access.TestNamespaceCommands |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12797591/hbase-15586-v2.patch |
| JIRA Issue | HBASE-15586 |
| Optional Tests | asflicense javac javadoc unit |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / ac8cd37 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/1332/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/1332/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/1332/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1332/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
> Unify human readable numbers in the web UI
> ---
>
> Key: HBASE-15586
> URL: https://issues.apache.org/jira/browse/HBASE-15586
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: Screen Shot 2016-04-01 at 3.52.00 PM.png, Screen Shot
> 2016-04-01 at 3.52.12 PM.png, Screen Shot 2016-04-01 at 3.52.32 PM.png,
> Screen Shot 2016-04-01 at 3.52.38 PM.png, Screen Shot 2016-04-01 at 3.52.49
> PM.png, hbase-15586-v1.patch, hbase-15586-v1.patch, hbase-15586-v2.patch
>
>
> I was looking at something else in the regionserver web ui trying to
> understand the WAL size, and we are reporting that as raw bytes, not in MB /
> GB range. Did a quick sweep to unify all the human
[jira] [Updated] (HBASE-15400) Use DateTieredCompactor for Date Tiered Compaction
[ https://issues.apache.org/jira/browse/HBASE-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15400: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patch, Clara. Thanks for the reviews, Enis and Duo. > Use DateTieredCompactor for Date Tiered Compaction > -- > > Key: HBASE-15400 > URL: https://issues.apache.org/jira/browse/HBASE-15400 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15400-15389-v12.patch, HBASE-15400-branch-1.patch, > HBASE-15400-v1.pa, HBASE-15400-v3-v3.patch, HBASE-15400-v3-v4.patch, > HBASE-15400-v3-v5.patch, HBASE-15400-v3.patch, HBASE-15400-v6.patch, > HBASE-15400-v7.patch, HBASE-15400.patch > > > When we compact, we can output multiple files along the current window > boundaries. There are two use cases: > 1. Major compaction: We want to output date tiered store files with data > older than max age archived in trunks of the window size on the higher tier. > Once a window is old enough, we don't combine the windows to promote to the > next tier any further. So files in these windows retain the same timespan as > they were minor-compacted last time, which is the window size of the highest > tier. Major compaction will touch these files and we want to maintain the > same layout. This way, TTL and archiving will be simpler and more efficient. > 2. Bulk load files and the old file generated by major compaction before > upgrading to DTCP. > Pros: > 1. Restore locality, process versioning, updates and deletes while > maintaining the tiered layout. > 2. The best way to fix a skewed layout. > > This work is based on a prototype of DateTieredCompactor from HBASE-15389 and > focused on the part to meet needs for these two use cases while supporting > others. I have to call out a few design decisions: > 1. We only want to output the files along all windows for major compaction. > And we want to output multiple files older than max age in the trunks of the > maximum tier window size determined by base window size, windows per tier and > max age. > 2. For minor compaction, we don't want to output too many files, which will > remain around because of current restriction of contiguous compaction by seq > id. I will only output two files if all the files in the windows are being > combined, one for the data within window and the other for the out-of-window > tail. If there is any file in the window excluded from compaction, only one > file will be output from compaction. When the windows are promoted, the > situation of out of order data will gradually improve. For the incoming > window, we need to accommodate the case with user-specified future data. > 3. We have to pass the boundaries with the list of store file as a complete > time snapshot instead of two separate calls because window layout is > determined by the time the computation is called. So we will need new type of > compaction request. > 4. Since we will assign the same seq id for all output files, we need to sort > by maxTimestamp subsequently. Right now all compaction policy gets the files > sorted for StoreFileManager which sorts by seq id and other criteria. I will > use this order for DTCP only, to avoid impacting other compaction policies. > 5. We need some cleanup of current design of StoreEngine and CompactionPolicy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15400) Use DateTieredCompactor for Date Tiered Compaction
[ https://issues.apache.org/jira/browse/HBASE-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231196#comment-15231196 ] Ted Yu commented on HBASE-15400: Ran TestModifyNamespaceProcedure locally on branch-1 with patch: it passed. Test failure not related to the patch. > Use DateTieredCompactor for Date Tiered Compaction > -- > > Key: HBASE-15400 > URL: https://issues.apache.org/jira/browse/HBASE-15400 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15400-15389-v12.patch, HBASE-15400-branch-1.patch, > HBASE-15400-v1.pa, HBASE-15400-v3-v3.patch, HBASE-15400-v3-v4.patch, > HBASE-15400-v3-v5.patch, HBASE-15400-v3.patch, HBASE-15400-v6.patch, > HBASE-15400-v7.patch, HBASE-15400.patch > > > When we compact, we can output multiple files along the current window > boundaries. There are two use cases: > 1. Major compaction: We want to output date tiered store files with data > older than max age archived in trunks of the window size on the higher tier. > Once a window is old enough, we don't combine the windows to promote to the > next tier any further. So files in these windows retain the same timespan as > they were minor-compacted last time, which is the window size of the highest > tier. Major compaction will touch these files and we want to maintain the > same layout. This way, TTL and archiving will be simpler and more efficient. > 2. Bulk load files and the old file generated by major compaction before > upgrading to DTCP. > Pros: > 1. Restore locality, process versioning, updates and deletes while > maintaining the tiered layout. > 2. The best way to fix a skewed layout. > > This work is based on a prototype of DateTieredCompactor from HBASE-15389 and > focused on the part to meet needs for these two use cases while supporting > others. I have to call out a few design decisions: > 1. We only want to output the files along all windows for major compaction. > And we want to output multiple files older than max age in the trunks of the > maximum tier window size determined by base window size, windows per tier and > max age. > 2. For minor compaction, we don't want to output too many files, which will > remain around because of current restriction of contiguous compaction by seq > id. I will only output two files if all the files in the windows are being > combined, one for the data within window and the other for the out-of-window > tail. If there is any file in the window excluded from compaction, only one > file will be output from compaction. When the windows are promoted, the > situation of out of order data will gradually improve. For the incoming > window, we need to accommodate the case with user-specified future data. > 3. We have to pass the boundaries with the list of store file as a complete > time snapshot instead of two separate calls because window layout is > determined by the time the computation is called. So we will need new type of > compaction request. > 4. Since we will assign the same seq id for all output files, we need to sort > by maxTimestamp subsequently. Right now all compaction policy gets the files > sorted for StoreFileManager which sorts by seq id and other criteria. I will > use this order for DTCP only, to avoid impacting other compaction policies. > 5. We need some cleanup of current design of StoreEngine and CompactionPolicy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15400) Use DateTieredCompactor for Date Tiered Compaction
[ https://issues.apache.org/jira/browse/HBASE-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15400: --- Hadoop Flags: Reviewed Fix Version/s: 1.4.0 1.3.0 > Use DateTieredCompactor for Date Tiered Compaction > -- > > Key: HBASE-15400 > URL: https://issues.apache.org/jira/browse/HBASE-15400 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15400-15389-v12.patch, HBASE-15400-branch-1.patch, > HBASE-15400-v1.pa, HBASE-15400-v3-v3.patch, HBASE-15400-v3-v4.patch, > HBASE-15400-v3-v5.patch, HBASE-15400-v3.patch, HBASE-15400-v6.patch, > HBASE-15400-v7.patch, HBASE-15400.patch > > > When we compact, we can output multiple files along the current window > boundaries. There are two use cases: > 1. Major compaction: We want to output date tiered store files with data > older than max age archived in trunks of the window size on the higher tier. > Once a window is old enough, we don't combine the windows to promote to the > next tier any further. So files in these windows retain the same timespan as > they were minor-compacted last time, which is the window size of the highest > tier. Major compaction will touch these files and we want to maintain the > same layout. This way, TTL and archiving will be simpler and more efficient. > 2. Bulk load files and the old file generated by major compaction before > upgrading to DTCP. > Pros: > 1. Restore locality, process versioning, updates and deletes while > maintaining the tiered layout. > 2. The best way to fix a skewed layout. > > This work is based on a prototype of DateTieredCompactor from HBASE-15389 and > focused on the part to meet needs for these two use cases while supporting > others. I have to call out a few design decisions: > 1. We only want to output the files along all windows for major compaction. > And we want to output multiple files older than max age in the trunks of the > maximum tier window size determined by base window size, windows per tier and > max age. > 2. For minor compaction, we don't want to output too many files, which will > remain around because of current restriction of contiguous compaction by seq > id. I will only output two files if all the files in the windows are being > combined, one for the data within window and the other for the out-of-window > tail. If there is any file in the window excluded from compaction, only one > file will be output from compaction. When the windows are promoted, the > situation of out of order data will gradually improve. For the incoming > window, we need to accommodate the case with user-specified future data. > 3. We have to pass the boundaries with the list of store file as a complete > time snapshot instead of two separate calls because window layout is > determined by the time the computation is called. So we will need new type of > compaction request. > 4. Since we will assign the same seq id for all output files, we need to sort > by maxTimestamp subsequently. Right now all compaction policy gets the files > sorted for StoreFileManager which sorts by seq id and other criteria. I will > use this order for DTCP only, to avoid impacting other compaction policies. > 5. We need some cleanup of current design of StoreEngine and CompactionPolicy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15400) Use DateTieredCompactor for Date Tiered Compaction
[
https://issues.apache.org/jira/browse/HBASE-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231181#comment-15231181
]
Hadoop QA commented on HBASE-15400:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
6s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} branch-1 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
38s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
11s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} branch-1 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s {color} | {color:green} hbase-server: patch generated 0 new + 32 unchanged
- 1 fixed = 32 total (was 33) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 54s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 116m 54s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 163m 50s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestModifyNamespaceProcedure |
| | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12797584/HBASE-15400-branch-1.patch
|
| JIRA Issue | HBASE-15400 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-1 / 0727d0f |
| Def
[jira] [Updated] (HBASE-15507) Online modification of enabled ReplicationPeerConfig
[ https://issues.apache.org/jira/browse/HBASE-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated HBASE-15507: Status: Patch Available (was: Open) Fix to the previous patch, which was missing two files. > Online modification of enabled ReplicationPeerConfig > > > Key: HBASE-15507 > URL: https://issues.apache.org/jira/browse/HBASE-15507 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 1.2.0, 2.0.0, 1.3.0 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby > Labels: Replication, shell > Fix For: 2.0.0 > > Attachments: HBASE-15507-v1.patch, HBASE-15507-v2.patch, > HBASE-15507.patch > > > It's currently possible to update the table CFs for a replication peer while > it's running, but not the peer configuration or data maps introduced as part > of custom replication endpoints in HBASE-11367. > This means that if you have a custom endpoint that depends on some > configuration parameter, you have to remove the peer and recreate it in order > to change the param. In some use cases that's not possible without risking > data loss. > HBASE-11393, which will consolidate tableCFs in the same znode as the rest of > ReplicationPeerConfig, may help here, but with or without it it still seems > like there needs to be further work to add update config/data API support to > ReplicationAdmin and a callback mechanism to notify the endpoints of config > parameter changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15507) Online modification of enabled ReplicationPeerConfig
[ https://issues.apache.org/jira/browse/HBASE-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated HBASE-15507: Attachment: HBASE-15507-v2.patch > Online modification of enabled ReplicationPeerConfig > > > Key: HBASE-15507 > URL: https://issues.apache.org/jira/browse/HBASE-15507 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby > Labels: Replication, shell > Fix For: 2.0.0 > > Attachments: HBASE-15507-v1.patch, HBASE-15507-v2.patch, > HBASE-15507.patch > > > It's currently possible to update the table CFs for a replication peer while > it's running, but not the peer configuration or data maps introduced as part > of custom replication endpoints in HBASE-11367. > This means that if you have a custom endpoint that depends on some > configuration parameter, you have to remove the peer and recreate it in order > to change the param. In some use cases that's not possible without risking > data loss. > HBASE-11393, which will consolidate tableCFs in the same znode as the rest of > ReplicationPeerConfig, may help here, but with or without it it still seems > like there needs to be further work to add update config/data API support to > ReplicationAdmin and a callback mechanism to notify the endpoints of config > parameter changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15507) Online modification of enabled ReplicationPeerConfig
[ https://issues.apache.org/jira/browse/HBASE-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated HBASE-15507: Status: Open (was: Patch Available) > Online modification of enabled ReplicationPeerConfig > > > Key: HBASE-15507 > URL: https://issues.apache.org/jira/browse/HBASE-15507 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 1.2.0, 2.0.0, 1.3.0 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby > Labels: Replication, shell > Fix For: 2.0.0 > > Attachments: HBASE-15507-v1.patch, HBASE-15507-v2.patch, > HBASE-15507.patch > > > It's currently possible to update the table CFs for a replication peer while > it's running, but not the peer configuration or data maps introduced as part > of custom replication endpoints in HBASE-11367. > This means that if you have a custom endpoint that depends on some > configuration parameter, you have to remove the peer and recreate it in order > to change the param. In some use cases that's not possible without risking > data loss. > HBASE-11393, which will consolidate tableCFs in the same znode as the rest of > ReplicationPeerConfig, may help here, but with or without it it still seems > like there needs to be further work to add update config/data API support to > ReplicationAdmin and a callback mechanism to notify the endpoints of config > parameter changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15093) Replication can report incorrect size of log queue for the global source when multiwal is enabled
[ https://issues.apache.org/jira/browse/HBASE-15093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashu Pachauri updated HBASE-15093: -- Attachment: HBASE-15093-V2.patch V2: Rebased on to master. > Replication can report incorrect size of log queue for the global source when > multiwal is enabled > - > > Key: HBASE-15093 > URL: https://issues.apache.org/jira/browse/HBASE-15093 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.2.0, 1.2.1 >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri >Priority: Minor > Attachments: HBASE-15093-V0.patch, HBASE-15093-V1.patch, > HBASE-15093-V1.patch, HBASE-15093-V2.patch > > > Replication can report incorrect size for the size of log queue for the > global source when multiwal is enabled. This happens because the method > MetricsSource#setSizeofLogQueue performs non-trivial operations in a > multithreaded world, even though it is not synchronized. > We can simply divide MetricsSource#setSizeofLogQueue into > MetricsSource#incrSizeofLogQueue and MetricsSource#decrSizeofLogQueue. Not > sure why we are currently directly setting the size instead of > incrementing/decrementing it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15400) Use DateTieredCompactor for Date Tiered Compaction
[
https://issues.apache.org/jira/browse/HBASE-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231119#comment-15231119
]
Hadoop QA commented on HBASE-15400:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
40s {color} | {color:green} hbase-server: patch generated 0 new + 111 unchanged
- 27 fixed = 111 total (was 138) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 8s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 110m 38s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 159m 42s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12797579/HBASE-15400-v7.patch |
| JIRA Issue | HBASE-15400 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ac8cd37 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0
[jira] [Commented] (HBASE-15606) Limit creating zk connection in HBaseAdmin#getCompactionState() only to case when 'hbase:meta' is checked.
[
https://issues.apache.org/jira/browse/HBASE-15606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231094#comment-15231094
]
Hadoop QA commented on HBASE-15606:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
6s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
45m 0s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 28s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 70m 6s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12797589/HBASE-15606_v0.patch |
| JIRA Issue | HBASE-15606 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf910.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / ac8cd37 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
[jira] [Commented] (HBASE-15605) Remove PB references from HCD and HTD for 2.0
[
https://issues.apache.org/jira/browse/HBASE-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231072#comment-15231072
]
Hadoop QA commented on HBASE-15605:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 55s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 27s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 14m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 11m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 35s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 36s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 7s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 47s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 47s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 26s
{color} | {color:red} hbase-client: patch generated 6 new + 1827 unchanged - 3
fixed = 1833 total (was 1830) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 14s
{color} | {color:red} hbase-client: patch generated 6 new + 1827 unchanged - 3
fixed = 1833 total (was 1830) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 15s
{color} | {color:red} hbase-server: patch generated 6 new + 1827 unchanged - 3
fixed = 1833 total (was 1830) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 54s
{color} | {color:red} hbase-server: patch generated 6 new + 1827 unchanged - 3
fixed = 1833 total (was 1830) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
51m 24s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 15m
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 14s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 6s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 25s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 24s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} |
[jira] [Commented] (HBASE-15610) Remove deprecated HConnection for 2.0 thus removing all PB references for 2.0
[
https://issues.apache.org/jira/browse/HBASE-15610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231011#comment-15231011
]
stack commented on HBASE-15610:
---
I vote for option #1, removing HConnection. We went by the book deprecating it
(Was first deprecated in 0.99). Deprecation went in here:
{code}
commit cad76a3431aabe2aee67fcf564cb56af53f48a37
Author: Enis Soztutar
Date: Tue Sep 16 00:09:31 2014 -0700
HBASE-11825 Create Connection and ConnectionManager (Solomon Duskis)
{code}
Another reason to purge it is because our Connection hierarchy has too much by
way of progeny... its a pain navigating it all... HConnection is the ugly
child. With it gone we'll be into a nice clean space where its Connection of
the internal ClusterConnection and thats it.
> Remove deprecated HConnection for 2.0 thus removing all PB references for 2.0
> -
>
> Key: HBASE-15610
> URL: https://issues.apache.org/jira/browse/HBASE-15610
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 2.0.0
>
>
> This is sub-task for HBASE-15174.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15606) Limit creating zk connection in HBaseAdmin#getCompactionState() only to case when 'hbase:meta' is checked.
[ https://issues.apache.org/jira/browse/HBASE-15606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231002#comment-15231002 ] stack commented on HBASE-15606: --- +1 Was going to ask for a test but the fix is minor and obvious, its good by me. Will wait on hadoopqa report before committing. > Limit creating zk connection in HBaseAdmin#getCompactionState() only to case > when 'hbase:meta' is checked. > -- > > Key: HBASE-15606 > URL: https://issues.apache.org/jira/browse/HBASE-15606 > Project: HBase > Issue Type: Improvement > Components: Admin >Affects Versions: 2.0.0 >Reporter: Samir Ahmic >Assignee: Samir Ahmic >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15606_v0.patch > > > We are creating new zk connection for every call to this function if we are > checking case of normal compaction. Connection is used to locate 'hbase:meta' > table. > Aim of this jira is to limit connection creation only to case when meta table > is checked. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
