[jira] [Commented] (HBASE-17215) Separate small/large file delete threads in HFileCleaner to accelerate archived hfile cleanup speed
[
https://issues.apache.org/jira/browse/HBASE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950418#comment-15950418
]
Hadoop QA commented on HBASE-17215:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 38s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 122m 39s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 167m 6s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12861376/HBASE-17215.v3.patch |
| JIRA Issue | HBASE-17215 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 30fad19be2ff 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / d033cbb |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6274/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6274/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Separate small/large file delete threads in HFileCleaner to accelerate
> archived hfile cleanup speed
> ---
>
> Key: HBASE-17215
> URL: https://issues.apache.org/jira/browse/HBASE-17215
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-17215
[jira] [Commented] (HBASE-16833) Implement asynchronous hbase client based on HBASE-15921
[ https://issues.apache.org/jira/browse/HBASE-16833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950348#comment-15950348 ] Duo Zhang commented on HBASE-16833: --- All non critical subtasks have been moved to HBASE-17856. The only unresolved one here is for documentation. > Implement asynchronous hbase client based on HBASE-15921 > > > Key: HBASE-16833 > URL: https://issues.apache.org/jira/browse/HBASE-16833 > Project: HBase > Issue Type: Umbrella > Components: Client >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Labels: asynchronous > Fix For: 2.0.0 > > > In HBASE-15921 we have introduced a simple AsyncTable only supports get, put > and delete. This issue aims to implement a complete async hbase client based > on it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17571) Add batch coprocessor service support
[ https://issues.apache.org/jira/browse/HBASE-17571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17571: -- Parent Issue: HBASE-17856 (was: HBASE-16833) > Add batch coprocessor service support > - > > Key: HBASE-17571 > URL: https://issues.apache.org/jira/browse/HBASE-17571 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Duo Zhang > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17356) Add replica read support
[ https://issues.apache.org/jira/browse/HBASE-17356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17356: -- Parent Issue: HBASE-17856 (was: HBASE-16833) > Add replica read support > > > Key: HBASE-17356 > URL: https://issues.apache.org/jira/browse/HBASE-17356 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Duo Zhang > Fix For: 2.0.0 > > > I think we can do better for scan at least as now we will pass the mvcc to > client. We can use the mvcc to determine if we can get a consistent view when > reading from replicas other than primary. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17358) Unify backoff calculation
[ https://issues.apache.org/jira/browse/HBASE-17358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17358: -- Parent Issue: HBASE-17856 (was: HBASE-16833) > Unify backoff calculation > - > > Key: HBASE-17358 > URL: https://issues.apache.org/jira/browse/HBASE-17358 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Duo Zhang > Fix For: 2.0.0 > > > For Async table the sleep pause is only determined by the retry number, at > least we should also take care of the exception(MultiActionResultTooLarge, > CallQueueTooBig...). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17856) Further improvements for asynchronous hbase client
Duo Zhang created HBASE-17856: - Summary: Further improvements for asynchronous hbase client Key: HBASE-17856 URL: https://issues.apache.org/jira/browse/HBASE-17856 Project: HBase Issue Type: Umbrella Reporter: Duo Zhang Open an umbrella issue to track the remainging non-critical issues for async client. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17855) Fix typo in async client implementation
[ https://issues.apache.org/jira/browse/HBASE-17855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17855: -- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. > Fix typo in async client implementation > --- > > Key: HBASE-17855 > URL: https://issues.apache.org/jira/browse/HBASE-17855 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Labels: trivial > Fix For: 2.0.0 > > Attachments: HBASE-17855.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17855) Fix typo in async client implementation
[ https://issues.apache.org/jira/browse/HBASE-17855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950325#comment-15950325 ] Duo Zhang commented on HBASE-17855: --- Will commit shortly. > Fix typo in async client implementation > --- > > Key: HBASE-17855 > URL: https://issues.apache.org/jira/browse/HBASE-17855 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Labels: trivial > Fix For: 2.0.0 > > Attachments: HBASE-17855.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17821) The CompoundConfiguration#toString is wrong
[
https://issues.apache.org/jira/browse/HBASE-17821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950317#comment-15950317
]
Yi Liang commented on HBASE-17821:
--
[~chia7712], could you help to commit? Thanks
> The CompoundConfiguration#toString is wrong
> ---
>
> Key: HBASE-17821
> URL: https://issues.apache.org/jira/browse/HBASE-17821
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Yi Liang
>Priority: Trivial
> Labels: beginner
> Attachments: HBase-17821-V1.patch, HBase-17821-V1.patch
>
>
> Find this bug when reading code. We dont use the API, so it is a trivial bug.
> sb.append(this.configs); -> sb.append(m);
> {noformat}
> @Override
> public String toString() {
> StringBuffer sb = new StringBuffer();
> sb.append("CompoundConfiguration: " + this.configs.size() + " configs");
> for (ImmutableConfigMap m : this.configs) {
> sb.append(this.configs);
> }
> return sb.toString();
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-16438: --- Status: Patch Available (was: Open) > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-16438_1.patch, > HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-16438: --- Attachment: HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch Lot of test cases needs to be changed because every one has its own initRegion. And we always need a ChunkCreator to make them run. > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-16438_1.patch, > HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-16438: --- Status: Open (was: Patch Available) > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-16438_1.patch, > HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17855) Fix typo in async client implementation
[
https://issues.apache.org/jira/browse/HBASE-17855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950299#comment-15950299
]
Hadoop QA commented on HBASE-17855:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 38s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
35s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 18s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 8s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 104m 42s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 151m 12s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12861362/HBASE-17855.patch |
| JIRA Issue | HBASE-17855 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 7280107deeb8 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d033cbb |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6273/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6273/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Fix typo in async client implementati
[jira] [Commented] (HBASE-17215) Separate small/large file delete threads in HFileCleaner to accelerate archived hfile cleanup speed
[
https://issues.apache.org/jira/browse/HBASE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950298#comment-15950298
]
Anoop Sam John commented on HBASE-17215:
Great point and careful advice.. So good on you Yu Li.
> Separate small/large file delete threads in HFileCleaner to accelerate
> archived hfile cleanup speed
> ---
>
> Key: HBASE-17215
> URL: https://issues.apache.org/jira/browse/HBASE-17215
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-17215.patch, HBASE-17215.v2.patch,
> HBASE-17215.v3.patch
>
>
> When using PCIe-SSD the flush speed will be really quick, and although we
> have per CF flush, we still have the
> {{hbase.regionserver.optionalcacheflushinterval}} setting and some other
> mechanism to avoid data kept in memory for too long to flush small hfiles. In
> our online environment we found the single thread cleaner kept cleaning
> earlier flushed small files while large files got no chance, which caused
> disk full then many other problems.
> Deleting hfiles in parallel with too many threads will also increase the
> workload of namenode, so here we propose to separate large/small hfile
> cleaner threads just like we do for compaction, and it turned out to work
> well in our cluster.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17215) Separate small/large file delete threads in HFileCleaner to accelerate archived hfile cleanup speed
[
https://issues.apache.org/jira/browse/HBASE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950291#comment-15950291
]
Yu Li commented on HBASE-17215:
---
Thanks for sharing [~huaxiang].
bq. First, hdfs operation is slow, it takes 120 ms ~ 140 ms to delete a file
from hdfs
Please carefully check and confirm whether this is caused by pressure on
Namenode, and if so the change here might worsen it (more requests in parallel,
although not that much). And good luck(smile).
> Separate small/large file delete threads in HFileCleaner to accelerate
> archived hfile cleanup speed
> ---
>
> Key: HBASE-17215
> URL: https://issues.apache.org/jira/browse/HBASE-17215
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-17215.patch, HBASE-17215.v2.patch,
> HBASE-17215.v3.patch
>
>
> When using PCIe-SSD the flush speed will be really quick, and although we
> have per CF flush, we still have the
> {{hbase.regionserver.optionalcacheflushinterval}} setting and some other
> mechanism to avoid data kept in memory for too long to flush small hfiles. In
> our online environment we found the single thread cleaner kept cleaning
> earlier flushed small files while large files got no chance, which caused
> disk full then many other problems.
> Deleting hfiles in parallel with too many threads will also increase the
> workload of namenode, so here we propose to separate large/small hfile
> cleaner threads just like we do for compaction, and it turned out to work
> well in our cluster.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (HBASE-17215) Separate small/large file delete threads in HFileCleaner to accelerate archived hfile cleanup speed
[
https://issues.apache.org/jira/browse/HBASE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950288#comment-15950288
]
Yu Li edited comment on HBASE-17215 at 3/31/17 4:08 AM:
Minor change to address review comment
was (Author: carp84):
Minor change to address review comment and trigger HadoopQA (strange that the
previous one didn't...)
> Separate small/large file delete threads in HFileCleaner to accelerate
> archived hfile cleanup speed
> ---
>
> Key: HBASE-17215
> URL: https://issues.apache.org/jira/browse/HBASE-17215
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-17215.patch, HBASE-17215.v2.patch,
> HBASE-17215.v3.patch
>
>
> When using PCIe-SSD the flush speed will be really quick, and although we
> have per CF flush, we still have the
> {{hbase.regionserver.optionalcacheflushinterval}} setting and some other
> mechanism to avoid data kept in memory for too long to flush small hfiles. In
> our online environment we found the single thread cleaner kept cleaning
> earlier flushed small files while large files got no chance, which caused
> disk full then many other problems.
> Deleting hfiles in parallel with too many threads will also increase the
> workload of namenode, so here we propose to separate large/small hfile
> cleaner threads just like we do for compaction, and it turned out to work
> well in our cluster.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17215) Separate small/large file delete threads in HFileCleaner to accelerate archived hfile cleanup speed
[
https://issues.apache.org/jira/browse/HBASE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated HBASE-17215:
--
Attachment: HBASE-17215.v3.patch
Minor change to address review comment and trigger HadoopQA (strange that the
previous one didn't...)
> Separate small/large file delete threads in HFileCleaner to accelerate
> archived hfile cleanup speed
> ---
>
> Key: HBASE-17215
> URL: https://issues.apache.org/jira/browse/HBASE-17215
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-17215.patch, HBASE-17215.v2.patch,
> HBASE-17215.v3.patch
>
>
> When using PCIe-SSD the flush speed will be really quick, and although we
> have per CF flush, we still have the
> {{hbase.regionserver.optionalcacheflushinterval}} setting and some other
> mechanism to avoid data kept in memory for too long to flush small hfiles. In
> our online environment we found the single thread cleaner kept cleaning
> earlier flushed small files while large files got no chance, which caused
> disk full then many other problems.
> Deleting hfiles in parallel with too many threads will also increase the
> workload of namenode, so here we propose to separate large/small hfile
> cleaner threads just like we do for compaction, and it turned out to work
> well in our cluster.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17828) Remove IS annotations from IA.Public classes
[ https://issues.apache.org/jira/browse/HBASE-17828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17828: -- Affects Version/s: 2.0.0 Priority: Critical (was: Major) Fix Version/s: 2.0.0 Component/s: API Issue Type: Task (was: Bug) Let's address this before cutting branch-2. > Remove IS annotations from IA.Public classes > > > Key: HBASE-17828 > URL: https://issues.apache.org/jira/browse/HBASE-17828 > Project: HBase > Issue Type: Task > Components: API >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Priority: Critical > Fix For: 2.0.0 > > > As discussed in the dev mailing list, we do not mention the IS annotations > for public API in our refguide so the IS annotations for IA.Public classes > only makes people confusing. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15314) Allow more than one backing file in bucketcache
[
https://issues.apache.org/jira/browse/HBASE-15314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-15314:
--
Fix Version/s: (was: 2.0)
2.0.0
> Allow more than one backing file in bucketcache
> ---
>
> Key: HBASE-15314
> URL: https://issues.apache.org/jira/browse/HBASE-15314
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache
>Reporter: stack
>Assignee: chunhui shen
> Fix For: 2.0.0
>
> Attachments: FileIOEngine.java, HBASE-15314.master.001.patch,
> HBASE-15314.master.001.patch, HBASE-15314.patch, HBASE-15314-v2.patch,
> HBASE-15314-v3.patch, HBASE-15314-v4.patch, HBASE-15314-v5.patch,
> HBASE-15314-v6.patch, HBASE-15314-v7.patch, HBASE-15314-v8.patch
>
>
> Allow bucketcache use more than just one backing file: e.g. chassis has more
> than one SSD in it.
> Usage (Setting the following configurations in hbase-site.xml):
> {quote}
>
> hbase.bucketcache.ioengine
>
> files:/mnt/disk1/bucketcache,/mnt/disk2/bucketcache,/mnt/disk3/bucketcache,/mnt/disk4/bucketcache
>
>
> hbase.bucketcache.size
> 1048576
>
> {quote}
> The above setting means the total capacity of cache is 1048576MB(1TB), each
> file length will be set to 0.25TB.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-14734) BindException when setting up MiniKdc
[ https://issues.apache.org/jira/browse/HBASE-14734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14734: -- Fix Version/s: (was: 2.0..) 2.0.0 > BindException when setting up MiniKdc > - > > Key: HBASE-14734 > URL: https://issues.apache.org/jira/browse/HBASE-14734 > Project: HBase > Issue Type: Sub-task > Components: flakey, test >Reporter: stack >Assignee: Appy > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.4 > > Attachments: HBASE-14734.master.001.patch > > > Root cause : Port for kdc service gets selected in the constructor, but we > bind to it later in MiniKdc.start()-->MiniKdc.initKDCServer() --> > KdcServer.start(). In meantime, some other service can capture the port which > results in BindException. The solution here is to catch the exception and > retry. > From > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/330/jdk=latest1.7,label=Hadoop/testReport/junit/org.apache.hadoop.hbase.security.token/TestGenerateDelegationToken/org_apache_hadoop_hbase_security_token_TestGenerateDelegationToken/ > Error Message > Address already in use > Stacktrace > java.net.BindException: Address already in use > at sun.nio.ch.Net.bind0(Native Method) > at sun.nio.ch.Net.bind(Net.java:444) > at sun.nio.ch.Net.bind(Net.java:436) > at > sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214) > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) > at > org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:198) > at > org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:51) > at > org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:547) > at > org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$400(AbstractPollingIoAcceptor.java:68) > at > org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:422) > at > org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Can this utility be made to not fail if address taken? Try another? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14417) Incremental backup and bulk loading
[ https://issues.apache.org/jira/browse/HBASE-14417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14417: -- Fix Version/s: (was: 2.0) 2.0.0 > Incremental backup and bulk loading > --- > > Key: HBASE-14417 > URL: https://issues.apache.org/jira/browse/HBASE-14417 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Ted Yu >Priority: Blocker > Labels: backup > Fix For: 2.0.0 > > Attachments: 14417-tbl-ext.v10.txt, 14417-tbl-ext.v11.txt, > 14417-tbl-ext.v14.txt, 14417-tbl-ext.v18.txt, 14417-tbl-ext.v19.txt, > 14417-tbl-ext.v20.txt, 14417-tbl-ext.v21.txt, 14417-tbl-ext.v22.txt, > 14417-tbl-ext.v23.txt, 14417-tbl-ext.v24.txt, 14417-tbl-ext.v9.txt, > 14417.v11.txt, 14417.v13.txt, 14417.v1.txt, 14417.v21.txt, 14417.v23.txt, > 14417.v24.txt, 14417.v25.txt, 14417.v2.txt, 14417.v6.txt > > > Currently, incremental backup is based on WAL files. Bulk data loading > bypasses WALs for obvious reasons, breaking incremental backups. The only way > to continue backups after bulk loading is to create new full backup of a > table. This may not be feasible for customers who do bulk loading regularly > (say, every day). > Here is the review board (out of date): > https://reviews.apache.org/r/54258/ > In order not to miss the hfiles which are loaded into region directories in a > situation where postBulkLoadHFile() hook is not called (bulk load being > interrupted), we record hfile names thru preCommitStoreFile() hook. > At time of incremental backup, we check the presence of such hfiles. If they > are present, they become part of the incremental backup image. > Here is review board: > https://reviews.apache.org/r/57790/ > Google doc for design: > https://docs.google.com/document/d/1ACCLsecHDvzVSasORgqqRNrloGx4mNYIbvAU7lq5lJE -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17287) Master becomes a zombie if filesystem object closes
[
https://issues.apache.org/jira/browse/HBASE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17287:
--
Fix Version/s: (was: 2.0)
2.0.0
> Master becomes a zombie if filesystem object closes
> ---
>
> Key: HBASE-17287
> URL: https://issues.apache.org/jira/browse/HBASE-17287
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Clay B.
>Assignee: Ted Yu
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.3.1, 1.1.9, 1.2.6
>
> Attachments: 17287.branch-1.1.v4.txt, 17287.branch-1.v3.txt,
> 17287.branch-1.v4.txt, 17287.master.v2.txt, 17287.master.v3.txt,
> 17287.master.v4.txt, 17287.master.v5.txt, 17287.v2.txt
>
>
> We have seen an issue whereby if the HDFS is unstable and the HBase master's
> HDFS client is unable to stabilize before
> {{dfs.client.failover.max.attempts}} then the master's filesystem object
> closes. This seems to result in an HBase master which will continue to run
> (process and znode exists) but no meaningful work can be done (e.g. assigning
> meta).What we saw in our HBase master logs was:{code}2016-12-01 19:19:08,192
> ERROR org.apache.hadoop.hbase.master.handler.MetaServerShutdownHandler:
> Caught M_META_SERVER_SHUTDOWN, count=1java.io.IOException: failed log
> splitting for cluster-r5n12.bloomberg.com,60200,1480632863218, will retryat
> org.apache.hadoop.hbase.master.handler.MetaServerShutdownHandler.process(MetaServerShutdownHandler.java:84)at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at
> java.lang.Thread.run(Thread.java:745)Caused by: java.io.IOException:
> Filesystem closed{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17674) Major compaction may be cancelled in CompactionChecker
[
https://issues.apache.org/jira/browse/HBASE-17674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17674:
--
Fix Version/s: (was: 2.0)
2.0.0
> Major compaction may be cancelled in CompactionChecker
> --
>
> Key: HBASE-17674
> URL: https://issues.apache.org/jira/browse/HBASE-17674
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 2.0.0, 1.3.0, 1.2.4, 0.98.24
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17674-master-v1.patch
>
>
> CompactionChecker will periodically check whether a region should perform a
> major compaction.
> If a region should perform a major compact, a request is submitted. But
> before the request is submitted, the variable forceMajor is not set to true
> by calling triggerMajorCompaction.
> When filtering a storefile, the large storefile may cause the request to be
> canceled or became minor compact.
> {code:title=HRegionServer.java|borderStyle=solid}
> @Override
> protected void chore() {
> for (Region r : this.instance.onlineRegions.values()) {
> if (r == null)
> continue;
> for (Store s : r.getStores()) {
> try {
> long multiplier = s.getCompactionCheckMultiplier();
> assert multiplier > 0;
> if (iteration % multiplier != 0) continue;
> if (s.needsCompaction()) {
> // Queue a compaction. Will recognize if major is needed.
> this.instance.compactSplitThread.requestSystemCompaction(r, s,
> getName()
> + " requests compaction");
> } else if (s.isMajorCompaction()) {
> if (majorCompactPriority == DEFAULT_PRIORITY
> || majorCompactPriority >
> ((HRegion)r).getCompactPriority()) {
> this.instance.compactSplitThread.requestCompaction(r, s,
> getName()
> + " requests major compaction; use default priority",
> null);
> } else {
> this.instance.compactSplitThread.requestCompaction(r, s,
> getName()
> + " requests major compaction; use configured priority",
> this.majorCompactPriority, null, null);
> }
> }
> } catch (IOException e) {
> LOG.warn("Failed major compaction check on " + r, e);
> }
> }
> }
> iteration = (iteration == Long.MAX_VALUE) ? 0 : (iteration + 1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17703) TestThriftServerCmdLine is flaky in master branch
[
https://issues.apache.org/jira/browse/HBASE-17703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17703:
--
Fix Version/s: (was: 2.0)
2.0.0
> TestThriftServerCmdLine is flaky in master branch
> -
>
> Key: HBASE-17703
> URL: https://issues.apache.org/jira/browse/HBASE-17703
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Jan Hentschel
> Fix For: 2.0.0
>
> Attachments: HBASE-17703.master.001.patch
>
>
> According to
> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
> , TestThriftServerCmdLine is the top flaky test.
> It fails in QA run sometimes.
> https://builds.apache.org/job/PreCommit-HBASE-Build/5848/artifact/patchprocess/patch-unit-hbase-thrift.txt
> {code}
> testRunThriftServer[0](org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine)
> Time elapsed: 10.057 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.testRunThriftServer(TestThriftServerCmdLine.java:187)
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 60.212 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine
> testRunThriftServer[0](org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine)
> Time elapsed: 0.166 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServer.checkTableList(TestThriftServer.java:254)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.talkToThriftServer(TestThriftServerCmdLine.java:227)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.testRunThriftServer(TestThriftServerCmdLine.java:190)
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 64.428 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine
> testRunThriftServer[0](org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine)
> Time elapsed: 0.144 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServer.checkTableList(TestThriftServer.java:254)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.talkToThriftServer(TestThriftServerCmdLine.java:227)
> at
> org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine.testRunThriftServer(TestThriftServerCmdLine.java:190)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17689) Add support for table.existsAll in thrift2 THBaseservice
[ https://issues.apache.org/jira/browse/HBASE-17689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17689: -- Fix Version/s: (was: 2.0) 2.0.0 > Add support for table.existsAll in thrift2 THBaseservice > > > Key: HBASE-17689 > URL: https://issues.apache.org/jira/browse/HBASE-17689 > Project: HBase > Issue Type: Improvement > Components: Thrift >Reporter: Yechao Chen >Assignee: Yechao Chen > Labels: thrift2 > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17689.patch > > > hbase thrift2 support existsAll(List gets) throws IOException; > hbase.thrift add a method to service THBaseService like this > list existsAll( > 1: required binary table, > 2: required list tgets > ) throws (1:TIOError io) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17706: -- Fix Version/s: (was: 2.0) 2.0.0 > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Fix For: 2.0.0 > > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706-06.patch, HBASE-17706-07.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17707) New More Accurate Table Skew cost function/generator
[ https://issues.apache.org/jira/browse/HBASE-17707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17707: -- Fix Version/s: (was: 2.0) 2.0.0 > New More Accurate Table Skew cost function/generator > > > Key: HBASE-17707 > URL: https://issues.apache.org/jira/browse/HBASE-17707 > Project: HBase > Issue Type: New Feature > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17707-00.patch, HBASE-17707-01.patch, > HBASE-17707-02.patch, HBASE-17707-03.patch, HBASE-17707-04.patch, > HBASE-17707-05.patch, HBASE-17707-06.patch, HBASE-17707-07.patch, > HBASE-17707-08.patch, HBASE-17707-09.patch, HBASE-17707-11.patch, > HBASE-17707-11.patch, HBASE-17707-12.patch, test-balancer2-13617.out > > > This patch includes new version of the TableSkewCostFunction and a new > TableSkewCandidateGenerator. > The new TableSkewCostFunction computes table skew by counting the minimal > number of region moves required for a given table to perfectly balance the > table across the cluster (i.e. as if the regions from that table had been > round-robin-ed across the cluster). This number of moves is computer for each > table, then normalized to a score between 0-1 by dividing by the number of > moves required in the absolute worst case (i.e. the entire table is stored on > one server), and stored in an array. The cost function then takes a weighted > average of the average and maximum value across all tables. The weights in > this average are configurable to allow for certain users to more strongly > penalize situations where one table is skewed versus where every table is a > little bit skewed. To better spread this value more evenly across the range > 0-1, we take the square root of the weighted average to get the final value. > The new TableSkewCandidateGenerator generates region moves/swaps to optimize > the above TableSkewCostFunction. It first simply tries to move regions until > each server has the right number of regions, then it swaps regions around > such that each region swap improves table skew across the cluster. > We tested the cost function and generator in our production clusters with > 100s of TBs of data and 100s of tables across dozens of servers and found > both to be very performant and accurate. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17760) HDFS Balancer doc is misleading
[ https://issues.apache.org/jira/browse/HBASE-17760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17760: -- Fix Version/s: (was: 2.0) 2.0.0 > HDFS Balancer doc is misleading > --- > > Key: HBASE-17760 > URL: https://issues.apache.org/jira/browse/HBASE-17760 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 2.0.0 >Reporter: Harsh J >Assignee: Harsh J >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17760.000.patch > > > HBASE-15332 added a doc note about how to use HDFS-6133, but the steps it > adds are incorrect. The specific balancer command provided in the doc note is > incorrect and not required. > Since HBase uses favored nodes features internally (HBASE-7932), and > HBASE-7942 extended that information to cover HDFS hinting too, the only step > required in the doc note is to enable the pinning feature DN-side. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17798) RpcServer.Listener.Reader can abort due to CancelledKeyException
[
https://issues.apache.org/jira/browse/HBASE-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17798:
--
Fix Version/s: (was: 2.0)
2.0.0
> RpcServer.Listener.Reader can abort due to CancelledKeyException
>
>
> Key: HBASE-17798
> URL: https://issues.apache.org/jira/browse/HBASE-17798
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0, 1.2.4, 0.98.24
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17798-master-v2.patch, connections.png,
> HBASE-17798-0.98-v1.patch, HBASE-17798-0.98-v2.patch,
> HBASE-17798-branch-1-v1.patch, HBASE-17798-branch-1-v2.patch,
> HBASE-17798-master-v1.patch, HBASE-17798-master-v2.patch
>
>
> In our production cluster(0.98), some of the requests were unacceptable
> because RpcServer.Listener.Reader were aborted.
> getReader() will return the next reader to deal with request.
> The implementation of getReader() as below：
> {code:title=RpcServer.java|borderStyle=solid}
> // The method that will return the next reader to work with
> // Simplistic implementation of round robin for now
> Reader getReader() {
> currentReader = (currentReader + 1) % readers.length;
> return readers[currentReader];
> }
> {code}
> If one of the readers abort, then it will lead to fall on the reader's
> request will never be dealt with.
> Why does RpcServer.Listener.Reader abort?We add the debug log to get it.
> After a while, we got the following exception:
> {code}
> 2017-03-10 08:05:13,247 ERROR [RpcServer.reader=3,port=60020] ipc.RpcServer:
> RpcServer.listener,port=60020: unexpectedly error in Reader(Throwable)
> java.nio.channels.CancelledKeyException
> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
> at java.nio.channels.SelectionKey.isReadable(SelectionKey.java:289)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:592)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:566)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> So, when deal with the request in reader, we should handle
> CanceledKeyException.
> --
> versions 1.x and 2.0 will log and retrun when dealing with the
> InterruptedException in Reader#doRunLoop after HBASE-10521. It will lead to
> the same problem.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17779) disable_table_replication returns misleading message and does not turn off replication
[ https://issues.apache.org/jira/browse/HBASE-17779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17779: -- Fix Version/s: (was: 2.0) 2.0.0 > disable_table_replication returns misleading message and does not turn off > replication > -- > > Key: HBASE-17779 > URL: https://issues.apache.org/jira/browse/HBASE-17779 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Janos Gub > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17779-branch-1.patch, HBASE-17779.patch, > HBASE-17779-v2.patch > > > Currently HBaseAdmin#isTableRepEnabled() returns false if ANY of the columns > families is not replicated. > Because of this if you have a table where replication is partially enabled, > calling disable_table_replication will not have any effect, but will report > that replication for a given table is disabled. > Workaround is enabling table replication before disabling. > As a solution quoted from HBASE-17460: > Admin#disableTableReplication() returns nothing and isTableRepEnabled() > returns boolean. For master branch, we can let isTableRepEnabled() return > enum (partially disabled, disabled, etc) This way, > Admin#disableTableReplication() can return meaningful value to the user. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17831) Support small scan in thrift2
[ https://issues.apache.org/jira/browse/HBASE-17831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17831: -- Fix Version/s: (was: 2.0) 2.0.0 > Support small scan in thrift2 > - > > Key: HBASE-17831 > URL: https://issues.apache.org/jira/browse/HBASE-17831 > Project: HBase > Issue Type: Improvement > Components: Thrift >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17831-branch-1.patch, > HBASE-17831-branch-1-v1.patch, HBASE-17831-master.patch, > HBASE-17831-master-v1.patch > > > Support small scan in thrift2 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17806) TestRSGroups#testMoveServersAndTables is flaky in master branch
[
https://issues.apache.org/jira/browse/HBASE-17806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17806:
--
Fix Version/s: (was: 2.0)
2.0.0
> TestRSGroups#testMoveServersAndTables is flaky in master branch
> ---
>
> Key: HBASE-17806
> URL: https://issues.apache.org/jira/browse/HBASE-17806
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Guangxu Cheng
> Fix For: 2.0.0
>
> Attachments: HBASE-17806-v1.patch
>
>
> According to https://builds.apache.org/job/HBASE-Flaky-Tests ,
> TestRSGroups#testMoveServersAndTables is the top flaky test.
> Recent test failure:
> https://builds.apache.org/job/HBASE-Flaky-Tests/14062/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/testMoveServersAndTables/
> {code}
> java.lang.AssertionError: expected:<4> but was:<5>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:834)
> at org.junit.Assert.assertEquals(Assert.java:645)
> at org.junit.Assert.assertEquals(Assert.java:631)
> at
> org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.testMoveServersAndTables(TestRSGroupsBase.java:747)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17782) Extend IdReadWriteLock to support using both weak and soft reference
[
https://issues.apache.org/jira/browse/HBASE-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17782:
--
Affects Version/s: (was: 2.0)
2.0.0
Fix Version/s: (was: 2.0)
2.0.0
> Extend IdReadWriteLock to support using both weak and soft reference
>
>
> Key: HBASE-17782
> URL: https://issues.apache.org/jira/browse/HBASE-17782
> Project: HBase
> Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0
>
> Attachments: HBASE-17782.patch, HBASE-17782.patch
>
>
> As per discussed in HBASE-17747, we need to make it configurable to decide
> whether to use weak or soft reference for {{IdReadWriteLock}}, with soft
> reference by default.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17854) Use StealJobQueue in HFileCleaner after HBASE-17215
[
https://issues.apache.org/jira/browse/HBASE-17854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17854:
--
Affects Version/s: (was: 2.0)
2.0.0
> Use StealJobQueue in HFileCleaner after HBASE-17215
> ---
>
> Key: HBASE-17854
> URL: https://issues.apache.org/jira/browse/HBASE-17854
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-17854.patch
>
>
> In HBASE-17215 we use specific threads for deleting large/small (archived)
> hfiles, and will improve it from below aspects in this JIRA:
> 1. Using {{StealJobQueue}} to allow large file deletion thread to steal jobs
> from small queue, based on the experience that in real world there'll be much
> more small hfiles
> 2. {{StealJobQueue}} is a kind of {{PriorityQueue}}, so we could also delete
> from the larger file in the queues.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17841) ServerCrashProcedure is not triggered when meta server with unflushed edits is aborted
[
https://issues.apache.org/jira/browse/HBASE-17841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17841:
--
Affects Version/s: (was: 2.0)
2.0.0
> ServerCrashProcedure is not triggered when meta server with unflushed edits
> is aborted
> --
>
> Key: HBASE-17841
> URL: https://issues.apache.org/jira/browse/HBASE-17841
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Ted Yu
> Attachments: 17841.tst
>
>
> When writing unit test for HBASE-17287, I noticed that the wait for master to
> come down after hdfs enters safe mode times out (where meta server still has
> unflushed edits).
> The same test in branch-1 passes fine.
> Looking at
> org.apache.hadoop.hbase.master.procedure.TestSafemodeBringsDownMaster-output.txt
> , I don't see occurrence of ServerCrashProcedure.
> While in branch-1, there is something similar to the following:
> {code}
> at org.apache.hadoop.hdfs.DFSClient.rename(DFSClient.java:1661)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.rename(DistributedFileSystem.java:525)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.getLogDirs(MasterFileSystem.java:364)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:429)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.splitMetaLog(MasterFileSystem.java:343)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.splitMetaLog(MasterFileSystem.java:334)
> at
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.processMeta(ServerCrashProcedure.java:351)
> at
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.executeFromState(ServerCrashProcedure.java:239)
> at
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.executeFromState(ServerCrashProcedure.java:73)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:506)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1152)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17747) Support both weak and soft object pool
[
https://issues.apache.org/jira/browse/HBASE-17747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17747:
--
Affects Version/s: (was: 2.0)
2.0.0
Fix Version/s: (was: 2.0)
2.0.0
> Support both weak and soft object pool
> --
>
> Key: HBASE-17747
> URL: https://issues.apache.org/jira/browse/HBASE-17747
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0
>
> Attachments: HBASE-17747.patch, HBASE-17747.v2.patch,
> HBASE-17747.v3.patch
>
>
> During YCSB testing on embedded mode after HBASE-17744, we found that under
> high read load GC is quite severe even with offheap L2 cache. After some
> investigation, we found it's caused by using weak reference in
> {{IdReadWriteLock}}. In embedded mode the read is so quick that the lock
> might already get promoted to the old generation when the weak reference is
> cleared, which causes dirty card table (old reference get removed and new
> lock object set into {{referenceCache}}, see {{WeakObjectPool#get}}) thus
> slowing YGC. In distributed mode there'll also be more lock object created
> with weak reference than soft reference that slowing down the processing.
> So we proposed to use soft reference for this {{IdReadWriteLock}} used in
> cache, which won't get cleared until JVM memory is not enough, and could
> resolve the issue mentioned above. What's more, we propose to extend the
> {{WeakObjectPool}} to be more generate to support both weak and soft
> reference.
> Note that the GC issue only emerges under embedded mode with DirectOperator,
> in which case all costs on the wire is removed thus produces extremely high
> concurrency.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17745) Support short circuit connection for master services
[ https://issues.apache.org/jira/browse/HBASE-17745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17745: -- Affects Version/s: (was: 2.0) 2.0.0 Fix Version/s: (was: 2.0) 2.0.0 > Support short circuit connection for master services > > > Key: HBASE-17745 > URL: https://issues.apache.org/jira/browse/HBASE-17745 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Yu Li >Assignee: Yu Li > Fix For: 2.0.0 > > Attachments: HBASE-17745.patch, HBASE-17745.v2.patch, > HBASE-17745.v2.trival.patch, HBASE-17745.v2.trival.patch, HBASE-17745.v3.patch > > > As titled, now we have short circuit connection, but no short circuit for > master services, and we propose to support it in this JIRA. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17742) Reverse scan with empty start row returns incorrect result
[ https://issues.apache.org/jira/browse/HBASE-17742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17742: -- Affects Version/s: (was: 2.0) 2.0.0 > Reverse scan with empty start row returns incorrect result > -- > > Key: HBASE-17742 > URL: https://issues.apache.org/jira/browse/HBASE-17742 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Yu Li >Assignee: Yu Li > Attachments: HBASE-17742.patch > > > As titled, reverse scan with empty start row should return empty result but > doesn't in latest master code base. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17676) Get class name once for all in AbstractFSWAL
[
https://issues.apache.org/jira/browse/HBASE-17676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17676:
--
Affects Version/s: (was: 2.0)
2.0.0
Fix Version/s: (was: 2.0)
2.0.0
> Get class name once for all in AbstractFSWAL
>
>
> Key: HBASE-17676
> URL: https://issues.apache.org/jira/browse/HBASE-17676
> Project: HBase
> Issue Type: Improvement
> Components: Performance
>Affects Versions: 2.0.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0
>
> Attachments: HBASE-17676.patch, HBASE-17676.v2.patch,
> HBASE-17676.v3.patch
>
>
> While verifying HBASE-17471 with high write workload, observed several
> handler thread at getting class name in jstack, as shown below:
> {noformat}
> "B.defaultRpcServer.handler=60,queue=3,port=16020" daemon prio=10
> tid=0x7f0673835800 nid=0x4dec runnable [0x7f06721b5000]
>java.lang.Thread.State: RUNNABLE
> at java.lang.Class.getEnclosingMethod0(Native Method)
> at java.lang.Class.getEnclosingMethodInfo(Class.java:964)
> at java.lang.Class.getEnclosingClass(Class.java:1137)
> at java.lang.Class.getSimpleBinaryName(Class.java:1282)
> at java.lang.Class.getSimpleName(Class.java:1174)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.stampSequenceIdAndPublishToRingBuffer(FSHLog.java:1251)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1238)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3173)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2874)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:823)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:785)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2259)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:848)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:756)
> {noformat}
> We could get the class name in constructor and use it for all places needed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16705) Eliminate long to Long auto boxing in LongComparator
[
https://issues.apache.org/jira/browse/HBASE-16705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-16705:
--
Affects Version/s: (was: 2.0..)
2.0.0
> Eliminate long to Long auto boxing in LongComparator
>
>
> Key: HBASE-16705
> URL: https://issues.apache.org/jira/browse/HBASE-16705
> Project: HBase
> Issue Type: Improvement
> Components: Filters
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 0.98.23
>
> Attachments: HBASE-16705-master.patch
>
>
> LongComparator
> @Override
> public int compareTo(byte[] value, int offset, int length) {
> Long that = Bytes.toLong(value, offset, length);
> return this.longValue.compareTo(that);
> }
> Every time need to convert long to Long, this is not necessary.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17482) mvcc mechanism fails when using mvccPreAssign
[ https://issues.apache.org/jira/browse/HBASE-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17482: -- Affects Version/s: (was: 2.0..) 2.0.0 > mvcc mechanism fails when using mvccPreAssign > - > > Key: HBASE-17482 > URL: https://issues.apache.org/jira/browse/HBASE-17482 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-17482.patch, HBASE-17482.v2.patch, > HBASE-17482.v3.patch > > > If mvccPreAssign and ASYNC_WAL is used, then cells may have been commited to > memstore before append thread can stamp seqid to them. The unit test shows > everything. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16694) Reduce garbage for onDiskChecksum in HFileBlock
[ https://issues.apache.org/jira/browse/HBASE-16694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-16694: -- Affects Version/s: (was: 2.0..) 2.0.0 > Reduce garbage for onDiskChecksum in HFileBlock > --- > > Key: HBASE-16694 > URL: https://issues.apache.org/jira/browse/HBASE-16694 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: binlijin >Assignee: binlijin >Priority: Minor > Fix For: 2.0.0, 1.4.0, 0.98.23 > > Attachments: HBASE-16694-master.patch > > > Current when finish a HFileBlock will create a new byte[] for onDiskChecksum, > we can reuse it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17855) Fix typo in async client implementation
[ https://issues.apache.org/jira/browse/HBASE-17855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17855: -- Status: Patch Available (was: Open) > Fix typo in async client implementation > --- > > Key: HBASE-17855 > URL: https://issues.apache.org/jira/browse/HBASE-17855 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Labels: trivial > Fix For: 2.0.0 > > Attachments: HBASE-17855.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17855) Fix typo in async client implementation
[ https://issues.apache.org/jira/browse/HBASE-17855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17855: -- Attachment: HBASE-17855.patch > Fix typo in async client implementation > --- > > Key: HBASE-17855 > URL: https://issues.apache.org/jira/browse/HBASE-17855 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Labels: trivial > Fix For: 2.0.0 > > Attachments: HBASE-17855.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17855) Fix typo in async client implementation
[ https://issues.apache.org/jira/browse/HBASE-17855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17855: -- Issue Type: Sub-task (was: Bug) Parent: HBASE-16833 > Fix typo in async client implementation > --- > > Key: HBASE-17855 > URL: https://issues.apache.org/jira/browse/HBASE-17855 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Labels: trivial > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17855) Fix typo in async client implementation
Duo Zhang created HBASE-17855: - Summary: Fix typo in async client implementation Key: HBASE-17855 URL: https://issues.apache.org/jira/browse/HBASE-17855 Project: HBase Issue Type: Bug Components: asyncclient, Client Affects Versions: 2.0.0 Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17633) Update unflushed sequence id in SequenceIdAccounting after flush with the minimum sequence id in memstore
[ https://issues.apache.org/jira/browse/HBASE-17633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950159#comment-15950159 ] Duo Zhang commented on HBASE-17633: --- And this is only an improvement for simplify our code, so I do not consider it as a blocker for 2.0. Feel free to change the fix version if we want to cut 2.0 sir [~stack]. > Update unflushed sequence id in SequenceIdAccounting after flush with the > minimum sequence id in memstore > - > > Key: HBASE-17633 > URL: https://issues.apache.org/jira/browse/HBASE-17633 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17633.patch, HBASE-17633-v1.patch > > > Now the tracking work is done by SequenceIdAccounting. And it is a little > tricky when dealing with flush. We should remove the mapping for the given > stores of a region from lowestUnflushedSequenceIds, so that we have space to > store the new lowest unflushed sequence id after flush. But we still need to > keep the old sequence ids in another map as we still need to use these values > when reporting to master to prevent data loss(think of the scenario that we > report the new lowest unflushed sequence id to master and we crashed before > actually flushed the data to disk). > And when reviewing HBASE-17407, I found that for CompactingMemStore, we have > to record the minimum sequence id.in memstore. We could just update the > mappings in SequenceIdAccounting using these values after flush. This means > we do not need to update the lowest unflushed sequence id in > SequenceIdAccounting, and also do not need to make space for the new lowest > unflushed when startCacheFlush, and also do not need the extra map to store > the old mappings. > This could simplify our logic a lot. But this is a fundamental change so I > need sometime to implement, especially for modifying tests... And I also need > sometime to check if I miss something. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17633) Update unflushed sequence id in SequenceIdAccounting after flush with the minimum sequence id in memstore
[ https://issues.apache.org/jira/browse/HBASE-17633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950155#comment-15950155 ] Duo Zhang commented on HBASE-17633: --- Finally I found that if we keep min sequence id in memstore then we do not need SequenceIdAccounting anymore(Of course the highestSequenceIds is still needed but it does not worth to have a separated class for it)... WAL could go to memstore to find out if it can safely purge an old wal file. Will give a try soon. > Update unflushed sequence id in SequenceIdAccounting after flush with the > minimum sequence id in memstore > - > > Key: HBASE-17633 > URL: https://issues.apache.org/jira/browse/HBASE-17633 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17633.patch, HBASE-17633-v1.patch > > > Now the tracking work is done by SequenceIdAccounting. And it is a little > tricky when dealing with flush. We should remove the mapping for the given > stores of a region from lowestUnflushedSequenceIds, so that we have space to > store the new lowest unflushed sequence id after flush. But we still need to > keep the old sequence ids in another map as we still need to use these values > when reporting to master to prevent data loss(think of the scenario that we > report the new lowest unflushed sequence id to master and we crashed before > actually flushed the data to disk). > And when reviewing HBASE-17407, I found that for CompactingMemStore, we have > to record the minimum sequence id.in memstore. We could just update the > mappings in SequenceIdAccounting using these values after flush. This means > we do not need to update the lowest unflushed sequence id in > SequenceIdAccounting, and also do not need to make space for the new lowest > unflushed when startCacheFlush, and also do not need the extra map to store > the old mappings. > This could simplify our logic a lot. But this is a fundamental change so I > need sometime to implement, especially for modifying tests... And I also need > sometime to check if I miss something. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17791) Locality should not be affected for non-faulty region servers at startup
[
https://issues.apache.org/jira/browse/HBASE-17791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950094#comment-15950094
]
Abhishek Singh Chouhan commented on HBASE-17791:
In HBASE-15251 failover is set to false even when we get an non empty list of
deadServers
{code}
if (failover) {
// This may not be a failover actually, especially if meta is on this
master.
if (LOG.isDebugEnabled()) {
LOG.debug("Found dead servers out on cluster " +
serverManager.getDeadServers());
}
// Check if there are any regions on these servers
failover = false;
{code}
Now this is set to true only when one of the many checks like region except
meta in transition etc. are true which look like valid scenarios , otherwise it
stays false and we go ahead with assigning regions as per meta. Won't this
already solve the problem ?
> Locality should not be affected for non-faulty region servers at startup
>
>
> Key: HBASE-17791
> URL: https://issues.apache.org/jira/browse/HBASE-17791
> Project: HBase
> Issue Type: Improvement
> Components: Balancer, Region Assignment
>Affects Versions: 1.3.0, 1.0.3, 1.2.4, 1.1.8
>Reporter: Lars George
>Priority: Blocker
>
> We seem to have an issue with store file locality as soon as a single server
> is missing or faulty upon restart. HBASE-15251 is addressing a subset of the
> problem, ensuring that some remaining files or an empty server do not trigger
> a full recovery, but is missing further, similar cases. Especially, the case
> where a server fails to report in before the active master is finished
> waiting for them to do so, or where servers have been decomissioned (but not
> removed from the {{regionservers}} file), and finally the true case of a dead
> node.
> In case a single node is faulty, the user regions are _not_ assigned as saved
> in the {{hbase:meta}} table, but completely randomized in a round-robin
> fashion. An additional factor is that in this case the regions are _not_
> assigned to the best matching node (the one with a copy of the data locally),
> but to any node, leaving the locality in shambles.
> What is also bad, if the {{hbase.hstore.min.locality.to.skip.major.compact}}
> property is left at the default {{0.0f}}, then an older region that had no
> writes since the last major compaction happened is just skipped (as expected,
> usually) and locality stays bad as-is. All reads for those aged-out regions
> will be network reads. But in any event, having to run a major compaction
> after a restart is not good anyways.
> The issue is the code in
> {{AssignmentManager.processDeadServersAndRegionsInTransition()}}, which is
> handed a list of dead servers. But it immediately sets the {{failover}} flag
> and the code
> {code}
> failoverCleanupDone();
> if (!failover) {
> // Fresh cluster startup.
> LOG.info("Clean cluster startup. Don't reassign user regions");
> assignAllUserRegions(allRegions);
> } else {
> LOG.info("Failover! Reassign user regions");
> }
> {code}
> is not triggering the assignment of the regions to those servers that are
> still present and have all their region data local. What should happen is
> that only the missing regions are reassigned, just like in the case of a
> server failing while the cluster is running.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950087#comment-15950087 ] Hudson commented on HBASE-14614: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2770 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2770/]) HBASE-17844 Subset of HBASE-14614, Procedure v2: Core Assignment Manager (stack: rev d033cbb715aa6153c4b764ef6638b7a1cecee64e) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateNamespaceProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestAsyncLogRolling.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/BalancedQueueRpcExecutor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterWalManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/TestStochasticBalancerJmxMetrics.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/locking/TestLockProcedure.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureEvent.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureScheduler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/locking/LockProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/LockAndQueue.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureScheduler.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/ServerTooBusyException.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestChangingEncoding.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcScheduler.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/StateMachineProcedure.java * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/ChoreService.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/master/RegionState.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/AbstractProcedureScheduler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/throttle/PressureAwareCompactionThroughputController.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/util/DelayedUtil.java * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/util/TestDelayedUtil.java > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >
[jira] [Commented] (HBASE-17844) Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical related changes)
[ https://issues.apache.org/jira/browse/HBASE-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950086#comment-15950086 ] Hudson commented on HBASE-17844: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2770 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2770/]) HBASE-17844 Subset of HBASE-14614, Procedure v2: Core Assignment Manager (stack: rev d033cbb715aa6153c4b764ef6638b7a1cecee64e) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/locking/LockProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/BalancedQueueRpcExecutor.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/throttle/PressureAwareCompactionThroughputController.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcScheduler.java * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/util/TestDelayedUtil.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestChangingEncoding.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/util/DelayedUtil.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateNamespaceProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureScheduler.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/ServerTooBusyException.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/ChoreService.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/locking/TestLockProcedure.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/AbstractProcedureScheduler.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureScheduler.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/master/RegionState.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureEvent.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/TestStochasticBalancerJmxMetrics.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterWalManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestAsyncLogRolling.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/LockAndQueue.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/StateMachineProcedure.java > Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical > related changes) > --- > > Key: HBASE-17844 > URL: https://issues.apache.org/jira/browse/HBASE-17844 > Project:
[jira] [Commented] (HBASE-17837) Backport HBASE-15314 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-17837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950029#comment-15950029 ] Zach York commented on HBASE-17837: --- Thanks [~ram_krish]! > Backport HBASE-15314 to branch-1.3 > -- > > Key: HBASE-17837 > URL: https://issues.apache.org/jira/browse/HBASE-17837 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Affects Versions: 1.3.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 1.4.0, 1.3.1 > > Attachments: HBASE-15314-branch-1.3.patch > > > Backport of HBASE-15314. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (HBASE-17844) Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical related changes)
[ https://issues.apache.org/jira/browse/HBASE-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949950#comment-15949950 ] Stephen Yuan Jiang edited comment on HBASE-17844 at 3/30/17 10:05 PM: -- [~stack], I like it! If you have time to break to smaller piece, that is better! was (Author: syuanjiang): [~saint@gmail.com], I like it! If you have time to break to smaller piece, that is better! > Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical > related changes) > --- > > Key: HBASE-17844 > URL: https://issues.apache.org/jira/browse/HBASE-17844 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-17844.master.001.patch, > HBASE-17844.master.002.patch, HBASE-17844.master.003.patch, > HBASE-17844.master.004.patch > > > Here is a patch that breaks out non-pertinent changes that made it into > HBASE-14614 in an attempt at shrinking its overall size. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17844) Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical related changes)
[ https://issues.apache.org/jira/browse/HBASE-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949950#comment-15949950 ] Stephen Yuan Jiang commented on HBASE-17844: [~saint@gmail.com], I like it! If you have time to break to smaller piece, that is better! > Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical > related changes) > --- > > Key: HBASE-17844 > URL: https://issues.apache.org/jira/browse/HBASE-17844 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-17844.master.001.patch, > HBASE-17844.master.002.patch, HBASE-17844.master.003.patch, > HBASE-17844.master.004.patch > > > Here is a patch that breaks out non-pertinent changes that made it into > HBASE-14614 in an attempt at shrinking its overall size. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17816) HRegion#mutateRowWithLocks should update writeRequestCount metric
[ https://issues.apache.org/jira/browse/HBASE-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949943#comment-15949943 ] Ashu Pachauri commented on HBASE-17816: --- Looks good to me; The failures also seem unrelated. +1 > HRegion#mutateRowWithLocks should update writeRequestCount metric > - > > Key: HBASE-17816 > URL: https://issues.apache.org/jira/browse/HBASE-17816 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Ashu Pachauri >Assignee: Weizhan Zeng > Attachments: HBASE-17816.master.001.patch, > HBASE-17816.master.002.patch > > > Currently, all the calls that use HRegion#mutateRowWithLocks miss > writeRequestCount metric. The mutateRowWithLocks base method should update > the metric. > Examples are checkAndMutate calls through RSRpcServices#multi, > Region#mutateRow api , MultiRowMutationProcessor coprocessor endpoint. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17732) Coprocessor Design Improvements
[ https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17732: - Description: Adding template for coprocessor type to CoprocessorEnvironment will: - get rid of 'instanceOf' checks we do in various coprocessor hosts - allow us to load only relevant coprocessors in hosts (right now i think each host loads all coprocs and it's only during execOperation that it checks if the coproc is of correct type ie XXCoprocessorHost will load XXObserver, YYObserver, and all others, and in will check in execOperation if coproc is of type XXObserver and ignore the rest). - allow sharing of a bunch functions/classes which are currently 99% duplicate of each other. For eg. CoprocessorOperations, CoprocessorOperationWithResult, execOperations(). Here's a more detailed design doc: https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-rOpQmZuixCDua-LT9X4/edit was: Adding template for coprocessor type to CoprocessorEnvironment will: - get rid of 'instanceOf' checks we do in various coprocessor hosts - allow us to load only relevant coprocessors in hosts (right now i think each host loads all coprocs and it's only during execOperation that it checks if the coproc is of correct type ie XXCoprocessorHost will load XXObserver, YYObserver, and all others, and in will check in execOperation if coproc is of type XXObserver and ignore the rest). - allow sharing of a bunch functions/classes which are currently 99% duplicate of each other. For eg. CoprocessorOperations, CoprocessorOperationWithResult, execOperations(). > Coprocessor Design Improvements > --- > > Key: HBASE-17732 > URL: https://issues.apache.org/jira/browse/HBASE-17732 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Attachments: HBASE-17732.master.001.patch, > HBASE-17732.master.002.patch > > > Adding template for coprocessor type to CoprocessorEnvironment will: > - get rid of 'instanceOf' checks we do in various coprocessor hosts > - allow us to load only relevant coprocessors in hosts (right now i think > each host loads all coprocs and it's only during execOperation that it checks > if the coproc is of correct type ie XXCoprocessorHost will load XXObserver, > YYObserver, and all others, and in will check in execOperation if coproc is > of type XXObserver and ignore the rest). > - allow sharing of a bunch functions/classes which are currently 99% > duplicate of each other. For eg. CoprocessorOperations, > CoprocessorOperationWithResult, execOperations(). > Here's a more detailed design doc: > https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-rOpQmZuixCDua-LT9X4/edit -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17732) Coprocessor Design Improvements
[ https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17732: - Summary: Coprocessor Design Improvements (was: Add CpType template to CoprocessorEnvironment) > Coprocessor Design Improvements > --- > > Key: HBASE-17732 > URL: https://issues.apache.org/jira/browse/HBASE-17732 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Attachments: HBASE-17732.master.001.patch, > HBASE-17732.master.002.patch > > > Adding template for coprocessor type to CoprocessorEnvironment will: > - get rid of 'instanceOf' checks we do in various coprocessor hosts > - allow us to load only relevant coprocessors in hosts (right now i think > each host loads all coprocs and it's only during execOperation that it checks > if the coproc is of correct type ie XXCoprocessorHost will load XXObserver, > YYObserver, and all others, and in will check in execOperation if coproc is > of type XXObserver and ignore the rest). > - allow sharing of a bunch functions/classes which are currently 99% > duplicate of each other. For eg. CoprocessorOperations, > CoprocessorOperationWithResult, execOperations(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15227) HBase Backup Phase 3: Fault tolerance (client/server) support
[ https://issues.apache.org/jira/browse/HBASE-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949895#comment-15949895 ] Ted Yu commented on HBASE-15227: Vlad: Can you summarize the high level plan for making backup fault tolerant ? > HBase Backup Phase 3: Fault tolerance (client/server) support > - > > Key: HBASE-15227 > URL: https://issues.apache.org/jira/browse/HBASE-15227 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15227-v3.patch, HBASE-15277-v1.patch > > > System must be tolerant to faults: > # Backup operations MUST be atomic (no partial completion state in system > table) > # Process must detect any type of failures which can result in a data loss > (partial backup) in a backup destination > # Proper system table state restore and cleanup must be done in case of a > failure > # Additional utility to repair backup system table and corresponding file > system cleanup must be implemented -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16961) FileSystem Quotas
[ https://issues.apache.org/jira/browse/HBASE-16961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949750#comment-15949750 ] Josh Elser commented on HBASE-16961: You happen to get anywhere with the above, [~apurtell]? :) > FileSystem Quotas > - > > Key: HBASE-16961 > URL: https://issues.apache.org/jira/browse/HBASE-16961 > Project: HBase > Issue Type: New Feature >Reporter: Josh Elser >Assignee: Josh Elser > Attachments: hbase-quota-test.sh > > > Umbrella issue for tracking the filesystem utilization of HBase data, > defining quotas on that utilization, and enforcement when utilization exceeds > the limits of the quota. > At a high level: we can define quotas on tables and namespaces. Region size > is computed by RegionServers and sent to the Master. The Master inspects the > sizes of Regions, rolling up to table and namespace sizes. Defined quotas in > the quota table are evaluated given the computed sizes, and, for those > tables/namespaces violating the quota, RegionServers are informed to take > some action to limit any further filesystem growth by that table/namespace. > Discuss: > https://lists.apache.org/thread.html/66a4b0c3725b5cbdd61dd6111c43847adaeef7b7da5f4cd045df30ef@%3Cdev.hbase.apache.org%3E > Design Doc: > http://home.apache.org/~elserj/hbase/FileSystemQuotasforApacheHBase.pdf or > https://docs.google.com/document/d/1VtLWDkB2tpwc_zgCNPE1ulZOeecF-YA2FYSK3TSs_bw/edit?usp=sharing -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17844) Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical related changes)
[ https://issues.apache.org/jira/browse/HBASE-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949692#comment-15949692 ] stack commented on HBASE-17844: --- [~syuanjiang] Hope you good w/ this. I'm trying to break up the big AMv2 patch into smaller pieces so it goes down easier. This patch is break out of doc and logging changes... > Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical > related changes) > --- > > Key: HBASE-17844 > URL: https://issues.apache.org/jira/browse/HBASE-17844 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-17844.master.001.patch, > HBASE-17844.master.002.patch, HBASE-17844.master.003.patch, > HBASE-17844.master.004.patch > > > Here is a patch that breaks out non-pertinent changes that made it into > HBASE-14614 in an attempt at shrinking its overall size. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17816) HRegion#mutateRowWithLocks should update writeRequestCount metric
[ https://issues.apache.org/jira/browse/HBASE-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949599#comment-15949599 ] Sean Busbey commented on HBASE-17816: - I'll leave this open for another 12-24 hours and then go ahead and push unless folks come in with more feedback. > HRegion#mutateRowWithLocks should update writeRequestCount metric > - > > Key: HBASE-17816 > URL: https://issues.apache.org/jira/browse/HBASE-17816 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Ashu Pachauri >Assignee: Weizhan Zeng > Attachments: HBASE-17816.master.001.patch, > HBASE-17816.master.002.patch > > > Currently, all the calls that use HRegion#mutateRowWithLocks miss > writeRequestCount metric. The mutateRowWithLocks base method should update > the metric. > Examples are checkAndMutate calls through RSRpcServices#multi, > Region#mutateRow api , MultiRowMutationProcessor coprocessor endpoint. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949578#comment-15949578 ] stack commented on HBASE-14614: --- HBASE-17844 landed in master so let me rebase and cut from this patch what has already gone in. > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: HBASE-14614.master.001.patch, > HBASE-14614.master.002.patch, HBASE-14614.master.003.patch, > HBASE-14614.master.004.patch, HBASE-14614.master.005.patch, > HBASE-14614.master.006.patch, HBASE-14614.master.007.patch, > HBASE-14614.master.008.patch, HBASE-14614.master.009.patch, > HBASE-14614.master.010.patch, HBASE-14614.master.011.patch, > HBASE-14614.master.012.patch, HBASE-14614.master.012.patch, > HBASE-14614.master.013.patch, HBASE-14614.master.014.patch, > HBASE-14614.master.015.patch, HBASE-14614.master.016.patch, > HBASE-14614.master.017.patch, HBASE-14614.master.018.patch, > HBASE-14614.master.019.patch, HBASE-14614.master.020.patch, > HBASE-14614.master.021.patch, HBASE-14614.master.022.patch, > HBASE-14614.master.023.patch, HBASE-14614.master.024.patch > > > New AssignmentManager implemented using proc-v2. > - AssignProcedure handle assignment operation > - UnassignProcedure handle unassign operation > - MoveRegionProcedure handle move/balance operation > Concurrent Assign operations are batched together and sent to the balancer > Concurrent Assign and Unassign operation ready to be sent to the RS are > batched together > This patch is an intermediate state where we add the new AM as > AssignmentManager2() to the master, to be reached by tests. but the new AM > will not be integrated with the rest of the system. Only new am unit-tests > will exercise the new assigment manager. The integration with the master code > is part of HBASE-14616 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17816) HRegion#mutateRowWithLocks should update writeRequestCount metric
[ https://issues.apache.org/jira/browse/HBASE-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949565#comment-15949565 ] Chia-Ping Tsai commented on HBASE-17816: +1 > HRegion#mutateRowWithLocks should update writeRequestCount metric > - > > Key: HBASE-17816 > URL: https://issues.apache.org/jira/browse/HBASE-17816 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Ashu Pachauri >Assignee: Weizhan Zeng > Attachments: HBASE-17816.master.001.patch, > HBASE-17816.master.002.patch > > > Currently, all the calls that use HRegion#mutateRowWithLocks miss > writeRequestCount metric. The mutateRowWithLocks base method should update > the metric. > Examples are checkAndMutate calls through RSRpcServices#multi, > Region#mutateRow api , MultiRowMutationProcessor coprocessor endpoint. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17816) HRegion#mutateRowWithLocks should update writeRequestCount metric
[ https://issues.apache.org/jira/browse/HBASE-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949538#comment-15949538 ] Jerry He commented on HBASE-17816: -- +1 > HRegion#mutateRowWithLocks should update writeRequestCount metric > - > > Key: HBASE-17816 > URL: https://issues.apache.org/jira/browse/HBASE-17816 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Ashu Pachauri >Assignee: Weizhan Zeng > Attachments: HBASE-17816.master.001.patch, > HBASE-17816.master.002.patch > > > Currently, all the calls that use HRegion#mutateRowWithLocks miss > writeRequestCount metric. The mutateRowWithLocks base method should update > the metric. > Examples are checkAndMutate calls through RSRpcServices#multi, > Region#mutateRow api , MultiRowMutationProcessor coprocessor endpoint. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17844) Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical related changes)
[ https://issues.apache.org/jira/browse/HBASE-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17844: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the reviews [~uagashe] and [~appy]. Pushed to master. > Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical > related changes) > --- > > Key: HBASE-17844 > URL: https://issues.apache.org/jira/browse/HBASE-17844 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-17844.master.001.patch, > HBASE-17844.master.002.patch, HBASE-17844.master.003.patch, > HBASE-17844.master.004.patch > > > Here is a patch that breaks out non-pertinent changes that made it into > HBASE-14614 in an attempt at shrinking its overall size. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14141) HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backup tables
[ https://issues.apache.org/jira/browse/HBASE-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949456#comment-15949456 ] Ted Yu commented on HBASE-14141: Can you fill out description with high level design ? > HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits > from backup tables > > > Key: HBASE-14141 > URL: https://issues.apache.org/jira/browse/HBASE-14141 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-14141.HBASE-14123.v1.patch, HBASE-14141.v1.patch, > HBASE-14141.v2.patch, HBASE-14141.v4.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17343) Make Compacting Memstore default in 2.0 with BASIC as the default type
[ https://issues.apache.org/jira/browse/HBASE-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949452#comment-15949452 ] Anoop Sam John commented on HBASE-17343: I just started to run some performance test comparing DefaultMemstore vs Compacting memstore (With BASIC). This is write only tests. Will say more soon > Make Compacting Memstore default in 2.0 with BASIC as the default type > -- > > Key: HBASE-17343 > URL: https://issues.apache.org/jira/browse/HBASE-17343 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-17343-V01.patch, HBASE-17343-V02.patch > > > FYI [~anastas], [~eshcar] and [~ebortnik]. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17844) Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical related changes)
[ https://issues.apache.org/jira/browse/HBASE-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949441#comment-15949441 ] Appy commented on HBASE-17844: -- +1 (Review details are in RB) > Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical > related changes) > --- > > Key: HBASE-17844 > URL: https://issues.apache.org/jira/browse/HBASE-17844 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-17844.master.001.patch, > HBASE-17844.master.002.patch, HBASE-17844.master.003.patch, > HBASE-17844.master.004.patch > > > Here is a patch that breaks out non-pertinent changes that made it into > HBASE-14614 in an attempt at shrinking its overall size. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17844) Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical related changes)
[ https://issues.apache.org/jira/browse/HBASE-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949431#comment-15949431 ] Umesh Agashe commented on HBASE-17844: -- +1 lgtm > Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical > related changes) > --- > > Key: HBASE-17844 > URL: https://issues.apache.org/jira/browse/HBASE-17844 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-17844.master.001.patch, > HBASE-17844.master.002.patch, HBASE-17844.master.003.patch, > HBASE-17844.master.004.patch > > > Here is a patch that breaks out non-pertinent changes that made it into > HBASE-14614 in an attempt at shrinking its overall size. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17215) Separate small/large file delete threads in HFileCleaner to accelerate archived hfile cleanup speed
[
https://issues.apache.org/jira/browse/HBASE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949419#comment-15949419
]
huaxiang sun commented on HBASE-17215:
--
Thanks [~carp84], it is definitely fine to go with a follow-up to address the
shared queue.
{quote}
I'm very much interested in this "some reason" (smile). Ever dig into it and
mind share if so? Thanks. huaxiang sun
{quote}
We have some rough idea about it. First, hdfs operation is slow, it takes 120
ms ~ 140 ms to delete a file from hdfs. I think this Jira will help to address
this issue (speed up + delete large files to free up the diskspace). Another
issue is related with SnapshotHFileCleaner, refresh snapshot file cache is
called very frequently, still looking into code to see if something can be done
there.
> Separate small/large file delete threads in HFileCleaner to accelerate
> archived hfile cleanup speed
> ---
>
> Key: HBASE-17215
> URL: https://issues.apache.org/jira/browse/HBASE-17215
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-17215.patch, HBASE-17215.v2.patch
>
>
> When using PCIe-SSD the flush speed will be really quick, and although we
> have per CF flush, we still have the
> {{hbase.regionserver.optionalcacheflushinterval}} setting and some other
> mechanism to avoid data kept in memory for too long to flush small hfiles. In
> our online environment we found the single thread cleaner kept cleaning
> earlier flushed small files while large files got no chance, which caused
> disk full then many other problems.
> Deleting hfiles in parallel with too many threads will also increase the
> workload of namenode, so here we propose to separate large/small hfile
> cleaner threads just like we do for compaction, and it turned out to work
> well in our cluster.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17343) Make Compacting Memstore default in 2.0 with BASIC as the default type
[ https://issues.apache.org/jira/browse/HBASE-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949414#comment-15949414 ] stack commented on HBASE-17343: --- There was the request for some long-time ITBLL runs before commit. Would also like to do some before/after compares by another party. That said, long-runs of ITBLL against master branch probably won't succeed given its messy state what with all the churn. Also, master branch is not representative of what 2.0 will be, not until HBASE-14614 lands at least. So, I'd suggest we commit this as is and then file a blocker task JIRA against hbase-2.0.0 to verify no general YCSB regression when BASIC is enabled. It can go unassigned for now but I'd probably pick it up later as part of the proofing of the hbase2 candidate. What do others think? > Make Compacting Memstore default in 2.0 with BASIC as the default type > -- > > Key: HBASE-17343 > URL: https://issues.apache.org/jira/browse/HBASE-17343 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-17343-V01.patch, HBASE-17343-V02.patch > > > FYI [~anastas], [~eshcar] and [~ebortnik]. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17816) HRegion#mutateRowWithLocks should update writeRequestCount metric
[ https://issues.apache.org/jira/browse/HBASE-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949378#comment-15949378 ] Sean Busbey commented on HBASE-17816: - +1. the new test looks fine. I don't see how the two tests that had failures would be impacted by this (and it looks like the failure is some issue with the build host cleaning up /tmp aggressively). > HRegion#mutateRowWithLocks should update writeRequestCount metric > - > > Key: HBASE-17816 > URL: https://issues.apache.org/jira/browse/HBASE-17816 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Ashu Pachauri >Assignee: Weizhan Zeng > Attachments: HBASE-17816.master.001.patch, > HBASE-17816.master.002.patch > > > Currently, all the calls that use HRegion#mutateRowWithLocks miss > writeRequestCount metric. The mutateRowWithLocks base method should update > the metric. > Examples are checkAndMutate calls through RSRpcServices#multi, > Region#mutateRow api , MultiRowMutationProcessor coprocessor endpoint. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17343) Make Compacting Memstore default in 2.0 with BASIC as the default type
[ https://issues.apache.org/jira/browse/HBASE-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949277#comment-15949277 ] Edward Bortnikov commented on HBASE-17343: -- Are we good to commit? :) > Make Compacting Memstore default in 2.0 with BASIC as the default type > -- > > Key: HBASE-17343 > URL: https://issues.apache.org/jira/browse/HBASE-17343 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-17343-V01.patch, HBASE-17343-V02.patch > > > FYI [~anastas], [~eshcar] and [~ebortnik]. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17343) Make Compacting Memstore default in 2.0 with BASIC as the default type
[
https://issues.apache.org/jira/browse/HBASE-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949213#comment-15949213
]
Hadoop QA commented on HBASE-17343:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
31s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 17s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 115m 59s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 165m 27s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12861211/HBASE-17343-V02.patch
|
| JIRA Issue | HBASE-17343 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux e344719a6fe5 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 752b258 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6272/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6272/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Make Compacting Memstore default in 2.0 with BASIC as the default type
> --
>
> Key: HBASE-17343
> URL: https://issues.apache.org/jira/browse/HBASE-17343
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>
[jira] [Issue Comment Deleted] (HBASE-17816) HRegion#mutateRowWithLocks should update writeRequestCount metric
[ https://issues.apache.org/jira/browse/HBASE-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weizhan Zeng updated HBASE-17816: - Comment: was deleted (was: [~ashu210890] please help me do review on that patch ?) > HRegion#mutateRowWithLocks should update writeRequestCount metric > - > > Key: HBASE-17816 > URL: https://issues.apache.org/jira/browse/HBASE-17816 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Ashu Pachauri >Assignee: Weizhan Zeng > Attachments: HBASE-17816.master.001.patch, > HBASE-17816.master.002.patch > > > Currently, all the calls that use HRegion#mutateRowWithLocks miss > writeRequestCount metric. The mutateRowWithLocks base method should update > the metric. > Examples are checkAndMutate calls through RSRpcServices#multi, > Region#mutateRow api , MultiRowMutationProcessor coprocessor endpoint. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17837) Backport HBASE-15314 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-17837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949161#comment-15949161 ] Hudson commented on HBASE-17837: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #135 (See [https://builds.apache.org/job/HBase-1.3-JDK7/135/]) HBASE-17837 Backport HBASE-15314 to branch-1.3 (Chunhui shen & (ramkrishna: rev 36da029edd596bf543a8f6eee3005c55eb313644) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestFileIOEngine.java > Backport HBASE-15314 to branch-1.3 > -- > > Key: HBASE-17837 > URL: https://issues.apache.org/jira/browse/HBASE-17837 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Affects Versions: 1.3.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 1.4.0, 1.3.1 > > Attachments: HBASE-15314-branch-1.3.patch > > > Backport of HBASE-15314. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15314) Allow more than one backing file in bucketcache
[
https://issues.apache.org/jira/browse/HBASE-15314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949162#comment-15949162
]
Hudson commented on HBASE-15314:
SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #135 (See
[https://builds.apache.org/job/HBase-1.3-JDK7/135/])
HBASE-17837 Backport HBASE-15314 to branch-1.3 (Chunhui shen & (ramkrishna:
rev 36da029edd596bf543a8f6eee3005c55eb313644)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestFileIOEngine.java
> Allow more than one backing file in bucketcache
> ---
>
> Key: HBASE-15314
> URL: https://issues.apache.org/jira/browse/HBASE-15314
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache
>Reporter: stack
>Assignee: chunhui shen
> Fix For: 2.0
>
> Attachments: FileIOEngine.java, HBASE-15314.master.001.patch,
> HBASE-15314.master.001.patch, HBASE-15314.patch, HBASE-15314-v2.patch,
> HBASE-15314-v3.patch, HBASE-15314-v4.patch, HBASE-15314-v5.patch,
> HBASE-15314-v6.patch, HBASE-15314-v7.patch, HBASE-15314-v8.patch
>
>
> Allow bucketcache use more than just one backing file: e.g. chassis has more
> than one SSD in it.
> Usage (Setting the following configurations in hbase-site.xml):
> {quote}
>
> hbase.bucketcache.ioengine
>
> files:/mnt/disk1/bucketcache,/mnt/disk2/bucketcache,/mnt/disk3/bucketcache,/mnt/disk4/bucketcache
>
>
> hbase.bucketcache.size
> 1048576
>
> {quote}
> The above setting means the total capacity of cache is 1048576MB(1TB), each
> file length will be set to 0.25TB.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15314) Allow more than one backing file in bucketcache
[
https://issues.apache.org/jira/browse/HBASE-15314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949130#comment-15949130
]
Hudson commented on HBASE-15314:
SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #144 (See
[https://builds.apache.org/job/HBase-1.3-JDK8/144/])
HBASE-17837 Backport HBASE-15314 to branch-1.3 (Chunhui shen & (ramkrishna:
rev 36da029edd596bf543a8f6eee3005c55eb313644)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestFileIOEngine.java
> Allow more than one backing file in bucketcache
> ---
>
> Key: HBASE-15314
> URL: https://issues.apache.org/jira/browse/HBASE-15314
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache
>Reporter: stack
>Assignee: chunhui shen
> Fix For: 2.0
>
> Attachments: FileIOEngine.java, HBASE-15314.master.001.patch,
> HBASE-15314.master.001.patch, HBASE-15314.patch, HBASE-15314-v2.patch,
> HBASE-15314-v3.patch, HBASE-15314-v4.patch, HBASE-15314-v5.patch,
> HBASE-15314-v6.patch, HBASE-15314-v7.patch, HBASE-15314-v8.patch
>
>
> Allow bucketcache use more than just one backing file: e.g. chassis has more
> than one SSD in it.
> Usage (Setting the following configurations in hbase-site.xml):
> {quote}
>
> hbase.bucketcache.ioengine
>
> files:/mnt/disk1/bucketcache,/mnt/disk2/bucketcache,/mnt/disk3/bucketcache,/mnt/disk4/bucketcache
>
>
> hbase.bucketcache.size
> 1048576
>
> {quote}
> The above setting means the total capacity of cache is 1048576MB(1TB), each
> file length will be set to 0.25TB.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17837) Backport HBASE-15314 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-17837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949129#comment-15949129 ] Hudson commented on HBASE-17837: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #144 (See [https://builds.apache.org/job/HBase-1.3-JDK8/144/]) HBASE-17837 Backport HBASE-15314 to branch-1.3 (Chunhui shen & (ramkrishna: rev 36da029edd596bf543a8f6eee3005c55eb313644) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestFileIOEngine.java > Backport HBASE-15314 to branch-1.3 > -- > > Key: HBASE-17837 > URL: https://issues.apache.org/jira/browse/HBASE-17837 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Affects Versions: 1.3.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 1.4.0, 1.3.1 > > Attachments: HBASE-15314-branch-1.3.patch > > > Backport of HBASE-15314. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17816) HRegion#mutateRowWithLocks should update writeRequestCount metric
[ https://issues.apache.org/jira/browse/HBASE-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949127#comment-15949127 ] Weizhan Zeng commented on HBASE-17816: -- [~ashu210890] Please have a look is that test OK ? > HRegion#mutateRowWithLocks should update writeRequestCount metric > - > > Key: HBASE-17816 > URL: https://issues.apache.org/jira/browse/HBASE-17816 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Ashu Pachauri >Assignee: Weizhan Zeng > Attachments: HBASE-17816.master.001.patch, > HBASE-17816.master.002.patch > > > Currently, all the calls that use HRegion#mutateRowWithLocks miss > writeRequestCount metric. The mutateRowWithLocks base method should update > the metric. > Examples are checkAndMutate calls through RSRpcServices#multi, > Region#mutateRow api , MultiRowMutationProcessor coprocessor endpoint. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17816) HRegion#mutateRowWithLocks should update writeRequestCount metric
[ https://issues.apache.org/jira/browse/HBASE-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949126#comment-15949126 ] Weizhan Zeng commented on HBASE-17816: -- [~ashu210890] please help me do review on that patch ? > HRegion#mutateRowWithLocks should update writeRequestCount metric > - > > Key: HBASE-17816 > URL: https://issues.apache.org/jira/browse/HBASE-17816 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Ashu Pachauri >Assignee: Weizhan Zeng > Attachments: HBASE-17816.master.001.patch, > HBASE-17816.master.002.patch > > > Currently, all the calls that use HRegion#mutateRowWithLocks miss > writeRequestCount metric. The mutateRowWithLocks base method should update > the metric. > Examples are checkAndMutate calls through RSRpcServices#multi, > Region#mutateRow api , MultiRowMutationProcessor coprocessor endpoint. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16851) User-facing documentation for the In-Memory Compaction feature
[ https://issues.apache.org/jira/browse/HBASE-16851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949098#comment-15949098 ] Edward Bortnikov commented on HBASE-16851: -- Updated the developer documentation in https://docs.google.com/document/d/1z1R-MdAxRvTC2NazxUmN3FOCFIknkxL2TFqVUhYBVbE. The modified parts are highlighted in yellow. Feel free to comment. > User-facing documentation for the In-Memory Compaction feature > -- > > Key: HBASE-16851 > URL: https://issues.apache.org/jira/browse/HBASE-16851 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Edward Bortnikov >Assignee: Edward Bortnikov > Attachments: Accordion HBase In-Memory Compaction - Nov 1 .pdf, > Accordion HBase In-Memory Compaction - Nov 23.pdf, Accordion_ HBase In-Memory > Compaction - Oct 27.pdf, HBaseAcceleratedHbaseConf-final.pptx > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15314) Allow more than one backing file in bucketcache
[
https://issues.apache.org/jira/browse/HBASE-15314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949094#comment-15949094
]
Hudson commented on HBASE-15314:
SUCCESS: Integrated in Jenkins build HBase-1.4 #684 (See
[https://builds.apache.org/job/HBase-1.4/684/])
HBASE-17837 Backport HBASE-15314 to branch-1.3 (Chunhui shen & (ramkrishna:
rev 589a0e2efb7b604d5f13f186143445b57438db0b)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestFileIOEngine.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java
> Allow more than one backing file in bucketcache
> ---
>
> Key: HBASE-15314
> URL: https://issues.apache.org/jira/browse/HBASE-15314
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache
>Reporter: stack
>Assignee: chunhui shen
> Fix For: 2.0
>
> Attachments: FileIOEngine.java, HBASE-15314.master.001.patch,
> HBASE-15314.master.001.patch, HBASE-15314.patch, HBASE-15314-v2.patch,
> HBASE-15314-v3.patch, HBASE-15314-v4.patch, HBASE-15314-v5.patch,
> HBASE-15314-v6.patch, HBASE-15314-v7.patch, HBASE-15314-v8.patch
>
>
> Allow bucketcache use more than just one backing file: e.g. chassis has more
> than one SSD in it.
> Usage (Setting the following configurations in hbase-site.xml):
> {quote}
>
> hbase.bucketcache.ioengine
>
> files:/mnt/disk1/bucketcache,/mnt/disk2/bucketcache,/mnt/disk3/bucketcache,/mnt/disk4/bucketcache
>
>
> hbase.bucketcache.size
> 1048576
>
> {quote}
> The above setting means the total capacity of cache is 1048576MB(1TB), each
> file length will be set to 0.25TB.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17837) Backport HBASE-15314 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-17837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949093#comment-15949093 ] Hudson commented on HBASE-17837: SUCCESS: Integrated in Jenkins build HBase-1.4 #684 (See [https://builds.apache.org/job/HBase-1.4/684/]) HBASE-17837 Backport HBASE-15314 to branch-1.3 (Chunhui shen & (ramkrishna: rev 589a0e2efb7b604d5f13f186143445b57438db0b) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestFileIOEngine.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java > Backport HBASE-15314 to branch-1.3 > -- > > Key: HBASE-17837 > URL: https://issues.apache.org/jira/browse/HBASE-17837 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Affects Versions: 1.3.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 1.4.0, 1.3.1 > > Attachments: HBASE-15314-branch-1.3.patch > > > Backport of HBASE-15314. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[
https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949018#comment-15949018
]
Hadoop QA commented on HBASE-16438:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 28s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 13 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
3s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
42s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 29s
{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 48s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 106m 24s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 153m 3s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestRecoveredEdits |
| | hadoop.hbase.regionserver.TestWALLockup |
| | hadoop.hbase.regionserver.TestStore |
| | hadoop.hbase.regionserver.TestHRegionReplayEvents |
| | hadoop.hbase.regionserver.wal.TestDurability |
| | hadoop.hbase.io.hfile.TestScannerFromBucketCache |
| | hadoop.hbase.regionserver.TestCompactionArchiveIOException |
| | hadoop.hbase.regionserver.TestCompactionArchiveConcurrentClose |
| | hadoop.hbase.regionserver.TestRegionIncrement |
| | hadoop.hbase.regionserver.TestFailedAppendAndSync |
| | hadoop.hbase.regionserver.TestHMobStore |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12861207/HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch
|
| JIRA Issue | HBASE-16438 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle comp
[jira] [Resolved] (HBASE-17377) MemStoreChunkAllocator
[ https://issues.apache.org/jira/browse/HBASE-17377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Braginsky resolved HBASE-17377. - Resolution: Duplicate Duplicate of HBASE-16438 > MemStoreChunkAllocator > -- > > Key: HBASE-17377 > URL: https://issues.apache.org/jira/browse/HBASE-17377 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky > Attachments: MemStoreChunkAllocator.pdf > > > Refactoring the separation between MemStoreChunkPool and (new) > MemStoreChunkAllocator. The latter allocates chunks either from heap or from > pool and assigns them the Chunk IDs. MemStoreChunkAllocator stores the > mapping between chunks and their IDs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17377) MemStoreChunkAllocator
[ https://issues.apache.org/jira/browse/HBASE-17377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949006#comment-15949006 ] Anastasia Braginsky commented on HBASE-17377: - This JIRA can be closed as HBASE-16438 takes care for everything that was planned to be done here... > MemStoreChunkAllocator > -- > > Key: HBASE-17377 > URL: https://issues.apache.org/jira/browse/HBASE-17377 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky > Attachments: MemStoreChunkAllocator.pdf > > > Refactoring the separation between MemStoreChunkPool and (new) > MemStoreChunkAllocator. The latter allocates chunks either from heap or from > pool and assigns them the Chunk IDs. MemStoreChunkAllocator stores the > mapping between chunks and their IDs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17520) Implement isTableEnabled/Disabled/Available methods
[ https://issues.apache.org/jira/browse/HBASE-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948968#comment-15948968 ] Hudson commented on HBASE-17520: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2767 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2767/]) HBASE-17520 Implement isTableEnabled/Disabled/Available methods (zghao: rev 752b258b7c30aa375b5bb7a33abf435f37e8c877) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/AsyncMetaTableAccessor.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncHBaseAdmin.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableAdminApi.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java > Implement isTableEnabled/Disabled/Available methods > --- > > Key: HBASE-17520 > URL: https://issues.apache.org/jira/browse/HBASE-17520 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17520-v1.patch, HBASE-17520-v2.patch, > HBASE-17520-v3.patch, HBASE-17520-v4.patch, HBASE-17520-v5.patch, > HBASE-17520-v6.patch, HBASE-17520-v6.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17343) Make Compacting Memstore default in 2.0 with BASIC as the default type
[ https://issues.apache.org/jira/browse/HBASE-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Braginsky updated HBASE-17343: Attachment: HBASE-17343-V02.patch > Make Compacting Memstore default in 2.0 with BASIC as the default type > -- > > Key: HBASE-17343 > URL: https://issues.apache.org/jira/browse/HBASE-17343 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-17343-V01.patch, HBASE-17343-V02.patch > > > FYI [~anastas], [~eshcar] and [~ebortnik]. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17343) Make Compacting Memstore default in 2.0 with BASIC as the default type
[ https://issues.apache.org/jira/browse/HBASE-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Braginsky updated HBASE-17343: Status: Patch Available (was: Open) > Make Compacting Memstore default in 2.0 with BASIC as the default type > -- > > Key: HBASE-17343 > URL: https://issues.apache.org/jira/browse/HBASE-17343 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-17343-V01.patch, HBASE-17343-V02.patch > > > FYI [~anastas], [~eshcar] and [~ebortnik]. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17343) Make Compacting Memstore default in 2.0 with BASIC as the default type
[ https://issues.apache.org/jira/browse/HBASE-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948922#comment-15948922 ] Anastasia Braginsky commented on HBASE-17343: - Hi Again! Under HBASE-16417 the performance results were presented. Also there was a decision to make BASIC+merge be the default. Here I upload a 2 lines patch for this change. Any comments? Thanks, Anastasia > Make Compacting Memstore default in 2.0 with BASIC as the default type > -- > > Key: HBASE-17343 > URL: https://issues.apache.org/jira/browse/HBASE-17343 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-17343-V01.patch > > > FYI [~anastas], [~eshcar] and [~ebortnik]. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15314) Allow more than one backing file in bucketcache
[
https://issues.apache.org/jira/browse/HBASE-15314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948921#comment-15948921
]
Hudson commented on HBASE-15314:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #17 (See
[https://builds.apache.org/job/HBase-1.3-IT/17/])
HBASE-17837 Backport HBASE-15314 to branch-1.3 (Chunhui shen & (ramkrishna:
rev 36da029edd596bf543a8f6eee3005c55eb313644)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestFileIOEngine.java
> Allow more than one backing file in bucketcache
> ---
>
> Key: HBASE-15314
> URL: https://issues.apache.org/jira/browse/HBASE-15314
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache
>Reporter: stack
>Assignee: chunhui shen
> Fix For: 2.0
>
> Attachments: FileIOEngine.java, HBASE-15314.master.001.patch,
> HBASE-15314.master.001.patch, HBASE-15314.patch, HBASE-15314-v2.patch,
> HBASE-15314-v3.patch, HBASE-15314-v4.patch, HBASE-15314-v5.patch,
> HBASE-15314-v6.patch, HBASE-15314-v7.patch, HBASE-15314-v8.patch
>
>
> Allow bucketcache use more than just one backing file: e.g. chassis has more
> than one SSD in it.
> Usage (Setting the following configurations in hbase-site.xml):
> {quote}
>
> hbase.bucketcache.ioengine
>
> files:/mnt/disk1/bucketcache,/mnt/disk2/bucketcache,/mnt/disk3/bucketcache,/mnt/disk4/bucketcache
>
>
> hbase.bucketcache.size
> 1048576
>
> {quote}
> The above setting means the total capacity of cache is 1048576MB(1TB), each
> file length will be set to 0.25TB.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17837) Backport HBASE-15314 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-17837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948919#comment-15948919 ] Hudson commented on HBASE-17837: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #17 (See [https://builds.apache.org/job/HBase-1.3-IT/17/]) HBASE-17837 Backport HBASE-15314 to branch-1.3 (Chunhui shen & (ramkrishna: rev 36da029edd596bf543a8f6eee3005c55eb313644) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestFileIOEngine.java > Backport HBASE-15314 to branch-1.3 > -- > > Key: HBASE-17837 > URL: https://issues.apache.org/jira/browse/HBASE-17837 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Affects Versions: 1.3.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 1.4.0, 1.3.1 > > Attachments: HBASE-15314-branch-1.3.patch > > > Backport of HBASE-15314. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17837) Backport HBASE-15314 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-17837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-17837: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-1 and branch-1.3. It applied cleanly on branch-1 and also compiled it to ensure there are no compilation issues. > Backport HBASE-15314 to branch-1.3 > -- > > Key: HBASE-17837 > URL: https://issues.apache.org/jira/browse/HBASE-17837 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Affects Versions: 1.3.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 1.4.0, 1.3.1 > > Attachments: HBASE-15314-branch-1.3.patch > > > Backport of HBASE-15314. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17215) Separate small/large file delete threads in HFileCleaner to accelerate archived hfile cleanup speed
[
https://issues.apache.org/jira/browse/HBASE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948885#comment-15948885
]
Hadoop QA commented on HBASE-17215:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
1s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 4s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 104m 30s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 144m 3s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12861199/HBASE-17215.v2.patch |
| JIRA Issue | HBASE-17215 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux a75f4c3d4757 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24
10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 752b258 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6270/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6270/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6270/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Separate small/large file delete threads in HFileCleaner to accelerate
> archived hfile cleanup speed
> ---
>
> Key: HBASE-17215
> URL: https://issues.apache.org/jira/browse/HBASE-17215
> Project: HBase
> Issue Ty
[jira] [Commented] (HBASE-17837) Backport HBASE-15314 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-17837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948860#comment-15948860 ] ramkrishna.s.vasudevan commented on HBASE-17837: Will commit this patch to branch-1 and branch-1.3 shortly. > Backport HBASE-15314 to branch-1.3 > -- > > Key: HBASE-17837 > URL: https://issues.apache.org/jira/browse/HBASE-17837 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Affects Versions: 1.3.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 1.4.0, 1.3.1 > > Attachments: HBASE-15314-branch-1.3.patch > > > Backport of HBASE-15314. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14614) Procedure v2: Core Assignment Manager
[
https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948828#comment-15948828
]
Hadoop QA commented on HBASE-14614:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 83 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 34s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 34s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
34s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 54s
{color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 3s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
16s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 130 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 20s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 2m
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 9m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 6s
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} hbase-hadoop2-compat generated 0 new + 1 unchanged - 1
fixed = 1 total (was 2) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 59s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {co
[jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948827#comment-15948827 ] Anastasia Braginsky commented on HBASE-16438: - Thanks [~ram_krish] for publishing a new patch! Please pay attention that I have published some replies on your comments on the previous version of the patch. It just happened simultaneously. Please take a look there. I'll look on your new patch soon. Thanks! > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-16438_1.patch, > HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-16438: --- Attachment: HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-16438_1.patch, > HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-16438: --- Status: Patch Available (was: Open) > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-16438_1.patch, > HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
