[jira] [Commented] (HBASE-14614) Procedure v2: Core Assignment Manager
[
https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960316#comment-15960316
]
Hadoop QA commented on HBASE-14614:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 38s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 82 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 30s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
37s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 8s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 25m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
11s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 0s
{color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 31s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 23s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m 23s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 23s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 26m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
9s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 130 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 41s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m
42s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 2s
{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 44s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 18s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 19s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 21s
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 162m 40s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 42s
{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
55s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} |
[jira] [Updated] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14614: -- Attachment: HBASE-14614.master.027.patch Fix a few more tests. > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: HBASE-14614.master.001.patch, > HBASE-14614.master.002.patch, HBASE-14614.master.003.patch, > HBASE-14614.master.004.patch, HBASE-14614.master.005.patch, > HBASE-14614.master.006.patch, HBASE-14614.master.007.patch, > HBASE-14614.master.008.patch, HBASE-14614.master.009.patch, > HBASE-14614.master.010.patch, HBASE-14614.master.011.patch, > HBASE-14614.master.012.patch, HBASE-14614.master.012.patch, > HBASE-14614.master.013.patch, HBASE-14614.master.014.patch, > HBASE-14614.master.015.patch, HBASE-14614.master.016.patch, > HBASE-14614.master.017.patch, HBASE-14614.master.018.patch, > HBASE-14614.master.019.patch, HBASE-14614.master.020.patch, > HBASE-14614.master.021.patch, HBASE-14614.master.022.patch, > HBASE-14614.master.023.patch, HBASE-14614.master.024.patch, > HBASE-14614.master.025.patch, HBASE-14614.master.026.patch, > HBASE-14614.master.027.patch > > > New AssignmentManager implemented using proc-v2. > - AssignProcedure handle assignment operation > - UnassignProcedure handle unassign operation > - MoveRegionProcedure handle move/balance operation > Concurrent Assign operations are batched together and sent to the balancer > Concurrent Assign and Unassign operation ready to be sent to the RS are > batched together > This patch is an intermediate state where we add the new AM as > AssignmentManager2() to the master, to be reached by tests. but the new AM > will not be integrated with the rest of the system. Only new am unit-tests > will exercise the new assigment manager. The integration with the master code > is part of HBASE-14616 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-16438: --- Status: Patch Available (was: Open) > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: > HBASE-16438_10_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_11_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_12_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-16438: --- Attachment: HBASE-16438_12_ChunkCreatorwrappingChunkPool_withchunkRef.patch Updated patch with all the comments fixed. In last patch I was still writing the chunkId as long though the actual chunkId was only int. > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: > HBASE-16438_10_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_11_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_12_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960268#comment-15960268
]
ramkrishna.s.vasudevan commented on HBASE-17872:
+1 on v3. Sorry for missing out on the volatile change.
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch, HBASE-17872.v3.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-16438: --- Status: Open (was: Patch Available) > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: > HBASE-16438_10_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_11_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960262#comment-15960262
]
Hadoop QA commented on HBASE-17872:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 35s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 51s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 4s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 16s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
56m 54s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 43s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 184m 20s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 277m 43s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
|
| | hadoop.hbase.client.TestAsyncBalancerAdminApi |
| | hadoop.hbase.client.TestAsyncTableAdminApi |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862402/HBASE-17872.v3.patch |
| JIRA Issue | HBASE-17872 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 5d608f7472f8 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 48b2502 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6357/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
[jira] [Commented] (HBASE-17889) ResultBoundedCompletionService's cancel() needs to interrupt the working thread and free it to the thread-pool
[ https://issues.apache.org/jira/browse/HBASE-17889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960246#comment-15960246 ] stack commented on HBASE-17889: --- [~huaxiang] s/getTaskFuture/getFuture/ ? You get and set the ask future. Will get and set be done by same thread? If not, you might want to make taskFuture volatile so other threads can see the change (or just synchronize get/set?) Want to describe how you tested sir and the difference you saw? Thanks [~huaxiang] > ResultBoundedCompletionService's cancel() needs to interrupt the working > thread and free it to the thread-pool > -- > > Key: HBASE-17889 > URL: https://issues.apache.org/jira/browse/HBASE-17889 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0, 1.4.0, 1.2.6, 1.3.2 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-17889-master-001.patch, jstack.txt > > > We run into one case with read-replica, when the server hosting the primary > region is shutdown, we see Get did not go to replica region and it paused for > about 50 seconds before Get was resumed. > More debugging finds out that when the server is down, one of the threads was > stuck at the write, it holds lock at > https://github.com/apache/hbase/blob/branch-1.3/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java#L916. > The later write threads were waiting on this lock until all threads in the > connection's thread pool were stuck on this lock. At that moment, no work > will be done. After socket write times out, it frees up all threads and it > continues. > When QueueingFuture#cancel() is called, it does not interrupt the working > thread and return the thread to the pool. > Attaching the jstack trace. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17889) ResultBoundedCompletionService's cancel() needs to interrupt the working thread and free it to the thread-pool
[ https://issues.apache.org/jira/browse/HBASE-17889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960240#comment-15960240 ] stack commented on HBASE-17889: --- [~enis] FYI. Nice find by [~huaxiang] debugging and replicating a nasty hangup in read replicas. > ResultBoundedCompletionService's cancel() needs to interrupt the working > thread and free it to the thread-pool > -- > > Key: HBASE-17889 > URL: https://issues.apache.org/jira/browse/HBASE-17889 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0, 1.4.0, 1.2.6, 1.3.2 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-17889-master-001.patch, jstack.txt > > > We run into one case with read-replica, when the server hosting the primary > region is shutdown, we see Get did not go to replica region and it paused for > about 50 seconds before Get was resumed. > More debugging finds out that when the server is down, one of the threads was > stuck at the write, it holds lock at > https://github.com/apache/hbase/blob/branch-1.3/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java#L916. > The later write threads were waiting on this lock until all threads in the > connection's thread pool were stuck on this lock. At that moment, no work > will be done. After socket write times out, it frees up all threads and it > continues. > When QueueingFuture#cancel() is called, it does not interrupt the working > thread and return the thread to the pool. > Attaching the jstack trace. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17816) HRegion#mutateRowWithLocks should update writeRequestCount metric
[ https://issues.apache.org/jira/browse/HBASE-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960239#comment-15960239 ] Hudson commented on HBASE-17816: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2813 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2813/]) HBASE-17816 HRegion#mutateRowWithLocks should update writeRequestCount (jerryjch: rev 48b2502a5fcd4d3cd954c3abf6703422da7cdc2f) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > HRegion#mutateRowWithLocks should update writeRequestCount metric > - > > Key: HBASE-17816 > URL: https://issues.apache.org/jira/browse/HBASE-17816 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Ashu Pachauri >Assignee: Weizhan Zeng > Attachments: HBASE-17816.master.001.patch, > HBASE-17816.master.002.patch > > > Currently, all the calls that use HRegion#mutateRowWithLocks miss > writeRequestCount metric. The mutateRowWithLocks base method should update > the metric. > Examples are checkAndMutate calls through RSRpcServices#multi, > Region#mutateRow api , MultiRowMutationProcessor coprocessor endpoint. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17869) UnsafeAvailChecker wrongly returns false on ppc
[ https://issues.apache.org/jira/browse/HBASE-17869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960238#comment-15960238 ] Hudson commented on HBASE-17869: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2813 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2813/]) HBASE-17869 UnsafeAvailChecker wrongly returns false on ppc (jerryjch: rev af604f0c0cf3c40c56746150ffa860aad07f128a) * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/UnsafeAvailChecker.java > UnsafeAvailChecker wrongly returns false on ppc > --- > > Key: HBASE-17869 > URL: https://issues.apache.org/jira/browse/HBASE-17869 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.4 >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17869.patch > > > On ppc64 arch, java.nio.Bits.unaligned() wrongly returns false due to a JDK > bug. > https://bugs.openjdk.java.net/browse/JDK-8165231 > This causes some problem for HBase. i.e. FuzzyRowFilter test fails. > Fix it by providing a hard-code workaround for the JDK bug. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17873) Change the IA.Public annotation to IA.Private for unstable API
[ https://issues.apache.org/jira/browse/HBASE-17873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960212#comment-15960212 ] Duo Zhang commented on HBASE-17873: --- Any comments [~busbey]? Thanks. > Change the IA.Public annotation to IA.Private for unstable API > -- > > Key: HBASE-17873 > URL: https://issues.apache.org/jira/browse/HBASE-17873 > Project: HBase > Issue Type: Sub-task > Components: API >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-17873.patch > > > As discussed in mailing list and HBASE-17857. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17876) Release 1.2.6
[ https://issues.apache.org/jira/browse/HBASE-17876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960193#comment-15960193 ] Yu Li commented on HBASE-17876: --- Mind take a look at HBASE-17886 and let me know whether you'd like it to go into 1.2.6 boss [~busbey]? Thanks. > Release 1.2.6 > - > > Key: HBASE-17876 > URL: https://issues.apache.org/jira/browse/HBASE-17876 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 1.2.6 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17889) ResultBoundedCompletionService's cancel() needs to interrupt the working thread and free it to the thread-pool
[ https://issues.apache.org/jira/browse/HBASE-17889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960183#comment-15960183 ] Ted Yu commented on HBASE-17889: Nice finding. Is getTaskFuture called anywhere ? Can setTaskFuture be private or package private ? Consider including trivial change in hbase-server module to trigger QA run. > ResultBoundedCompletionService's cancel() needs to interrupt the working > thread and free it to the thread-pool > -- > > Key: HBASE-17889 > URL: https://issues.apache.org/jira/browse/HBASE-17889 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0, 1.4.0, 1.2.6, 1.3.2 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-17889-master-001.patch, jstack.txt > > > We run into one case with read-replica, when the server hosting the primary > region is shutdown, we see Get did not go to replica region and it paused for > about 50 seconds before Get was resumed. > More debugging finds out that when the server is down, one of the threads was > stuck at the write, it holds lock at > https://github.com/apache/hbase/blob/branch-1.3/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java#L916. > The later write threads were waiting on this lock until all threads in the > connection's thread pool were stuck on this lock. At that moment, no work > will be done. After socket write times out, it frees up all threads and it > continues. > When QueueingFuture#cancel() is called, it does not interrupt the working > thread and return the thread to the pool. > Attaching the jstack trace. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17889) ResultBoundedCompletionService's cancel() needs to interrupt the working thread and free it to the thread-pool
[
https://issues.apache.org/jira/browse/HBASE-17889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960181#comment-15960181
]
Hadoop QA commented on HBASE-17889:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
58m 47s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 45s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 84m 6s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862412/HBASE-17889-master-001.patch
|
| JIRA Issue | HBASE-17889 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux e3dda8b23138 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 48b2502 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6358/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6358/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> ResultBoundedCompletionService's cancel() needs to interrupt the working
> thread and free it to the thread-pool
> --
>
> Key: HBASE-17889
> URL: https://issues.apache.org/jira/browse/HBASE-17889
> Project:
[jira] [Commented] (HBASE-17869) UnsafeAvailChecker wrongly returns false on ppc
[ https://issues.apache.org/jira/browse/HBASE-17869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960171#comment-15960171 ] Hudson commented on HBASE-17869: FAILURE: Integrated in Jenkins build HBase-1.4 #690 (See [https://builds.apache.org/job/HBase-1.4/690/]) HBASE-17869 UnsafeAvailChecker wrongly returns false on ppc (jerryjch: rev b6a2c02b935ab22ec4c86accc3b1015fe715c675) * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/UnsafeAvailChecker.java > UnsafeAvailChecker wrongly returns false on ppc > --- > > Key: HBASE-17869 > URL: https://issues.apache.org/jira/browse/HBASE-17869 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.4 >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17869.patch > > > On ppc64 arch, java.nio.Bits.unaligned() wrongly returns false due to a JDK > bug. > https://bugs.openjdk.java.net/browse/JDK-8165231 > This causes some problem for HBase. i.e. FuzzyRowFilter test fails. > Fix it by providing a hard-code workaround for the JDK bug. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17836) CellUtil#estimatedSerializedSizeOf is slow when input is ByteBufferCell
[ https://issues.apache.org/jira/browse/HBASE-17836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-17836: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the review. [~anoop.hbase] > CellUtil#estimatedSerializedSizeOf is slow when input is ByteBufferCell > --- > > Key: HBASE-17836 > URL: https://issues.apache.org/jira/browse/HBASE-17836 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17836.v0.patch, HBASE-17836.v1.patch > > > We call CellUtil#estimatedSerializedSize to calculate the size of rows when > scanning. If the input is ByteBufferCell, the > CellUtil#estimatedSerializedSizeOf parses many length components to get the > qualifierLength stored in the backing buffer. > We should consider using the KeyValueUtil#getSerializedSize. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HBASE-16575) unify the semantic of RRCI#callWithRetries and RRCI#callWithoutRetries when the maxAttempts is configured to one
[ https://issues.apache.org/jira/browse/HBASE-16575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-16575. Resolution: Not A Problem > unify the semantic of RRCI#callWithRetries and RRCI#callWithoutRetries when > the maxAttempts is configured to one > > > Key: HBASE-16575 > URL: https://issues.apache.org/jira/browse/HBASE-16575 > Project: HBase > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Priority: Minor > > It seems to me that RRCI#callWithRetries and RRCI#callWithoutRetries should > have the same logic if the maxAttempts is configured to one. But there are > some difference are shown below: > 1) timeout > 2) failure handle > The quick solution is that we always call the RRCI#callWithRetries in the > RRCI#callWithoutRetries when the maxAttempts is configured to one. > Any comment? Thanks. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16346) The cell cannot be incremented with null qualifier
[
https://issues.apache.org/jira/browse/HBASE-16346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-16346:
---
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
No use cases. Close it.
> The cell cannot be incremented with null qualifier
> --
>
> Key: HBASE-16346
> URL: https://issues.apache.org/jira/browse/HBASE-16346
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Trivial
> Attachments: HBASE-16346.v0.patch, HBASE-16346-v0.txt
>
>
> {code:title=Increment.java|borderStyle=solid}
> // Is it necessary to prevent the null qualifier
> public Increment addColumn(byte [] family, byte [] qualifier, long amount) {
> if (family == null) {
> throw new IllegalArgumentException("family cannot be null");
> }
> if (qualifier == null) {
> throw new IllegalArgumentException("qualifier cannot be null");
> }
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts,
> Bytes.toBytes(amount));
> list.add(kv);
> familyMap.put(CellUtil.cloneFamily(kv), list);
> return this;
> }
> {code}
> I use the add(Cell) method to add the cell with null qualifier and it works
> fine.
> It seems to me that the check should be removed
> any command? thanks
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17881) Remove the ByteBufferCellImpl
[ https://issues.apache.org/jira/browse/HBASE-17881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960130#comment-15960130 ] Chia-Ping Tsai commented on HBASE-17881: Any more comment (smile)? [~stack] > Remove the ByteBufferCellImpl > - > > Key: HBASE-17881 > URL: https://issues.apache.org/jira/browse/HBASE-17881 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17881.v0.patch > > > We should substitute ByteBufferKeyValue for ByteBufferCellImpl -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17835) Spelling mistakes in the Java source
[ https://issues.apache.org/jira/browse/HBASE-17835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960118#comment-15960118 ] Qilin Cao commented on HBASE-17835: --- [~elserj] Can assign to me? thanks! > Spelling mistakes in the Java source > > > Key: HBASE-17835 > URL: https://issues.apache.org/jira/browse/HBASE-17835 > Project: HBase > Issue Type: Improvement >Reporter: Qilin Cao >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-17835-001.patch > > > I found spelling mistakes in the hbase java source files viz. recieved > instead of received, and SpanReciever instead of SpanReceiver -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960114#comment-15960114 ] stack commented on HBASE-14614: --- v026 Fix in DisableTable seems to fix a bunch of test failures. Trying (Findbugs is from pb classes). > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: HBASE-14614.master.001.patch, > HBASE-14614.master.002.patch, HBASE-14614.master.003.patch, > HBASE-14614.master.004.patch, HBASE-14614.master.005.patch, > HBASE-14614.master.006.patch, HBASE-14614.master.007.patch, > HBASE-14614.master.008.patch, HBASE-14614.master.009.patch, > HBASE-14614.master.010.patch, HBASE-14614.master.011.patch, > HBASE-14614.master.012.patch, HBASE-14614.master.012.patch, > HBASE-14614.master.013.patch, HBASE-14614.master.014.patch, > HBASE-14614.master.015.patch, HBASE-14614.master.016.patch, > HBASE-14614.master.017.patch, HBASE-14614.master.018.patch, > HBASE-14614.master.019.patch, HBASE-14614.master.020.patch, > HBASE-14614.master.021.patch, HBASE-14614.master.022.patch, > HBASE-14614.master.023.patch, HBASE-14614.master.024.patch, > HBASE-14614.master.025.patch, HBASE-14614.master.026.patch > > > New AssignmentManager implemented using proc-v2. > - AssignProcedure handle assignment operation > - UnassignProcedure handle unassign operation > - MoveRegionProcedure handle move/balance operation > Concurrent Assign operations are batched together and sent to the balancer > Concurrent Assign and Unassign operation ready to be sent to the RS are > batched together > This patch is an intermediate state where we add the new AM as > AssignmentManager2() to the master, to be reached by tests. but the new AM > will not be integrated with the rest of the system. Only new am unit-tests > will exercise the new assigment manager. The integration with the master code > is part of HBASE-14616 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14614: -- Attachment: HBASE-14614.master.026.patch > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: HBASE-14614.master.001.patch, > HBASE-14614.master.002.patch, HBASE-14614.master.003.patch, > HBASE-14614.master.004.patch, HBASE-14614.master.005.patch, > HBASE-14614.master.006.patch, HBASE-14614.master.007.patch, > HBASE-14614.master.008.patch, HBASE-14614.master.009.patch, > HBASE-14614.master.010.patch, HBASE-14614.master.011.patch, > HBASE-14614.master.012.patch, HBASE-14614.master.012.patch, > HBASE-14614.master.013.patch, HBASE-14614.master.014.patch, > HBASE-14614.master.015.patch, HBASE-14614.master.016.patch, > HBASE-14614.master.017.patch, HBASE-14614.master.018.patch, > HBASE-14614.master.019.patch, HBASE-14614.master.020.patch, > HBASE-14614.master.021.patch, HBASE-14614.master.022.patch, > HBASE-14614.master.023.patch, HBASE-14614.master.024.patch, > HBASE-14614.master.025.patch, HBASE-14614.master.026.patch > > > New AssignmentManager implemented using proc-v2. > - AssignProcedure handle assignment operation > - UnassignProcedure handle unassign operation > - MoveRegionProcedure handle move/balance operation > Concurrent Assign operations are batched together and sent to the balancer > Concurrent Assign and Unassign operation ready to be sent to the RS are > batched together > This patch is an intermediate state where we add the new AM as > AssignmentManager2() to the master, to be reached by tests. but the new AM > will not be integrated with the rest of the system. Only new am unit-tests > will exercise the new assigment manager. The integration with the master code > is part of HBASE-14616 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17889) ResultBoundedCompletionService's cancel() needs to interrupt the working thread and free it to the thread-pool
[ https://issues.apache.org/jira/browse/HBASE-17889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-17889: - Status: Patch Available (was: Open) > ResultBoundedCompletionService's cancel() needs to interrupt the working > thread and free it to the thread-pool > -- > > Key: HBASE-17889 > URL: https://issues.apache.org/jira/browse/HBASE-17889 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0, 1.4.0, 1.2.6, 1.3.2 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-17889-master-001.patch, jstack.txt > > > We run into one case with read-replica, when the server hosting the primary > region is shutdown, we see Get did not go to replica region and it paused for > about 50 seconds before Get was resumed. > More debugging finds out that when the server is down, one of the threads was > stuck at the write, it holds lock at > https://github.com/apache/hbase/blob/branch-1.3/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java#L916. > The later write threads were waiting on this lock until all threads in the > connection's thread pool were stuck on this lock. At that moment, no work > will be done. After socket write times out, it frees up all threads and it > continues. > When QueueingFuture#cancel() is called, it does not interrupt the working > thread and return the thread to the pool. > Attaching the jstack trace. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17889) ResultBoundedCompletionService's cancel() needs to interrupt the working thread and free it to the thread-pool
[ https://issues.apache.org/jira/browse/HBASE-17889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960082#comment-15960082 ] huaxiang sun commented on HBASE-17889: -- It will be hard to add an unittest case as the server needs to be physically shutdown to reproduce the issue. > ResultBoundedCompletionService's cancel() needs to interrupt the working > thread and free it to the thread-pool > -- > > Key: HBASE-17889 > URL: https://issues.apache.org/jira/browse/HBASE-17889 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0, 1.4.0, 1.2.6, 1.3.2 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-17889-master-001.patch, jstack.txt > > > We run into one case with read-replica, when the server hosting the primary > region is shutdown, we see Get did not go to replica region and it paused for > about 50 seconds before Get was resumed. > More debugging finds out that when the server is down, one of the threads was > stuck at the write, it holds lock at > https://github.com/apache/hbase/blob/branch-1.3/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java#L916. > The later write threads were waiting on this lock until all threads in the > connection's thread pool were stuck on this lock. At that moment, no work > will be done. After socket write times out, it frees up all threads and it > continues. > When QueueingFuture#cancel() is called, it does not interrupt the working > thread and return the thread to the pool. > Attaching the jstack trace. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17889) ResultBoundedCompletionService's cancel() needs to interrupt the working thread and free it to the thread-pool
[ https://issues.apache.org/jira/browse/HBASE-17889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-17889: - Attachment: HBASE-17889-master-001.patch Attach the first version of the patch. > ResultBoundedCompletionService's cancel() needs to interrupt the working > thread and free it to the thread-pool > -- > > Key: HBASE-17889 > URL: https://issues.apache.org/jira/browse/HBASE-17889 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0, 1.4.0, 1.2.6, 1.3.2 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-17889-master-001.patch, jstack.txt > > > We run into one case with read-replica, when the server hosting the primary > region is shutdown, we see Get did not go to replica region and it paused for > about 50 seconds before Get was resumed. > More debugging finds out that when the server is down, one of the threads was > stuck at the write, it holds lock at > https://github.com/apache/hbase/blob/branch-1.3/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java#L916. > The later write threads were waiting on this lock until all threads in the > connection's thread pool were stuck on this lock. At that moment, no work > will be done. After socket write times out, it frees up all threads and it > continues. > When QueueingFuture#cancel() is called, it does not interrupt the working > thread and return the thread to the pool. > Attaching the jstack trace. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17800) [C++] handle exceptions in client RPC
[
https://issues.apache.org/jira/browse/HBASE-17800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960032#comment-15960032
]
Hadoop QA commented on HBASE-17800:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 33m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 15s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s
{color} | {color:green} HBASE-14850 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s
{color} | {color:green} HBASE-14850 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 2s
{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
7s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
61m 16s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
4s {color} | {color:green} the patch passed {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 95m 7s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:date2017-04-06
|
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862385/HBASE-17800-HBASE-14850.003.patch
|
| JIRA Issue | HBASE-17800 |
| Optional Tests | shellcheck shelldocs cc compile |
| uname | Linux 80606c078be0 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | nobuild |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-14850 / 66f8f36 |
| shellcheck | v0.4.6 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6356/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> [C++] handle exceptions in client RPC
> -
>
> Key: HBASE-17800
> URL: https://issues.apache.org/jira/browse/HBASE-17800
> Project: HBase
> Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HBASE-17800-HBASE-14850.000.patch,
> HBASE-17800-HBASE-14850.001.patch, HBASE-17800-HBASE-14850.002.patch,
> HBASE-17800-HBASE-14850.003.patch
>
>
> Exceptions are ignored in current client RPC. They should be handled properly
> to be consumed by RPC retry or propagated up to APIs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17863) Procedure V2: Proc Executor cleanup. Split FINISHED state to two states: SUCCESS and FAILED.
[ https://issues.apache.org/jira/browse/HBASE-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960050#comment-15960050 ] Hudson commented on HBASE-17863: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2812 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2812/]) HBASE-17863: Procedure V2: Some cleanup around Procedure.isFinished() (stack: rev 9109803891e256f8c047af72572f07695e604a3f) * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureUtil.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/ProcedureStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestProcedureAdmin.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/ProcedureState.java * (edit) hbase-protocol-shaded/src/main/protobuf/Procedure.proto * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALLoaderPerformanceEvaluation.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/ProcedureProtos.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALFormatReader.java > Procedure V2: Proc Executor cleanup. Split FINISHED state to two states: > SUCCESS and FAILED. > - > > Key: HBASE-17863 > URL: https://issues.apache.org/jira/browse/HBASE-17863 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-17863.v1.patch, HBASE-17863.v2.patch, > HBASE-17863.v3.patch, HBASE-17863.v3.patch, HBASE-17863.v4.patch, > HBASE-17863.v4.patch > > > Clean up around isFinished() and procedure executor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17863) Procedure V2: Proc Executor cleanup. Split FINISHED state to two states: SUCCESS and FAILED.
[ https://issues.apache.org/jira/browse/HBASE-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960038#comment-15960038 ] stack commented on HBASE-17863: --- On the second change, [~uagashe] put it under my nose and I thought it made sense but you raise a good point [~appy] > Procedure V2: Proc Executor cleanup. Split FINISHED state to two states: > SUCCESS and FAILED. > - > > Key: HBASE-17863 > URL: https://issues.apache.org/jira/browse/HBASE-17863 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-17863.v1.patch, HBASE-17863.v2.patch, > HBASE-17863.v3.patch, HBASE-17863.v3.patch, HBASE-17863.v4.patch, > HBASE-17863.v4.patch > > > Clean up around isFinished() and procedure executor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17863) Procedure V2: Proc Executor cleanup. Split FINISHED state to two states: SUCCESS and FAILED.
[ https://issues.apache.org/jira/browse/HBASE-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960037#comment-15960037 ] Umesh Agashe commented on HBASE-17863: -- working on addendum, will attach it soon. > Procedure V2: Proc Executor cleanup. Split FINISHED state to two states: > SUCCESS and FAILED. > - > > Key: HBASE-17863 > URL: https://issues.apache.org/jira/browse/HBASE-17863 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-17863.v1.patch, HBASE-17863.v2.patch, > HBASE-17863.v3.patch, HBASE-17863.v3.patch, HBASE-17863.v4.patch, > HBASE-17863.v4.patch > > > Clean up around isFinished() and procedure executor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HBASE-10145) Table creation should proceed in the presence of a stale znode
[ https://issues.apache.org/jira/browse/HBASE-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-10145. --- Resolution: Won't Fix Closing old issue. > Table creation should proceed in the presence of a stale znode > -- > > Key: HBASE-10145 > URL: https://issues.apache.org/jira/browse/HBASE-10145 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Priority: Minor > > HBASE-7600 fixed a race condition where concurrent attempts to create the > same table could succeed. > An unfortunate side effect is that it is now impossible to create a table as > long as the table's znode is around, which is an issue when a cluster was > wiped at the HDFS level. > Minor issue as we have discussed this many times before, but it ought to be > possible to check whether the table directory exists and if not either create > it or remove the corresponding znode. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HBASE-6492) Remove Reflection based Hadoop abstractions
[ https://issues.apache.org/jira/browse/HBASE-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-6492. -- Resolution: Won't Fix Closing old issue. > Remove Reflection based Hadoop abstractions > --- > > Key: HBASE-6492 > URL: https://issues.apache.org/jira/browse/HBASE-6492 > Project: HBase > Issue Type: Improvement >Reporter: Lars Hofhansl > > In 0.96 we now have the Hadoop1-compat and Hadoop2-compat projects. > The reflection we're using to deal with different versions of Hadoop should > be removed in favour of using the compact projects. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HBASE-10028) Cleanup metrics documentation
[ https://issues.apache.org/jira/browse/HBASE-10028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-10028. --- Resolution: Won't Fix Closing old issue. > Cleanup metrics documentation > - > > Key: HBASE-10028 > URL: https://issues.apache.org/jira/browse/HBASE-10028 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > > The current documentation of the metrics is incomplete and at point incorrect > (HDFS latencies are in ns rather than ms for example). > We should clean this up and add other related metrics as well. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HBASE-5475) Allow importtsv and Import to work truly offline when using bulk import option
[ https://issues.apache.org/jira/browse/HBASE-5475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-5475. -- Resolution: Won't Fix Closing old issue. > Allow importtsv and Import to work truly offline when using bulk import option > -- > > Key: HBASE-5475 > URL: https://issues.apache.org/jira/browse/HBASE-5475 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Lars Hofhansl >Priority: Minor > > Currently importtsv (and now also Import with HBASE-5440) support using > HFileOutputFormat for later bulk loading. > However, currently that cannot be without having access to the table we're > going to import to, because both importtsv and Import need to lookup the > split points, and find the compression setting. > It would be nice if there would be an offline way to provide the split point > and compression setting. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HBASE-5311) Allow inmemory Memstore compactions
[ https://issues.apache.org/jira/browse/HBASE-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-5311. -- Resolution: Won't Fix Closing old issue. > Allow inmemory Memstore compactions > --- > > Key: HBASE-5311 > URL: https://issues.apache.org/jira/browse/HBASE-5311 > Project: HBase > Issue Type: Improvement >Reporter: Lars Hofhansl > Attachments: InternallyLayeredMap.java > > > Just like we periodically compact the StoreFiles we should also periodically > compact the MemStore. > During these compactions we eliminate deleted cells, expired cells, cells to > removed because of version count, etc, before we even do a memstore flush. > Besides the optimization that we could get from this, it should also allow us > to remove the special handling of ICV, Increment, and Append (all of which > use upsert logic to avoid accumulating excessive cells in the Memstore). > Not targeting this. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15453) [Performance] Considering reverting HBASE-10015 - reinstate synchronized in StoreScanner
[ https://issues.apache.org/jira/browse/HBASE-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959998#comment-15959998 ] Lars Hofhansl commented on HBASE-15453: --- I guess I dropped the ball on this one. If there's still interest, this can go to all branches but master. > [Performance] Considering reverting HBASE-10015 - reinstate synchronized in > StoreScanner > > > Key: HBASE-15453 > URL: https://issues.apache.org/jira/browse/HBASE-15453 > Project: HBase > Issue Type: Improvement > Components: Performance >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Critical > Attachments: 15453-0.98.txt > > > In HBASE-10015 back then I found that intrinsic locks (synchronized) in > StoreScanner are slower that explicit locks. > I was surprised by this. To make sure I added a simple perf test and many > folks ran it on their machines. All found that explicit locks were faster. > Now... I just ran that test again. On the latest JDK8 I find that now the > intrinsic locks are significantly faster: > (OpenJDK Runtime Environment (build 1.8.0_72-b15)) > Explicit locks: > 10 runs mean:2223.6 sigma:72.29412147609237 > Intrinsic locks: > 10 runs mean:1865.3 sigma:32.63755505548784 > I confirmed the same with timing some Phoenix scans. We can save a bunch of > time by changing this back > Arrghhh... So maybe it's time to revert this now...? > (Note that in trunk due to [~ram_krish]'s work, we do not lock in > StoreScanner anymore) > I'll attach the perf test and a patch that changes lock to synchronized, if > some folks could run this on 0.98, that'd be great. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17521) Avoid stopping the load balancer in graceful stop
[ https://issues.apache.org/jira/browse/HBASE-17521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959996#comment-15959996 ] Lars Hofhansl commented on HBASE-17521: --- [~sandeep.guggilam], [~mnpoonia], any update on this? > Avoid stopping the load balancer in graceful stop > - > > Key: HBASE-17521 > URL: https://issues.apache.org/jira/browse/HBASE-17521 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > > ... instead setting the regionserver in question to draining. > [~sandeep.guggilam], FYI -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-17872:
---
Attachment: HBASE-17872.v3.patch
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch, HBASE-17872.v3.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-17872:
---
Status: Open (was: Patch Available)
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch, HBASE-17872.v3.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-17872:
---
Status: Patch Available (was: Open)
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch, HBASE-17872.v3.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959990#comment-15959990
]
Chia-Ping Tsai commented on HBASE-17872:
bq. Then it becomes how to change a static final setting at test time
The issue is not that complicated. We change the UNSAFE_AVAIL/UNSAFE_UNALIGNED
before starting up a mini cluster when testing, so they shouldn't be declared
volatile.
Please see the v3 patch. Thanks. [~anoop.hbase] [~stack]
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17816) HRegion#mutateRowWithLocks should update writeRequestCount metric
[ https://issues.apache.org/jira/browse/HBASE-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959986#comment-15959986 ] Jerry He commented on HBASE-17816: -- Pushed to master. [~qgxiaozhan] If you provide a branch-1 patch, I'll commit it. > HRegion#mutateRowWithLocks should update writeRequestCount metric > - > > Key: HBASE-17816 > URL: https://issues.apache.org/jira/browse/HBASE-17816 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Ashu Pachauri >Assignee: Weizhan Zeng > Attachments: HBASE-17816.master.001.patch, > HBASE-17816.master.002.patch > > > Currently, all the calls that use HRegion#mutateRowWithLocks miss > writeRequestCount metric. The mutateRowWithLocks base method should update > the metric. > Examples are checkAndMutate calls through RSRpcServices#multi, > Region#mutateRow api , MultiRowMutationProcessor coprocessor endpoint. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17869) UnsafeAvailChecker wrongly returns false on ppc
[ https://issues.apache.org/jira/browse/HBASE-17869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959980#comment-15959980 ] Jerry He commented on HBASE-17869: -- Filed HBASE-17890. > UnsafeAvailChecker wrongly returns false on ppc > --- > > Key: HBASE-17869 > URL: https://issues.apache.org/jira/browse/HBASE-17869 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.4 >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17869.patch > > > On ppc64 arch, java.nio.Bits.unaligned() wrongly returns false due to a JDK > bug. > https://bugs.openjdk.java.net/browse/JDK-8165231 > This causes some problem for HBase. i.e. FuzzyRowFilter test fails. > Fix it by providing a hard-code workaround for the JDK bug. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17890) FuzzyRow tests fail if unaligned support is false
Jerry He created HBASE-17890:
Summary: FuzzyRow tests fail if unaligned support is false
Key: HBASE-17890
URL: https://issues.apache.org/jira/browse/HBASE-17890
Project: HBase
Issue Type: Bug
Affects Versions: 1.2.5, 2.0.0
Reporter: Jerry He
When unaligned support is false, FuzzyRow tests fail:
{noformat}
Failed tests:
TestFuzzyRowAndColumnRangeFilter.Test:134->runTest:157->runScanner:186
expected:<10> but was:<0>
TestFuzzyRowFilter.testSatisfiesForward:81 expected: but was:
TestFuzzyRowFilter.testSatisfiesReverse:121 expected: but
was:
TestFuzzyRowFilterEndToEnd.testEndToEnd:247->runTest1:278->runScanner:343
expected:<6250> but was:<0>
TestFuzzyRowFilterEndToEnd.testFilterList:385->runTest:417->runScanner:445
expected:<5> but was:<0>
TestFuzzyRowFilterEndToEnd.testHBASE14782:204 expected:<6> but was:<0>
{noformat}
This can be reproduced in the case described in HBASE-17869. Or on a platform
really without unaligned support.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17869) UnsafeAvailChecker wrongly returns false on ppc
[ https://issues.apache.org/jira/browse/HBASE-17869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry He updated HBASE-17869: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > UnsafeAvailChecker wrongly returns false on ppc > --- > > Key: HBASE-17869 > URL: https://issues.apache.org/jira/browse/HBASE-17869 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.4 >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17869.patch > > > On ppc64 arch, java.nio.Bits.unaligned() wrongly returns false due to a JDK > bug. > https://bugs.openjdk.java.net/browse/JDK-8165231 > This causes some problem for HBase. i.e. FuzzyRowFilter test fails. > Fix it by providing a hard-code workaround for the JDK bug. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17869) UnsafeAvailChecker wrongly returns false on ppc
[
https://issues.apache.org/jira/browse/HBASE-17869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959964#comment-15959964
]
Jerry He commented on HBASE-17869:
--
Pushed to master and branch-1. (Fixed Ted's nit.)
Tested on ppc64le:
Without the patch:
{code}
Failed tests:
TestFuzzyRowAndColumnRangeFilter.Test:134->runTest:157->runScanner:186
expected:<10> but was:<0>
TestFuzzyRowFilter.testSatisfiesForward:81 expected: but was:
TestFuzzyRowFilter.testSatisfiesReverse:121 expected: but
was:
TestFuzzyRowFilterEndToEnd.testEndToEnd:247->runTest1:278->runScanner:343
expected:<6250> but was:<0>
TestFuzzyRowFilterEndToEnd.testFilterList:385->runTest:417->runScanner:445
expected:<5> but was:<0>
TestFuzzyRowFilterEndToEnd.testHBASE14782:204 expected:<6> but was:<0>
{code}
With the patch, the full 'mvn clean test' is clean on master :-)
Thanks for the review.
> UnsafeAvailChecker wrongly returns false on ppc
> ---
>
> Key: HBASE-17869
> URL: https://issues.apache.org/jira/browse/HBASE-17869
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.4
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17869.patch
>
>
> On ppc64 arch, java.nio.Bits.unaligned() wrongly returns false due to a JDK
> bug.
> https://bugs.openjdk.java.net/browse/JDK-8165231
> This causes some problem for HBase. i.e. FuzzyRowFilter test fails.
> Fix it by providing a hard-code workaround for the JDK bug.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17800) [C++] handle exceptions in client RPC
[ https://issues.apache.org/jira/browse/HBASE-17800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-17800: -- Status: Patch Available (was: Open) > [C++] handle exceptions in client RPC > - > > Key: HBASE-17800 > URL: https://issues.apache.org/jira/browse/HBASE-17800 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17800-HBASE-14850.000.patch, > HBASE-17800-HBASE-14850.001.patch, HBASE-17800-HBASE-14850.002.patch, > HBASE-17800-HBASE-14850.003.patch > > > Exceptions are ignored in current client RPC. They should be handled properly > to be consumed by RPC retry or propagated up to APIs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17889) ResultBoundedCompletionService's cancel() needs to interrupt the working thread and free it to the thread-pool
[ https://issues.apache.org/jira/browse/HBASE-17889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-17889: - Attachment: jstack.txt jstack dump. > ResultBoundedCompletionService's cancel() needs to interrupt the working > thread and free it to the thread-pool > -- > > Key: HBASE-17889 > URL: https://issues.apache.org/jira/browse/HBASE-17889 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0, 1.4.0, 1.2.6, 1.3.2 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: jstack.txt > > > We run into one case with read-replica, when the server hosting the primary > region is shutdown, we see Get did not go to replica region and it paused for > about 50 seconds before Get was resumed. > More debugging finds out that when the server is down, one of the threads was > stuck at the write, it holds lock at > https://github.com/apache/hbase/blob/branch-1.3/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java#L916. > The later write threads were waiting on this lock until all threads in the > connection's thread pool were stuck on this lock. At that moment, no work > will be done. After socket write times out, it frees up all threads and it > continues. > When QueueingFuture#cancel() is called, it does not interrupt the working > thread and return the thread to the pool. > Attaching the jstack trace. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17800) [C++] handle exceptions in client RPC
[ https://issues.apache.org/jira/browse/HBASE-17800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959907#comment-15959907 ] Xiaobing Zhou commented on HBASE-17800: --- v3 fixed a bunch of compile issues, and simplified call to do_not_retry. Still working on unit tests. > [C++] handle exceptions in client RPC > - > > Key: HBASE-17800 > URL: https://issues.apache.org/jira/browse/HBASE-17800 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17800-HBASE-14850.000.patch, > HBASE-17800-HBASE-14850.001.patch, HBASE-17800-HBASE-14850.002.patch, > HBASE-17800-HBASE-14850.003.patch > > > Exceptions are ignored in current client RPC. They should be handled properly > to be consumed by RPC retry or propagated up to APIs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17800) [C++] handle exceptions in client RPC
[ https://issues.apache.org/jira/browse/HBASE-17800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-17800: -- Attachment: HBASE-17800-HBASE-14850.003.patch > [C++] handle exceptions in client RPC > - > > Key: HBASE-17800 > URL: https://issues.apache.org/jira/browse/HBASE-17800 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17800-HBASE-14850.000.patch, > HBASE-17800-HBASE-14850.001.patch, HBASE-17800-HBASE-14850.002.patch, > HBASE-17800-HBASE-14850.003.patch > > > Exceptions are ignored in current client RPC. They should be handled properly > to be consumed by RPC retry or propagated up to APIs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17889) ResultBoundedCompletionService's cancel() needs to interrupt the working thread and free it to the thread-pool
huaxiang sun created HBASE-17889: Summary: ResultBoundedCompletionService's cancel() needs to interrupt the working thread and free it to the thread-pool Key: HBASE-17889 URL: https://issues.apache.org/jira/browse/HBASE-17889 Project: HBase Issue Type: Bug Components: Client Affects Versions: 2.0.0, 1.4.0, 1.2.6, 1.3.2 Reporter: huaxiang sun Assignee: huaxiang sun We run into one case with read-replica, when the server hosting the primary region is shutdown, we see Get did not go to replica region and it paused for about 50 seconds before Get was resumed. More debugging finds out that when the server is down, one of the threads was stuck at the write, it holds lock at https://github.com/apache/hbase/blob/branch-1.3/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java#L916. The later write threads were waiting on this lock until all threads in the connection's thread pool were stuck on this lock. At that moment, no work will be done. After socket write times out, it frees up all threads and it continues. When QueueingFuture#cancel() is called, it does not interrupt the working thread and return the thread to the pool. Attaching the jstack trace. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17863) Procedure V2: Proc Executor cleanup. Split FINISHED state to two states: SUCCESS and FAILED.
[ https://issues.apache.org/jira/browse/HBASE-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17863: - Summary: Procedure V2: Proc Executor cleanup. Split FINISHED state to two states: SUCCESS and FAILED. (was: Procedure V2: Some cleanup around Procedure.isFinished() and procedure executor) > Procedure V2: Proc Executor cleanup. Split FINISHED state to two states: > SUCCESS and FAILED. > - > > Key: HBASE-17863 > URL: https://issues.apache.org/jira/browse/HBASE-17863 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-17863.v1.patch, HBASE-17863.v2.patch, > HBASE-17863.v3.patch, HBASE-17863.v3.patch, HBASE-17863.v4.patch, > HBASE-17863.v4.patch > > > Clean up around isFinished() and procedure executor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17888) Add generic methods for updating metrics on start and end of a procedure execution
Umesh Agashe created HBASE-17888: Summary: Add generic methods for updating metrics on start and end of a procedure execution Key: HBASE-17888 URL: https://issues.apache.org/jira/browse/HBASE-17888 Project: HBase Issue Type: Improvement Components: proc-v2 Reporter: Umesh Agashe Assignee: Umesh Agashe For all procedures in Procv2 framework, Procedure class can have generic methods to update metrics on start and end of a procedure execution. Specific procedure can override these and implement/ update respective metrics. Default implementation needs to be provided so override and implementation is optional. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17863) Procedure V2: Some cleanup around Procedure.isFinished() and procedure executor
[
https://issues.apache.org/jira/browse/HBASE-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959811#comment-15959811
]
Appy commented on HBASE-17863:
--
so afaiu,
||old semantic||new semantic||
|finished and exception|FAILED|
|finished and no exception|SUCCESS|
They SUCCESS state implies no exception, then do we need the and condition below
{noformat}
public synchronized boolean isSuccess() {
return state == ProcedureState.SUCCESS && !hasException();
}
{noformat}
Why are we switching the order here?
{noformat}
- // Commit the transaction
- updateStoreOnExec(procStack, procedure, subprocs);
-
// if the store is not running we are aborting
if (!store.isRunning()) return;
+ // Commit the transaction
+ updateStoreOnExec(procStack, procedure, subprocs);
+
{noformat}
nit: (only in case there's an addendum) rename
ProcedureTestingUtility#setFinishedState to setSuccessState()
> Procedure V2: Some cleanup around Procedure.isFinished() and procedure
> executor
> ---
>
> Key: HBASE-17863
> URL: https://issues.apache.org/jira/browse/HBASE-17863
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
> Fix For: 2.0.0
>
> Attachments: HBASE-17863.v1.patch, HBASE-17863.v2.patch,
> HBASE-17863.v3.patch, HBASE-17863.v3.patch, HBASE-17863.v4.patch,
> HBASE-17863.v4.patch
>
>
> Clean up around isFinished() and procedure executor
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17871) scan#setBatch(int) call leads wrong result of VerifyReplication
[
https://issues.apache.org/jira/browse/HBASE-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959654#comment-15959654
]
Hudson commented on HBASE-17871:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2811 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2811/])
HBASE-17871 scan#setBatch(int) call leads wrong result of (tedyu: rev
ec5188df3090d42088b6f4cb8f0c2fd49425f8c1)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
> scan#setBatch(int) call leads wrong result of VerifyReplication
> ---
>
> Key: HBASE-17871
> URL: https://issues.apache.org/jira/browse/HBASE-17871
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0
>Reporter: Tomu Tsuruhara
>Assignee: Tomu Tsuruhara
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after.png, beforethepatch.png,
> HBASE-17871.master.001.patch, HBASE-17871.master.002.patch,
> HBASE-17871.master.003.patch, HBASE-17871.master.003.patch,
> HBASE-17871.master.004.patch
>
>
> VerifyReplication tool printed weird logs.
> {noformat}
> 2017-04-03 23:30:50,252 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> CONTENT_DIFFERENT_ROWS, rowkey=a100193
> 2017-04-03 23:30:50,280 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> ONLY_IN_PEER_TABLE_ROWS, rowkey=a100193
> 2017-04-03 23:30:50,387 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> CONTENT_DIFFERENT_ROWS, rowkey=a100385
> 2017-04-03 23:30:50,414 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> ONLY_IN_PEER_TABLE_ROWS, rowkey=a100385
> 2017-04-03 23:30:50,480 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> CONTENT_DIFFERENT_ROWS, rowkey=a100532
> 2017-04-03 23:30:50,508 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> ONLY_IN_PEER_TABLE_ROWS, rowkey=a100532
> {noformat}
> Here, each bad rows were marked as both {{CONTENT_DIFFERENT_ROWS}} and
> {{ONLY_IN_PEER_TABLE_ROWS}}.
> This should never happen so I took a look at code and found scan.setBatch
> call.
> {code}
> @Override
> public void map(ImmutableBytesWritable row, final Result value,
> Context context)
> throws IOException {
> if (replicatedScanner == null) {
> ...
> final Scan scan = new Scan();
> scan.setBatch(batch);
> {code}
> As stated in HBASE-16376, {{scan#setBatch(int)}} call implicitly allows scan
> results to be partial.
> Since {{VerifyReplication}} is assuming each {{scanner.next()}} call returns
> entire row,
> partial results break compare logic.
> We should avoid setBatch call here.
> Thanks to RPC chunking (explained in this blog
> https://blogs.apache.org/hbase/entry/scan_improvements_in_hbase_1),
> it's safe and acceptable I think.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17863) Procedure V2: Some cleanup around Procedure.isFinished() and procedure executor
[ https://issues.apache.org/jira/browse/HBASE-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959638#comment-15959638 ] Umesh Agashe commented on HBASE-17863: -- I tested it on my laptop. Each test runs 3 times in a single run and I ran it 10 times with and without the patch. > Procedure V2: Some cleanup around Procedure.isFinished() and procedure > executor > --- > > Key: HBASE-17863 > URL: https://issues.apache.org/jira/browse/HBASE-17863 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-17863.v1.patch, HBASE-17863.v2.patch, > HBASE-17863.v3.patch, HBASE-17863.v3.patch, HBASE-17863.v4.patch, > HBASE-17863.v4.patch > > > Clean up around isFinished() and procedure executor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959577#comment-15959577 ] Edward Bortnikov commented on HBASE-16438: -- [~anastas] - is this a +1? > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: > HBASE-16438_10_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_11_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959565#comment-15959565 ] Anastasia Braginsky commented on HBASE-16438: - OK guys, I understand that seqID is not in ByteBuffer and this is how it was before this JIRA. If you don't like to write it on data-chunk I accept it. I will write seqID as part of the cell-representation in the CellChunkMap. > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: > HBASE-16438_10_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_11_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17863) Procedure V2: Some cleanup around Procedure.isFinished() and procedure executor
[ https://issues.apache.org/jira/browse/HBASE-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17863: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) Pushed. > Procedure V2: Some cleanup around Procedure.isFinished() and procedure > executor > --- > > Key: HBASE-17863 > URL: https://issues.apache.org/jira/browse/HBASE-17863 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-17863.v1.patch, HBASE-17863.v2.patch, > HBASE-17863.v3.patch, HBASE-17863.v3.patch, HBASE-17863.v4.patch, > HBASE-17863.v4.patch > > > Clean up around isFinished() and procedure executor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17863) Procedure V2: Some cleanup around Procedure.isFinished() and procedure executor
[ https://issues.apache.org/jira/browse/HBASE-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959542#comment-15959542 ] stack commented on HBASE-17863: --- I do not see how this change which pertains to macro ops -- i.e. procedures -- can disturb a data read messing w/ row consistency view. Let me push it. Thanks for testing [~uagashe] Did you do any work to compare before and after and if so, what was it sir! Thanks. > Procedure V2: Some cleanup around Procedure.isFinished() and procedure > executor > --- > > Key: HBASE-17863 > URL: https://issues.apache.org/jira/browse/HBASE-17863 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Attachments: HBASE-17863.v1.patch, HBASE-17863.v2.patch, > HBASE-17863.v3.patch, HBASE-17863.v3.patch, HBASE-17863.v4.patch, > HBASE-17863.v4.patch > > > Clean up around isFinished() and procedure executor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17863) Procedure V2: Some cleanup around Procedure.isFinished() and procedure executor
[ https://issues.apache.org/jira/browse/HBASE-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959502#comment-15959502 ] Umesh Agashe commented on HBASE-17863: -- TestAcidGuarantees fail similar number of times with or without the patch for this JIRA. > Procedure V2: Some cleanup around Procedure.isFinished() and procedure > executor > --- > > Key: HBASE-17863 > URL: https://issues.apache.org/jira/browse/HBASE-17863 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Attachments: HBASE-17863.v1.patch, HBASE-17863.v2.patch, > HBASE-17863.v3.patch, HBASE-17863.v3.patch, HBASE-17863.v4.patch, > HBASE-17863.v4.patch > > > Clean up around isFinished() and procedure executor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17887) TestAcidGuarantees fails frequently
Umesh Agashe created HBASE-17887: Summary: TestAcidGuarantees fails frequently Key: HBASE-17887 URL: https://issues.apache.org/jira/browse/HBASE-17887 Project: HBase Issue Type: Bug Components: regionserver Reporter: Umesh Agashe Priority: Blocker As per the flaky tests dashboard here: https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html, It fails 30% of the time. While working on HBASE-17863, a few verification builds on patch failed due to TestAcidGuarantees didn't pass. IMHO, the changes for HBASE-17863 are unlikely to affect get/ put path. I ran the test with and without the patch several times locally and found that TestAcidGuarantees fails without the patch similar number of times. Opening blocker, considering acid guarantees are critical to HBase. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14925) Develop HBase shell command/tool to list table's region info through command line
[
https://issues.apache.org/jira/browse/HBASE-14925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959491#comment-15959491
]
Karan Mehta commented on HBASE-14925:
-
bq. I could not find the start_time variable in the patch and also end_time
variable value not being used anywhere in the patch. What am I missing ? Can
you point me ?
Here is the code snippet from the {{commands.rb}} file. The method
{{command_safe()}} is a wrapper to the actual command call which initializes
the global variable {{start_time}}. It also automatically computes the value of
{{end_time}} and displays the total execution time. We can override these
values. Since I didn't want the output formatting and display time to be
included in the actual execution of the command, I provided a value to that
variable before the output stuff. Does that seem okay?
{code}
#wrap an execution of cmd to catch hbase exceptions
# cmd - command name to execute
# args - arguments to pass to the command
def command_safe(debug, cmd = :command, *args)
# Commands can overwrite start_time to skip time used in some kind of
setup.
# See count.rb for example.
@start_time = Time.now
# send is internal ruby method to call 'cmd' with *args
#(everything is a message, so this is just the formal semantics to
support that idiom)
translate_hbase_exceptions(*args) { send(cmd, *args) }
rescue => e
rootCause = e
while rootCause != nil && rootCause.respond_to?(:cause) &&
rootCause.cause != nil
rootCause = rootCause.cause
end
if @shell.interactive?
puts
puts "ERROR: #{rootCause}"
puts "Backtrace: #{rootCause.backtrace.join("\n ")}" if
debug
puts
puts help
puts
else
raise rootCause
end
ensure
# If end_time is not already set by the command, use current time.
@end_time ||= Time.now
formatter.output_str("Took %.4f seconds" % [@end_time - @start_time])
{code}
> Develop HBase shell command/tool to list table's region info through command
> line
> -
>
> Key: HBASE-14925
> URL: https://issues.apache.org/jira/browse/HBASE-14925
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Romil Choksi
>Assignee: Karan Mehta
> Attachments: HBASE-14925.002.patch, HBASE-14925.003.patch,
> HBASE-14925.patch
>
>
> I am going through the hbase shell commands to see if there is anything I can
> use to get all the regions info just for a particular table. I don’t see any
> such command that provides me that information.
> It would be better to have a command that provides region info, start key,
> end key etc taking a table name as the input parameter. This is available
> through HBase UI on clicking on a particular table's link
> A tool/shell command to get a list of regions for a table or all tables in a
> tabular structured output (that is machine readable)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959485#comment-15959485
]
Hadoop QA commented on HBASE-17872:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 32m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 16s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 41s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
7s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 33s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
61m 53s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 18s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 194m 30s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 329m 16s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
|
| | hadoop.hbase.client.TestAsyncTableAdminApi |
| Timed out junit tests |
org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862277/HBASE-17872.v2.patch |
| JIRA Issue | HBASE-17872 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 164342f47bc1 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d7e3116 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6353/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
[jira] [Commented] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959307#comment-15959307
]
Chia-Ping Tsai commented on HBASE-17872:
bq. Volatile reads are way more expensive than a local cache read. The
UNSAFE_AVAIL check is called frequently.
My bad. I overlook the Perf issue.
bq. We can not have volatile.
copy that.
I will fix it tomorrow. Thanks for the feedback. [~stack] and [~anoop.hbase]
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959268#comment-15959268
]
Anoop Sam John commented on HBASE-17872:
Thanks Stack.. My bad.. I did not really realize that the boolean is now
volatile. Only thought the change is private.. changing my +1. We can not
have volatile.
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959250#comment-15959250
]
stack commented on HBASE-17872:
---
Seems a pity making runtime pay the price of test-time convenience. Volatile
reads are way more expensive than a local cache read. The UNSAFE_AVAIL check is
called frequently.
Make UNSAFE_AVAIL a final static?
Then it becomes how to change a static final setting at test time. Reflection?
A delegating class that makes use of a subclass of ByteBufferUtils ? Could
move the test clutter of detectAvailabilityOfUnsafe and disableUnsafe out of
this vital class and into this new test class.
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17871) scan#setBatch(int) call leads wrong result of VerifyReplication
[
https://issues.apache.org/jira/browse/HBASE-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959237#comment-15959237
]
Hudson commented on HBASE-17871:
FAILURE: Integrated in Jenkins build HBase-1.4 #689 (See
[https://builds.apache.org/job/HBase-1.4/689/])
HBASE-17871 scan#setBatch(int) call leads wrong result of (tedyu: rev
a6e9de3a0edd8b1d9320e48d47f594c75b80f4e3)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
> scan#setBatch(int) call leads wrong result of VerifyReplication
> ---
>
> Key: HBASE-17871
> URL: https://issues.apache.org/jira/browse/HBASE-17871
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0
>Reporter: Tomu Tsuruhara
>Assignee: Tomu Tsuruhara
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after.png, beforethepatch.png,
> HBASE-17871.master.001.patch, HBASE-17871.master.002.patch,
> HBASE-17871.master.003.patch, HBASE-17871.master.003.patch,
> HBASE-17871.master.004.patch
>
>
> VerifyReplication tool printed weird logs.
> {noformat}
> 2017-04-03 23:30:50,252 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> CONTENT_DIFFERENT_ROWS, rowkey=a100193
> 2017-04-03 23:30:50,280 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> ONLY_IN_PEER_TABLE_ROWS, rowkey=a100193
> 2017-04-03 23:30:50,387 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> CONTENT_DIFFERENT_ROWS, rowkey=a100385
> 2017-04-03 23:30:50,414 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> ONLY_IN_PEER_TABLE_ROWS, rowkey=a100385
> 2017-04-03 23:30:50,480 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> CONTENT_DIFFERENT_ROWS, rowkey=a100532
> 2017-04-03 23:30:50,508 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> ONLY_IN_PEER_TABLE_ROWS, rowkey=a100532
> {noformat}
> Here, each bad rows were marked as both {{CONTENT_DIFFERENT_ROWS}} and
> {{ONLY_IN_PEER_TABLE_ROWS}}.
> This should never happen so I took a look at code and found scan.setBatch
> call.
> {code}
> @Override
> public void map(ImmutableBytesWritable row, final Result value,
> Context context)
> throws IOException {
> if (replicatedScanner == null) {
> ...
> final Scan scan = new Scan();
> scan.setBatch(batch);
> {code}
> As stated in HBASE-16376, {{scan#setBatch(int)}} call implicitly allows scan
> results to be partial.
> Since {{VerifyReplication}} is assuming each {{scanner.next()}} call returns
> entire row,
> partial results break compare logic.
> We should avoid setBatch call here.
> Thanks to RPC chunking (explained in this blog
> https://blogs.apache.org/hbase/entry/scan_improvements_in_hbase_1),
> it's safe and acceptable I think.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17878) java.lang.NoSuchMethodError: org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter when starting HBase with hbase.rootdir on S3
[
https://issues.apache.org/jira/browse/HBASE-17878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959236#comment-15959236
]
Zach York commented on HBASE-17878:
---
[~water] You should still be able to exclude it, but I think there might be a
better solution.
>From a quick grep (on branch-1.3):
grep -nr "jruby-complete" . --include "*pom.xml"
./hbase-shell/pom.xml:248: jruby-complete
./pom.xml:1558:jruby-complete
I'm not sure if any of the other modules depend on jruby-complete. Otherwise
you could likely move this dependency into the Hbase-shell module which I
believe will remove the conflict in hbase-server (where the Master code is).
One question. Where are you trying to include the hadoop-aws? Or are you just
including it on the classpath at runtime?
> java.lang.NoSuchMethodError:
> org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter
> when starting HBase with hbase.rootdir on S3
> -
>
> Key: HBASE-17878
> URL: https://issues.apache.org/jira/browse/HBASE-17878
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> When setting up HBASE-17437 (Support specifying a WAL directory outside of
> the root directory), we specify
> (1) hbase.rootdir on s3a
> (2) hbase.wal.dir on HDFS
> When starting HBase, the following exception is thrown:
> {code}
> Caused by: java.lang.NoSuchMethodError:
> org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter;
> at
> com.amazonaws.auth.internal.AWS4SignerUtils.(AWS4SignerUtils.java:26)
> at
> com.amazonaws.auth.internal.AWS4SignerRequestParams.(AWS4SignerRequestParams.java:85)
> at com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:184)
> at
> com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:709)
> at
> com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
> at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
> at
> com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
> at
> com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1107)
> at
> com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1070)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:232)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
> at org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:1007)
> at
> org.apache.hadoop.hbase.util.FSUtils.isValidWALRootDir(FSUtils.java:1050)
> at
> org.apache.hadoop.hbase.util.FSUtils.getWALRootDir(FSUtils.java:1032)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeFileSystem(HRegionServer.java:627)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:570)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:393)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2456)
> ... 5 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17886) Fix compatibility of ServerSideScanMetrics
[
https://issues.apache.org/jira/browse/HBASE-17886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959204#comment-15959204
]
stack commented on HBASE-17886:
---
Thanks for the clarification [~samarth.j...@gmail.com]
> Fix compatibility of ServerSideScanMetrics
> --
>
> Key: HBASE-17886
> URL: https://issues.apache.org/jira/browse/HBASE-17886
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0, 1.3.1
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: compatibility_check_1.3.1RC0.png, HBASE-17886.patch,
> HBASE-17886.v2.patch
>
>
> In HBASE-17716 we have changed the public field name in
> {{ServerSideScanMetrics}} which is IA.Public, which causes source
> compatibility issue, and we propose to fix it in this JIRA.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17886) Fix compatibility of ServerSideScanMetrics
[
https://issues.apache.org/jira/browse/HBASE-17886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959201#comment-15959201
]
Hudson commented on HBASE-17886:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2810 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2810/])
HBASE-17886 Fix compatibility of ServerSideScanMetrics (liyu: rev
d7e3116a1744057359ca48d94aa50d7fdf0db974)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/metrics/ServerSideScanMetrics.java
> Fix compatibility of ServerSideScanMetrics
> --
>
> Key: HBASE-17886
> URL: https://issues.apache.org/jira/browse/HBASE-17886
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0, 1.3.1
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: compatibility_check_1.3.1RC0.png, HBASE-17886.patch,
> HBASE-17886.v2.patch
>
>
> In HBASE-17716 we have changed the public field name in
> {{ServerSideScanMetrics}} which is IA.Public, which causes source
> compatibility issue, and we propose to fix it in this JIRA.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (HBASE-17878) java.lang.NoSuchMethodError: org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter when starting HBase with hbase.rootdir on
[
https://issues.apache.org/jira/browse/HBASE-17878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958966#comment-15958966
]
Xiang Li edited comment on HBASE-17878 at 4/6/17 4:09 PM:
--
Hi [~zyork]
Thanks for the comments!
jruby-complete is weird, as it packages a lot of classes of joda-time in its
jar
directly,(http://central.maven.org/maven2/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar),
that is, joda-time classes is not introduced by transitive (hbase depends on
jruby-complete, while jruby-complete depends on joda-time). So it can not be
excluded by Please correct me if I am wrong.
was (Author: water):
Hi [~zyork]
Thanks for the comments!
jruby-complete is weird, as it packages a lot of classes of joda-time in its
jar
directly,(http://central.maven.org/maven2/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar),
that is, joda-time classes is not introduced by transitive (hbase depends on
jruby-complete, while jruby-complete depends on joda-time). So it can not be
excluded by Please correct me if I am wrong.
> java.lang.NoSuchMethodError:
> org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter
> when starting HBase with hbase.rootdir on S3
> -
>
> Key: HBASE-17878
> URL: https://issues.apache.org/jira/browse/HBASE-17878
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> When setting up HBASE-17437 (Support specifying a WAL directory outside of
> the root directory), we specify
> (1) hbase.rootdir on s3a
> (2) hbase.wal.dir on HDFS
> When starting HBase, the following exception is thrown:
> {code}
> Caused by: java.lang.NoSuchMethodError:
> org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter;
> at
> com.amazonaws.auth.internal.AWS4SignerUtils.(AWS4SignerUtils.java:26)
> at
> com.amazonaws.auth.internal.AWS4SignerRequestParams.(AWS4SignerRequestParams.java:85)
> at com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:184)
> at
> com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:709)
> at
> com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
> at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
> at
> com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
> at
> com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1107)
> at
> com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1070)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:232)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
> at org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:1007)
> at
> org.apache.hadoop.hbase.util.FSUtils.isValidWALRootDir(FSUtils.java:1050)
> at
> org.apache.hadoop.hbase.util.FSUtils.getWALRootDir(FSUtils.java:1032)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeFileSystem(HRegionServer.java:627)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:570)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:393)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2456)
> ... 5 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17886) Fix compatibility of ServerSideScanMetrics
[
https://issues.apache.org/jira/browse/HBASE-17886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959143#comment-15959143
]
Samarth Jain commented on HBASE-17886:
--
We don't have to have HBASE-17716 in 1.3. We can use the workaround of using
the metric names directly with the caveat that we would have to look into hbase
source code to get hold of them.
> Fix compatibility of ServerSideScanMetrics
> --
>
> Key: HBASE-17886
> URL: https://issues.apache.org/jira/browse/HBASE-17886
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0, 1.3.1
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: compatibility_check_1.3.1RC0.png, HBASE-17886.patch,
> HBASE-17886.v2.patch
>
>
> In HBASE-17716 we have changed the public field name in
> {{ServerSideScanMetrics}} which is IA.Public, which causes source
> compatibility issue, and we propose to fix it in this JIRA.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16477) Remove Writable interface and related code from WALEdit/WALKey
[ https://issues.apache.org/jira/browse/HBASE-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959122#comment-15959122 ] Chia-Ping Tsai commented on HBASE-16477: +1 > Remove Writable interface and related code from WALEdit/WALKey > -- > > Key: HBASE-16477 > URL: https://issues.apache.org/jira/browse/HBASE-16477 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0 > > Attachments: hbase-16477_v1.patch, hbase-16477-v2.patch > > > Writables are gone, and SequenceFile based WAL will be gone in parent. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959127#comment-15959127 ] Anoop Sam John commented on HBASE-16438: We were never doing this way of writing seqId bytes onto MSLAB chunks (Data chunks).. we keep it as state. And FYI, when we flush cells to HFiles, then also we write seqId but as a VLong. > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: > HBASE-16438_10_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_11_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16477) Remove Writable interface and related code from WALEdit/WALKey
[
https://issues.apache.org/jira/browse/HBASE-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959116#comment-15959116
]
Hadoop QA commented on HBASE-16477:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 24s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 108m 9s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 150m 53s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862282/hbase-16477-v2.patch |
| JIRA Issue | HBASE-16477 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 17839df71b5d 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d7e3116 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6354/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6354/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Remove Writable interface and related code from WALEdit/WALKey
> --
>
> Key: HBASE-16477
> URL: https://issues.apache.org/jira/browse/HBASE-16477
> Project: HBase
> Issue Type: Sub-task
> Components: wal
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0
>
> Attachments: hbase-16477_v1.patch,
[jira] [Commented] (HBASE-16469) Several log refactoring/improvement suggestions
[
https://issues.apache.org/jira/browse/HBASE-16469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959075#comment-15959075
]
Nemo Chen commented on HBASE-16469:
---
Hi [~busbey], I uploaded the patch with correction. Somehow the test failed due
to time out. After reviewing the test results, I think it is not due to this
patch. How do you think we can proceed from here? Thanks!
> Several log refactoring/improvement suggestions
> ---
>
> Key: HBASE-16469
> URL: https://issues.apache.org/jira/browse/HBASE-16469
> Project: HBase
> Issue Type: Improvement
> Components: Operability
>Affects Versions: 1.2.5
>Reporter: Nemo Chen
>Assignee: Nemo Chen
> Labels: easyfix, easytest
> Fix For: 2.0.0, 1.5.0
>
> Attachments: HBASE-16469.master.001.patch
>
>
> *method invocation replaced by variable*
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java
> {code}String name = regionInfo.getRegionNameAsString();{code}
> {code}LOG.warn("Can't close region: was already closed during close(): " +
> regionInfo.getRegionNameAsString()); {code}
> In the above two examples, the method invocations are assigned to the
> variables before the logging code. These method invocations should be
> replaced by variables in case of simplicity and readability
>
> *method invocation in return statement*
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> {code}
> public String toString() {
> return getRegionInfo().getRegionNameAsString();
> }
> {code}
> {code}
> LOG.debug("Region " + getRegionInfo().getRegionNameAsString()
> + " is not mergeable because it is closing or closed");
> {code}
> {code}
> LOG.debug("Region " + getRegionInfo().getRegionNameAsString()
> + " is not mergeable because it has references");
> {code}
> {code}
> LOG.info("Running close preflush of " +
> getRegionInfo().getRegionNameAsString());
> {code}
> In these above examples, the "getRegionInfo().getRegionNameAsString())" is
> the return statement of method "toString" in the same class. They should be
> replaced with “this” in case of simplicity and readability.
>
> *check the logged variable if it is null*
> hbase-it/src/test/java/org/apache/hadoop/hbase/HBaseClusterManager.java
> {code}
> if ((sshUserName != null && sshUserName.length() > 0) ||
> (sshOptions != null && sshOptions.length() > 0)) {
> LOG.info("Running with SSH user [" + sshUserName + "] and options [" +
> sshOptions + "]");
> }
> {code}
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
> {code}
> if ((regionState == null && latestState != null)
> || (regionState != null && latestState == null)
> || (regionState != null && latestState != null
> && latestState.getState() != regionState.getState())) {
> LOG.warn("Region state changed from " + regionState + " to "
> + latestState + ", while acquiring lock");
> }
> {code}
> In the above example, the logging variable could null at run time. It is a
> bad practice to include null variables inside logs.
>
> *variable in byte printed directly*
> hbase-server/src/test/java/org/apache/hadoop/hbase/util/MultiThreadedUpdater.java
> {code}
> byte[] rowKey = dataGenerator.getDeterministicUniqueKey(rowKeyBase);
> {code}
> {code}
> LOG.error("Failed to update the row with key = [" + rowKey
> + "], since we could not get the original row");
> {code}
> rowKey should be printed as Bytes.toString(rowKey).
>
> *object toString contain mi*
> The toString method returns getServerName(), so the "server.getServerName()"
> should be replaced with "server" in case of simplicity and readability.
> hbase-client/src/main/java/org/apache/hadoop/hbase/client/PreemptiveFastFailInterceptor.java
> {code}
> LOG.info("Clearing out PFFE for server " + server.getServerName());
> return getServerName();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17879) Avoid NPE in snapshot.jsp when accessing without any request parameter
[ https://issues.apache.org/jira/browse/HBASE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959066#comment-15959066 ] Ted Yu commented on HBASE-17879: lgtm > Avoid NPE in snapshot.jsp when accessing without any request parameter > -- > > Key: HBASE-17879 > URL: https://issues.apache.org/jira/browse/HBASE-17879 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 1.3.0 >Reporter: Abhishek Kumar >Priority: Trivial > Attachments: HBASE-17879.patch > > > When accessing snapshot jsp with below url inadvertently NPE comes in UI: > Requested URL: > http://:/snapshot.jsp? > Response: > java.lang.NullPointerException > at > org.apache.hadoop.hbase.generated.master.snapshot_jsp._jspService(snapshot_jsp.java:66) > at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17879) Avoid NPE in snapshot.jsp when accessing without any request parameter
[ https://issues.apache.org/jira/browse/HBASE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Kumar updated HBASE-17879: --- Attachment: HBASE-17879.patch A simple patch for null check, pls take a look. > Avoid NPE in snapshot.jsp when accessing without any request parameter > -- > > Key: HBASE-17879 > URL: https://issues.apache.org/jira/browse/HBASE-17879 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 1.3.0 >Reporter: Abhishek Kumar >Priority: Trivial > Attachments: HBASE-17879.patch > > > When accessing snapshot jsp with below url inadvertently NPE comes in UI: > Requested URL: > http://:/snapshot.jsp? > Response: > java.lang.NullPointerException > at > org.apache.hadoop.hbase.generated.master.snapshot_jsp._jspService(snapshot_jsp.java:66) > at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959036#comment-15959036 ] Anastasia Braginsky commented on HBASE-16438: - Hey once again! I got now the idea why the seqID shouldn't probably be in the data-chunk's byte-buffer upon Cells creation. If upon snapshot the BBKV is streamed out directly to the hfileblocks so seqIDs must move to disk as well, then this is bad enough to place it in the CellChunkMap and probably decrease its performance. So if this is the issue I agree to accommodate the seqIDs in the CellChunkMap. Please let me know what do you think... > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: > HBASE-16438_10_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_11_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16459) Remove unused hbase shell --format option
[
https://issues.apache.org/jira/browse/HBASE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959027#comment-15959027
]
Hadoop QA commented on HBASE-16459:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 7s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 7s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 13s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 55s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12824905/HBASE-16459_v2.patch |
| JIRA Issue | HBASE-16459 |
| Optional Tests | asflicense rubocop ruby_lint |
| uname | Linux 1db470d99f88 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ec5188d |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6355/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Remove unused hbase shell --format option
> -
>
> Key: HBASE-16459
> URL: https://issues.apache.org/jira/browse/HBASE-16459
> Project: HBase
> Issue Type: Task
> Components: shell
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-16459.patch, HBASE-16459_v2.patch
>
>
> The usage information when running {{hbase shell}} refers to a formatter
> option that has yet to implemented in over 8 years and which will ostensibly
> never be implemented. As such, let's cleanup the [help
> message|https://github.com/apache/hbase/blob/master/bin/hirb.rb#L57-L59] and
> remove some extraneous lines of code from
> {{[hirb.rb|https://github.com/apache/hbase/blob/master/bin/hirb.rb#L74-L83]}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16459) Remove unused hbase shell --format option
[
https://issues.apache.org/jira/browse/HBASE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958983#comment-15958983
]
Chia-Ping Tsai commented on HBASE-16459:
+1
> Remove unused hbase shell --format option
> -
>
> Key: HBASE-16459
> URL: https://issues.apache.org/jira/browse/HBASE-16459
> Project: HBase
> Issue Type: Task
> Components: shell
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-16459.patch, HBASE-16459_v2.patch
>
>
> The usage information when running {{hbase shell}} refers to a formatter
> option that has yet to implemented in over 8 years and which will ostensibly
> never be implemented. As such, let's cleanup the [help
> message|https://github.com/apache/hbase/blob/master/bin/hirb.rb#L57-L59] and
> remove some extraneous lines of code from
> {{[hirb.rb|https://github.com/apache/hbase/blob/master/bin/hirb.rb#L74-L83]}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17871) scan#setBatch(int) call leads wrong result of VerifyReplication
[
https://issues.apache.org/jira/browse/HBASE-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-17871:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 1.4.0
2.0.0
Status: Resolved (was: Patch Available)
Thanks for the patch, Tomu.
Thanks for the review, Phil.
> scan#setBatch(int) call leads wrong result of VerifyReplication
> ---
>
> Key: HBASE-17871
> URL: https://issues.apache.org/jira/browse/HBASE-17871
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0
>Reporter: Tomu Tsuruhara
>Assignee: Tomu Tsuruhara
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: after.png, beforethepatch.png,
> HBASE-17871.master.001.patch, HBASE-17871.master.002.patch,
> HBASE-17871.master.003.patch, HBASE-17871.master.003.patch,
> HBASE-17871.master.004.patch
>
>
> VerifyReplication tool printed weird logs.
> {noformat}
> 2017-04-03 23:30:50,252 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> CONTENT_DIFFERENT_ROWS, rowkey=a100193
> 2017-04-03 23:30:50,280 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> ONLY_IN_PEER_TABLE_ROWS, rowkey=a100193
> 2017-04-03 23:30:50,387 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> CONTENT_DIFFERENT_ROWS, rowkey=a100385
> 2017-04-03 23:30:50,414 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> ONLY_IN_PEER_TABLE_ROWS, rowkey=a100385
> 2017-04-03 23:30:50,480 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> CONTENT_DIFFERENT_ROWS, rowkey=a100532
> 2017-04-03 23:30:50,508 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> ONLY_IN_PEER_TABLE_ROWS, rowkey=a100532
> {noformat}
> Here, each bad rows were marked as both {{CONTENT_DIFFERENT_ROWS}} and
> {{ONLY_IN_PEER_TABLE_ROWS}}.
> This should never happen so I took a look at code and found scan.setBatch
> call.
> {code}
> @Override
> public void map(ImmutableBytesWritable row, final Result value,
> Context context)
> throws IOException {
> if (replicatedScanner == null) {
> ...
> final Scan scan = new Scan();
> scan.setBatch(batch);
> {code}
> As stated in HBASE-16376, {{scan#setBatch(int)}} call implicitly allows scan
> results to be partial.
> Since {{VerifyReplication}} is assuming each {{scanner.next()}} call returns
> entire row,
> partial results break compare logic.
> We should avoid setBatch call here.
> Thanks to RPC chunking (explained in this blog
> https://blogs.apache.org/hbase/entry/scan_improvements_in_hbase_1),
> it's safe and acceptable I think.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17878) java.lang.NoSuchMethodError: org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter when starting HBase with hbase.rootdir on S3
[
https://issues.apache.org/jira/browse/HBASE-17878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958966#comment-15958966
]
Xiang Li commented on HBASE-17878:
--
Hi [~zyork]
Thanks for the comments!
jruby-complete is weird, as it packages a lot of classes of joda-time in its
jar
directly,(http://central.maven.org/maven2/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar),
that is, joda-time classes is not introduced by transitive (hbase depends on
jruby-complete, while jruby-complete depends on joda-time). So it can not be
excluded by Please correct me if I am wrong.
> java.lang.NoSuchMethodError:
> org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter
> when starting HBase with hbase.rootdir on S3
> -
>
> Key: HBASE-17878
> URL: https://issues.apache.org/jira/browse/HBASE-17878
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> When setting up HBASE-17437 (Support specifying a WAL directory outside of
> the root directory), we specify
> (1) hbase.rootdir on s3a
> (2) hbase.wal.dir on HDFS
> When starting HBase, the following exception is thrown:
> {code}
> Caused by: java.lang.NoSuchMethodError:
> org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter;
> at
> com.amazonaws.auth.internal.AWS4SignerUtils.(AWS4SignerUtils.java:26)
> at
> com.amazonaws.auth.internal.AWS4SignerRequestParams.(AWS4SignerRequestParams.java:85)
> at com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:184)
> at
> com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:709)
> at
> com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
> at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
> at
> com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
> at
> com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1107)
> at
> com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1070)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:232)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
> at org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:1007)
> at
> org.apache.hadoop.hbase.util.FSUtils.isValidWALRootDir(FSUtils.java:1050)
> at
> org.apache.hadoop.hbase.util.FSUtils.getWALRootDir(FSUtils.java:1032)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeFileSystem(HRegionServer.java:627)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:570)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:393)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2456)
> ... 5 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958968#comment-15958968
]
Ted Yu commented on HBASE-17872:
+1, pending QA.
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17871) scan#setBatch(int) call leads wrong result of VerifyReplication
[
https://issues.apache.org/jira/browse/HBASE-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958955#comment-15958955
]
Hadoop QA commented on HBASE-17871:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 33s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 39s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 99m 23s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 137m 34s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862274/HBASE-17871.master.004.patch
|
| JIRA Issue | HBASE-17871 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 9a7b3bf649c3 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d7e3116 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6352/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6352/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> scan#setBatch(int) call leads wrong result of VerifyReplication
> ---
>
> Key: HBASE-17871
> URL: https://issues.apache.org/jira/browse/HBASE-17871
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0
>Reporter:
[jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
[ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958938#comment-15958938 ] Anastasia Braginsky commented on HBASE-16438: - Hi Guys! I really apologize for this late comment. If you haven't yet committed, please consider this comment. I just had time to went over the last version of the patch once again. I left some comment in the RB, please take a look there, but most of them are not very important. The very important thing is that I have just realized that seqID is not written anywhere in the ByteBuffer! Please correct me if I am wrong... I was sure (that even before this patch) the serialization of the Cell included writing of the seqID to the ByteBuffer and we are talking only about CellChunkMap. Obviously, if seqID is not written in MSLAB Chunk upon cell creation (when the cell is still indexed with CSLM) it can not be added there later when we have the transfer to CellChunkMap. So the only two options are (1) to add seqID in the index-chunk when CellChunkMap is created (in-memory-flush) or (2) to write seqID into the data-chunk when a cell is added to the memstore IN ADDITION to seqID continue existing in the Cell Object. I strongly suggest to do the second option. First, if seqID remains in the Cell object, there is completely no new work being done in the DefaultMemStore or active segment. The seqID in the data chunk are not going to be accessed by anyone except CellChunkMap. Second, addition of another 8 bytes to key+value, which is anyway big sized is not significant. While adding those 8 bytes to cell-representation in the ChunkMap is increasing the metadata per cell almost twice. I really appreciate all the hard work that you are doing! You were talking about this issue even before, I am sorry I didn't understand that. But I think this task is not complete before we finalize this seqID issue... > Create a cell type so that chunk id is embedded in it > - > > Key: HBASE-16438 > URL: https://issues.apache.org/jira/browse/HBASE-16438 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: > HBASE-16438_10_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_11_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, > HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch, > HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch, > MemstoreChunkCell_trunk.patch > > > For CellChunkMap we may need a cell such that the chunk out of which it was > created, the id of the chunk be embedded in it so that when doing flattening > we can use the chunk id as a meta data. More details will follow once the > initial tasks are completed. > Why we need to embed the chunkid in the Cell is described by [~anastas] in > this remark over in parent issue > https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16477) Remove Writable interface and related code from WALEdit/WALKey
[ https://issues.apache.org/jira/browse/HBASE-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-16477: -- Status: Patch Available (was: Open) > Remove Writable interface and related code from WALEdit/WALKey > -- > > Key: HBASE-16477 > URL: https://issues.apache.org/jira/browse/HBASE-16477 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0 > > Attachments: hbase-16477_v1.patch, hbase-16477-v2.patch > > > Writables are gone, and SequenceFile based WAL will be gone in parent. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16477) Remove Writable interface and related code from WALEdit/WALKey
[ https://issues.apache.org/jira/browse/HBASE-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-16477: -- Attachment: hbase-16477-v2.patch Thanks, I've removed KVCompression as well. Let's get this in. > Remove Writable interface and related code from WALEdit/WALKey > -- > > Key: HBASE-16477 > URL: https://issues.apache.org/jira/browse/HBASE-16477 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0 > > Attachments: hbase-16477_v1.patch, hbase-16477-v2.patch > > > Writables are gone, and SequenceFile based WAL will be gone in parent. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17886) Fix compatibility of ServerSideScanMetrics
[
https://issues.apache.org/jira/browse/HBASE-17886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958874#comment-15958874
]
Hudson commented on HBASE-17886:
SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #141 (See
[https://builds.apache.org/job/HBase-1.3-JDK7/141/])
HBASE-17886 Fix compatibility of ServerSideScanMetrics (liyu: rev
627f0796af07f176c6df1501529c39b7bccd8044)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/metrics/ServerSideScanMetrics.java
> Fix compatibility of ServerSideScanMetrics
> --
>
> Key: HBASE-17886
> URL: https://issues.apache.org/jira/browse/HBASE-17886
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0, 1.3.1
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: compatibility_check_1.3.1RC0.png, HBASE-17886.patch,
> HBASE-17886.v2.patch
>
>
> In HBASE-17716 we have changed the public field name in
> {{ServerSideScanMetrics}} which is IA.Public, which causes source
> compatibility issue, and we propose to fix it in this JIRA.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-17872:
---
Status: Patch Available (was: Open)
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-17872:
---
Attachment: HBASE-17872.v2.patch
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-17872:
---
Status: Open (was: Patch Available)
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17872) The MSLABImpl generates the invaild cells when unsafe is not availble
[
https://issues.apache.org/jira/browse/HBASE-17872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958849#comment-15958849
]
Chia-Ping Tsai commented on HBASE-17872:
bq. I suggest changing the above method to detectAvailabilityOfUnsafe().
Good point. rename it.
bq. Can TestFromClientSide3WoUnsafe be structured in such a way that the two
helper methods can be declared package private ?
Copy that. I make they in the same package.
See v2 patch. Thanks for the feedback. [~yuzhih...@gmail.com]
> The MSLABImpl generates the invaild cells when unsafe is not availble
> -
>
> Key: HBASE-17872
> URL: https://issues.apache.org/jira/browse/HBASE-17872
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17872.v0.patch, HBASE-17872.v1.patch,
> HBASE-17872.v2.patch
>
>
> We will get the wrong position of buffer in multithreaded environment, so the
> method makes the invalid cell in MSLAB.
> {noformat}
> public static int copyFromBufferToBuffer(ByteBuffer in, ByteBuffer out, int
> sourceOffset,
> int destinationOffset, int length) {
> if (in.hasArray() && out.hasArray()) {
> // ...
> } else if (UNSAFE_AVAIL) {
> // ...
> } else {
> int outOldPos = out.position();
> out.position(destinationOffset);
> ByteBuffer inDup = in.duplicate();
> inDup.position(sourceOffset).limit(sourceOffset + length);
> out.put(inDup);
> out.position(outOldPos);
> }
> return destinationOffset + length;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-14614) Procedure v2: Core Assignment Manager
[
https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958847#comment-15958847
]
Hadoop QA commented on HBASE-14614:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 81 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 49s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 4s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 25m
35s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
8s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 57s
{color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 32s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 25m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
8s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 130 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 14s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m
40s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 57s
{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 27s
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 47s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 15s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 18s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 121m 52s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 34s
{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
41s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} |
[jira] [Commented] (HBASE-17886) Fix compatibility of ServerSideScanMetrics
[
https://issues.apache.org/jira/browse/HBASE-17886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958836#comment-15958836
]
Hudson commented on HBASE-17886:
FAILURE: Integrated in Jenkins build HBase-1.4 #688 (See
[https://builds.apache.org/job/HBase-1.4/688/])
HBASE-17886 Fix compatibility of ServerSideScanMetrics (liyu: rev
96890d64d41a3263a54becabd0c0079d67a1a8b7)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/metrics/ServerSideScanMetrics.java
> Fix compatibility of ServerSideScanMetrics
> --
>
> Key: HBASE-17886
> URL: https://issues.apache.org/jira/browse/HBASE-17886
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0, 1.3.1
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: compatibility_check_1.3.1RC0.png, HBASE-17886.patch,
> HBASE-17886.v2.patch
>
>
> In HBASE-17716 we have changed the public field name in
> {{ServerSideScanMetrics}} which is IA.Public, which causes source
> compatibility issue, and we propose to fix it in this JIRA.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17886) Fix compatibility of ServerSideScanMetrics
[
https://issues.apache.org/jira/browse/HBASE-17886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958814#comment-15958814
]
Hudson commented on HBASE-17886:
SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #152 (See
[https://builds.apache.org/job/HBase-1.3-JDK8/152/])
HBASE-17886 Fix compatibility of ServerSideScanMetrics (liyu: rev
627f0796af07f176c6df1501529c39b7bccd8044)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/metrics/ServerSideScanMetrics.java
> Fix compatibility of ServerSideScanMetrics
> --
>
> Key: HBASE-17886
> URL: https://issues.apache.org/jira/browse/HBASE-17886
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0, 1.3.1
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: compatibility_check_1.3.1RC0.png, HBASE-17886.patch,
> HBASE-17886.v2.patch
>
>
> In HBASE-17716 we have changed the public field name in
> {{ServerSideScanMetrics}} which is IA.Public, which causes source
> compatibility issue, and we propose to fix it in this JIRA.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16555) JUnit dependency not scoped as test
[
https://issues.apache.org/jira/browse/HBASE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958807#comment-15958807
]
Hadoop QA commented on HBASE-16555:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 32s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 55s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 58s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 56s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 112m 22s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 155m 32s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12839553/HBASE-16555.master.001.patch
|
| JIRA Issue | HBASE-16555 |
| Optional Tests | asflicense javac javadoc unit xml compile |
| uname | Linux e0c63c93717a 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d7e3116 |
| Default Java | 1.8.0_121 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6351/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6351/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6351/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> JUnit dependency not scoped as test
> ---
>
> Key: HBASE-16555
> URL: https://issues.apache.org/jira/browse/HBASE-16555
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.2.2
>Reporter: Robert G Duncan
>Assignee: Jan Hentschel
> Attachments: HBASE-16555.master.001.patch
>
>
> *Issue*
> JUnit is included as a compile scope dependency. This increases the size of
> dependent project shaded/Uber JARs unnecessarily.
> *Actual Entry*
> {code:xml}
>
> junit
> junit
> ${junit.version}
>
> {code}
> *Expected Entry*
> {code:xml}
>
> junit
> junit
> ${junit.version}
> test
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17849) PE tool random read is not totally random
[
https://issues.apache.org/jira/browse/HBASE-17849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958773#comment-15958773
]
Hadoop QA commented on HBASE-17849:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 54s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 106m 50s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 147m 49s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862259/HBASE-17849_1.patch |
| JIRA Issue | HBASE-17849 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 11566c2f4bbe 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 17737b2 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6350/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6350/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> PE tool random read is not totally random
> -
>
> Key: HBASE-17849
> URL: https://issues.apache.org/jira/browse/HBASE-17849
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-17849_1.patch,
[jira] [Commented] (HBASE-17871) scan#setBatch(int) call leads wrong result of VerifyReplication
[
https://issues.apache.org/jira/browse/HBASE-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958766#comment-15958766
]
Tomu Tsuruhara commented on HBASE-17871:
[~yangzhe1991] Thanks! Yes, the patch can be applied to the branch-1 cleanly.
> scan#setBatch(int) call leads wrong result of VerifyReplication
> ---
>
> Key: HBASE-17871
> URL: https://issues.apache.org/jira/browse/HBASE-17871
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0
>Reporter: Tomu Tsuruhara
>Assignee: Tomu Tsuruhara
>Priority: Minor
> Attachments: after.png, beforethepatch.png,
> HBASE-17871.master.001.patch, HBASE-17871.master.002.patch,
> HBASE-17871.master.003.patch, HBASE-17871.master.003.patch,
> HBASE-17871.master.004.patch
>
>
> VerifyReplication tool printed weird logs.
> {noformat}
> 2017-04-03 23:30:50,252 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> CONTENT_DIFFERENT_ROWS, rowkey=a100193
> 2017-04-03 23:30:50,280 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> ONLY_IN_PEER_TABLE_ROWS, rowkey=a100193
> 2017-04-03 23:30:50,387 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> CONTENT_DIFFERENT_ROWS, rowkey=a100385
> 2017-04-03 23:30:50,414 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> ONLY_IN_PEER_TABLE_ROWS, rowkey=a100385
> 2017-04-03 23:30:50,480 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> CONTENT_DIFFERENT_ROWS, rowkey=a100532
> 2017-04-03 23:30:50,508 ERROR [main]
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication:
> ONLY_IN_PEER_TABLE_ROWS, rowkey=a100532
> {noformat}
> Here, each bad rows were marked as both {{CONTENT_DIFFERENT_ROWS}} and
> {{ONLY_IN_PEER_TABLE_ROWS}}.
> This should never happen so I took a look at code and found scan.setBatch
> call.
> {code}
> @Override
> public void map(ImmutableBytesWritable row, final Result value,
> Context context)
> throws IOException {
> if (replicatedScanner == null) {
> ...
> final Scan scan = new Scan();
> scan.setBatch(batch);
> {code}
> As stated in HBASE-16376, {{scan#setBatch(int)}} call implicitly allows scan
> results to be partial.
> Since {{VerifyReplication}} is assuming each {{scanner.next()}} call returns
> entire row,
> partial results break compare logic.
> We should avoid setBatch call here.
> Thanks to RPC chunking (explained in this blog
> https://blogs.apache.org/hbase/entry/scan_improvements_in_hbase_1),
> it's safe and acceptable I think.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
