[jira] [Commented] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472924#comment-16472924
]
Hadoop QA commented on HBASE-20567:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
19s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
3s{color} | {color:red} hbase-server: The patch generated 5 new + 238 unchanged
- 3 fixed = 243 total (was 241) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
23s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 26s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}178m
39s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}224m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20567 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12923125/HBASE-20567.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 97d7f2a21d90 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11
12:16:42 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 021f66d11d |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/12797/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12797/testReport/ |
| Max.
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472917#comment-16472917 ] huaxiang sun commented on HBASE-20411: -- Thanks [~stack]. You should be the assigned one as you contributed most of the ideas/codes, :) > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: huaxiang sun >Priority: Major > Fix For: 2.0.1 > > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > HBASE-20411.branch-2.0.011.patch, HBASE-20411.branch-2.0.012.patch, > HBASE-20411.branch-2.0.013.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20574) prevRowLock seems not working
[
https://issues.apache.org/jira/browse/HBASE-20574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Zheng updated HBASE-20574:
-
Description:
In HRegion.java, getRowLockInternal using prevRowLock to check whether this
lock is obtained before, but it may not work as imagine.
Because this parameter passed as the last lock the thread obtained, and the
code always be like this:
{code:java}
for (byte[] rows: rowsToLock) {
rowLock = getRowLockInternal(row, prevRowLock);
if (rowLock != prevRowLock){
acquiredRowLocks.add(rowLock);
prevRowLock = rowLock;
}
}{code}
prevRowLock will never equal to the rowLockContext's lock in
getRowLockInternal. I think this parameter is redundant.
was:
In HRegion.java, getRowLockInternal using prevRowLock to check whether this
lock is obtained before, but it may not work as imagine.
Because this parameter passed as the last lock the thread obtained, and the
code always be like this:
for (byte[] rows: rowsToLock) {
rowLock = getRowLockInternal(row, prevRowLock);
if (rowLock != prevRowLock) {
acquiredRowLocks.add(rowLock);
prevRowLock = rowLock;
}
}
prevRowLock will never equal to the rowLockContext's lock in
getRowLockInternal. I think this parameter is redundant.
> prevRowLock seems not working
> -
>
> Key: HBASE-20574
> URL: https://issues.apache.org/jira/browse/HBASE-20574
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Yi Zheng
>Priority: Major
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> In HRegion.java, getRowLockInternal using prevRowLock to check whether this
> lock is obtained before, but it may not work as imagine.
> Because this parameter passed as the last lock the thread obtained, and the
> code always be like this:
> {code:java}
> for (byte[] rows: rowsToLock) {
> rowLock = getRowLockInternal(row, prevRowLock);
> if (rowLock != prevRowLock){
> acquiredRowLocks.add(rowLock);
> prevRowLock = rowLock;
> }
> }{code}
> prevRowLock will never equal to the rowLockContext's lock in
> getRowLockInternal. I think this parameter is redundant.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20574) prevRowLock seems not working
[
https://issues.apache.org/jira/browse/HBASE-20574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Zheng updated HBASE-20574:
-
Description:
In HRegion.java, getRowLockInternal using prevRowLock to check whether this
lock is obtained before, but it may not work as imagine.
Because this parameter passed as the last lock the thread obtained, and the
code always be like this:
for (byte[] rows: rowsToLock) {
rowLock = getRowLockInternal(row, prevRowLock);
if (rowLock != prevRowLock) {
acquiredRowLocks.add(rowLock);
prevRowLock = rowLock;
}
}
prevRowLock will never equal to the rowLockContext's lock in
getRowLockInternal. I think this parameter is redundant.
was:
In HRegion.java, getRowLockInternal using prevRowLock to check whether this
lock is obtained before, but it may not work as imagine.
Because this parameter passed as the last lock the thread obtained, and the
code always be like this:
for (byte[] rows: rowsToLock) {
rowLock = getRowLockInternal(row, prevRowLock);
if (rowLock != prevRowLock) {
acquiredRowLocks.add(rowLock);
prevRowLock = rowLock;
}
}
prevRowLock will never equal to the rowLockContext's lock in
getRowLockInternal. I think this parameter is redundant.
> prevRowLock seems not working
> -
>
> Key: HBASE-20574
> URL: https://issues.apache.org/jira/browse/HBASE-20574
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Yi Zheng
>Priority: Major
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> In HRegion.java, getRowLockInternal using prevRowLock to check whether this
> lock is obtained before, but it may not work as imagine.
> Because this parameter passed as the last lock the thread obtained, and the
> code always be like this:
> for (byte[] rows: rowsToLock) {
> rowLock = getRowLockInternal(row, prevRowLock);
> if (rowLock != prevRowLock) {
> acquiredRowLocks.add(rowLock);
> prevRowLock = rowLock;
> }
> }
> prevRowLock will never equal to the rowLockContext's lock in
> getRowLockInternal. I think this parameter is redundant.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20574) prevRowLock seems not working
Yi Zheng created HBASE-20574:
Summary: prevRowLock seems not working
Key: HBASE-20574
URL: https://issues.apache.org/jira/browse/HBASE-20574
Project: HBase
Issue Type: Improvement
Components: regionserver
Reporter: Yi Zheng
In HRegion.java, getRowLockInternal using prevRowLock to check whether this
lock is obtained before, but it may not work as imagine.
Because this parameter passed as the last lock the thread obtained, and the
code always be like this:
for (byte[] rows: rowsToLock) {
rowLock = getRowLockInternal(row, prevRowLock);
if (rowLock != prevRowLock) {
acquiredRowLocks.add(rowLock);
prevRowLock = rowLock;
}
}
prevRowLock will never equal to the rowLockContext's lock in
getRowLockInternal. I think this parameter is redundant.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20572) HBase2 does not compile against Hadoop3 after HADOOP-10768
[ https://issues.apache.org/jira/browse/HBASE-20572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472908#comment-16472908 ] Wei-Chiu Chuang commented on HBASE-20572: - I should probably use reflection to make the fix backward compatible. But I'll worry about that after HADOOP-10768 is done. > HBase2 does not compile against Hadoop3 after HADOOP-10768 > -- > > Key: HBASE-20572 > URL: https://issues.apache.org/jira/browse/HBASE-20572 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Blocker > Attachments: HBASE-20572.001.patch, HBASE-20572.002.patch > > > This is a non-issue yet, because HADOOP-10768 is not committed in Hadoop > codebase yet. But I am reviewing that jira and trying to get it in. > FanOutOneBlockAsyncDFSOutput does not compile because protobuf message > CipherOptionProto was moved from hdfs.proto (in hadoop-hdfs) to > Security.proto (in hadoop-common). > File this jiras as a placeholder. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20556) Backport HBASE-16490 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472898#comment-16472898 ] Tak Lon (Stephen) Wu commented on HBASE-20556: -- I looked the [build|https://builds.apache.org/job/PreCommit-HBASE-Build/12782/console] and didn't find unit tests (test classes) failed. > Backport HBASE-16490 to branch-1 > > > Key: HBASE-20556 > URL: https://issues.apache.org/jira/browse/HBASE-20556 > Project: HBase > Issue Type: Sub-task > Components: HFile, snapshots >Affects Versions: 1.4.4, 1.4.5 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Major > Attachments: HBASE-20556.branch-1.002.patch > > > As part of HBASE-20555, HBASE-16490 is the first patch that is needed for > backporting HBASE-18083 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20572) HBase2 does not compile against Hadoop3 after HADOOP-10768
[ https://issues.apache.org/jira/browse/HBASE-20572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HBASE-20572: Attachment: HBASE-20572.002.patch > HBase2 does not compile against Hadoop3 after HADOOP-10768 > -- > > Key: HBASE-20572 > URL: https://issues.apache.org/jira/browse/HBASE-20572 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Blocker > Attachments: HBASE-20572.001.patch, HBASE-20572.002.patch > > > This is a non-issue yet, because HADOOP-10768 is not committed in Hadoop > codebase yet. But I am reviewing that jira and trying to get it in. > FanOutOneBlockAsyncDFSOutput does not compile because protobuf message > CipherOptionProto was moved from hdfs.proto (in hadoop-hdfs) to > Security.proto (in hadoop-common). > File this jiras as a placeholder. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472885#comment-16472885
]
Umesh Agashe commented on HBASE-20552:
--
I think its real problem in the code. Working on repro and the patch.
> HBase RegionServer was shutdown due to UnexpectedStateException
> ---
>
> Key: HBASE-20552
> URL: https://issues.apache.org/jira/browse/HBASE-20552
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Umesh Agashe
>Priority: Critical
> Attachments:
> 102143-master-ctr-e138-1518143905142-279227-01-03.hwx.site.log,
> 102143-master-ctr-e138-1518143905142-279227-01-05.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-02.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-07.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-08.hwx.site.log
>
>
> This was observed during cluster testing (source code sync'ed with hbase-2.0,
> built May 2nd):
> {code}
> 2018-05-02 05:44:10,089 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=2]
> master.MasterRpcServices: Region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474 reported
> a fatal error:
> * ABORTING region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN, location=ctr-e138-
> 1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on server=ctr-e138-
> 1518143905142-279227-01-02.hwx.site,16020,1525239334474 but state has
> otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7 more
> *
> Cause:
> org.apache.hadoop.hbase.YouAreDeadException:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
>table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
>
[jira] [Commented] (HBASE-20545) Improve performance of BaseLoadBalancer.retainAssignment
[
https://issues.apache.org/jira/browse/HBASE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472881#comment-16472881
]
Hadoop QA commented on HBASE-20545:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1.4 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
58s{color} | {color:green} branch-1.4 passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
20s{color} | {color:red} hbase-server in branch-1.4 failed with JDK v1.8.0_172.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
15s{color} | {color:red} hbase-server in branch-1.4 failed with JDK v1.7.0_181.
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
12s{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_172.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 12s{color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0_172. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
15s{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_181.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 15s{color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_181. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} hbase-server: The patch generated 0 new + 77
unchanged - 1 fixed = 77 total (was 78) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
15s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 20s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}100m 13s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}128m 4s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:74e3133 |
| JIRA Issue |
[jira] [Updated] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-20567:
-
Attachment: HBASE-20567.master.002.patch
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-20567.master.001.patch,
> HBASE-20567.master.002.patch
>
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-20567:
-
Attachment: (was: HBASE-20567.branch-2.001.patch)
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-20567.master.001.patch
>
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472874#comment-16472874
]
Appy commented on HBASE-20567:
--
Dammit, it was stuck at the password phase of my submit-patch.py script. Should
be there now :D
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-20567.branch-2.001.patch,
> HBASE-20567.master.001.patch
>
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-20567:
-
Attachment: HBASE-20567.branch-2.001.patch
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-20567.branch-2.001.patch,
> HBASE-20567.master.001.patch
>
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472867#comment-16472867
]
stack commented on HBASE-20552:
---
Go [~uagashe]!
Usually we'll complain if we fail to read procedures from Master WAL. Any
evidence of us skipping Procedure steps?
(Seems like a good one!!)
> HBase RegionServer was shutdown due to UnexpectedStateException
> ---
>
> Key: HBASE-20552
> URL: https://issues.apache.org/jira/browse/HBASE-20552
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Umesh Agashe
>Priority: Critical
> Attachments:
> 102143-master-ctr-e138-1518143905142-279227-01-03.hwx.site.log,
> 102143-master-ctr-e138-1518143905142-279227-01-05.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-02.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-07.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-08.hwx.site.log
>
>
> This was observed during cluster testing (source code sync'ed with hbase-2.0,
> built May 2nd):
> {code}
> 2018-05-02 05:44:10,089 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=2]
> master.MasterRpcServices: Region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474 reported
> a fatal error:
> * ABORTING region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN, location=ctr-e138-
> 1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on server=ctr-e138-
> 1518143905142-279227-01-02.hwx.site,16020,1525239334474 but state has
> otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7 more
> *
> Cause:
> org.apache.hadoop.hbase.YouAreDeadException:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
>table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
>
[jira] [Commented] (HBASE-20483) [PERFORMANCE] Flushing is 2x slower in hbase2.
[ https://issues.apache.org/jira/browse/HBASE-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472866#comment-16472866 ] stack commented on HBASE-20483: --- [~anastas] HBASE-20411 removes the worst 'locking' offender in the write-path. There are other 'frictions' but this one overwhelmed the others. It does not make for much of an improvement in throughput; that is an ongoing project (IMC is not the problem; IMC is unable to shine, in my opinion, until we figure this basic perf drop-off). Yes, MSLAB is on in 1.4. If you want to follow-along, https://docs.google.com/document/d/1vZ_k6_pNR1eQxID5u1xFihuPC7FkPaJQW8c4M5eA2AQ/edit is where I"m doing my investigations (HBASE-20188). Thanks. > [PERFORMANCE] Flushing is 2x slower in hbase2. > -- > > Key: HBASE-20483 > URL: https://issues.apache.org/jira/browse/HBASE-20483 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Attachments: > 0001-HBASE-20483-perpertual-flush-and-memstore-fill-branch-2.0.patch, > 0001-HBASE-20483-perpetual-flush.branch-1.4.patch, > 0001-HBASE-20483-perpetual-flush.branch-1.4.patch, > PerpetualFlushingProgram.patch, PerpetualFlushingProgram1.2.patch, > SnapshotSegmentScanner_2.0.0.patch > > > When flush is slow, memstores backup and go slower. See "1.2.7 vs. 2.0.0 > Flush History" > https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826 > for compare. First noted by [~anoop.hbase] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Resolution: Fixed Assignee: huaxiang sun Fix Version/s: 2.0.1 Status: Resolved (was: Patch Available) Pushed to branch-2.0+. Assigned to [~huaxiang] since what was committed was his idea (Anoop had same suggestion but can only assign to one). Thanks for reviews. > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: huaxiang sun >Priority: Major > Fix For: 2.0.1 > > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > HBASE-20411.branch-2.0.011.patch, HBASE-20411.branch-2.0.012.patch, > HBASE-20411.branch-2.0.013.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472862#comment-16472862 ] stack commented on HBASE-20411: --- Thank you [~huaxiang] (and [~mdrob]) for reviews. > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > HBASE-20411.branch-2.0.011.patch, HBASE-20411.branch-2.0.012.patch, > HBASE-20411.branch-2.0.013.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472859#comment-16472859
]
Mike Drob commented on HBASE-20567:
---
bq. Uploaded v2.
I... don't see it?
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-20567.master.001.patch
>
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20573) Add to prefix-tree removal section in upgrade doc, how to undo dependency before upgrade
stack created HBASE-20573: - Summary: Add to prefix-tree removal section in upgrade doc, how to undo dependency before upgrade Key: HBASE-20573 URL: https://issues.apache.org/jira/browse/HBASE-20573 Project: HBase Issue Type: Bug Components: documentation Reporter: stack Fix For: 2.0.1 Add a note to upgrade section that user has to undo dependency on prefix-tree before they upgrade to hbase2. To get over prefix-encoding, you need to do the below before upgrade (From [~appy]) 1. Change the encoding to something other than PREFIX_TREE and which is supported by hbase2 2. Major ompact the tables which were changed so they are rewritten into the new encoding. 3. Retry upgrade -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20545) Improve performance of BaseLoadBalancer.retainAssignment
[ https://issues.apache.org/jira/browse/HBASE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472831#comment-16472831 ] Thiruvel Thirumoolan commented on HBASE-20545: -- Doesn't look the failure is related to my patch, have resubmitted 002 patch to trigger the build again. > Improve performance of BaseLoadBalancer.retainAssignment > > > Key: HBASE-20545 > URL: https://issues.apache.org/jira/browse/HBASE-20545 > Project: HBase > Issue Type: Improvement > Components: Balancer >Affects Versions: 1.4.4, 2.0.0 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.5 > > Attachments: HBASE-20545.branch-1.4.001.patch, > HBASE-20545.branch-1.4.002.patch, HBASE-20545.branch-2.001.patch > > > I was measuring perf at scale with a 1m region table and noticed some > improvements can be made to BaseLoadBalancer.retainAssignment(). > retainAssignment() spends a few mins to enable a 1m regions table and also > generates a lot of objects unnecessarily. This jira is to make the most > common case go faster with very minimal changes. A slightly modified version > of this patch takes about 5 seconds for a 1m region table ignoring the time > spent in createCluster(). I think locality can be refreshed during master > startup in different ways without taking time in retainAssignment, but will > follow up on that in subsequent jiras. Leaving it untouched here, but wanted > to call out the time taken without that method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20545) Improve performance of BaseLoadBalancer.retainAssignment
[ https://issues.apache.org/jira/browse/HBASE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvel Thirumoolan updated HBASE-20545: - Attachment: HBASE-20545.branch-1.4.002.patch > Improve performance of BaseLoadBalancer.retainAssignment > > > Key: HBASE-20545 > URL: https://issues.apache.org/jira/browse/HBASE-20545 > Project: HBase > Issue Type: Improvement > Components: Balancer >Affects Versions: 1.4.4, 2.0.0 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.5 > > Attachments: HBASE-20545.branch-1.4.001.patch, > HBASE-20545.branch-1.4.002.patch, HBASE-20545.branch-2.001.patch > > > I was measuring perf at scale with a 1m region table and noticed some > improvements can be made to BaseLoadBalancer.retainAssignment(). > retainAssignment() spends a few mins to enable a 1m regions table and also > generates a lot of objects unnecessarily. This jira is to make the most > common case go faster with very minimal changes. A slightly modified version > of this patch takes about 5 seconds for a 1m region table ignoring the time > spent in createCluster(). I think locality can be refreshed during master > startup in different ways without taking time in retainAssignment, but will > follow up on that in subsequent jiras. Leaving it untouched here, but wanted > to call out the time taken without that method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20565) ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result since 1.4
[ https://issues.apache.org/jira/browse/HBASE-20565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472775#comment-16472775 ] Jerry He commented on HBASE-20565: -- You can see the results from branch-1.2 makes more sense and have correctly identify the ColumnPaginationFilter's offset of either 0 or 1 relative to the ColumnRangeFilter's range. > ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect > result since 1.4 > - > > Key: HBASE-20565 > URL: https://issues.apache.org/jira/browse/HBASE-20565 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 1.4.4 >Reporter: Jerry He >Assignee: Zheng Hu >Priority: Major > Attachments: debug.diff, debug.log, test-branch-1.4.patch > > > When ColumnPaginationFilter is combined with ColumnRangeFilter, we may see > incorrect result. > Here is a simple example. > One row with 10 columns c0, c1, c2, .., c9. I have a ColumnRangeFilter for > range c2 to c9. Then I have a ColumnPaginationFilter with limit 5 and offset > 0. FileterList is FilterList(Operator.MUST_PASS_ALL, ColumnRangeFilter, > ColumnPaginationFilter). > We expect 5 columns being returned. But in HBase 1.4 and after, 4 columns > are returned. > In 1.2.x, the correct 5 columns are returned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20565) ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result since 1.4
[ https://issues.apache.org/jira/browse/HBASE-20565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472774#comment-16472774 ] Jerry He commented on HBASE-20565: -- Hi, [~openinx] Thanks for taking a look! i did a little more testing with the test patch. Here is what I got: Branch 1.4: --- ColumnRangeFilter StringRange(“1”, true, "9", false), ColumnPaginationFilter(5, 0) Result: 0, 1, 2, 3, 4 StringRange(“1”, true, "9", false), ColumnPaginationFilter(5, 0) Result: 1, 2, 3, 4 StringRange(“3”, true, "9", false), ColumnPaginationFilter(5, 0) Result: 3, 4, 5, 6 StringRange(“0”, true, "9", false), ColumnPaginationFilter(5, 1) Result: 1, 2, 3, 4, 5 StringRange(“1”, true, "9", false), ColumnPaginationFilter(5, 1) Result: 1, 2, 3, 4 , 5 StringRange(“3”, true, "9", false), ColumnPaginationFilter(5, 1) Result: 3, 4, 5, 6, 7 Branch 1.2 --- ColumnRangeFilter StringRange(“0”, true, "9", false), ColumnPaginationFilter(5, 0) Result: 0, 1, 2, 3, 4 StringRange(“1”, true, "9", false), ColumnPaginationFilter(5, 0) Result: 1, 2, 3, 4, 5 StringRange(“3”, true, "9", false), ColumnPaginationFilter(5, 0) Result: 3, 4, 5, 6, 7 StringRange(“0”, true, "9", false), ColumnPaginationFilter(5, 1) Result: 1, 2, 3, 4, 5 StringRange(“1”, true, "9", false), ColumnPaginationFilter(5, 1) Result: 2, 3, 4 , 5, 6 StringRange(“3”, true, "9", false), ColumnPaginationFilter(5, 1) Result: 4, 5, 6, 7, 8 > ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect > result since 1.4 > - > > Key: HBASE-20565 > URL: https://issues.apache.org/jira/browse/HBASE-20565 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 1.4.4 >Reporter: Jerry He >Assignee: Zheng Hu >Priority: Major > Attachments: debug.diff, debug.log, test-branch-1.4.patch > > > When ColumnPaginationFilter is combined with ColumnRangeFilter, we may see > incorrect result. > Here is a simple example. > One row with 10 columns c0, c1, c2, .., c9. I have a ColumnRangeFilter for > range c2 to c9. Then I have a ColumnPaginationFilter with limit 5 and offset > 0. FileterList is FilterList(Operator.MUST_PASS_ALL, ColumnRangeFilter, > ColumnPaginationFilter). > We expect 5 columns being returned. But in HBase 1.4 and after, 4 columns > are returned. > In 1.2.x, the correct 5 columns are returned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-20552 started by Umesh Agashe.
> HBase RegionServer was shutdown due to UnexpectedStateException
> ---
>
> Key: HBASE-20552
> URL: https://issues.apache.org/jira/browse/HBASE-20552
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Umesh Agashe
>Priority: Critical
> Attachments:
> 102143-master-ctr-e138-1518143905142-279227-01-03.hwx.site.log,
> 102143-master-ctr-e138-1518143905142-279227-01-05.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-02.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-07.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-08.hwx.site.log
>
>
> This was observed during cluster testing (source code sync'ed with hbase-2.0,
> built May 2nd):
> {code}
> 2018-05-02 05:44:10,089 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=2]
> master.MasterRpcServices: Region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474 reported
> a fatal error:
> * ABORTING region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN, location=ctr-e138-
> 1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on server=ctr-e138-
> 1518143905142-279227-01-02.hwx.site,16020,1525239334474 but state has
> otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7 more
> *
> Cause:
> org.apache.hadoop.hbase.YouAreDeadException:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
>table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
>
[jira] [Commented] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472760#comment-16472760
]
Appy commented on HBASE-20567:
--
Makes sense.
Uploaded v2.
Added back old methods for backward compatibility. Now we can backport this
patch to 2.1 too.
I took the liberty of renaming arguments in old methods to make it clear what
they are and how they relate to new methods.
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-20567.master.001.patch
>
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472755#comment-16472755
]
Ted Yu commented on HBASE-20566:
Can this problem be re-created with unit test ?
Please see also HBASE-19757
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20483) [PERFORMANCE] Flushing is 2x slower in hbase2.
[
https://issues.apache.org/jira/browse/HBASE-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472684#comment-16472684
]
Anastasia Braginsky commented on HBASE-20483:
-
{quote}bq.I'm back to this issue now via HBASE-20411 and other explorations
bq.Yeah, its after. (regarding IMC disabled)
{quote}
I see that you are working without IMC and that you believe the problem is due
to locking around the sizing (HBASE-20411). Let's see if it helps.
On the other note, is there MSLAB in the hbase 1.4 (the version you are
comparing to)?
> [PERFORMANCE] Flushing is 2x slower in hbase2.
> --
>
> Key: HBASE-20483
> URL: https://issues.apache.org/jira/browse/HBASE-20483
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Reporter: stack
>Assignee: stack
>Priority: Major
> Attachments:
> 0001-HBASE-20483-perpertual-flush-and-memstore-fill-branch-2.0.patch,
> 0001-HBASE-20483-perpetual-flush.branch-1.4.patch,
> 0001-HBASE-20483-perpetual-flush.branch-1.4.patch,
> PerpetualFlushingProgram.patch, PerpetualFlushingProgram1.2.patch,
> SnapshotSegmentScanner_2.0.0.patch
>
>
> When flush is slow, memstores backup and go slower. See "1.2.7 vs. 2.0.0
> Flush History"
> https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826
> for compare. First noted by [~anoop.hbase]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-20572) HBase2 does not compile against Hadoop3 after HADOOP-10768
[ https://issues.apache.org/jira/browse/HBASE-20572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HBASE-20572: --- Assignee: Wei-Chiu Chuang > HBase2 does not compile against Hadoop3 after HADOOP-10768 > -- > > Key: HBASE-20572 > URL: https://issues.apache.org/jira/browse/HBASE-20572 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Blocker > Attachments: HBASE-20572.001.patch > > > This is a non-issue yet, because HADOOP-10768 is not committed in Hadoop > codebase yet. But I am reviewing that jira and trying to get it in. > FanOutOneBlockAsyncDFSOutput does not compile because protobuf message > CipherOptionProto was moved from hdfs.proto (in hadoop-hdfs) to > Security.proto (in hadoop-common). > File this jiras as a placeholder. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472628#comment-16472628 ] huaxiang sun commented on HBASE-20411: -- +1 for .13 version, looks good to me. > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > HBASE-20411.branch-2.0.011.patch, HBASE-20411.branch-2.0.012.patch, > HBASE-20411.branch-2.0.013.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20526) multithreads bulkload performance
[
https://issues.apache.org/jira/browse/HBASE-20526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472625#comment-16472625
]
Ted Yu commented on HBASE-20526:
Test failure seems to be related to patch:
{code}
[ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 174.114
s <<< FAILURE! - in
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientWithRegionReplicas
[ERROR]
testRestoreSnapshotAfterSplit(org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientWithRegionReplicas)
Time elapsed: 17.969 s <<< ERROR!
org.apache.hadoop.hbase.snapshot.HBaseSnapshotException:
org.apache.hadoop.hbase.snapshot.HBaseSnapshotException: Snapshot {
ss=testRestoreSnapshotAfterSplit-snap table=testtb-1525617745131 type=FLUSH }
had an error. Procedure testRestoreSnapshotAfterSplit-snap { waiting=[]
done=[] }
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:354)
at
org.apache.hadoop.hbase.master.MasterRpcServices.isSnapshotDone(MasterRpcServices.java:1030)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:58585)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2349)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException via
Failed taking snapshot { ss=testRestoreSnapshotAfterSplit-snap
table=testtb-1525617745131 type=FLUSH } due to exception:Manifest region info
{ENCODED => 7e089fb3359462cea181a607beba185a, NAME =>
'testtb-1525617745131,8,1525617745132_0002.7e089fb3359462cea181a607beba185a.',
STARTKEY => '8', ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID =>
2}doesn't match expected region:{ENCODED => 9ca827b8e313d4b010ae10accb02a970,
NAME =>
'testtb-1525617745131,8,1525617745132.9ca827b8e313d4b010ae10accb02a970.',
STARTKEY => '8', ENDKEY => '', OFFLINE => true, SPLIT =>
true}:org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Manifest
region info {ENCODED => 7e089fb3359462cea181a607beba185a, NAME =>
'testtb-1525617745131,8,1525617745132_0002.7e089fb3359462cea181a607beba185a.',
STARTKEY => '8', ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID =>
2}doesn't match expected region:{ENCODED => 9ca827b8e313d4b010ae10accb02a970,
NAME =>
'testtb-1525617745131,8,1525617745132.9ca827b8e313d4b010ae10accb02a970.',
STARTKEY => '8', ENDKEY => '', OFFLINE => true, SPLIT => true}
at
org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:83)
at
org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.rethrowExceptionIfFailed(TakeSnapshotHandler.java:315)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:344)
... 6 more
{code}
Please correct the test failure before posting patch for master.
> multithreads bulkload performance
> -
>
> Key: HBASE-20526
> URL: https://issues.apache.org/jira/browse/HBASE-20526
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce, Zookeeper
>Affects Versions: 1.2.5, 1.3.2
> Environment: hbase-server-1.2.0-cdh5.12.1
> spark version 1.6
>Reporter: Key Hutu
>Assignee: Key Hutu
>Priority: Minor
> Labels: performance
> Fix For: 1.3.2
>
> Attachments: HBASE-20526-branch-1.3.V1.patch
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> When doing bulkload , some interactive with zookeeper to getting region key
> range may be cost more time.
> In multithreads enviorment, the duration maybe cost 5 minute or more.
> From the executor log, like 'Reading reply sessionid:0x262fb37f4a07080 ,
> packet:: clientPath:null server ...' contents appear many times.
>
> It likely to provide new method for bulkload, caching the key range outside
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472614#comment-16472614
]
Hudson commented on HBASE-20004:
Results for branch branch-1.4
[build #317 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/317/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/317//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/317//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/317//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Client is not able to execute REST queries in a secure cluster
> --
>
> Key: HBASE-20004
> URL: https://issues.apache.org/jira/browse/HBASE-20004
> Project: HBase
> Issue Type: Bug
> Components: REST, security
>Affects Versions: 1.3.1
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20004.branch-1.patch,
> HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch,
> HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch,
> HBASE-20004.v2.patch
>
>
> Firefox browser is not able to negotiate REST queries with server in secure
> mode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18383) Upgrade cyrus SASL library to 2.1.27
[ https://issues.apache.org/jira/browse/HBASE-18383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472613#comment-16472613 ] Ted Yu commented on HBASE-18383: SASL 2.1.27 rc8 has been posted > Upgrade cyrus SASL library to 2.1.27 > > > Key: HBASE-18383 > URL: https://issues.apache.org/jira/browse/HBASE-18383 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > > On cyrus mailing list, Ken Murchison indicated 2.1.27 would be released in > August. > We should upgrade to the new release - 2.1.26 was several years old. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20572) HBase2 does not compile against Hadoop3 after HADOOP-10768
[ https://issues.apache.org/jira/browse/HBASE-20572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HBASE-20572: Description: This is a non-issue yet, because HADOOP-10768 is not committed in Hadoop codebase yet. But I am reviewing that jira and trying to get it in. FanOutOneBlockAsyncDFSOutput does not compile because protobuf message CipherOptionProto was moved from hdfs.proto (in hadoop-hdfs) to Security.proto (in hadoop-common). File this jiras as a placeholder. was: This is a non-issue yet, because HADOOP-10768 is not committed in Hadoop codebase yet. But I am reviewing that jira and trying to get it in. FanOutOneBlockAsyncDFSOutput does not compile because protobuf message CipherOptionProto was moved from hdfs.proto to Security.proto. File this jiras as a placeholder. > HBase2 does not compile against Hadoop3 after HADOOP-10768 > -- > > Key: HBASE-20572 > URL: https://issues.apache.org/jira/browse/HBASE-20572 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Wei-Chiu Chuang >Priority: Blocker > Attachments: HBASE-20572.001.patch > > > This is a non-issue yet, because HADOOP-10768 is not committed in Hadoop > codebase yet. But I am reviewing that jira and trying to get it in. > FanOutOneBlockAsyncDFSOutput does not compile because protobuf message > CipherOptionProto was moved from hdfs.proto (in hadoop-hdfs) to > Security.proto (in hadoop-common). > File this jiras as a placeholder. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472482#comment-16472482
]
Umesh Agashe commented on HBASE-20552:
--
Further, M003 starts SCP with pid=507 for R007:
{code:java}
2018-05-02 05:44:08,413 INFO [PEWorker-6] procedure.ServerCrashProcedure:
Start pid=507, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure
server=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525238558502,
splitWal=true, meta=false{code}
This starts AssignProcedure with pid=508 for region
94f6ca283dbb4445b2bcdc321b734d28:
{code:java}
2018-05-02 05:44:08,480 INFO [PEWorker-6] assignment.AssignProcedure: Starting
pid=508, ppid=507, state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure
table=test_hbase_ha_load_test_tool_hbase,
region=94f6ca283dbb4445b2bcdc321b734d28; rit=OFFLINE,
location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525238558502;
forceNewPlan=false, retain=true
2018-05-02 05:44:08,659 INFO [PEWorker-11] assignment.RegionStateStore:
pid=508 updating hbase:meta row=94f6ca283dbb4445b2bcdc321b734d28,
regionState=OPENING,
regionLocation=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353
2018-05-02 05:44:08,727 INFO [PEWorker-11]
assignment.RegionTransitionProcedure: Dispatch pid=508, ppid=507,
state=RUNNABLE:REGION_TRANSITION_DISPATCH; AssignProcedure
table=test_hbase_ha_load_test_tool_hbase,
region=94f6ca283dbb4445b2bcdc321b734d28; rit=OPENING,
location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353
...
2018-05-02 05:44:09,213 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=2]
assignment.RegionTransitionProcedure: Received report OPENED seqId=13402,
pid=508, ppid=507, state=RUNNABLE:REGION_TRANSITION_DISPATCH; AssignProcedure
table=test_hbase_ha_load_test_tool_hbase,
region=94f6ca283dbb4445b2bcdc321b734d28; rit=OPENING,
location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353
2018-05-02 05:44:09,213 DEBUG [PEWorker-12]
assignment.RegionTransitionProcedure: Finishing pid=508, ppid=507,
state=RUNNABLE:REGION_TRANSITION_FINISH; AssignProcedure
table=test_hbase_ha_load_test_tool_hbase,
region=94f6ca283dbb4445b2bcdc321b734d28; rit=OPENING,
location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353
2018-05-02 05:44:09,214 INFO [PEWorker-12] assignment.RegionStateStore: pid=508
updating hbase:meta row=94f6ca283dbb4445b2bcdc321b734d28, regionState=OPEN,
openSeqNum=13402,
regionLocation=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353
2018-05-02 05:44:09,258 INFO [PEWorker-12] procedure2.ProcedureExecutor:
Finished subprocedure(s) of pid=507, state=RUNNABLE:SERVER_CRASH_HANDLE_RIT2;
ServerCrashProcedure
server=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525238558502,
splitWal=true, meta=false; resume parent processing.
2018-05-02 05:44:09,258 INFO [PEWorker-12] procedure2.ProcedureExecutor:
Finished pid=508, ppid=507, state=SUCCESS; AssignProcedure
table=test_hbase_ha_load_test_tool_hbase,
region=94f6ca283dbb4445b2bcdc321b734d28 in 764msec
2018-05-02 05:44:09,273 INFO [PEWorker-14] procedure2.ProcedureExecutor:
Finished pid=507, state=SUCCESS; ServerCrashProcedure
server=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525238558502,
splitWal=true, meta=false in 975msec{code}
Strange thing is SCP for R007 is assigning region back to R007!
> HBase RegionServer was shutdown due to UnexpectedStateException
> ---
>
> Key: HBASE-20552
> URL: https://issues.apache.org/jira/browse/HBASE-20552
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Umesh Agashe
>Priority: Critical
> Attachments:
> 102143-master-ctr-e138-1518143905142-279227-01-03.hwx.site.log,
> 102143-master-ctr-e138-1518143905142-279227-01-05.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-02.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-07.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-08.hwx.site.log
>
>
> This was observed during cluster testing (source code sync'ed with hbase-2.0,
> built May 2nd):
> {code}
> 2018-05-02 05:44:10,089 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=2]
> master.MasterRpcServices: Region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474 reported
> a fatal error:
> * ABORTING region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN, location=ctr-e138-
> 1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
>
[jira] [Commented] (HBASE-20480) Multiple one-time cell objects are allocated and de-allocated when working with CCM
[
https://issues.apache.org/jira/browse/HBASE-20480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472480#comment-16472480
]
Anastasia Braginsky commented on HBASE-20480:
-
{quote}bq.ThreadLocal based accessing will have some perf penalty though!
{quote}
Yes, I am checking the performance. It might be that due to that we will switch
to not-ThreadLocal solution. Thanks!
> Multiple one-time cell objects are allocated and de-allocated when working
> with CCM
> ---
>
> Key: HBASE-20480
> URL: https://issues.apache.org/jira/browse/HBASE-20480
> Project: HBase
> Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>Priority: Major
> Attachments: HBASE-20480-V02.patch
>
>
> We believe that the cause for some read performance degradation while working
> with CellChunkMap (CCM). Multiple one-time cell objects are allocated and
> de-allocated when performing multiple reads and working with CCM MemStore. We
> have a couple of ideas for solution. More details will follow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20572) HBase2 does not compile against Hadoop3 after HADOOP-10768
[ https://issues.apache.org/jira/browse/HBASE-20572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HBASE-20572: Attachment: HBASE-20572.001.patch > HBase2 does not compile against Hadoop3 after HADOOP-10768 > -- > > Key: HBASE-20572 > URL: https://issues.apache.org/jira/browse/HBASE-20572 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Wei-Chiu Chuang >Priority: Blocker > Attachments: HBASE-20572.001.patch > > > This is a non-issue yet, because HADOOP-10768 is not committed in Hadoop > codebase yet. But I am reviewing that jira and trying to get it in. > FanOutOneBlockAsyncDFSOutput does not compile because protobuf message > CipherOptionProto was moved from hdfs.proto to Security.proto. > File this jiras as a placeholder. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20572) HBase2 does not compile against Hadoop3 after HADOOP-10768
Wei-Chiu Chuang created HBASE-20572: --- Summary: HBase2 does not compile against Hadoop3 after HADOOP-10768 Key: HBASE-20572 URL: https://issues.apache.org/jira/browse/HBASE-20572 Project: HBase Issue Type: Bug Affects Versions: 2.0.0-beta-1 Reporter: Wei-Chiu Chuang Attachments: HBASE-20572.001.patch This is a non-issue yet, because HADOOP-10768 is not committed in Hadoop codebase yet. But I am reviewing that jira and trying to get it in. FanOutOneBlockAsyncDFSOutput does not compile because protobuf message CipherOptionProto was moved from hdfs.proto to Security.proto. File this jiras as a placeholder. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20552:
---
Attachment:
102143-regionserver-ctr-e138-1518143905142-279227-01-08.hwx.site.log
> HBase RegionServer was shutdown due to UnexpectedStateException
> ---
>
> Key: HBASE-20552
> URL: https://issues.apache.org/jira/browse/HBASE-20552
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Umesh Agashe
>Priority: Critical
> Attachments:
> 102143-master-ctr-e138-1518143905142-279227-01-03.hwx.site.log,
> 102143-master-ctr-e138-1518143905142-279227-01-05.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-02.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-07.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-08.hwx.site.log
>
>
> This was observed during cluster testing (source code sync'ed with hbase-2.0,
> built May 2nd):
> {code}
> 2018-05-02 05:44:10,089 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=2]
> master.MasterRpcServices: Region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474 reported
> a fatal error:
> * ABORTING region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN, location=ctr-e138-
> 1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on server=ctr-e138-
> 1518143905142-279227-01-02.hwx.site,16020,1525239334474 but state has
> otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7 more
> *
> Cause:
> org.apache.hadoop.hbase.YouAreDeadException:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
>table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
>
[jira] [Updated] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20552:
---
Attachment:
102143-regionserver-ctr-e138-1518143905142-279227-01-07.hwx.site.log
> HBase RegionServer was shutdown due to UnexpectedStateException
> ---
>
> Key: HBASE-20552
> URL: https://issues.apache.org/jira/browse/HBASE-20552
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Umesh Agashe
>Priority: Critical
> Attachments:
> 102143-master-ctr-e138-1518143905142-279227-01-03.hwx.site.log,
> 102143-master-ctr-e138-1518143905142-279227-01-05.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-02.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-07.hwx.site.log
>
>
> This was observed during cluster testing (source code sync'ed with hbase-2.0,
> built May 2nd):
> {code}
> 2018-05-02 05:44:10,089 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=2]
> master.MasterRpcServices: Region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474 reported
> a fatal error:
> * ABORTING region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN, location=ctr-e138-
> 1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on server=ctr-e138-
> 1518143905142-279227-01-02.hwx.site,16020,1525239334474 but state has
> otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7 more
> *
> Cause:
> org.apache.hadoop.hbase.YouAreDeadException:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
>table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7
[jira] [Commented] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472456#comment-16472456
]
Umesh Agashe commented on HBASE-20552:
--
bq. Log for server 0002 was attached already.
Thanks! and also for 007?
> HBase RegionServer was shutdown due to UnexpectedStateException
> ---
>
> Key: HBASE-20552
> URL: https://issues.apache.org/jira/browse/HBASE-20552
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Umesh Agashe
>Priority: Critical
> Attachments:
> 102143-master-ctr-e138-1518143905142-279227-01-03.hwx.site.log,
> 102143-master-ctr-e138-1518143905142-279227-01-05.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-02.hwx.site.log
>
>
> This was observed during cluster testing (source code sync'ed with hbase-2.0,
> built May 2nd):
> {code}
> 2018-05-02 05:44:10,089 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=2]
> master.MasterRpcServices: Region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474 reported
> a fatal error:
> * ABORTING region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN, location=ctr-e138-
> 1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on server=ctr-e138-
> 1518143905142-279227-01-02.hwx.site,16020,1525239334474 but state has
> otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7 more
> *
> Cause:
> org.apache.hadoop.hbase.YouAreDeadException:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
>table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7 more
> at
[jira] [Commented] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472438#comment-16472438
]
Umesh Agashe commented on HBASE-20552:
--
bq. Was there any region on 0008 you're interested in ?
670f6b815d2acac905130e5440d59304
1d954f21d711345a9587d995cecea136
91f73e76bbe7bc8a61b1b1299d34c6ab
> HBase RegionServer was shutdown due to UnexpectedStateException
> ---
>
> Key: HBASE-20552
> URL: https://issues.apache.org/jira/browse/HBASE-20552
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Umesh Agashe
>Priority: Critical
> Attachments:
> 102143-master-ctr-e138-1518143905142-279227-01-03.hwx.site.log,
> 102143-master-ctr-e138-1518143905142-279227-01-05.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-02.hwx.site.log
>
>
> This was observed during cluster testing (source code sync'ed with hbase-2.0,
> built May 2nd):
> {code}
> 2018-05-02 05:44:10,089 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=2]
> master.MasterRpcServices: Region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474 reported
> a fatal error:
> * ABORTING region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN, location=ctr-e138-
> 1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on server=ctr-e138-
> 1518143905142-279227-01-02.hwx.site,16020,1525239334474 but state has
> otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7 more
> *
> Cause:
> org.apache.hadoop.hbase.YouAreDeadException:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
>table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
>
[jira] [Commented] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472422#comment-16472422
]
Ted Yu commented on HBASE-20552:
Log for server 0002 was attached already.
Was there any region on 0008 you're interested in ?
> HBase RegionServer was shutdown due to UnexpectedStateException
> ---
>
> Key: HBASE-20552
> URL: https://issues.apache.org/jira/browse/HBASE-20552
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Umesh Agashe
>Priority: Critical
> Attachments:
> 102143-master-ctr-e138-1518143905142-279227-01-03.hwx.site.log,
> 102143-master-ctr-e138-1518143905142-279227-01-05.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-02.hwx.site.log
>
>
> This was observed during cluster testing (source code sync'ed with hbase-2.0,
> built May 2nd):
> {code}
> 2018-05-02 05:44:10,089 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=2]
> master.MasterRpcServices: Region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474 reported
> a fatal error:
> * ABORTING region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN, location=ctr-e138-
> 1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on server=ctr-e138-
> 1518143905142-279227-01-02.hwx.site,16020,1525239334474 but state has
> otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7 more
> *
> Cause:
> org.apache.hadoop.hbase.YouAreDeadException:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
>table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7 more
> at
[jira] [Comment Edited] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472409#comment-16472409
]
Umesh Agashe edited comment on HBASE-20552 at 5/11/18 6:23 PM:
---
Usually following warnings can be ignored. But these messages followed by
"Completed pid=" looks trouble. When M003 became active at around 2018-05-02
05:43:33, there are a few warnings while reading master proc wal:
{code:java}
2018-05-02 05:43:33,529 WARN
[master/ctr-e138-1518143905142-279227-01-03:2] wal.WALProcedureStore:
Unable to read tracker for
hdfs://mycluster/apps/hbase/data/MasterProcWALs/pv2-0004.log -
Invalid Trailer version. got 8 expected 1
2018-05-02 05:43:33,638 DEBUG
[master/ctr-e138-1518143905142-279227-01-03:2] wal.WALProcedureStore:
Roll new state log: 5
2018-05-02 05:43:33,655 INFO
[master/ctr-e138-1518143905142-279227-01-03:2]
procedure2.ProcedureExecutor: Recovered WALProcedureStore lease in 219msec
2018-05-02 05:43:33,681 INFO
[master/ctr-e138-1518143905142-279227-01-03:2]
wal.ProcedureWALFormatReader: Rebuilding tracker for
hdfs://mycluster/apps/hbase/data/MasterProcWALs/pv2-0004.log
2018-05-02 05:43:33,816 WARN
[master/ctr-e138-1518143905142-279227-01-03:2]
wal.ProcedureWALFormatReader: Nothing left to decode. Exiting with missing EOF,
log=hdfs://mycluster/apps/hbase/data/MasterProcWALs/pv2-0004.log
2018-05-02 05:43:33,875 DEBUG
[master/ctr-e138-1518143905142-279227-01-03:2]
procedure2.ProcedureExecutor: Completed pid=467, state=SUCCESS;
MoveRegionProcedure hri=4c37ee7a4e1210e481debdc2933fc4d2,
source=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474,
destination=ctr-e138-1518143905142-279227-01-03.hwx.site,16020,15252394258262018-05-02
05:43:33,876 DEBUG [master/ctr-e138-1518143905142-279227-01-03:2]
procedure2.ProcedureExecutor: Completed pid=465, state=SUCCESS;
MoveRegionProcedure hri=94f6ca283dbb4445b2bcdc321b734d28,
source=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474,
destination=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525238558502
2018-05-02 05:43:33,876 DEBUG
[master/ctr-e138-1518143905142-279227-01-03:2]
procedure2.ProcedureExecutor: Completed pid=462, state=SUCCESS;
MoveRegionProcedure hri=a8ff96226d546f0ea151823ae73e5a1b,
source=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474,
destination=ctr-e138-1518143905142-279227-01-08.hwx.site,16020,1525238658606{code}
M003 during startup has no log messages for procedures with ids 468 to 504 even
though they ran and completed on M005. This is unusual. RecoverMetaProcedure on
M003 starts with id 505 which is correct.
Orthogonal to above observation we have meta update issue as well. On M005,
pid=471 is SCP for R007 which also hosts meta. Meta is re-assigned with pid=472
to R002 which is followed by other region assignments
{code:java}
pid=478 e75a388bc2011feed75bdc1a0e99a9a9
regionLocation=ctr-e138-1518143905142-279227-01-02.hwx.site
pid=474 670f6b815d2acac905130e5440d59304
regionLocation=ctr-e138-1518143905142-279227-01-08.hwx.site
pid=479 c963eb77dbdc6dbab886dbe4eebba5ad
regionLocation=ctr-e138-1518143905142-279227-01-06.hwx.site
pid=481 b5180eee96b616afdf79578309c66a11
regionLocation=ctr-e138-1518143905142-279227-01-02.hwx.site
pid=486 8dc6fd2022c2fdf8c065fbd16cadaaca
regionLocation=ctr-e138-1518143905142-279227-01-03.hwx.site
pid=480 f3db9f9879ed03f488dcb89bea834237
regionLocation=ctr-e138-1518143905142-279227-01-02.hwx.site
pid=484 c078deb2474e9c19b85b5fdb9efaa47d
regionLocation=ctr-e138-1518143905142-279227-01-06.hwx.site
pid=475 94f6ca283dbb4445b2bcdc321b734d28
regionLocation=ctr-e138-1518143905142-279227-01-02.hwx.site
pid=483 1d954f21d711345a9587d995cecea136
regionLocation=ctr-e138-1518143905142-279227-01-08.hwx.site
pid=476 1595f38ee901be7c67b997fe2fc95951
regionLocation=ctr-e138-1518143905142-279227-01-06.hwx.site
pid=482 a6e0d7561c4f19e78f94d37462588281
regionLocation=ctr-e138-1518143905142-279227-01-06.hwx.site
pid=485 91f73e76bbe7bc8a61b1b1299d34c6ab
regionLocation=ctr-e138-1518143905142-279227-01-08.hwx.site
pid=477 a0620fc83de532a37f6a9bb8f99cc6c4
regionLocation=ctr-e138-1518143905142-279227-01-03.hwx.site{code}
>From the logs all the procedures finished successfully without skipping steps.
>Meta doesn't seem to be updated for 4 of these assignments. When M003 logs all
>regions from meta at startup, locations for following 4 regions don't match
>with the target locations in above procedures:
{code:java}
670f6b815d2acac905130e5440d59304
ctr-e138-1518143905142-279227-01-08.hwx.site
lastHost=ctr-e138-1518143905142-279227-01-07.hwx.site
regionLocation=ctr-e138-1518143905142-279227-01-07.hwx.site
[jira] [Commented] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472409#comment-16472409
]
Umesh Agashe commented on HBASE-20552:
--
Usually following warnings can be ignored. But these messages followed by
"Completed pid=" looks trouble. When M003 became active at around 2018-05-02
05:43:33, there are a few warnings while reading master proc wal:
{code:java}
2018-05-02 05:43:33,529 WARN
[master/ctr-e138-1518143905142-279227-01-03:2] wal.WALProcedureStore:
Unable to read tracker for
hdfs://mycluster/apps/hbase/data/MasterProcWALs/pv2-0004.log -
Invalid Trailer version. got 8 expected 1
2018-05-02 05:43:33,638 DEBUG
[master/ctr-e138-1518143905142-279227-01-03:2] wal.WALProcedureStore:
Roll new state log: 5
2018-05-02 05:43:33,655 INFO
[master/ctr-e138-1518143905142-279227-01-03:2]
procedure2.ProcedureExecutor: Recovered WALProcedureStore lease in 219msec
2018-05-02 05:43:33,681 INFO
[master/ctr-e138-1518143905142-279227-01-03:2]
wal.ProcedureWALFormatReader: Rebuilding tracker for
hdfs://mycluster/apps/hbase/data/MasterProcWALs/pv2-0004.log
2018-05-02 05:43:33,816 WARN
[master/ctr-e138-1518143905142-279227-01-03:2]
wal.ProcedureWALFormatReader: Nothing left to decode. Exiting with missing EOF,
log=hdfs://mycluster/apps/hbase/data/MasterProcWALs/pv2-0004.log
2018-05-02 05:43:33,875 DEBUG
[master/ctr-e138-1518143905142-279227-01-03:2]
procedure2.ProcedureExecutor: Completed pid=467, state=SUCCESS;
MoveRegionProcedure hri=4c37ee7a4e1210e481debdc2933fc4d2,
source=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474,
destination=ctr-e138-1518143905142-279227-01-03.hwx.site,16020,15252394258262018-05-02
05:43:33,876 DEBUG [master/ctr-e138-1518143905142-279227-01-03:2]
procedure2.ProcedureExecutor: Completed pid=465, state=SUCCESS;
MoveRegionProcedure hri=94f6ca283dbb4445b2bcdc321b734d28,
source=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474,
destination=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525238558502
2018-05-02 05:43:33,876 DEBUG
[master/ctr-e138-1518143905142-279227-01-03:2]
procedure2.ProcedureExecutor: Completed pid=462, state=SUCCESS;
MoveRegionProcedure hri=a8ff96226d546f0ea151823ae73e5a1b,
source=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474,
destination=ctr-e138-1518143905142-279227-01-08.hwx.site,16020,1525238658606{code}
M003 during startup has no log messages for procedures with ids 468 to 504 even
though they are ran and completed on M005. This is unusual.
RecoverMetaProcedure on M003 starts with id 505 which is correct.
Orthogonal to above observation we have meta update issue as well. On M005,
pid=471 is SCP for R007 which also hosts meta. Meta is re-assigned with pid=472
to R002 which is followed by other region assignments
{code:java}
pid=478 e75a388bc2011feed75bdc1a0e99a9a9
regionLocation=ctr-e138-1518143905142-279227-01-02.hwx.site
pid=474 670f6b815d2acac905130e5440d59304
regionLocation=ctr-e138-1518143905142-279227-01-08.hwx.site
pid=479 c963eb77dbdc6dbab886dbe4eebba5ad
regionLocation=ctr-e138-1518143905142-279227-01-06.hwx.site
pid=481 b5180eee96b616afdf79578309c66a11
regionLocation=ctr-e138-1518143905142-279227-01-02.hwx.site
pid=486 8dc6fd2022c2fdf8c065fbd16cadaaca
regionLocation=ctr-e138-1518143905142-279227-01-03.hwx.site
pid=480 f3db9f9879ed03f488dcb89bea834237
regionLocation=ctr-e138-1518143905142-279227-01-02.hwx.site
pid=484 c078deb2474e9c19b85b5fdb9efaa47d
regionLocation=ctr-e138-1518143905142-279227-01-06.hwx.site
pid=475 94f6ca283dbb4445b2bcdc321b734d28
regionLocation=ctr-e138-1518143905142-279227-01-02.hwx.site
pid=483 1d954f21d711345a9587d995cecea136
regionLocation=ctr-e138-1518143905142-279227-01-08.hwx.site
pid=476 1595f38ee901be7c67b997fe2fc95951
regionLocation=ctr-e138-1518143905142-279227-01-06.hwx.site
pid=482 a6e0d7561c4f19e78f94d37462588281
regionLocation=ctr-e138-1518143905142-279227-01-06.hwx.site
pid=485 91f73e76bbe7bc8a61b1b1299d34c6ab
regionLocation=ctr-e138-1518143905142-279227-01-08.hwx.site
pid=477 a0620fc83de532a37f6a9bb8f99cc6c4
regionLocation=ctr-e138-1518143905142-279227-01-03.hwx.site{code}
>From the logs all the procedures finished successfully without skipping steps.
>Meta doesn't seem to be updated for 4 of these assignments. When M003 logs all
>regions from meta at startup, locations for following 4 regions don't match
>with the target locations in above procedures:
{code:java}
670f6b815d2acac905130e5440d59304
ctr-e138-1518143905142-279227-01-08.hwx.site
lastHost=ctr-e138-1518143905142-279227-01-07.hwx.site
regionLocation=ctr-e138-1518143905142-279227-01-07.hwx.site
94f6ca283dbb4445b2bcdc321b734d28
[jira] [Commented] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472384#comment-16472384
]
Umesh Agashe commented on HBASE-20544:
--
+1 for addendum
> downstream HBaseTestingUtility fails with invalid port
> --
>
> Key: HBASE-20544
> URL: https://issues.apache.org/jira/browse/HBASE-20544
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20544.0.patch, HBASE-20544.1.patch,
> HBASE-20544.2.patch, HBASE-20544.addendum.0.patch
>
>
> Attempting to update hbase-downstreamer to use our 2.0.0 release fails with
> an invalid port in the event that {{hbase.localcluster.assign.random.ports}}
> isn't set (or is set to false, specifically):
> {code}
> 2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer
> (HRegionServer.java:(631)) - Failed construction RegionServer
> java.lang.IllegalArgumentException: port out of range:-1
> at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
> at java.net.InetSocketAddress.(InetSocketAddress.java:224)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1217)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1184)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:147)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:121)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
> at
> org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
[jira] [Commented] (HBASE-20571) JMXJsonServlet generates invalid JSON if it has NaN in metrics
[
https://issues.apache.org/jira/browse/HBASE-20571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472299#comment-16472299
]
Hadoop QA commented on HBASE-20571:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
7s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
7s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 18s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}108m
16s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}148m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:369877d |
| JIRA Issue | HBASE-20571 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12923030/HBASE-20571.branch-2.0.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 4629976c8f70 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.0 / f46f70921c |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12794/testReport/ |
| Max. process+thread count | 4059 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[
https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472232#comment-16472232
]
Hadoop QA commented on HBASE-20369:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
26s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 4 line(s) with tabs. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
36s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20369 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12923052/HBASE-20369_v5.patch |
| Optional Tests | asflicense refguide |
| uname | Linux c3dff9d8ff94 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c60578d982 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/12795/artifact/patchprocess/branch-site/book.html
|
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/12795/artifact/patchprocess/whitespace-tabs.txt
|
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/12795/artifact/patchprocess/patch-site/book.html
|
| Max. process+thread count | 83 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12795/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Document incompatibilities between HBase 1.1.2 and HBase 2.0
>
>
> Key: HBASE-20369
> URL: https://issues.apache.org/jira/browse/HBASE-20369
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Thiriguna Bharat Rao
>Assignee: Thiriguna Bharat Rao
>Priority: Critical
> Labels: patch
> Attachments: HBASE-20369.patch, HBASE-20369_v1.patch,
> HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch,
> HBASE-20369_v5.patch, book.adoc
>
>
> Hi,
> I compiled a draft document for the HBase incompatibilities from the raw
> source content that was available in HBase Beta 1 site. Can someone please
> review and provide a feedback or share your comments on this document?
> Appreciate your support and time.
>
> Best Regards,
> Triguna
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472180#comment-16472180 ] Thiriguna Bharat Rao commented on HBASE-20369: -- Hi, Uploaded a new patch HBase-20369_v5.patch. Removed the General information section that mentions about Rolling upgrade. Please validate it. Appreciate your support and time. Best, Triguna > Document incompatibilities between HBase 1.1.2 and HBase 2.0 > > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch, > HBASE-20369_v5.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiriguna Bharat Rao updated HBASE-20369: - Attachment: HBASE-20369_v5.patch > Document incompatibilities between HBase 1.1.2 and HBase 2.0 > > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch, > HBASE-20369_v5.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiriguna Bharat Rao updated HBASE-20369: - Attachment: (was: HBASE-20369_v5.patch) > Document incompatibilities between HBase 1.1.2 and HBase 2.0 > > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiriguna Bharat Rao updated HBASE-20369: - Attachment: HBASE-20369_v5.patch > Document incompatibilities between HBase 1.1.2 and HBase 2.0 > > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch, > HBASE-20369_v5.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20480) Multiple one-time cell objects are allocated and de-allocated when working with CCM
[ https://issues.apache.org/jira/browse/HBASE-20480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472111#comment-16472111 ] Anoop Sam John commented on HBASE-20480: bq.We suggest to use and reuse the ThreadLocal variables per CCM, ThreadLocal based accessing will have some perf penalty though! > Multiple one-time cell objects are allocated and de-allocated when working > with CCM > --- > > Key: HBASE-20480 > URL: https://issues.apache.org/jira/browse/HBASE-20480 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Priority: Major > Attachments: HBASE-20480-V02.patch > > > We believe that the cause for some read performance degradation while working > with CellChunkMap (CCM). Multiple one-time cell objects are allocated and > de-allocated when performing multiple reads and working with CCM MemStore. We > have a couple of ideas for solution. More details will follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-16191) Add stop_regionserver and stop_master to shell
[
https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472069#comment-16472069
]
Hadoop QA commented on HBASE-16191:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} rubocop {color} | {color:green} 0m
13s{color} | {color:green} The patch generated 0 new + 410 unchanged - 2 fixed
= 410 total (was 412) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 5s{color} | {color:orange} The patch generated 4 new + 725 unchanged - 0
fixed = 729 total (was 725) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
50s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-16191 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12923024/HBASE-16191.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux 64e7aff44335 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c60578d982 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| rubocop | v0.55.0 |
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/12793/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12793/testReport/ |
| Max. process+thread count | 2071 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12793/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Add stop_regionserver and stop_master to shell
> --
>
> Key: HBASE-16191
> URL: https://issues.apache.org/jira/browse/HBASE-16191
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-16191.master.001.patch,
> HBASE-16191.master.002.patch, hbase-16191_v1.patch, hbase-16191_v1.patch
>
>
> Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed
> that for testing something. Seems useful to add.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20545) Improve performance of BaseLoadBalancer.retainAssignment
[ https://issues.apache.org/jira/browse/HBASE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472065#comment-16472065 ] Ted Yu commented on HBASE-20545: Please check branch-1 QA test failures. > Improve performance of BaseLoadBalancer.retainAssignment > > > Key: HBASE-20545 > URL: https://issues.apache.org/jira/browse/HBASE-20545 > Project: HBase > Issue Type: Improvement > Components: Balancer >Affects Versions: 1.4.4, 2.0.0 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.5 > > Attachments: HBASE-20545.branch-1.4.001.patch, > HBASE-20545.branch-2.001.patch > > > I was measuring perf at scale with a 1m region table and noticed some > improvements can be made to BaseLoadBalancer.retainAssignment(). > retainAssignment() spends a few mins to enable a 1m regions table and also > generates a lot of objects unnecessarily. This jira is to make the most > common case go faster with very minimal changes. A slightly modified version > of this patch takes about 5 seconds for a 1m region table ignoring the time > spent in createCluster(). I think locality can be refreshed during master > startup in different ways without taking time in retainAssignment, but will > follow up on that in subsequent jiras. Leaving it untouched here, but wanted > to call out the time taken without that method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-16191) Add stop_regionserver and stop_master to shell
[ https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472052#comment-16472052 ] Ted Yu commented on HBASE-16191: lgtm > Add stop_regionserver and stop_master to shell > -- > > Key: HBASE-16191 > URL: https://issues.apache.org/jira/browse/HBASE-16191 > Project: HBase > Issue Type: Improvement >Reporter: Enis Soztutar >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-16191.master.001.patch, > HBASE-16191.master.002.patch, hbase-16191_v1.patch, hbase-16191_v1.patch > > > Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed > that for testing something. Seems useful to add. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472039#comment-16472039
]
Hudson commented on HBASE-20004:
Results for branch master
[build #327 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/327/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/327//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/327//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/327//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Client is not able to execute REST queries in a secure cluster
> --
>
> Key: HBASE-20004
> URL: https://issues.apache.org/jira/browse/HBASE-20004
> Project: HBase
> Issue Type: Bug
> Components: REST, security
>Affects Versions: 1.3.1
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20004.branch-1.patch,
> HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch,
> HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch,
> HBASE-20004.v2.patch
>
>
> Firefox browser is not able to negotiate REST queries with server in secure
> mode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-16191) Add stop_regionserver and stop_master to shell
[
https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472013#comment-16472013
]
Hadoop QA commented on HBASE-16191:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} rubocop {color} | {color:green} 0m
14s{color} | {color:green} The patch generated 0 new + 410 unchanged - 2 fixed
= 410 total (was 412) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 5s{color} | {color:orange} The patch generated 4 new + 725 unchanged - 0
fixed = 729 total (was 725) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
46s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-16191 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12923024/HBASE-16191.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux d2fae362233c 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c60578d982 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| rubocop | v0.55.0 |
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/12792/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12792/testReport/ |
| Max. process+thread count | 2097 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12792/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Add stop_regionserver and stop_master to shell
> --
>
> Key: HBASE-16191
> URL: https://issues.apache.org/jira/browse/HBASE-16191
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-16191.master.001.patch,
> HBASE-16191.master.002.patch, hbase-16191_v1.patch, hbase-16191_v1.patch
>
>
> Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed
> that for testing something. Seems useful to add.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20571) JMXJsonServlet generates invalid JSON if it has NaN in metrics
[
https://issues.apache.org/jira/browse/HBASE-20571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-20571:
Fix Version/s: 2.0.1
Status: Patch Available (was: Open)
> JMXJsonServlet generates invalid JSON if it has NaN in metrics
> --
>
> Key: HBASE-20571
> URL: https://issues.apache.org/jira/browse/HBASE-20571
> Project: HBase
> Issue Type: Bug
> Components: UI
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: HBASE-20571.branch-2.0.001.patch
>
>
> {{/jmx}} servlet responses invalid JSON, if some metrics are NaN:
> {code}
> "l1CacheHitCount" : 0,
> "l1CacheMissCount" : 0,
> "l1CacheHitRatio" : NaN,
> "l1CacheMissRatio" : NaN,
> "l2CacheHitCount" : 0,
> "l2CacheMissCount" : 0,
> "l2CacheHitRatio" : 0.0,
> "l2CacheMissRatio" : 0.0,
> {code}
> NaN is an invalid character sequence in JSON. We should not response NaN in
> metrics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20571) JMXJsonServlet generates invalid JSON if it has NaN in metrics
[
https://issues.apache.org/jira/browse/HBASE-20571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-20571:
Attachment: HBASE-20571.branch-2.0.001.patch
> JMXJsonServlet generates invalid JSON if it has NaN in metrics
> --
>
> Key: HBASE-20571
> URL: https://issues.apache.org/jira/browse/HBASE-20571
> Project: HBase
> Issue Type: Bug
> Components: UI
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
>Priority: Major
> Attachments: HBASE-20571.branch-2.0.001.patch
>
>
> {{/jmx}} servlet responses invalid JSON, if some metrics are NaN:
> {code}
> "l1CacheHitCount" : 0,
> "l1CacheMissCount" : 0,
> "l1CacheHitRatio" : NaN,
> "l1CacheMissRatio" : NaN,
> "l2CacheHitCount" : 0,
> "l2CacheMissCount" : 0,
> "l2CacheHitRatio" : 0.0,
> "l2CacheMissRatio" : 0.0,
> {code}
> NaN is an invalid character sequence in JSON. We should not response NaN in
> metrics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20571) JMXJsonServlet generates invalid JSON if it has NaN in metrics
Balazs Meszaros created HBASE-20571:
---
Summary: JMXJsonServlet generates invalid JSON if it has NaN in
metrics
Key: HBASE-20571
URL: https://issues.apache.org/jira/browse/HBASE-20571
Project: HBase
Issue Type: Bug
Components: UI
Reporter: Balazs Meszaros
Assignee: Balazs Meszaros
{{/jmx}} servlet responses invalid JSON, if some metrics are NaN:
{code}
"l1CacheHitCount" : 0,
"l1CacheMissCount" : 0,
"l1CacheHitRatio" : NaN,
"l1CacheMissRatio" : NaN,
"l2CacheHitCount" : 0,
"l2CacheMissCount" : 0,
"l2CacheHitRatio" : 0.0,
"l2CacheMissRatio" : 0.0,
{code}
NaN is an invalid character sequence in JSON. We should not response NaN in
metrics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-16191) Add stop_regionserver and stop_master to shell
[ https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-16191: -- Status: In Progress (was: Patch Available) > Add stop_regionserver and stop_master to shell > -- > > Key: HBASE-16191 > URL: https://issues.apache.org/jira/browse/HBASE-16191 > Project: HBase > Issue Type: Improvement >Reporter: Enis Soztutar >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-16191.master.001.patch, > HBASE-16191.master.002.patch, hbase-16191_v1.patch, hbase-16191_v1.patch > > > Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed > that for testing something. Seems useful to add. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16191) Add stop_regionserver and stop_master to shell
[ https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-16191: -- Status: Patch Available (was: In Progress) > Add stop_regionserver and stop_master to shell > -- > > Key: HBASE-16191 > URL: https://issues.apache.org/jira/browse/HBASE-16191 > Project: HBase > Issue Type: Improvement >Reporter: Enis Soztutar >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-16191.master.001.patch, > HBASE-16191.master.002.patch, hbase-16191_v1.patch, hbase-16191_v1.patch > > > Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed > that for testing something. Seems useful to add. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20570) CLONE - Integrate SparkOnHBase into HBase
[ https://issues.apache.org/jira/browse/HBASE-20570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471990#comment-16471990 ] Sean Busbey commented on HBASE-20570: - Please read HBASE-18405, which has a scope document of the spark support that is expected to be released by the Apache HBase project. > CLONE - Integrate SparkOnHBase into HBase > - > > Key: HBASE-20570 > URL: https://issues.apache.org/jira/browse/HBASE-20570 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: ujjawal kumar >Assignee: Theodore michael Malaska >Priority: Major > Fix For: 3.0.0 > > > This Jira is to ask if SparkOnHBase can find a home in side HBase core. > Here is the github: > https://github.com/cloudera-labs/SparkOnHBase > I am the core author of this project and the license is Apache 2.0 > A blog explaining this project is here > http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/ > A spark Streaming example is here > http://blog.cloudera.com/blog/2014/11/how-to-do-near-real-time-sessionization-with-spark-streaming-and-apache-hadoop/ > A real customer using this in produce is blogged here > http://blog.cloudera.com/blog/2015/03/how-edmunds-com-used-spark-streaming-to-build-a-near-real-time-dashboard/ > Please debate and let me know what I can do to make this happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20570) CLONE - Integrate SparkOnHBase into HBase
[ https://issues.apache.org/jira/browse/HBASE-20570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob resolved HBASE-20570. --- Resolution: Duplicate This looks like a duplicate of HBASE-13992, which is currently on track to be included in HBase 3.0 (the next major release). > CLONE - Integrate SparkOnHBase into HBase > - > > Key: HBASE-20570 > URL: https://issues.apache.org/jira/browse/HBASE-20570 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: ujjawal kumar >Assignee: Theodore michael Malaska >Priority: Major > Fix For: 3.0.0 > > > This Jira is to ask if SparkOnHBase can find a home in side HBase core. > Here is the github: > https://github.com/cloudera-labs/SparkOnHBase > I am the core author of this project and the license is Apache 2.0 > A blog explaining this project is here > http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/ > A spark Streaming example is here > http://blog.cloudera.com/blog/2014/11/how-to-do-near-real-time-sessionization-with-spark-streaming-and-apache-hadoop/ > A real customer using this in produce is blogged here > http://blog.cloudera.com/blog/2015/03/how-edmunds-com-used-spark-streaming-to-build-a-near-real-time-dashboard/ > Please debate and let me know what I can do to make this happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-16191) Add stop_regionserver and stop_master to shell
[
https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471986#comment-16471986
]
Peter Somogyi commented on HBASE-16191:
---
This is how it looks like on a cluster:
{noformat}
hbase(main):006:0* stop_regionserver 'psom-shell-2.gce.cloudera.com:22101'
Took 0.7779 seconds
hbase(main):007:0> stop_master
Took 0.4967 seconds
hbase(main):008:0> stop_master
ERROR: KeeperErrorCode = NoNode for /hbase/master
For usage try 'help "stop_master"'
Took 11.5361 seconds
hbase(main):009:0> stop_regionserver 'psom-shell-2.gce.cloudera.com:22101'
ERROR: Connection refused
For usage try 'help "stop_regionserver"'
Took 0.2501 seconds
{noformat}
> Add stop_regionserver and stop_master to shell
> --
>
> Key: HBASE-16191
> URL: https://issues.apache.org/jira/browse/HBASE-16191
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-16191.master.001.patch,
> HBASE-16191.master.002.patch, hbase-16191_v1.patch, hbase-16191_v1.patch
>
>
> Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed
> that for testing something. Seems useful to add.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-16191) Add stop_regionserver and stop_master to shell
[
https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471980#comment-16471980
]
Mike Drob commented on HBASE-16191:
---
>From the javadoc for {{Admin::stopMaster}}:
{quote}Notice that, a success stopMaster call may ends with an error since the
remote server has already been shutdown.{quote}
Is this important to consider for the shell? What does success look like? Will
we print a lot of output to the console?
> Add stop_regionserver and stop_master to shell
> --
>
> Key: HBASE-16191
> URL: https://issues.apache.org/jira/browse/HBASE-16191
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-16191.master.001.patch,
> HBASE-16191.master.002.patch, hbase-16191_v1.patch, hbase-16191_v1.patch
>
>
> Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed
> that for testing something. Seems useful to add.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471963#comment-16471963
]
Mike Drob commented on HBASE-20567:
---
Hmm, I'm concerned about changing the CP API here, do we think there is a way
to do the same thing by adding a new method and having default implementation
of that call the old?
So we end up with something like:
{code}
default void preModifyTable(final
ObserverContext ctx,
final TableName tableName, TableDescriptor newDescriptor) throws
IOException {}
default void preModifyTable(final
ObserverContext ctx,
final TableName tableName, TableDescriptor currentDescriptor,
TableDescriptor newDescriptor)
throws IOException {
preModifyTable(ctx, tableName, newDescriptor);
}
{code}
This way folks that are currently implementing preModifyTable don't break and
folks that need the new method can use that. Would have to doc that If you
override both, then only the first will get called.
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-20567.master.001.patch
>
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-16191) Add stop_regionserver and stop_master to shell
[ https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-16191: -- Attachment: HBASE-16191.master.002.patch > Add stop_regionserver and stop_master to shell > -- > > Key: HBASE-16191 > URL: https://issues.apache.org/jira/browse/HBASE-16191 > Project: HBase > Issue Type: Improvement >Reporter: Enis Soztutar >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-16191.master.001.patch, > HBASE-16191.master.002.patch, hbase-16191_v1.patch, hbase-16191_v1.patch > > > Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed > that for testing something. Seems useful to add. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-16191) Add stop_regionserver and stop_master to shell
[
https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471868#comment-16471868
]
Hadoop QA commented on HBASE-16191:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 13 new + 412 unchanged - 0 fixed =
425 total (was 412) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 5s{color} | {color:orange} The patch generated 4 new + 725 unchanged - 0
fixed = 729 total (was 725) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
28s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-16191 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12923018/HBASE-16191.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux 41c21ca69c37 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c60578d982 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| rubocop | v0.55.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/12791/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/12791/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12791/testReport/ |
| Max. process+thread count | 2193 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12791/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Add stop_regionserver and stop_master to shell
> --
>
> Key: HBASE-16191
> URL: https://issues.apache.org/jira/browse/HBASE-16191
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-16191.master.001.patch, hbase-16191_v1.patch,
> hbase-16191_v1.patch
>
>
> Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed
> that for testing something. Seems useful to add.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20424) Allow writing WAL to local and remote cluster concurrently
[ https://issues.apache.org/jira/browse/HBASE-20424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471863#comment-16471863 ] Duo Zhang commented on HBASE-20424: --- Talked with [~zghaobac] offline, the basic idea is: 1. When splitting wals, we need to copy the wals which are ended with '.syncrep' to the remote wal directory if we are in A state. If it is not possible then the split will fail. 2. If we are transiting to DA, then give up copying and split the wals in the normal way. 3. If we are transiting to S, then the remote cluster should have been transited from S to DA, so a. When transiting from S to DA, we will make a snapshot of the current remote wals(actually we only need the name). For example, for peer '1', we first rename the directory to '1-replay', and finally rename it to '1-snapshot'. b. When transiting to S, we will check the snapshot directory on the remote cluster, the wals which are placed in the directory will be skipped when splitting. This is because that we know these wals will be replicated back to us. c. When transiting to S, we need to confirm that there is no wal ended with '.syncrep' under the wal directory or being splitting(under the old wal directory is allowed, although it will also be deleted soon). This is used to make sure we do not replay the wals which are replayed at the remote cluster when transiting from S to DA. > Allow writing WAL to local and remote cluster concurrently > -- > > Key: HBASE-20424 > URL: https://issues.apache.org/jira/browse/HBASE-20424 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > > For better performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20551) Can't find bundle for base name org.apache.jasper.resources.LocalStrings, locale en_US
[
https://issues.apache.org/jira/browse/HBASE-20551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471862#comment-16471862
]
Peter Somogyi commented on HBASE-20551:
---
Thanks [~adityap] for checking. Resolved as duplicate.
> Can't find bundle for base name org.apache.jasper.resources.LocalStrings,
> locale en_US
> --
>
> Key: HBASE-20551
> URL: https://issues.apache.org/jira/browse/HBASE-20551
> Project: HBase
> Issue Type: Bug
> Components: build, java, test
>Affects Versions: 1.2.6
> Environment: Ubuntu 18.04
> OpenJDK version "1.8.0_162"
> maven - 3.3.9
>Reporter: Aditya Pawaskar
>Priority: Major
>
> While building HBase 1.2.6 on Ubuntu 18.04, I observe build failure with
> error :
> *Can't find bundle for base name org.apache.jasper.resources.LocalStrings,
> locale en_US*
> Steps to produce:
> git clone git://github.com/apache/hbase.git
> {color:#005cc5}cd{color} hbase
> git checkout rel/1.2.6
> mvn package
>
> Error logs:
> [INFO] Logging to
> org.slf4j.impl.MavenSimpleLogger(org.mortbay.log) via org.mortbay.log.Slf4jLog
> java.util.MissingResourceException: Can't find bundle for base name
> org.apache.jasper.resources.LocalStrings, locale en_US
> at
> java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:1573)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-20551) Can't find bundle for base name org.apache.jasper.resources.LocalStrings, locale en_US
[
https://issues.apache.org/jira/browse/HBASE-20551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-20551.
---
Resolution: Duplicate
Duplicate of HBASE-19188.
> Can't find bundle for base name org.apache.jasper.resources.LocalStrings,
> locale en_US
> --
>
> Key: HBASE-20551
> URL: https://issues.apache.org/jira/browse/HBASE-20551
> Project: HBase
> Issue Type: Bug
> Components: build, java, test
>Affects Versions: 1.2.6
> Environment: Ubuntu 18.04
> OpenJDK version "1.8.0_162"
> maven - 3.3.9
>Reporter: Aditya Pawaskar
>Priority: Major
>
> While building HBase 1.2.6 on Ubuntu 18.04, I observe build failure with
> error :
> *Can't find bundle for base name org.apache.jasper.resources.LocalStrings,
> locale en_US*
> Steps to produce:
> git clone git://github.com/apache/hbase.git
> {color:#005cc5}cd{color} hbase
> git checkout rel/1.2.6
> mvn package
>
> Error logs:
> [INFO] Logging to
> org.slf4j.impl.MavenSimpleLogger(org.mortbay.log) via org.mortbay.log.Slf4jLog
> java.util.MissingResourceException: Can't find bundle for base name
> org.apache.jasper.resources.LocalStrings, locale en_US
> at
> java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:1573)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20551) Can't find bundle for base name org.apache.jasper.resources.LocalStrings, locale en_US
[
https://issues.apache.org/jira/browse/HBASE-20551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471857#comment-16471857
]
Aditya Pawaskar commented on HBASE-20551:
-
Thank you for help [~psomogyi]. patches given in HBASE-19188 worked for HBase
1.2.6.--
> Can't find bundle for base name org.apache.jasper.resources.LocalStrings,
> locale en_US
> --
>
> Key: HBASE-20551
> URL: https://issues.apache.org/jira/browse/HBASE-20551
> Project: HBase
> Issue Type: Bug
> Components: build, java, test
>Affects Versions: 1.2.6
> Environment: Ubuntu 18.04
> OpenJDK version "1.8.0_162"
> maven - 3.3.9
>Reporter: Aditya Pawaskar
>Priority: Major
>
> While building HBase 1.2.6 on Ubuntu 18.04, I observe build failure with
> error :
> *Can't find bundle for base name org.apache.jasper.resources.LocalStrings,
> locale en_US*
> Steps to produce:
> git clone git://github.com/apache/hbase.git
> {color:#005cc5}cd{color} hbase
> git checkout rel/1.2.6
> mvn package
>
> Error logs:
> [INFO] Logging to
> org.slf4j.impl.MavenSimpleLogger(org.mortbay.log) via org.mortbay.log.Slf4jLog
> java.util.MissingResourceException: Can't find bundle for base name
> org.apache.jasper.resources.LocalStrings, locale en_US
> at
> java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:1573)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-16191) Add stop_regionserver and stop_master to shell
[ https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471851#comment-16471851 ] Peter Somogyi commented on HBASE-16191: --- HBASE-16191.master.001.patch: Rebased [~enis]'s patch and added comment for stop_regionserver to use graceful_stop.sh instead. > Add stop_regionserver and stop_master to shell > -- > > Key: HBASE-16191 > URL: https://issues.apache.org/jira/browse/HBASE-16191 > Project: HBase > Issue Type: Improvement >Reporter: Enis Soztutar >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-16191.master.001.patch, hbase-16191_v1.patch, > hbase-16191_v1.patch > > > Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed > that for testing something. Seems useful to add. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16191) Add stop_regionserver and stop_master to shell
[ https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-16191: -- Fix Version/s: 3.0.0 Status: Patch Available (was: Open) > Add stop_regionserver and stop_master to shell > -- > > Key: HBASE-16191 > URL: https://issues.apache.org/jira/browse/HBASE-16191 > Project: HBase > Issue Type: Improvement >Reporter: Enis Soztutar >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-16191.master.001.patch, hbase-16191_v1.patch, > hbase-16191_v1.patch > > > Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed > that for testing something. Seems useful to add. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16191) Add stop_regionserver and stop_master to shell
[ https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-16191: -- Attachment: HBASE-16191.master.001.patch > Add stop_regionserver and stop_master to shell > -- > > Key: HBASE-16191 > URL: https://issues.apache.org/jira/browse/HBASE-16191 > Project: HBase > Issue Type: Improvement >Reporter: Enis Soztutar >Assignee: Peter Somogyi >Priority: Major > Attachments: HBASE-16191.master.001.patch, hbase-16191_v1.patch, > hbase-16191_v1.patch > > > Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed > that for testing something. Seems useful to add. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-16191) Add stop_regionserver and stop_master to shell
[ https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi reassigned HBASE-16191: - Assignee: Peter Somogyi (was: Enis Soztutar) > Add stop_regionserver and stop_master to shell > -- > > Key: HBASE-16191 > URL: https://issues.apache.org/jira/browse/HBASE-16191 > Project: HBase > Issue Type: Improvement >Reporter: Enis Soztutar >Assignee: Peter Somogyi >Priority: Major > Attachments: HBASE-16191.master.001.patch, hbase-16191_v1.patch, > hbase-16191_v1.patch > > > Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed > that for testing something. Seems useful to add. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20570) CLONE - Integrate SparkOnHBase into HBase
[ https://issues.apache.org/jira/browse/HBASE-20570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471822#comment-16471822 ] Ted Yu commented on HBASE-20570: There is hbase-spark module in master branch. Can your work be integrated to this module ? > CLONE - Integrate SparkOnHBase into HBase > - > > Key: HBASE-20570 > URL: https://issues.apache.org/jira/browse/HBASE-20570 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: ujjawal kumar >Assignee: Theodore michael Malaska >Priority: Major > Fix For: 3.0.0 > > > This Jira is to ask if SparkOnHBase can find a home in side HBase core. > Here is the github: > https://github.com/cloudera-labs/SparkOnHBase > I am the core author of this project and the license is Apache 2.0 > A blog explaining this project is here > http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/ > A spark Streaming example is here > http://blog.cloudera.com/blog/2014/11/how-to-do-near-real-time-sessionization-with-spark-streaming-and-apache-hadoop/ > A real customer using this in produce is blogged here > http://blog.cloudera.com/blog/2015/03/how-edmunds-com-used-spark-streaming-to-build-a-near-real-time-dashboard/ > Please debate and let me know what I can do to make this happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471742#comment-16471742
]
Hudson commented on HBASE-20004:
Results for branch branch-1
[build #311 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/311/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/311//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/311//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/311//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Client is not able to execute REST queries in a secure cluster
> --
>
> Key: HBASE-20004
> URL: https://issues.apache.org/jira/browse/HBASE-20004
> Project: HBase
> Issue Type: Bug
> Components: REST, security
>Affects Versions: 1.3.1
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20004.branch-1.patch,
> HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch,
> HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch,
> HBASE-20004.v2.patch
>
>
> Firefox browser is not able to negotiate REST queries with server in secure
> mode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[
https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471690#comment-16471690
]
Hadoop QA commented on HBASE-20411:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
9s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
30s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 30s{color}
| {color:red} hbase-server generated 6 new + 182 unchanged - 6 fixed = 188
total (was 188) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} hbase-server: The patch generated 0 new + 493
unchanged - 5 fixed = 493 total (was 498) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
39s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 58s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} hbase-server generated 0 new + 0 unchanged - 2 fixed
= 0 total (was 2) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}109m
23s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}144m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:369877d |
| JIRA Issue | HBASE-20411 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922960/HBASE-20411.branch-2.0.013.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 66ec4a34a23f 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.0 / f46f70921c |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/12789/artifact/patchprocess/diff-compile-javac-hbase-server.txt
|
| Test Results |
[jira] [Assigned] (HBASE-20549) Dead MBean: Hadoop:service=HBase,sub=Memory,name=RegionServer
[
https://issues.apache.org/jira/browse/HBASE-20549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros reassigned HBASE-20549:
---
Assignee: Balazs Meszaros
> Dead MBean: Hadoop:service=HBase,sub=Memory,name=RegionServer
> -
>
> Key: HBASE-20549
> URL: https://issues.apache.org/jira/browse/HBASE-20549
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Major
> Attachments: Screen Shot 2018-05-08 at 10.05.57 PM.png
>
>
> I'm studying flushing and blocking memstores. I came across the MBean
> Hadoop:service=HBase,sub=Memory,name=RegionServer It looks promising with
> metrics on blocked flush time, etc. See below. All metrics are zero. I tried
> to get it going. It is connected to the heap tuner facitlity that we might
> want to turn on. I managed to get the latter going w/ below configs (need
> to dump out more on what this thing is doing because it is tricky enabling it)
> {code:java}
>
>
>
> hbase.regionserver.global.memstore.size.min.range
>
> 0.39
>
>
>
> hbase.regionserver.global.memstore.size.max.range
>
> 0.41
>
>
>
> hfile.block.cache.size.max.range
>
> 0.41
>
>
>
> hfile.block.cache.size.min.range
>
> 0.39
>
>
> {code}
> ... but still the metrics are not updated.
> We should make this bean work whether tuning is on or off or remove it.
> !Screen Shot 2018-05-08 at 10.05.57 PM.png|width=440,height=692!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HBASE-20549) Dead MBean: Hadoop:service=HBase,sub=Memory,name=RegionServer
[
https://issues.apache.org/jira/browse/HBASE-20549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-20549 started by Balazs Meszaros.
---
> Dead MBean: Hadoop:service=HBase,sub=Memory,name=RegionServer
> -
>
> Key: HBASE-20549
> URL: https://issues.apache.org/jira/browse/HBASE-20549
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Major
> Attachments: Screen Shot 2018-05-08 at 10.05.57 PM.png
>
>
> I'm studying flushing and blocking memstores. I came across the MBean
> Hadoop:service=HBase,sub=Memory,name=RegionServer It looks promising with
> metrics on blocked flush time, etc. See below. All metrics are zero. I tried
> to get it going. It is connected to the heap tuner facitlity that we might
> want to turn on. I managed to get the latter going w/ below configs (need
> to dump out more on what this thing is doing because it is tricky enabling it)
> {code:java}
>
>
>
> hbase.regionserver.global.memstore.size.min.range
>
> 0.39
>
>
>
> hbase.regionserver.global.memstore.size.max.range
>
> 0.41
>
>
>
> hfile.block.cache.size.max.range
>
> 0.41
>
>
>
> hfile.block.cache.size.min.range
>
> 0.39
>
>
> {code}
> ... but still the metrics are not updated.
> We should make this bean work whether tuning is on or off or remove it.
> !Screen Shot 2018-05-08 at 10.05.57 PM.png|width=440,height=692!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20549) Dead MBean: Hadoop:service=HBase,sub=Memory,name=RegionServer
[
https://issues.apache.org/jira/browse/HBASE-20549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-20549:
Description:
I'm studying flushing and blocking memstores. I came across the MBean
Hadoop:service=HBase,sub=Memory,name=RegionServer It looks promising with
metrics on blocked flush time, etc. See below. All metrics are zero. I tried to
get it going. It is connected to the heap tuner facitlity that we might want to
turn on. I managed to get the latter going w/ below configs (need to dump
out more on what this thing is doing because it is tricky enabling it)
{code:java}
hbase.regionserver.global.memstore.size.min.range
0.39
hbase.regionserver.global.memstore.size.max.range
0.41
hfile.block.cache.size.max.range
0.41
hfile.block.cache.size.min.range
0.39
{code}
... but still the metrics are not updated.
We should make this bean work whether tuning is on or off or remove it.
!Screen Shot 2018-05-08 at 10.05.57 PM.png|width=440,height=692!
was:
I'm studying flushing and blocking memstores. I came across the MBean
Hadoop:service=HBase,sub=Memory,name=RegionServer It looks promising with
metrics on blocked flush time, etc. See below. All metrics are zero. I tried to
get it going. It is connected to the heap tuner facitlity that we might want to
turn on. I managed to get the latter going w/ below configs (need to dump
out more on what this thing is doing because it is tricky enabling it)
{code}
hbase.regionserver.global.memstore.size.min.range
0.39
hbase.regionserver.global.memstore.size.max.range
0.41
hfile.block.cache.size.max.range
0.41
hfile.block.cache.size.min.range
0.39
{code}
... but still the metrics are not updated.
We should make this bean work whether tuning is on or off or remove it.
!Screen Shot 2018-05-08 at 10.05.57 PM.png!
> Dead MBean: Hadoop:service=HBase,sub=Memory,name=RegionServer
> -
>
> Key: HBASE-20549
> URL: https://issues.apache.org/jira/browse/HBASE-20549
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Major
> Attachments: Screen Shot 2018-05-08 at 10.05.57 PM.png
>
>
> I'm studying flushing and blocking memstores. I came across the MBean
> Hadoop:service=HBase,sub=Memory,name=RegionServer It looks promising with
> metrics on blocked flush time, etc. See below. All metrics are zero. I tried
> to get it going. It is connected to the heap tuner facitlity that we might
> want to turn on. I managed to get the latter going w/ below configs (need
> to dump out more on what this thing is doing because it is tricky enabling it)
> {code:java}
>
>
>
> hbase.regionserver.global.memstore.size.min.range
>
> 0.39
>
>
>
> hbase.regionserver.global.memstore.size.max.range
>
> 0.41
>
>
>
> hfile.block.cache.size.max.range
>
> 0.41
>
>
>
> hfile.block.cache.size.min.range
>
> 0.39
>
>
> {code}
> ... but still the metrics are not updated.
> We should make this bean work whether tuning is on or off or remove it.
> !Screen Shot 2018-05-08 at 10.05.57 PM.png|width=440,height=692!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20568) Concurrency issue for CombinedAsyncWriter
[ https://issues.apache.org/jira/browse/HBASE-20568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471667#comment-16471667 ] Duo Zhang commented on HBASE-20568: --- It is not easy to fix, as now we only depend on the public method of AsyncWriter interface. And after HBASE-20424, we will not need the sequential mode any more, so maybe we can just resolve this issue by resolving HBASE-20424... > Concurrency issue for CombinedAsyncWriter > - > > Key: HBASE-20568 > URL: https://issues.apache.org/jira/browse/HBASE-20568 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > > In sequential mode, we need to wait for the previous writer's sync completes > before issue the sync for the next writer. But the sync request is > asynchronous, so we will return immediately and then we can call append, so > it is possible that we call append and sync at the same time and cause > concurrency issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20570) CLONE - Integrate SparkOnHBase into HBase
ujjawal kumar created HBASE-20570: - Summary: CLONE - Integrate SparkOnHBase into HBase Key: HBASE-20570 URL: https://issues.apache.org/jira/browse/HBASE-20570 Project: HBase Issue Type: New Feature Components: spark Reporter: ujjawal kumar Assignee: Theodore michael Malaska Fix For: 3.0.0 This Jira is to ask if SparkOnHBase can find a home in side HBase core. Here is the github: https://github.com/cloudera-labs/SparkOnHBase I am the core author of this project and the license is Apache 2.0 A blog explaining this project is here http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/ A spark Streaming example is here http://blog.cloudera.com/blog/2014/11/how-to-do-near-real-time-sessionization-with-spark-streaming-and-apache-hadoop/ A real customer using this in produce is blogged here http://blog.cloudera.com/blog/2015/03/how-edmunds-com-used-spark-streaming-to-build-a-near-real-time-dashboard/ Please debate and let me know what I can do to make this happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20569) NPE in RecoverStandbyProcedure.execute
Duo Zhang created HBASE-20569: - Summary: NPE in RecoverStandbyProcedure.execute Key: HBASE-20569 URL: https://issues.apache.org/jira/browse/HBASE-20569 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang We call ReplaySyncReplicationWALManager.initPeerWorkers in INIT_WORKERS state and then use it in DISPATCH_TASKS. But if we restart the master and the procedure is restarted from state DISPATCH_TASKS, no one will call the initPeerWorkers method and we will get NPE. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471662#comment-16471662
]
Nihal Jain commented on HBASE-20566:
Dug a little deep. Points to note:
* During "hbase:quota" table creation, the CreateTableProcedure stays in
WAITING state as the child procedure AssignProcedure tries to assign table
regions
* The postCreateTable() hook is not reached which actually does default RS
group assignment for each table.
* None of the CreateTableProcedure hooks seem to be idle place for the logic
to add the current system table to rsgroup
Wondering where this logic should go!
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20564) Tighter ByteBufferKeyValue Cell Comparator
[ https://issues.apache.org/jira/browse/HBASE-20564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471636#comment-16471636 ] ramkrishna.s.vasudevan commented on HBASE-20564: As per the patch some values are taken out seperately and avoid deserialization again. Not that we are explicitly caching it and in 1.4 also we were not caching. I confirmed it once again. > Tighter ByteBufferKeyValue Cell Comparator > -- > > Key: HBASE-20564 > URL: https://issues.apache.org/jira/browse/HBASE-20564 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20564.branch-2.0.001.patch, > HBASE-20564.branch-2.patch > > > Comparing Cells in hbase2 takes almost 3x the CPU. > In hbase1, its a keyValue backed by a byte array caching a few important > values.. In hbase2, its a NoTagByteBufferChunkKeyValue(?) deserializing the > row/family/qualifier lengths repeatedly. > I tried making a purposed comparator -- one that was not generic -- and it > seemed to have a nicer profile coming close to hbase1 in percentage used > (I'll post graphs) when I ran it in my perpetual memstore filler (See scripts > attached to HBASE-20483). It doesn't work when I try to run it on cluster. > Let me run unit tests to see if it can figure what I have wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20568) Concurrency issue for CombinedAsyncWriter
Duo Zhang created HBASE-20568: - Summary: Concurrency issue for CombinedAsyncWriter Key: HBASE-20568 URL: https://issues.apache.org/jira/browse/HBASE-20568 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang In sequential mode, we need to wait for the previous writer's sync completes before issue the sync for the next writer. But the sync request is asynchronous, so we will return immediately and then we can call append, so it is possible that we call append and sync at the same time and cause concurrency issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20564) Tighter ByteBufferKeyValue Cell Comparator
[ https://issues.apache.org/jira/browse/HBASE-20564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471631#comment-16471631 ] Anoop Sam John commented on HBASE-20564: I can see the changes in the Comparator and do the type check only once. But in 1.4 KV which len values are getting cached? I don't think we were doing any. May be with the refactoring in BBKV, you try avoid deserializing of few items? > Tighter ByteBufferKeyValue Cell Comparator > -- > > Key: HBASE-20564 > URL: https://issues.apache.org/jira/browse/HBASE-20564 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20564.branch-2.0.001.patch, > HBASE-20564.branch-2.patch > > > Comparing Cells in hbase2 takes almost 3x the CPU. > In hbase1, its a keyValue backed by a byte array caching a few important > values.. In hbase2, its a NoTagByteBufferChunkKeyValue(?) deserializing the > row/family/qualifier lengths repeatedly. > I tried making a purposed comparator -- one that was not generic -- and it > seemed to have a nicer profile coming close to hbase1 in percentage used > (I'll post graphs) when I ran it in my perpetual memstore filler (See scripts > attached to HBASE-20483). It doesn't work when I try to run it on cluster. > Let me run unit tests to see if it can figure what I have wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20564) Tighter ByteBufferKeyValue Cell Comparator
[
https://issues.apache.org/jira/browse/HBASE-20564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471605#comment-16471605
]
Hadoop QA commented on HBASE-20564:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
25s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
5s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} hbase-common: The patch generated 0 new + 7
unchanged - 6 fixed = 7 total (was 13) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
22s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:369877d |
| JIRA Issue | HBASE-20564 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922966/HBASE-20564.branch-2.0.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 16ea904f2439 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.0 / f46f70921c |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12790/testReport/ |
| Max. process+thread count | 319 (vs. ulimit of 1) |
| modules | C: hbase-common U: hbase-common |
[jira] [Commented] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471581#comment-16471581
]
Hadoop QA commented on HBASE-20567:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
46s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
14s{color} | {color:red} hbase-server: The patch generated 5 new + 303
unchanged - 3 fixed = 308 total (was 306) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
50s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}117m
1s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
37s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}170m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20567 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922958/HBASE-20567.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 3816cad18110 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c60578d982 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297;
[jira] [Commented] (HBASE-19064) Synchronous replication for HBase
[
https://issues.apache.org/jira/browse/HBASE-19064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471579#comment-16471579
]
Hudson commented on HBASE-19064:
Results for branch HBASE-19064
[build #125 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/125/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/125//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/125//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/125//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Synchronous replication for HBase
> -
>
> Key: HBASE-19064
> URL: https://issues.apache.org/jira/browse/HBASE-19064
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> The guys from Alibaba made a presentation on HBaseCon Asia about the
> synchronous replication for HBase. We(Xiaomi) think this is a very useful
> feature for HBase so we want to bring it into the community version.
> This is a big feature so we plan to do it in a feature branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20565) ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result since 1.4
[
https://issues.apache.org/jira/browse/HBASE-20565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471531#comment-16471531
]
Zheng Hu commented on HBASE-20565:
--
Oh, change my mind now, it's not bug for branch-1.4, see below:
{code}
ColumnPaginationFilter
^
|---|
0 1 2 3 4 5 6 7 8 9
|---|
|
v
ColumnRangeFilter
{code}
The union set of ColumnPaginationFilter and ColumnRangeFilter should be
[1,2,3,4], so the size of results should be 4.
[~jinghe], I think the correct filterList to meet your requirement would be:
{code}
// Column range from 1 to 9.
StringRange stringRange = new StringRange("1", true, "9", false);
ColumnRangeFilter filter1 =
new ColumnRangeFilter(Bytes.toBytes(stringRange.getStart()),
stringRange.isStartInclusive(),
Bytes.toBytes(stringRange.getEnd()), stringRange.isEndInclusive());
// ColumnPaginationFilter limit 5
ColumnPaginationFilter filter2 = new ColumnPaginationFilter(5,
Bytes.toBytes("1")); // here, we use columnOffset instead of offset=0.
FilterList filterList = new FilterList(Operator.MUST_PASS_ALL, filter1,
filter2);
{code}
> ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect
> result since 1.4
> -
>
> Key: HBASE-20565
> URL: https://issues.apache.org/jira/browse/HBASE-20565
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.4.4
>Reporter: Jerry He
>Assignee: Zheng Hu
>Priority: Major
> Attachments: debug.diff, debug.log, test-branch-1.4.patch
>
>
> When ColumnPaginationFilter is combined with ColumnRangeFilter, we may see
> incorrect result.
> Here is a simple example.
> One row with 10 columns c0, c1, c2, .., c9. I have a ColumnRangeFilter for
> range c2 to c9. Then I have a ColumnPaginationFilter with limit 5 and offset
> 0. FileterList is FilterList(Operator.MUST_PASS_ALL, ColumnRangeFilter,
> ColumnPaginationFilter).
> We expect 5 columns being returned. But in HBase 1.4 and after, 4 columns
> are returned.
> In 1.2.x, the correct 5 columns are returned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20564) Tighter ByteBufferKeyValue Cell Comparator
[ https://issues.apache.org/jira/browse/HBASE-20564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471524#comment-16471524 ] stack commented on HBASE-20564: --- .002 Fix build complaints. This patch doesn't work on cluster though. Suspect concurrency messing with backing BB buffer. Digging. > Tighter ByteBufferKeyValue Cell Comparator > -- > > Key: HBASE-20564 > URL: https://issues.apache.org/jira/browse/HBASE-20564 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20564.branch-2.0.001.patch, > HBASE-20564.branch-2.patch > > > Comparing Cells in hbase2 takes almost 3x the CPU. > In hbase1, its a keyValue backed by a byte array caching a few important > values.. In hbase2, its a NoTagByteBufferChunkKeyValue(?) deserializing the > row/family/qualifier lengths repeatedly. > I tried making a purposed comparator -- one that was not generic -- and it > seemed to have a nicer profile coming close to hbase1 in percentage used > (I'll post graphs) when I ran it in my perpetual memstore filler (See scripts > attached to HBASE-20483). It doesn't work when I try to run it on cluster. > Let me run unit tests to see if it can figure what I have wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20564) Tighter ByteBufferKeyValue Cell Comparator
[ https://issues.apache.org/jira/browse/HBASE-20564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20564: -- Attachment: HBASE-20564.branch-2.0.001.patch > Tighter ByteBufferKeyValue Cell Comparator > -- > > Key: HBASE-20564 > URL: https://issues.apache.org/jira/browse/HBASE-20564 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20564.branch-2.0.001.patch, > HBASE-20564.branch-2.patch > > > Comparing Cells in hbase2 takes almost 3x the CPU. > In hbase1, its a keyValue backed by a byte array caching a few important > values.. In hbase2, its a NoTagByteBufferChunkKeyValue(?) deserializing the > row/family/qualifier lengths repeatedly. > I tried making a purposed comparator -- one that was not generic -- and it > seemed to have a nicer profile coming close to hbase1 in percentage used > (I'll post graphs) when I ran it in my perpetual memstore filler (See scripts > attached to HBASE-20483). It doesn't work when I try to run it on cluster. > Let me run unit tests to see if it can figure what I have wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
