[jira] [Commented] (HBASE-21907) Should set priority for rpc request
[ https://issues.apache.org/jira/browse/HBASE-21907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770033#comment-16770033 ] Duo Zhang commented on HBASE-21907: --- Review board link: https://reviews.apache.org/r/69996/ > Should set priority for rpc request > --- > > Key: HBASE-21907 > URL: https://issues.apache.org/jira/browse/HBASE-21907 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21907.patch > > > Now in async client we just ignored the priority for RpcController. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21907) Should set priority for rpc request
[ https://issues.apache.org/jira/browse/HBASE-21907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21907: -- Attachment: HBASE-21907.patch > Should set priority for rpc request > --- > > Key: HBASE-21907 > URL: https://issues.apache.org/jira/browse/HBASE-21907 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21907.patch > > > Now in async client we just ignored the priority for RpcController. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21907) Should set priority for rpc request
[ https://issues.apache.org/jira/browse/HBASE-21907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770032#comment-16770032 ] Duo Zhang commented on HBASE-21907: --- Will add UT in the next patch. > Should set priority for rpc request > --- > > Key: HBASE-21907 > URL: https://issues.apache.org/jira/browse/HBASE-21907 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21907.patch > > > Now in async client we just ignored the priority for RpcController. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21907) Should set priority for rpc request
[ https://issues.apache.org/jira/browse/HBASE-21907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21907: -- Status: Patch Available (was: Open) > Should set priority for rpc request > --- > > Key: HBASE-21907 > URL: https://issues.apache.org/jira/browse/HBASE-21907 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21907.patch > > > Now in async client we just ignored the priority for RpcController. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21667) Move to latest ASF Parent POM
[ https://issues.apache.org/jira/browse/HBASE-21667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770025#comment-16770025 ] Peter Somogyi commented on HBASE-21667: --- TestReadOnlyZKClient passes locally on branch-2 patch. > Move to latest ASF Parent POM > - > > Key: HBASE-21667 > URL: https://issues.apache.org/jira/browse/HBASE-21667 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21667.branch-2.0.001.patch, > HBASE-21667.branch-2.0.002.patch, HBASE-21667.branch-2.001.patch, > HBASE-21667.branch-2.003.patch, HBASE-21667.branch-2.2.003.patch, > HBASE-21667.master.001.patch, HBASE-21667.master.002.patch, > HBASE-21667.master.003.patch > > > Currently HBase depends on version 18 which was released on 2016-05-18. > Version 21 was released in August 2018. > Relevant dependency upgrades > > ||Name||Currently used version||New version||Notes|| > |surefire.version|2.21.0|2.22.0| | > |maven-compiler-plugin|3.6.1|3.7| | > |maven-dependency-plugin|3.0.1|3.1.1| | > |maven-jar-plugin|3.0.0|3.0.2| | > |maven-javadoc-plugin|3.0.0|3.0.1| | > |maven-resources-plugin|2.7|3.1.0| | > |maven-site-plugin|3.4|3.7.1|Currently not relying on ASF version. See: > HBASE-18333| > |maven-source-plugin|3.0.0|3.0.1| | > |maven-shade-plugin|3.0.0|3.1.1|Newly added to ASF pom| > |maven-clean-plugin|3.0.0|3.1.0| | > |maven-project-info-reports-plugin |2.9|3.0.0| | > Version 21 added net.nicoulaj.maven.plugins:checksum-maven-plugin which > introduced SHA512 checksum instead of SHA1. Should verify if we can rely on > that for releases or breaks our current processes. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21667) Move to latest ASF Parent POM
[ https://issues.apache.org/jira/browse/HBASE-21667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-21667: -- Attachment: HBASE-21667.branch-2.2.003.patch > Move to latest ASF Parent POM > - > > Key: HBASE-21667 > URL: https://issues.apache.org/jira/browse/HBASE-21667 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21667.branch-2.0.001.patch, > HBASE-21667.branch-2.0.002.patch, HBASE-21667.branch-2.001.patch, > HBASE-21667.branch-2.003.patch, HBASE-21667.branch-2.2.003.patch, > HBASE-21667.master.001.patch, HBASE-21667.master.002.patch, > HBASE-21667.master.003.patch > > > Currently HBase depends on version 18 which was released on 2016-05-18. > Version 21 was released in August 2018. > Relevant dependency upgrades > > ||Name||Currently used version||New version||Notes|| > |surefire.version|2.21.0|2.22.0| | > |maven-compiler-plugin|3.6.1|3.7| | > |maven-dependency-plugin|3.0.1|3.1.1| | > |maven-jar-plugin|3.0.0|3.0.2| | > |maven-javadoc-plugin|3.0.0|3.0.1| | > |maven-resources-plugin|2.7|3.1.0| | > |maven-site-plugin|3.4|3.7.1|Currently not relying on ASF version. See: > HBASE-18333| > |maven-source-plugin|3.0.0|3.0.1| | > |maven-shade-plugin|3.0.0|3.1.1|Newly added to ASF pom| > |maven-clean-plugin|3.0.0|3.1.0| | > |maven-project-info-reports-plugin |2.9|3.0.0| | > Version 21 added net.nicoulaj.maven.plugins:checksum-maven-plugin which > introduced SHA512 checksum instead of SHA1. Should verify if we can rely on > that for releases or breaks our current processes. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21915) FileLink$FileLinkInputStream doesn't implement CanUnbuffer
[
https://issues.apache.org/jira/browse/HBASE-21915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770022#comment-16770022
]
stack commented on HBASE-21915:
---
Good one lads.
Whats up w/ the call to tryOpen inside the unbuffer? Are we opening the stream
when we do this? If fresh open, an unbuffer makes sense? Thanks.
> FileLink$FileLinkInputStream doesn't implement CanUnbuffer
> --
>
> Key: HBASE-21915
> URL: https://issues.apache.org/jira/browse/HBASE-21915
> Project: HBase
> Issue Type: Bug
> Components: Filesystem Integration
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Attachments: HBASE-21915.001.patch
>
>
> FileLinkInputStream is an InputStream which handles the indirection of where
> the real HFile lives. This implementation is wrapped via
> FSDataInputStreamWrapper and is transparent when it's being used by a caller.
> Often, we have an FSDataInputStreamWrapper wrapping a FileLinkInputStream
> which wraps an FSDataInputStream.
> The problem is that FileLinkInputStream does not implement the
> \{{CanUnbuffer}} interface, which means that the underlying
> {{FSDataInputStream}} for the HFile the link refers to doesn't get
> {{unbuffer()}} called on it. This can cause an open Socket to hang around, as
> described in HBASE-9393.
> Both [~wchevreuil] and myself have run into this, each for different users.
> We think the commonality as to why these users saw this (but we haven't run
> into it on our own) is that it requires a very large snapshot to be brought
> into a new system. Big kudos to [~esteban] for his help in diagnosing this as
> well!
> If this analysis is accurate, it would affect all branches.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21908) Remove Scan.getScanMetrics
[ https://issues.apache.org/jira/browse/HBASE-21908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770019#comment-16770019 ] stack commented on HBASE-21908: --- HBASE-13030 is the issue that deprecated Scan.getScanMetrics (you get them from Result instead). > Remove Scan.getScanMetrics > -- > > Key: HBASE-21908 > URL: https://issues.apache.org/jira/browse/HBASE-21908 > Project: HBase > Issue Type: Task > Components: Client, scan >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21908-v1.patch, HBASE-21908-v2.patch, > HBASE-21908.patch > > > It has been deprecated in 2.x, so remove it from 3.0.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21908) Remove Scan.getScanMetrics
[ https://issues.apache.org/jira/browse/HBASE-21908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770018#comment-16770018 ] stack commented on HBASE-21908: --- +1 > Remove Scan.getScanMetrics > -- > > Key: HBASE-21908 > URL: https://issues.apache.org/jira/browse/HBASE-21908 > Project: HBase > Issue Type: Task > Components: Client, scan >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21908-v1.patch, HBASE-21908-v2.patch, > HBASE-21908.patch > > > It has been deprecated in 2.x, so remove it from 3.0.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21908) Remove Scan.getScanMetrics
[ https://issues.apache.org/jira/browse/HBASE-21908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21908: -- Attachment: HBASE-21908-v2.patch > Remove Scan.getScanMetrics > -- > > Key: HBASE-21908 > URL: https://issues.apache.org/jira/browse/HBASE-21908 > Project: HBase > Issue Type: Task > Components: Client, scan >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21908-v1.patch, HBASE-21908-v2.patch, > HBASE-21908.patch > > > It has been deprecated in 2.x, so remove it from 3.0.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21783) Support exceed user/table/ns throttle quota if region server has available quota
[ https://issues.apache.org/jira/browse/HBASE-21783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21783: --- Attachment: HBASE-21783.master.004.patch > Support exceed user/table/ns throttle quota if region server has available > quota > > > Key: HBASE-21783 > URL: https://issues.apache.org/jira/browse/HBASE-21783 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21783.master.001.patch, > HBASE-21783.master.002.patch, HBASE-21783.master.003.patch, > HBASE-21783.master.004.patch > > > Currently, all types of rpc throttle quota (include region server, namespace, > table and user quota) are hard limit, which means once requests exceed the > amount, they will be throttled. > In some situation, user use out of all their own quotas but region server > still has available quota because other users don't consume at the same time, > in this case, we can allow user consume additional quota. So add a switch to > enable or disable other quotas(except region server quota) exceed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21783) Support exceed user/table/ns throttle quota if region server has available quota
[ https://issues.apache.org/jira/browse/HBASE-21783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21783: --- Attachment: (was: HBASE-21783.master.004.patch) > Support exceed user/table/ns throttle quota if region server has available > quota > > > Key: HBASE-21783 > URL: https://issues.apache.org/jira/browse/HBASE-21783 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21783.master.001.patch, > HBASE-21783.master.002.patch, HBASE-21783.master.003.patch > > > Currently, all types of rpc throttle quota (include region server, namespace, > table and user quota) are hard limit, which means once requests exceed the > amount, they will be throttled. > In some situation, user use out of all their own quotas but region server > still has available quota because other users don't consume at the same time, > in this case, we can allow user consume additional quota. So add a switch to > enable or disable other quotas(except region server quota) exceed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21908) Remove Scan.getScanMetrics
[
https://issues.apache.org/jira/browse/HBASE-21908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769996#comment-16769996
]
Hadoop QA commented on HBASE-21908:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m
25s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
28s{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 28s{color}
| {color:red} hbase-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} hbase-client: The patch generated 0 new + 5
unchanged - 4 fixed = 5 total (was 9) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} The patch passed checkstyle in hbase-server {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} The patch passed checkstyle in hbase-mapreduce
{color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 2m
11s{color} | {color:red} patch has 16 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
21s{color} | {color:red} The patch causes 16 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
41s{color} | {color:red} The patch causes 16 errors with Hadoop v3.0.0. {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
19s{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 21s{color}
| {color:red} hbase-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}128m
37s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m
32s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}183m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue |
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[ https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769991#comment-16769991 ] Anoop Sam John commented on HBASE-21874: bq.So in this case, assuming device is mounted in "memory mode" described here, it works as an extension of main memory? But we still have to access it as a mmap? Yes in that mode, the device will act as an extension of DRAM. In this case no need to do any mmap. It is like the DRAM. So ByteBufferIOEngine is enough then. We can configure a huge size off heap cache there. The ByteBuffer.allocateDirect() calls would be allocating the chunks over the Pmem device rather than over DRAM. As said, this patch is for the other mode (App Direct).. In this mode, we have to create file over the device mount path. We can always do file based operation also. The device is to be with DAX FS. And the OS should be capable to identify the pmem device. All the file based OS calls work normally here. The extra is we can do mmap and the OS will not try to bring the pages into its Page cache (DRAM). Instead the CPU can directly access bytes over the device with its load/store instructions. For this the pmem library APIs also not a mandatory item. > Bucket cache on Persistent memory > - > > Key: HBASE-21874 > URL: https://issues.apache.org/jira/browse/HBASE-21874 > Project: HBase > Issue Type: New Feature > Components: BucketCache >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21874.patch, HBASE-21874.patch, > HBASE-21874_V2.patch, Pmem_BC.png > > > Non volatile persistent memory devices are byte addressable like DRAM (for > eg. Intel DCPMM). Bucket cache implementation can take advantage of this new > memory type and can make use of the existing offheap data structures to serve > data directly from this memory area without having to bring the data to > onheap. > The patch is a new IOEngine implementation that works with the persistent > memory. > Note : Here we don't make use of the persistence nature of the device and > just make use of the big memory it provides. > Performance numbers to follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21910) The nonce implementation is wrong for AsyncTable
[
https://issues.apache.org/jira/browse/HBASE-21910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769986#comment-16769986
]
Hudson commented on HBASE-21910:
Results for branch branch-2.0
[build #1352 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1352/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1352//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1352//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1352//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> The nonce implementation is wrong for AsyncTable
>
>
> Key: HBASE-21910
> URL: https://issues.apache.org/jira/browse/HBASE-21910
> Project: HBase
> Issue Type: Bug
> Components: asyncclient, Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4
>
> Attachments: HBASE-21910.patch
>
>
> We recreate the nonce every time when retry, which is not correct.
> And it is strange that we even do not have a UT for this...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21913) src/main/asciidoc/images link is incorrect
[
https://issues.apache.org/jira/browse/HBASE-21913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769987#comment-16769987
]
Hudson commented on HBASE-21913:
Results for branch branch-2.0
[build #1352 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1352/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1352//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1352//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1352//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> src/main/asciidoc/images link is incorrect
> --
>
> Key: HBASE-21913
> URL: https://issues.apache.org/jira/browse/HBASE-21913
> Project: HBase
> Issue Type: Bug
> Components: website
>Affects Versions: 2.0.4
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Trivial
> Fix For: 2.0.5
>
> Attachments: HBASE-21913.branch-2.0.001.patch
>
>
> The link for src/main/asciidoc/images points to ../../site/resources/images/
> but the site is one folder up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21782) LoadIncrementalHFiles should not be IA.Public
[
https://issues.apache.org/jira/browse/HBASE-21782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769979#comment-16769979
]
Hudson commented on HBASE-21782:
Results for branch master
[build #800 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/800/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> LoadIncrementalHFiles should not be IA.Public
> -
>
> Key: HBASE-21782
> URL: https://issues.apache.org/jira/browse/HBASE-21782
> Project: HBase
> Issue Type: Task
> Components: mapreduce
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21782-v1.patch, HBASE-21782.patch
>
>
> It is an implementation class, so some of the methods which are only supposed
> to be used by replication sink are also public to users. And it exposes
> methods which take Table and Connection as parameter and inside the
> implementation we assume that they are HTable and ConnectionImplementation,
> which will be a pain when we want to replace the sync client implementation
> with async client.
> Here I think we should make the implementation class as
> IA.LimitPrivate(TOOL), and introduce an interface for bulking hfiles
> programmatically.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21872) Clean up getBytes() calls without charsets provided
[
https://issues.apache.org/jira/browse/HBASE-21872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769980#comment-16769980
]
Hudson commented on HBASE-21872:
Results for branch master
[build #800 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/800/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Clean up getBytes() calls without charsets provided
> ---
>
> Key: HBASE-21872
> URL: https://issues.apache.org/jira/browse/HBASE-21872
> Project: HBase
> Issue Type: Task
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Trivial
> Fix For: 3.0.0
>
> Attachments: HBASE-21782.001.patch, HBASE-21782.002.patch,
> HBASE-21782.003.patch, HBASE-21872.004.patch
>
>
> As we saw over in HBASE-21201, the use of {{String.getBytes()}} without a
> Charset can result is some compiler warnings. Let's just get rid of these
> calls. There are only a handful anymore in master.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21686) Release 1.2.10
[
https://issues.apache.org/jira/browse/HBASE-21686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769978#comment-16769978
]
Hudson commented on HBASE-21686:
Results for branch master
[build #800 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/800/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Release 1.2.10
> --
>
> Key: HBASE-21686
> URL: https://issues.apache.org/jira/browse/HBASE-21686
> Project: HBase
> Issue Type: Task
> Components: community
>Affects Versions: 1.2.10
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 1.2.10
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21910) The nonce implementation is wrong for AsyncTable
[
https://issues.apache.org/jira/browse/HBASE-21910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769977#comment-16769977
]
Hudson commented on HBASE-21910:
Results for branch master
[build #800 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/800/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The nonce implementation is wrong for AsyncTable
>
>
> Key: HBASE-21910
> URL: https://issues.apache.org/jira/browse/HBASE-21910
> Project: HBase
> Issue Type: Bug
> Components: asyncclient, Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4
>
> Attachments: HBASE-21910.patch
>
>
> We recreate the nonce every time when retry, which is not correct.
> And it is strange that we even do not have a UT for this...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21814) Remove the TODO in AccessControlLists#addUserPermission
[
https://issues.apache.org/jira/browse/HBASE-21814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769976#comment-16769976
]
Hudson commented on HBASE-21814:
Results for branch master
[build #800 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/800/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/800//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove the TODO in AccessControlLists#addUserPermission
> ---
>
> Key: HBASE-21814
> URL: https://issues.apache.org/jira/browse/HBASE-21814
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21814.master.001.patch,
> HBASE-21814.master.001.patch, HBASE-21814.master.002.patch,
> HBASE-21814.master.002.patch
>
>
> The TODO was added by me. Because this method happens within the RS. The old
> impl use a login user(User.runAsLoginUser where the login user is the user
> who started RS process) to call Table.put(). And it will check the permission
> when put record to ACL table. At RpcServer we have a ThreadLocal where we
> keep the CallContext and inside that the current RPC called user info is set.
> We need Table.put(List) to change to a new thread and and so old
> ThreadLocal variable is not accessible and so it looks as if no Rpc context
> and we were relying on the super user who starts the RS process.
>
> {code:java}
> User.runAsLoginUser(new PrivilegedExceptionAction() {
> @Override
> public Void run() throws Exception {
>
> AccessControlLists.addUserPermission(regionEnv.getConfiguration(), perm,
> regionEnv.getTable(AccessControlLists.ACL_TABLE_NAME),
> request.getMergeExistingPermissions());
> return null;
> }
> });
> {code}
>
> But after HBASE-21739, no need to User.runAsLoginUser. Because we will call
> Admin method to grant/revoke. And this will be execute in master and use the
> master user(the user who started master process) to call Table.put. So this
> is not a problem now.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21914) Set assert to check if exception is set in procedures
[
https://issues.apache.org/jira/browse/HBASE-21914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769927#comment-16769927
]
Xiang Li edited comment on HBASE-21914 at 2/16/19 1:42 AM:
---
Hi [~xucang], I got your idea.
Do you mind check the latest master branch
{code:title=Procedure.java|borderStyle=solid}
public synchronized boolean isFailed() {
return state == ProcedureState.FAILED || state == ProcedureState.ROLLEDBACK;
}
{code}
Checking in git log, that was changed from the version in your comment to the
one above by HBASE-17863
was (Author: water):
Hi [~xucang], I got your idea.
Do you mind check the latest master branch
{code:title=Procedure.java|borderStyle=solid}
public synchronized boolean isFailed() {
return state == ProcedureState.FAILED || state == ProcedureState.ROLLEDBACK;
}
{code}
Checking in git log, that was changed from the version in comment to the one
above by HBASE-17863
> Set assert to check if exception is set in procedures
> -
>
> Key: HBASE-21914
> URL: https://issues.apache.org/jira/browse/HBASE-21914
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Critical
>
> Take CreateTableProcedure as an example, in executeFromState()
> {code:java}
> case CREATE_TABLE_PRE_OPERATION:
> // Verify if we can create the table
> boolean exists = !prepareCreate(env);
> releaseSyncLatch();
>
> if (exists) {
> assert isFailed() : "the delete should have an exception here";
> return Flow.NO_MORE_STATE;
> }
> {code}
> The following assertion:
> {code}
> assert isFailed() : "the delete should have an exception here";
> {code}
> If I get the idea behind "the delete should have an exception here"
> correctly, it is to make sure that when setting the state to FAILED, the
> exception must be set. (or must call setFailure()). But the assertion only
> check isFailed() but no "!hasException()"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[ https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769964#comment-16769964 ] ramkrishna.s.vasudevan commented on HBASE-21874: bq. Can you point the exact place in the patch where you control this? We need not control at the Java level. It will controlled at the OS level. These devices are configured with DAX (Direct Access mode) at the OS level. As said in the link - here we use the App Direct mode and not the memory mode. Memory mode does not give us control as where the cache or the address space could reside. It may be on DRAM or Pmem address space. But here we specifically ask our cache to reside only on the Pmem area and once it is mapped in the Pmem address space everything is transparent to us. > Bucket cache on Persistent memory > - > > Key: HBASE-21874 > URL: https://issues.apache.org/jira/browse/HBASE-21874 > Project: HBase > Issue Type: New Feature > Components: BucketCache >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21874.patch, HBASE-21874.patch, > HBASE-21874_V2.patch, Pmem_BC.png > > > Non volatile persistent memory devices are byte addressable like DRAM (for > eg. Intel DCPMM). Bucket cache implementation can take advantage of this new > memory type and can make use of the existing offheap data structures to serve > data directly from this memory area without having to bring the data to > onheap. > The patch is a new IOEngine implementation that works with the persistent > memory. > Note : Here we don't make use of the persistence nature of the device and > just make use of the big memory it provides. > Performance numbers to follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21814) Remove the TODO in AccessControlLists#addUserPermission
[
https://issues.apache.org/jira/browse/HBASE-21814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769923#comment-16769923
]
Hudson commented on HBASE-21814:
Results for branch branch-2
[build #1687 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1687/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1687//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1687//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1687//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove the TODO in AccessControlLists#addUserPermission
> ---
>
> Key: HBASE-21814
> URL: https://issues.apache.org/jira/browse/HBASE-21814
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21814.master.001.patch,
> HBASE-21814.master.001.patch, HBASE-21814.master.002.patch,
> HBASE-21814.master.002.patch
>
>
> The TODO was added by me. Because this method happens within the RS. The old
> impl use a login user(User.runAsLoginUser where the login user is the user
> who started RS process) to call Table.put(). And it will check the permission
> when put record to ACL table. At RpcServer we have a ThreadLocal where we
> keep the CallContext and inside that the current RPC called user info is set.
> We need Table.put(List) to change to a new thread and and so old
> ThreadLocal variable is not accessible and so it looks as if no Rpc context
> and we were relying on the super user who starts the RS process.
>
> {code:java}
> User.runAsLoginUser(new PrivilegedExceptionAction() {
> @Override
> public Void run() throws Exception {
>
> AccessControlLists.addUserPermission(regionEnv.getConfiguration(), perm,
> regionEnv.getTable(AccessControlLists.ACL_TABLE_NAME),
> request.getMergeExistingPermissions());
> return null;
> }
> });
> {code}
>
> But after HBASE-21739, no need to User.runAsLoginUser. Because we will call
> Admin method to grant/revoke. And this will be execute in master and use the
> master user(the user who started master process) to call Table.put. So this
> is not a problem now.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21914) Set assert to check if exception is set in procedures
[
https://issues.apache.org/jira/browse/HBASE-21914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769927#comment-16769927
]
Xiang Li commented on HBASE-21914:
--
Hi [~xucang], I got your idea.
Do you mind check the latest master branch
{code:title=Procedure.java|borderStyle=solid}
public synchronized boolean isFailed() {
return state == ProcedureState.FAILED || state == ProcedureState.ROLLEDBACK;
}
{code}
Checking in git log, that was changed from the version in comment to the one
above by HBASE-17863
> Set assert to check if exception is set in procedures
> -
>
> Key: HBASE-21914
> URL: https://issues.apache.org/jira/browse/HBASE-21914
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Critical
>
> Take CreateTableProcedure as an example, in executeFromState()
> {code:java}
> case CREATE_TABLE_PRE_OPERATION:
> // Verify if we can create the table
> boolean exists = !prepareCreate(env);
> releaseSyncLatch();
>
> if (exists) {
> assert isFailed() : "the delete should have an exception here";
> return Flow.NO_MORE_STATE;
> }
> {code}
> The following assertion:
> {code}
> assert isFailed() : "the delete should have an exception here";
> {code}
> If I get the idea behind "the delete should have an exception here"
> correctly, it is to make sure that when setting the state to FAILED, the
> exception must be set. (or must call setFailure()). But the assertion only
> check isFailed() but no "!hasException()"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21914) Set assert to check if exception is set in procedures
[
https://issues.apache.org/jira/browse/HBASE-21914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769930#comment-16769930
]
Xu Cang commented on HBASE-21914:
-
you are right Xiang, I was checking branch-1. Please go ahead with your
proposal. Thanks!
> Set assert to check if exception is set in procedures
> -
>
> Key: HBASE-21914
> URL: https://issues.apache.org/jira/browse/HBASE-21914
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Critical
>
> Take CreateTableProcedure as an example, in executeFromState()
> {code:java}
> case CREATE_TABLE_PRE_OPERATION:
> // Verify if we can create the table
> boolean exists = !prepareCreate(env);
> releaseSyncLatch();
>
> if (exists) {
> assert isFailed() : "the delete should have an exception here";
> return Flow.NO_MORE_STATE;
> }
> {code}
> The following assertion:
> {code}
> assert isFailed() : "the delete should have an exception here";
> {code}
> If I get the idea behind "the delete should have an exception here"
> correctly, it is to make sure that when setting the state to FAILED, the
> exception must be set. (or must call setFailure()). But the assertion only
> check isFailed() but no "!hasException()"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21914) Set assert to check if exception is set in procedures
[
https://issues.apache.org/jira/browse/HBASE-21914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769927#comment-16769927
]
Xiang Li edited comment on HBASE-21914 at 2/16/19 1:03 AM:
---
Hi [~xucang], I got your idea.
Do you mind check the latest master branch
{code:title=Procedure.java|borderStyle=solid}
public synchronized boolean isFailed() {
return state == ProcedureState.FAILED || state == ProcedureState.ROLLEDBACK;
}
{code}
Checking in git log, that was changed from the version in comment to the one
above by HBASE-17863
was (Author: water):
Hi [~xucang], I got your idea.
Do you mind check the latest master branch
{code:title=Procedure.java|borderStyle=solid}
public synchronized boolean isFailed() {
return state == ProcedureState.FAILED || state == ProcedureState.ROLLEDBACK;
}
{code}
Checking in git log, that was changed from the version in comment to the one
above by HBASE-17863
> Set assert to check if exception is set in procedures
> -
>
> Key: HBASE-21914
> URL: https://issues.apache.org/jira/browse/HBASE-21914
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Critical
>
> Take CreateTableProcedure as an example, in executeFromState()
> {code:java}
> case CREATE_TABLE_PRE_OPERATION:
> // Verify if we can create the table
> boolean exists = !prepareCreate(env);
> releaseSyncLatch();
>
> if (exists) {
> assert isFailed() : "the delete should have an exception here";
> return Flow.NO_MORE_STATE;
> }
> {code}
> The following assertion:
> {code}
> assert isFailed() : "the delete should have an exception here";
> {code}
> If I get the idea behind "the delete should have an exception here"
> correctly, it is to make sure that when setting the state to FAILED, the
> exception must be set. (or must call setFailure()). But the assertion only
> check isFailed() but no "!hasException()"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21910) The nonce implementation is wrong for AsyncTable
[
https://issues.apache.org/jira/browse/HBASE-21910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769924#comment-16769924
]
Hudson commented on HBASE-21910:
Results for branch branch-2
[build #1687 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1687/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1687//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1687//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1687//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The nonce implementation is wrong for AsyncTable
>
>
> Key: HBASE-21910
> URL: https://issues.apache.org/jira/browse/HBASE-21910
> Project: HBase
> Issue Type: Bug
> Components: asyncclient, Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4
>
> Attachments: HBASE-21910.patch
>
>
> We recreate the nonce every time when retry, which is not correct.
> And it is strange that we even do not have a UT for this...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21908) Remove Scan.getScanMetrics
[ https://issues.apache.org/jira/browse/HBASE-21908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21908: -- Attachment: HBASE-21908-v1.patch > Remove Scan.getScanMetrics > -- > > Key: HBASE-21908 > URL: https://issues.apache.org/jira/browse/HBASE-21908 > Project: HBase > Issue Type: Task > Components: Client, scan >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21908-v1.patch, HBASE-21908.patch > > > It has been deprecated in 2.x, so remove it from 3.0.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21909) Validate the put instance before executing in AsyncTable.put method
[ https://issues.apache.org/jira/browse/HBASE-21909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769907#comment-16769907 ] Duo Zhang commented on HBASE-21909: --- OK, good. All green. Ping [~zghaobac]. > Validate the put instance before executing in AsyncTable.put method > --- > > Key: HBASE-21909 > URL: https://issues.apache.org/jira/browse/HBASE-21909 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21909.patch > > > Align with the sync client implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21909) Validate the put instance before executing in AsyncTable.put method
[ https://issues.apache.org/jira/browse/HBASE-21909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769914#comment-16769914 ] Duo Zhang commented on HBASE-21909: --- Review board link: https://reviews.apache.org/r/69995/ > Validate the put instance before executing in AsyncTable.put method > --- > > Key: HBASE-21909 > URL: https://issues.apache.org/jira/browse/HBASE-21909 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21909.patch > > > Align with the sync client implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21895) Take more care on the error prone plugin and warnings
[ https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769908#comment-16769908 ] Duo Zhang commented on HBASE-21895: --- I think error prone will be executed as part of the compile? Let me try it locally to see what is going on here. > Take more care on the error prone plugin and warnings > - > > Key: HBASE-21895 > URL: https://issues.apache.org/jira/browse/HBASE-21895 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Priority: Major > > As shown in HBASE-21890 , the error prone warnings are truly useful. > But now, the javac warnings section in the pre commit result is messed up, it > always reports unrelated warnings. Need to take a look at it. > And also, we should try out best to clean up the warnings. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21744) timeout for server list refresh calls
[ https://issues.apache.org/jira/browse/HBASE-21744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769906#comment-16769906 ] Duo Zhang commented on HBASE-21744: --- The patch is fine, and is it possible to add a UT for it? > timeout for server list refresh calls > -- > > Key: HBASE-21744 > URL: https://issues.apache.org/jira/browse/HBASE-21744 > Project: HBase > Issue Type: Improvement > Components: Zookeeper >Affects Versions: 3.0.0, 2.2.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HBASE-21744.01.patch, HBASE-21744.02.patch, > HBASE-21744.03.patch, HBASE-21744.04.patch, HBASE-21744.patch > > > Not sure why yet, but we are seeing the case when cluster is in overall a bad > state, where after RS dies and deletes its znode, the notification looks like > it's lost, so the master doesn't detect the failure. ZK itself appears to be > healthy and doesn't report anything special. > After some other change is made to the server list, master rescans the list > and picks up the stale change. Might make sense to add a config that would > trigger the refresh if it hasn't happened for a while (e.g. 1 minute). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21788) OpenRegionProcedure (after recovery?) is unreliable and needs to be improved
[ https://issues.apache.org/jira/browse/HBASE-21788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769902#comment-16769902 ] Duo Zhang commented on HBASE-21788: --- [~zghaobac] Also observed similar conditions in an ITBLL run for branch-2.2, all the regions are in OFFLINE state and after restarting there are bunch of corrupted procedures. It seems easy to reproduce so I think we will find out the reason soon. > OpenRegionProcedure (after recovery?) is unreliable and needs to be improved > > > Key: HBASE-21788 > URL: https://issues.apache.org/jira/browse/HBASE-21788 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Assignee: stack >Priority: Critical > Attachments: WAL-Orphan.log > > > Not much for this one yet. > I repeatedly see the cases when the region is stuck in OPENING, and after > master restart RIT is recovered, and stays WAITING; its OpenRegionProcedure > (also recovered) is stuck in Runnable and never does anything for hours. I > cannot find logs on the target server indicating that it ever tried to do > anything after master restart. > This procedure needs at the very least logging of what it's trying to do, and > maybe a timeout so it unconditionally fails after a configurable period (1 > hour?). > I may also investigate why it doesn't do anything and file a separate bug. I > wonder if it's somehow related to the region status check, but this is just a > hunch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[
https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769893#comment-16769893
]
Wellington Chevreuil commented on HBASE-21874:
--
So in this case, assuming device is mounted in "memory mode" described
[here|https://software.intel.com/en-us/blogs/2018/10/30/intel-optane-dc-persistent-memory-a-major-advance-in-memory-and-storage-architecture],
it works as an extension of main memory? But we still have to access it as a
mmap?
{noformat}
the operating system sees the persistent memory module capacity as the system
main memory. For example, on a common two-socket system, the Memory Mode can
provide 6 TB of main memory, something very difficult and expensive to do with
DRAM (if it is even possible). In Memory Mode, the DRAM installed in the system
acts as a cache to deliver DRAM-like performance for this high-capacity main
memory.
{noformat}
> Bucket cache on Persistent memory
> -
>
> Key: HBASE-21874
> URL: https://issues.apache.org/jira/browse/HBASE-21874
> Project: HBase
> Issue Type: New Feature
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21874.patch, HBASE-21874.patch,
> HBASE-21874_V2.patch, Pmem_BC.png
>
>
> Non volatile persistent memory devices are byte addressable like DRAM (for
> eg. Intel DCPMM). Bucket cache implementation can take advantage of this new
> memory type and can make use of the existing offheap data structures to serve
> data directly from this memory area without having to bring the data to
> onheap.
> The patch is a new IOEngine implementation that works with the persistent
> memory.
> Note : Here we don't make use of the persistence nature of the device and
> just make use of the big memory it provides.
> Performance numbers to follow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21915) FileLink$FileLinkInputStream doesn't implement CanUnbuffer
[
https://issues.apache.org/jira/browse/HBASE-21915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769891#comment-16769891
]
Hadoop QA commented on HBASE-21915:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
4s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
1s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 23s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}133m
2s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}180m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21915 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12958934/HBASE-21915.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 193a50f4b299 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ae0198084c |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15996/testReport/ |
| Max. process+thread count | 5514 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-21788) OpenRegionProcedure (after recovery?) is unreliable and needs to be improved
[ https://issues.apache.org/jira/browse/HBASE-21788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769814#comment-16769814 ] Bahram Chehrazy commented on HBASE-21788: - pid=44140 is the SCP for the hosting server that died eariler: 2019-02-06 07:39:56,776 INFO [RegionServerTracker-0] assignment.AssignmentManager: Added ,16020,1549465415156 to dead servers which carryingMeta=false, submitted ServerCrashProcedure pid=44140 > OpenRegionProcedure (after recovery?) is unreliable and needs to be improved > > > Key: HBASE-21788 > URL: https://issues.apache.org/jira/browse/HBASE-21788 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Assignee: stack >Priority: Critical > Attachments: WAL-Orphan.log > > > Not much for this one yet. > I repeatedly see the cases when the region is stuck in OPENING, and after > master restart RIT is recovered, and stays WAITING; its OpenRegionProcedure > (also recovered) is stuck in Runnable and never does anything for hours. I > cannot find logs on the target server indicating that it ever tried to do > anything after master restart. > This procedure needs at the very least logging of what it's trying to do, and > maybe a timeout so it unconditionally fails after a configurable period (1 > hour?). > I may also investigate why it doesn't do anything and file a separate bug. I > wonder if it's somehow related to the region status check, but this is just a > hunch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21866) Do not move the table to null rsgroup when creating an existing table
[
https://issues.apache.org/jira/browse/HBASE-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769810#comment-16769810
]
Xu Cang commented on HBASE-21866:
-
Before I commit this, [~nihaljain.cs]
Do you have more comments on this? Thanks in advance.
> Do not move the table to null rsgroup when creating an existing table
> -
>
> Key: HBASE-21866
> URL: https://issues.apache.org/jira/browse/HBASE-21866
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Major
> Attachments: HBASE-21866.master.000.patch,
> HBASE-21866.master.001.patch, HBASE-21866.master.002.patch
>
>
> By using the latest HBase master branch, the bug could be re-produced as:
> # create 't1', 'cf1'
> # create 't1', 'cf1'
> The following message is logged into HMaster's log:
> {code}
> INFO [PEWorker-12] rsgroup.RSGroupAdminServer: Moving table t1 to RSGroup
> null
> {code}
> This is a wrong action and instead, we should keep t1 as where it originally
> is.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21667) Move to latest ASF Parent POM
[
https://issues.apache.org/jira/browse/HBASE-21667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769802#comment-16769802
]
Hadoop QA commented on HBASE-21667:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
55s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 22m
29s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 15m
5s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
27s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
5s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
28s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 25m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 25m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 31m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 6m
54s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 51s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 32m 47s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}175m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.zookeeper.TestReadOnlyZKClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:42ca976 |
| JIRA Issue | HBASE-21667 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12958926/HBASE-21667.branch-2.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile refguide mvnsite |
| uname | Linux 328cf00dc599 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 48f2ef432b |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| refguide |
[jira] [Commented] (HBASE-21814) Remove the TODO in AccessControlLists#addUserPermission
[
https://issues.apache.org/jira/browse/HBASE-21814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769797#comment-16769797
]
Hudson commented on HBASE-21814:
Results for branch branch-2.2
[build #45 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/45/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/45//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/45//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/45//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove the TODO in AccessControlLists#addUserPermission
> ---
>
> Key: HBASE-21814
> URL: https://issues.apache.org/jira/browse/HBASE-21814
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21814.master.001.patch,
> HBASE-21814.master.001.patch, HBASE-21814.master.002.patch,
> HBASE-21814.master.002.patch
>
>
> The TODO was added by me. Because this method happens within the RS. The old
> impl use a login user(User.runAsLoginUser where the login user is the user
> who started RS process) to call Table.put(). And it will check the permission
> when put record to ACL table. At RpcServer we have a ThreadLocal where we
> keep the CallContext and inside that the current RPC called user info is set.
> We need Table.put(List) to change to a new thread and and so old
> ThreadLocal variable is not accessible and so it looks as if no Rpc context
> and we were relying on the super user who starts the RS process.
>
> {code:java}
> User.runAsLoginUser(new PrivilegedExceptionAction() {
> @Override
> public Void run() throws Exception {
>
> AccessControlLists.addUserPermission(regionEnv.getConfiguration(), perm,
> regionEnv.getTable(AccessControlLists.ACL_TABLE_NAME),
> request.getMergeExistingPermissions());
> return null;
> }
> });
> {code}
>
> But after HBASE-21739, no need to User.runAsLoginUser. Because we will call
> Admin method to grant/revoke. And this will be execute in master and use the
> master user(the user who started master process) to call Table.put. So this
> is not a problem now.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21910) The nonce implementation is wrong for AsyncTable
[
https://issues.apache.org/jira/browse/HBASE-21910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769798#comment-16769798
]
Hudson commented on HBASE-21910:
Results for branch branch-2.2
[build #45 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/45/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/45//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/45//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/45//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The nonce implementation is wrong for AsyncTable
>
>
> Key: HBASE-21910
> URL: https://issues.apache.org/jira/browse/HBASE-21910
> Project: HBase
> Issue Type: Bug
> Components: asyncclient, Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4
>
> Attachments: HBASE-21910.patch
>
>
> We recreate the nonce every time when retry, which is not correct.
> And it is strange that we even do not have a UT for this...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21322) Add a scheduleServerCrashProcedure() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21322: Priority: Critical (was: Major) > Add a scheduleServerCrashProcedure() API to HbckService > --- > > Key: HBASE-21322 > URL: https://issues.apache.org/jira/browse/HBASE-21322 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.0.3, 2.1.2 > > Attachments: HBASE-21322.branch-2.1.001.patch, > HBASE-21322.branch-2.addendum.patch, HBASE-21322.master.001.patch, > HBASE-21322.master.002.patch, HBASE-21322.master.003.patch, > HBASE-21322.master.004.patch, HBASE-21322.master.005.patch, > HBASE-21322.master.006.patch, Screenshot from 2018-10-17 13-35-58.png, > Screenshot from 2018-10-17 13-38-41.png, Screenshot from 2018-10-17 > 13-47-06.png > > > According to my test, if one RS is down, then all procedure logs are deleted, > it will lead to that no ServerCrashProcedure is scheduled. And restarting > master cannot help. Thus we need to schedule a ServerCrashProcedure manually > to solve the problem. I plan to add a scheduleServerCrashProcedure() API to > HbckService, then add this API to HBCK2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable (and RS may run after losing its znode)
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769782#comment-16769782
]
Sergey Shelukhin edited comment on HBASE-14498 at 2/15/19 9:56 PM:
---
I'll have to look a little more, but it doesn't look like this happened in this
case... long time after the loss RS was still doing flushes and compactions.
Is it possible for master to fail to revoke the lease, die, and then for the
following master to not revoke it? I'll check later.
Why no curator? It's used in Hive and seems to be fine.
was (Author: sershe):
I'll have to look a little more, but it doesn't look like this happened in this
case... long time after the loss RS was still doing flushes and compactions.
Is it possible for master to fail to revoke the lease, die, and then for the
following master to not revoke it? I'll check later
> Master stuck in infinite loop when all Zookeeper servers are unreachable (and
> RS may run after losing its znode)
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0, 2.2.0
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch,
> HBASE-14498-V4.patch, HBASE-14498-V5.patch, HBASE-14498-V6.patch,
> HBASE-14498-V6.patch, HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.2.patch, HBASE-14498-branch-1.3-V2.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.007.patch, HBASE-14498.008.patch,
> HBASE-14498.master.001.patch, HBASE-14498.master.002.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
>
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[
https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769598#comment-16769598
]
ramkrishna.s.vasudevan commented on HBASE-21874:
{quote}Where are your going to keep bucket cache? Not in DRAM definitely, hence
in NVDIMM (PMEM)?
{quote}
Yes. The cache will reside in Pmem only.
{quote}If you keep data in PMEM and use extended FileMmapIOEngine, where do yo
you mmap it into? into DRAM? That is strange
{quote}
We use extended FileMmapIOEngine, but the mmap won't do the memmory mapping to
DRAM, it will mmap to a different address space maintained by the NVDIMM. So
even if you have less DRAM capacity still your data is served from PMEM's
address space. That is why you can use the SHARED mode in the IOEngine. Where
as in a file mmap case you will go with EXCLUSIVE - where you need to copy the
content to the onheap memory.
{quote}My question, regarding file system required on top PMEM has remained
unanswered.
You rely on file system on top of PMEM
{quote}
Pls check the description from [http://pmem.io.|http://pmem.io./]
NVDIMMs are going to be addressed as an mmap files only unlike DRAM where you
directly access the memory addresses.
{quote}You mmap PMEM resided file into RAM
{quote}
No as explained previously.
Some related links which is there in public
https://software.intel.com/en-us/blogs/2018/10/30/intel-optane-dc-persistent-memory-a-major-advance-in-memory-and-storage-architecture
> Bucket cache on Persistent memory
> -
>
> Key: HBASE-21874
> URL: https://issues.apache.org/jira/browse/HBASE-21874
> Project: HBase
> Issue Type: New Feature
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21874.patch, HBASE-21874.patch,
> HBASE-21874_V2.patch, Pmem_BC.png
>
>
> Non volatile persistent memory devices are byte addressable like DRAM (for
> eg. Intel DCPMM). Bucket cache implementation can take advantage of this new
> memory type and can make use of the existing offheap data structures to serve
> data directly from this memory area without having to bring the data to
> onheap.
> The patch is a new IOEngine implementation that works with the persistent
> memory.
> Note : Here we don't make use of the persistence nature of the device and
> just make use of the big memory it provides.
> Performance numbers to follow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable (and RS may run after losing its znode)
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769782#comment-16769782
]
Sergey Shelukhin commented on HBASE-14498:
--
I'll have to look a little more, but it doesn't look like this happened in this
case... long time after the loss RS was still doing flushes and compactions.
Is it possible for master to fail to revoke the lease, die, and then for the
following master to not revoke it? I'll check later
> Master stuck in infinite loop when all Zookeeper servers are unreachable (and
> RS may run after losing its znode)
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0, 2.2.0
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch,
> HBASE-14498-V4.patch, HBASE-14498-V5.patch, HBASE-14498-V6.patch,
> HBASE-14498-V6.patch, HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.2.patch, HBASE-14498-branch-1.3-V2.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.007.patch, HBASE-14498.008.patch,
> HBASE-14498.master.001.patch, HBASE-14498.master.002.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO |
[jira] [Comment Edited] (HBASE-21874) Bucket cache on Persistent memory
[ https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769571#comment-16769571 ] Vladimir Rodionov edited comment on HBASE-21874 at 2/15/19 6:30 PM: This patch requires good write up, it is not clear how this thing work, [~ram_krish]. From what I see, you extends FileMmapEngine, which works with persistent cache by mmapping it into RAM, so your new engine does exactly the same thing? As far as I understand, the major selling point of PMEM is providing RAM-like capacity beyond what DRAM can do. So we are supposed to have some relatively small DRAM and large size of NVDIMM installed into a server. This raises the questions: # Where are your going to keep bucket cache? Not in DRAM definitely, hence in NVDIMM (PMEM)? # If you keep data in PMEM and use extended FileMmapIOEngine, where do yo you mmap it into? into DRAM? That is strange # My question, regarding file system required on top PMEM has remained unanswered. What I see in this patch (may be I am wrong) # You rely on file system on top of PMEM # You mmap PMEM resided file into DRAM Am I right? was (Author: vrodionov): This patch requires good write up, it is not clear how this thing work, [~ram_krish]. From what I see, you extends FileMmapEngine, which works with persistent cache by mmapping it into RAM, so your new engine does exactly the same thing? As far as I understand, the major selling point of PMEM is providing RAM-like capacity beyond what DRAM can do. So we are supposed to have some relatively small DRAM and large size of NVDIMM installed into a server. This raises the questions: # Where are your going to keep bucket cache? Not in DRAM definitely, hence in NVDIMM (PMEM)? # If you keep data in PMEM and use extended FileMmapIOEngine, where do yo you mmap it into? into DRAM? That is strange # My question, regarding file system required on top PMEM has remained unanswered. What I see in this patch (may be I am wrong) # You rely on file system on top of PMEM # You mmap PMEM resided file into DRAM This does not look right to me. > Bucket cache on Persistent memory > - > > Key: HBASE-21874 > URL: https://issues.apache.org/jira/browse/HBASE-21874 > Project: HBase > Issue Type: New Feature > Components: BucketCache >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21874.patch, HBASE-21874.patch, > HBASE-21874_V2.patch, Pmem_BC.png > > > Non volatile persistent memory devices are byte addressable like DRAM (for > eg. Intel DCPMM). Bucket cache implementation can take advantage of this new > memory type and can make use of the existing offheap data structures to serve > data directly from this memory area without having to bring the data to > onheap. > The patch is a new IOEngine implementation that works with the persistent > memory. > Note : Here we don't make use of the persistence nature of the device and > just make use of the big memory it provides. > Performance numbers to follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable (and RS may run after losing its znode)
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769772#comment-16769772
]
Sean Busbey commented on HBASE-14498:
-
how do we end up with dataloss? when the master sees the loss of zk node, it
takes the HDFS lease out from under the WAL so the RS can't accept any
additional writes or do anything that would require persisting a WAL edit.
given the history, I recommend you try running ITBLL on a cluster with the
patch in place before commit. if you commit this, please pay attention to the
nightly runs that follow so it can be reverted if it causes issues again.
No curator please.
> Master stuck in infinite loop when all Zookeeper servers are unreachable (and
> RS may run after losing its znode)
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0, 2.2.0
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch,
> HBASE-14498-V4.patch, HBASE-14498-V5.patch, HBASE-14498-V6.patch,
> HBASE-14498-V6.patch, HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.2.patch, HBASE-14498-branch-1.3-V2.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.007.patch, HBASE-14498.008.patch,
> HBASE-14498.master.001.patch, HBASE-14498.master.002.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of
[jira] [Updated] (HBASE-21915) FileLink$FileLinkInputStream doesn't implement CanUnbuffer
[
https://issues.apache.org/jira/browse/HBASE-21915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21915:
Component/s: Filesystem Integration
> FileLink$FileLinkInputStream doesn't implement CanUnbuffer
> --
>
> Key: HBASE-21915
> URL: https://issues.apache.org/jira/browse/HBASE-21915
> Project: HBase
> Issue Type: Bug
> Components: Filesystem Integration
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Attachments: HBASE-21915.001.patch
>
>
> FileLinkInputStream is an InputStream which handles the indirection of where
> the real HFile lives. This implementation is wrapped via
> FSDataInputStreamWrapper and is transparent when it's being used by a caller.
> Often, we have an FSDataInputStreamWrapper wrapping a FileLinkInputStream
> which wraps an FSDataInputStream.
> The problem is that FileLinkInputStream does not implement the
> \{{CanUnbuffer}} interface, which means that the underlying
> {{FSDataInputStream}} for the HFile the link refers to doesn't get
> {{unbuffer()}} called on it. This can cause an open Socket to hang around, as
> described in HBASE-9393.
> Both [~wchevreuil] and myself have run into this, each for different users.
> We think the commonality as to why these users saw this (but we haven't run
> into it on our own) is that it requires a very large snapshot to be brought
> into a new system. Big kudos to [~esteban] for his help in diagnosing this as
> well!
> If this analysis is accurate, it would affect all branches.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769759#comment-16769759
]
Sergey Shelukhin commented on HBASE-14498:
--
This is actually a critical data loss issue, because if RS is network split
from ZK, master could see its znode expire and reassign the region.
We've just had a ZK outage (not network) and some RSes kept running for hours
without having a connection, and doing region stuff.
+1 on the patch for now, I will commit Monday if no objections. nit: should use
HConstants.DEFAULT_ZK_SESSION_TIMEOUT, not 9.
I think this area needs to be hardened; if the server disconnects just before
ZK ping update that was somehow delayed for almost the whole session timeout,
and then waits 2/3rd of the timeout to abort, that means master has to wait
conntimeout + some time after the znode is gone before reassigning regions.
Also, I am not sure abort is good enough because it also takes time and may do
cleanup and such that affects region directories and WALs. Ideally the server
should kill -9 itself when losing the lock, or smth close enough.
Also, I wonder if we should use Curator; e.g. use a lock, with master trying to
steal all server locks - the server is dead as soon as the lock is stolen.
Although I guess there, SUSPENDED still needs to be handled in a similar way;
but at least I hope it sends LOST on the lack of connection. Also, it would
reduce HBase ZK code and benefit from Curator wisdom :)
I'll file a JIRA or two after this patch.
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.5.0, 2.0.0
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch,
> HBASE-14498-V4.patch, HBASE-14498-V5.patch, HBASE-14498-V6.patch,
> HBASE-14498-V6.patch, HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.2.patch, HBASE-14498-branch-1.3-V2.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.007.patch, HBASE-14498.008.patch,
> HBASE-14498.master.001.patch, HBASE-14498.master.002.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26
[jira] [Updated] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HBASE-14498:
-
Affects Version/s: 2.2.0
3.0.0
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0, 2.2.0
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch,
> HBASE-14498-V4.patch, HBASE-14498-V5.patch, HBASE-14498-V6.patch,
> HBASE-14498-V6.patch, HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.2.patch, HBASE-14498-branch-1.3-V2.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.007.patch, HBASE-14498.008.patch,
> HBASE-14498.master.001.patch, HBASE-14498.master.002.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 483913
> ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms,
> interval of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> {noformat}
> > At other end, region servers are reporting
[jira] [Commented] (HBASE-21914) Set assert to check if exception is set in procedures
[
https://issues.apache.org/jira/browse/HBASE-21914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769770#comment-16769770
]
Xu Cang commented on HBASE-21914:
-
Actually #isFailed() does this:
{code:java}
return exception != null || state == ProcedureState.ROLLEDBACK;
{code}
So, what you want is already covered. Please correct me if I misunderstood your
idea.
> Set assert to check if exception is set in procedures
> -
>
> Key: HBASE-21914
> URL: https://issues.apache.org/jira/browse/HBASE-21914
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Critical
>
> Take CreateTableProcedure as an example, in executeFromState()
> {code:java}
> case CREATE_TABLE_PRE_OPERATION:
> // Verify if we can create the table
> boolean exists = !prepareCreate(env);
> releaseSyncLatch();
>
> if (exists) {
> assert isFailed() : "the delete should have an exception here";
> return Flow.NO_MORE_STATE;
> }
> {code}
> The following assertion:
> {code}
> assert isFailed() : "the delete should have an exception here";
> {code}
> If I get the idea behind "the delete should have an exception here"
> correctly, it is to make sure that when setting the state to FAILED, the
> exception must be set. (or must call setFailure()). But the assertion only
> check isFailed() but no "!hasException()"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable (and RS may run after losing its znode)
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769759#comment-16769759
]
Sergey Shelukhin edited comment on HBASE-14498 at 2/15/19 9:32 PM:
---
This is actually a critical data loss issue, because if RS is network split
from ZK, master could see its znode expire and reassign the region while RS ZK
watcher is in the loop forever.
We've just had a ZK outage (not network) and some RSes kept running for hours
without having a connection, and doing region stuff.
+1 on the patch for now, I will commit Monday if no objections. nit: should use
HConstants.DEFAULT_ZK_SESSION_TIMEOUT, not 9.
I think this area needs to be hardened; if the server disconnects just before
ZK ping update that was somehow delayed for almost the whole session timeout,
and then waits 2/3rd of the timeout to abort, that means master has to wait
conntimeout + some time after the znode is gone before reassigning regions.
Also, I am not sure abort is good enough because it also takes time and may do
cleanup and such that affects region directories and WALs. Ideally the server
should kill -9 itself when losing the lock, or smth close enough.
Also, I wonder if we should use Curator; e.g. use a lock, with master trying to
steal all server locks - the server is dead as soon as the lock is stolen.
Although I guess there, SUSPENDED still needs to be handled in a similar way;
but at least I hope it sends LOST on the lack of connection. Also, it would
reduce HBase ZK code and benefit from Curator wisdom :)
I'll file a JIRA or two after this patch.
was (Author: sershe):
This is actually a critical data loss issue, because if RS is network split
from ZK, master could see its znode expire and reassign the region.
We've just had a ZK outage (not network) and some RSes kept running for hours
without having a connection, and doing region stuff.
+1 on the patch for now, I will commit Monday if no objections. nit: should use
HConstants.DEFAULT_ZK_SESSION_TIMEOUT, not 9.
I think this area needs to be hardened; if the server disconnects just before
ZK ping update that was somehow delayed for almost the whole session timeout,
and then waits 2/3rd of the timeout to abort, that means master has to wait
conntimeout + some time after the znode is gone before reassigning regions.
Also, I am not sure abort is good enough because it also takes time and may do
cleanup and such that affects region directories and WALs. Ideally the server
should kill -9 itself when losing the lock, or smth close enough.
Also, I wonder if we should use Curator; e.g. use a lock, with master trying to
steal all server locks - the server is dead as soon as the lock is stolen.
Although I guess there, SUSPENDED still needs to be handled in a similar way;
but at least I hope it sends LOST on the lack of connection. Also, it would
reduce HBase ZK code and benefit from Curator wisdom :)
I'll file a JIRA or two after this patch.
> Master stuck in infinite loop when all Zookeeper servers are unreachable (and
> RS may run after losing its znode)
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0, 2.2.0
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch,
> HBASE-14498-V4.patch, HBASE-14498-V5.patch, HBASE-14498-V6.patch,
> HBASE-14498-V6.patch, HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.2.patch, HBASE-14498-branch-1.3-V2.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.007.patch, HBASE-14498.008.patch,
> HBASE-14498.master.001.patch, HBASE-14498.master.002.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879
[jira] [Updated] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable (and RS may run after losing its znode)
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HBASE-14498:
-
Summary: Master stuck in infinite loop when all Zookeeper servers are
unreachable (and RS may run after losing its znode) (was: Master stuck in
infinite loop when all Zookeeper servers are unreachable)
> Master stuck in infinite loop when all Zookeeper servers are unreachable (and
> RS may run after losing its znode)
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0, 2.2.0
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch,
> HBASE-14498-V4.patch, HBASE-14498-V5.patch, HBASE-14498-V6.patch,
> HBASE-14498-V6.patch, HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.2.patch, HBASE-14498-branch-1.3-V2.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.007.patch, HBASE-14498.008.patch,
> HBASE-14498.master.001.patch, HBASE-14498.master.002.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 483913
> ms, expecting
[jira] [Updated] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HBASE-14498:
-
Priority: Blocker (was: Critical)
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0, 2.2.0
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch,
> HBASE-14498-V4.patch, HBASE-14498-V5.patch, HBASE-14498-V6.patch,
> HBASE-14498-V6.patch, HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.2.patch, HBASE-14498-branch-1.3-V2.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.007.patch, HBASE-14498.008.patch,
> HBASE-14498.master.001.patch, HBASE-14498.master.002.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 483913
> ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms,
> interval of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> {noformat}
> > At other end, region servers are reporting to HM1 on 3 sec
[jira] [Commented] (HBASE-21866) Do not move the table to null rsgroup when creating an existing table
[
https://issues.apache.org/jira/browse/HBASE-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769760#comment-16769760
]
Xu Cang commented on HBASE-21866:
-
HBASE-21914 sounds good to me. Though I think it's still good to check before
using it in this patch as you are doing now.
and the patch looks good to me now. I will check it in today. Thank you.
And, would you mind to take a look if this issue is in branch-1 or not?
> Do not move the table to null rsgroup when creating an existing table
> -
>
> Key: HBASE-21866
> URL: https://issues.apache.org/jira/browse/HBASE-21866
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Major
> Attachments: HBASE-21866.master.000.patch,
> HBASE-21866.master.001.patch, HBASE-21866.master.002.patch
>
>
> By using the latest HBase master branch, the bug could be re-produced as:
> # create 't1', 'cf1'
> # create 't1', 'cf1'
> The following message is logged into HMaster's log:
> {code}
> INFO [PEWorker-12] rsgroup.RSGroupAdminServer: Moving table t1 to RSGroup
> null
> {code}
> This is a wrong action and instead, we should keep t1 as where it originally
> is.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21915) FileLink$FileLinkInputStream doesn't implement CanUnbuffer
[
https://issues.apache.org/jira/browse/HBASE-21915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-21915:
---
Status: Patch Available (was: Open)
> FileLink$FileLinkInputStream doesn't implement CanUnbuffer
> --
>
> Key: HBASE-21915
> URL: https://issues.apache.org/jira/browse/HBASE-21915
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Attachments: HBASE-21915.001.patch
>
>
> FileLinkInputStream is an InputStream which handles the indirection of where
> the real HFile lives. This implementation is wrapped via
> FSDataInputStreamWrapper and is transparent when it's being used by a caller.
> Often, we have an FSDataInputStreamWrapper wrapping a FileLinkInputStream
> which wraps an FSDataInputStream.
> The problem is that FileLinkInputStream does not implement the
> \{{CanUnbuffer}} interface, which means that the underlying
> {{FSDataInputStream}} for the HFile the link refers to doesn't get
> {{unbuffer()}} called on it. This can cause an open Socket to hang around, as
> described in HBASE-9393.
> Both [~wchevreuil] and myself have run into this, each for different users.
> We think the commonality as to why these users saw this (but we haven't run
> into it on our own) is that it requires a very large snapshot to be brought
> into a new system. Big kudos to [~esteban] for his help in diagnosing this as
> well!
> If this analysis is accurate, it would affect all branches.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21915) FileLink$FileLinkInputStream doesn't implement CanUnbuffer
[
https://issues.apache.org/jira/browse/HBASE-21915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-21915:
---
Attachment: HBASE-21915.001.patch
> FileLink$FileLinkInputStream doesn't implement CanUnbuffer
> --
>
> Key: HBASE-21915
> URL: https://issues.apache.org/jira/browse/HBASE-21915
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Attachments: HBASE-21915.001.patch
>
>
> FileLinkInputStream is an InputStream which handles the indirection of where
> the real HFile lives. This implementation is wrapped via
> FSDataInputStreamWrapper and is transparent when it's being used by a caller.
> Often, we have an FSDataInputStreamWrapper wrapping a FileLinkInputStream
> which wraps an FSDataInputStream.
> The problem is that FileLinkInputStream does not implement the
> \{{CanUnbuffer}} interface, which means that the underlying
> {{FSDataInputStream}} for the HFile the link refers to doesn't get
> {{unbuffer()}} called on it. This can cause an open Socket to hang around, as
> described in HBASE-9393.
> Both [~wchevreuil] and myself have run into this, each for different users.
> We think the commonality as to why these users saw this (but we haven't run
> into it on our own) is that it requires a very large snapshot to be brought
> into a new system. Big kudos to [~esteban] for his help in diagnosing this as
> well!
> If this analysis is accurate, it would affect all branches.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21915) FileLink$FileLinkInputStream doesn't implement CanUnbuffer
Josh Elser created HBASE-21915:
--
Summary: FileLink$FileLinkInputStream doesn't implement CanUnbuffer
Key: HBASE-21915
URL: https://issues.apache.org/jira/browse/HBASE-21915
Project: HBase
Issue Type: Bug
Reporter: Josh Elser
Assignee: Josh Elser
FileLinkInputStream is an InputStream which handles the indirection of where
the real HFile lives. This implementation is wrapped via
FSDataInputStreamWrapper and is transparent when it's being used by a caller.
Often, we have an FSDataInputStreamWrapper wrapping a FileLinkInputStream which
wraps an FSDataInputStream.
The problem is that FileLinkInputStream does not implement the \{{CanUnbuffer}}
interface, which means that the underlying {{FSDataInputStream}} for the HFile
the link refers to doesn't get {{unbuffer()}} called on it. This can cause an
open Socket to hang around, as described in HBASE-9393.
Both [~wchevreuil] and myself have run into this, each for different users. We
think the commonality as to why these users saw this (but we haven't run into
it on our own) is that it requires a very large snapshot to be brought into a
new system. Big kudos to [~esteban] for his help in diagnosing this as well!
If this analysis is accurate, it would affect all branches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21866) Do not move the table to null rsgroup when creating an existing table
[
https://issues.apache.org/jira/browse/HBASE-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769705#comment-16769705
]
Hadoop QA commented on HBASE-21866:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
40s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 33s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}131m
4s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
13s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}179m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21866 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12958915/HBASE-21866.master.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 608baabfccc5 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ae0198084c |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[
https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769679#comment-16769679
]
Vladimir Rodionov commented on HBASE-21874:
---
{quote}
We use extended FileMmapIOEngine, but the mmap won't do the memmory mapping to
DRAM, it will mmap to a different address space maintained by the NVDIMM.
{quote}
Can you point the exact place in the patch where you control this?
> Bucket cache on Persistent memory
> -
>
> Key: HBASE-21874
> URL: https://issues.apache.org/jira/browse/HBASE-21874
> Project: HBase
> Issue Type: New Feature
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21874.patch, HBASE-21874.patch,
> HBASE-21874_V2.patch, Pmem_BC.png
>
>
> Non volatile persistent memory devices are byte addressable like DRAM (for
> eg. Intel DCPMM). Bucket cache implementation can take advantage of this new
> memory type and can make use of the existing offheap data structures to serve
> data directly from this memory area without having to bring the data to
> onheap.
> The patch is a new IOEngine implementation that works with the persistent
> memory.
> Note : Here we don't make use of the persistence nature of the device and
> just make use of the big memory it provides.
> Performance numbers to follow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21817) handle corrupted cells like other corrupted WAL cases
[ https://issues.apache.org/jira/browse/HBASE-21817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769631#comment-16769631 ] Sergey Shelukhin commented on HBASE-21817: -- I'll fix the final checkstyle issue on commit provided someone +1s this :) > handle corrupted cells like other corrupted WAL cases > - > > Key: HBASE-21817 > URL: https://issues.apache.org/jira/browse/HBASE-21817 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: HBASE-21817.01.patch, HBASE-21817.02.patch, > HBASE-21817.03.patch, HBASE-21817.04.patch, HBASE-21817.04.patch, > HBASE-21817.05.patch, HBASE-21817.patch > > > See HBASE-21601 for context. > I looked at the code a bit but it will take a while to understand, so for now > I'm going to mitigate it by handling such cases like any other corrupted WAL > (via either skipErrors or CorruptedHLog exception) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21623) ServerCrashProcedure can stomp on a RIT for a wrong server
[
https://issues.apache.org/jira/browse/HBASE-21623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769625#comment-16769625
]
Sergey Shelukhin commented on HBASE-21623:
--
[~wchevreuil] the problem is "for (RegionInfo region : regions) {" loop.
The "regions" are the regions assumed to be on the server; they are obtained on
previous procedure step, without any locks. So, if between getting "regions"
and running the loop RIT makes a change (transitions the region from OPENING
on server1 to OPENING on server2), SCP still has this region in the list.
> ServerCrashProcedure can stomp on a RIT for a wrong server
> --
>
> Key: HBASE-21623
> URL: https://issues.apache.org/jira/browse/HBASE-21623
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Critical
> Attachments: HBASE-21623.patch
>
>
> A server died while some region was being opened on it; eventually the open
> failed, and the RIT procedure started retrying on a different server.
> However, by then SCP for the dying server had already obtained the region
> from the list of regions on the old server, and proceeded to overwrite
> whatever the RIT was doing with a new server.
> {noformat}
> 2018-12-18 23:06:03,160 INFO [PEWorker-14] procedure2.ProcedureExecutor:
> Initialized subprocedures=[{pid=151404, ppid=151104, state=RUNNABLE,
> hasLock=false; org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure}]
> ...
> 2018-12-18 23:06:38,208 INFO [PEWorker-10] procedure.ServerCrashProcedure:
> Start pid=151632, state=RUNNABLE:SERVER_CRASH_START, hasLock=true;
> ServerCrashProcedure server=oldServer,17020,1545202098577, splitWal=true,
> meta=false
> ...
> 2018-12-18 23:06:41,953 WARN [RSProcedureDispatcher-pool4-t115]
> assignment.RegionRemoteProcedureBase: The remote operation pid=151404,
> ppid=151104, state=RUNNABLE, hasLock=false;
> org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure for region
> {ENCODED => region1, ... } to server oldServer,17020,1545202098577 failed
> org.apache.hadoop.hbase.regionserver.RegionServerAbortedException:
> org.apache.hadoop.hbase.regionserver.RegionServerAbortedException: Server
> oldServer,17020,1545202098577 aborting
> 2018-12-18 23:06:42,485 INFO [PEWorker-5] procedure2.ProcedureExecutor:
> Finished subprocedure(s) of pid=151104, ppid=150875,
> state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_OPENED, hasLock=true;
> TransitRegionStateProcedure table=t1, region=region1, ASSIGN; resume parent
> processing.
> 2018-12-18 23:06:42,485 INFO [PEWorker-13]
> assignment.TransitRegionStateProcedure: Retry=1 of max=2147483647;
> pid=151104, ppid=150875,
> state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_OPENED, hasLock=true;
> TransitRegionStateProcedure table=t1, region=region1, ASSIGN; rit=OPENING,
> location=oldServer,17020,1545202098577
> 2018-12-18 23:06:42,500 INFO [PEWorker-13]
> assignment.TransitRegionStateProcedure: Starting pid=151104, ppid=150875,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true;
> TransitRegionStateProcedure table=t1, region=region1, ASSIGN; rit=OPENING,
> location=null; forceNewPlan=true, retain=false
> 2018-12-18 23:06:42,657 INFO [PEWorker-2] assignment.RegionStateStore:
> pid=151104 updating hbase:meta row=region1, regionState=OPENING,
> regionLocation=newServer,17020,1545202111238
> ...
> 2018-12-18 23:06:43,094 INFO [PEWorker-4] procedure.ServerCrashProcedure:
> pid=151632, state=RUNNABLE:SERVER_CRASH_ASSIGN, hasLock=true;
> ServerCrashProcedure server=oldServer,17020,1545202098577, splitWal=true,
> meta=false found RIT pid=151104, ppid=150875,
> state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, hasLock=true;
> TransitRegionStateProcedure table=t1, region=region1, ASSIGN; rit=OPENING,
> location=newServer,17020,1545202111238, table=t1, region=region1
> 2018-12-18 23:06:43,094 INFO [PEWorker-4] assignment.RegionStateStore:
> pid=151104 updating hbase:meta row=region1, regionState=ABNORMALLY_CLOSED
> {noformat}
> Later, the RIT overwrote the state again, it seems, and then the region got
> stuck in OPENING state forever, but I'm not sure yet if that's just due to
> this bug or if there was another bug after that. For now this can be
> addressed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21817) handle corrupted cells like other corrupted WAL cases
[ https://issues.apache.org/jira/browse/HBASE-21817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769631#comment-16769631 ] Sergey Shelukhin edited comment on HBASE-21817 at 2/15/19 7:33 PM: --- I'll fix the final checkstyle issue on commit provided someone reviews and +1s this :) was (Author: sershe): I'll fix the final checkstyle issue on commit provided someone +1s this :) > handle corrupted cells like other corrupted WAL cases > - > > Key: HBASE-21817 > URL: https://issues.apache.org/jira/browse/HBASE-21817 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: HBASE-21817.01.patch, HBASE-21817.02.patch, > HBASE-21817.03.patch, HBASE-21817.04.patch, HBASE-21817.04.patch, > HBASE-21817.05.patch, HBASE-21817.patch > > > See HBASE-21601 for context. > I looked at the code a bit but it will take a while to understand, so for now > I'm going to mitigate it by handling such cases like any other corrupted WAL > (via either skipErrors or CorruptedHLog exception) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21744) timeout for server list refresh calls
[ https://issues.apache.org/jira/browse/HBASE-21744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769629#comment-16769629 ] Sergey Shelukhin commented on HBASE-21744: -- [~Apache9] does the updated patch look ok to you? I will also test it on a cluster before committing I think > timeout for server list refresh calls > -- > > Key: HBASE-21744 > URL: https://issues.apache.org/jira/browse/HBASE-21744 > Project: HBase > Issue Type: Improvement > Components: Zookeeper >Affects Versions: 3.0.0, 2.2.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HBASE-21744.01.patch, HBASE-21744.02.patch, > HBASE-21744.03.patch, HBASE-21744.04.patch, HBASE-21744.patch > > > Not sure why yet, but we are seeing the case when cluster is in overall a bad > state, where after RS dies and deletes its znode, the notification looks like > it's lost, so the master doesn't detect the failure. ZK itself appears to be > healthy and doesn't report anything special. > After some other change is made to the server list, master rescans the list > and picks up the stale change. Might make sense to add a config that would > trigger the refresh if it hasn't happened for a while (e.g. 1 minute). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21667) Move to latest ASF Parent POM
[ https://issues.apache.org/jira/browse/HBASE-21667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-21667: -- Attachment: HBASE-21667.branch-2.003.patch > Move to latest ASF Parent POM > - > > Key: HBASE-21667 > URL: https://issues.apache.org/jira/browse/HBASE-21667 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21667.branch-2.0.001.patch, > HBASE-21667.branch-2.0.002.patch, HBASE-21667.branch-2.001.patch, > HBASE-21667.branch-2.003.patch, HBASE-21667.master.001.patch, > HBASE-21667.master.002.patch, HBASE-21667.master.003.patch > > > Currently HBase depends on version 18 which was released on 2016-05-18. > Version 21 was released in August 2018. > Relevant dependency upgrades > > ||Name||Currently used version||New version||Notes|| > |surefire.version|2.21.0|2.22.0| | > |maven-compiler-plugin|3.6.1|3.7| | > |maven-dependency-plugin|3.0.1|3.1.1| | > |maven-jar-plugin|3.0.0|3.0.2| | > |maven-javadoc-plugin|3.0.0|3.0.1| | > |maven-resources-plugin|2.7|3.1.0| | > |maven-site-plugin|3.4|3.7.1|Currently not relying on ASF version. See: > HBASE-18333| > |maven-source-plugin|3.0.0|3.0.1| | > |maven-shade-plugin|3.0.0|3.1.1|Newly added to ASF pom| > |maven-clean-plugin|3.0.0|3.1.0| | > |maven-project-info-reports-plugin |2.9|3.0.0| | > Version 21 added net.nicoulaj.maven.plugins:checksum-maven-plugin which > introduced SHA512 checksum instead of SHA1. Should verify if we can rely on > that for releases or breaks our current processes. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-21913) src/main/asciidoc/images link is incorrect
[ https://issues.apache.org/jira/browse/HBASE-21913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-21913. --- Resolution: Fixed > src/main/asciidoc/images link is incorrect > -- > > Key: HBASE-21913 > URL: https://issues.apache.org/jira/browse/HBASE-21913 > Project: HBase > Issue Type: Bug > Components: website >Affects Versions: 2.0.4 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Trivial > Fix For: 2.0.5 > > Attachments: HBASE-21913.branch-2.0.001.patch > > > The link for src/main/asciidoc/images points to ../../site/resources/images/ > but the site is one folder up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[ https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769596#comment-16769596 ] Anoop Sam John commented on HBASE-21874: bq.If you keep data in PMEM and use extended FileMmapIOEngine, where do yo you mmap it into? into DRAM? That is strange bq. You mmap PMEM resided file into RAM No.. The mmap call will NOT bring data into DRAM. The Pmem device is direct accessible for CPU with load/store instructions. The mmap will just give a ByteBuffer with the address in Pmem. > Bucket cache on Persistent memory > - > > Key: HBASE-21874 > URL: https://issues.apache.org/jira/browse/HBASE-21874 > Project: HBase > Issue Type: New Feature > Components: BucketCache >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21874.patch, HBASE-21874.patch, > HBASE-21874_V2.patch, Pmem_BC.png > > > Non volatile persistent memory devices are byte addressable like DRAM (for > eg. Intel DCPMM). Bucket cache implementation can take advantage of this new > memory type and can make use of the existing offheap data structures to serve > data directly from this memory area without having to bring the data to > onheap. > The patch is a new IOEngine implementation that works with the persistent > memory. > Note : Here we don't make use of the persistence nature of the device and > just make use of the big memory it provides. > Performance numbers to follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21866) Do not move the table to null rsgroup when creating an existing table
[
https://issues.apache.org/jira/browse/HBASE-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769563#comment-16769563
]
Xiang Li commented on HBASE-21866:
--
Hi [~xucang], does HBASE-21914 make any sense to you?
> Do not move the table to null rsgroup when creating an existing table
> -
>
> Key: HBASE-21866
> URL: https://issues.apache.org/jira/browse/HBASE-21866
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Major
> Attachments: HBASE-21866.master.000.patch,
> HBASE-21866.master.001.patch, HBASE-21866.master.002.patch
>
>
> By using the latest HBase master branch, the bug could be re-produced as:
> # create 't1', 'cf1'
> # create 't1', 'cf1'
> The following message is logged into HMaster's log:
> {code}
> INFO [PEWorker-12] rsgroup.RSGroupAdminServer: Moving table t1 to RSGroup
> null
> {code}
> This is a wrong action and instead, we should keep t1 as where it originally
> is.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21874) Bucket cache on Persistent memory
[ https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769571#comment-16769571 ] Vladimir Rodionov edited comment on HBASE-21874 at 2/15/19 6:31 PM: This patch requires good write up, it is not clear how this thing work, [~ram_krish]. From what I see, you extends FileMmapEngine, which works with persistent cache by mmapping it into RAM, so your new engine does exactly the same thing? As far as I understand, the major selling point of PMEM is providing RAM-like capacity beyond what DRAM can do. So we are supposed to have some relatively small DRAM and large size of NVDIMM installed into a server. This raises the questions: # Where are your going to keep bucket cache? Not in DRAM definitely, hence in NVDIMM (PMEM)? # If you keep data in PMEM and use extended FileMmapIOEngine, where do yo you mmap it into? into DRAM? That is strange # My question, regarding file system required on top PMEM has remained unanswered. What I see in this patch (may be I am wrong) # You rely on file system on top of PMEM # You mmap PMEM resided file into RAM Am I right? was (Author: vrodionov): This patch requires good write up, it is not clear how this thing work, [~ram_krish]. From what I see, you extends FileMmapEngine, which works with persistent cache by mmapping it into RAM, so your new engine does exactly the same thing? As far as I understand, the major selling point of PMEM is providing RAM-like capacity beyond what DRAM can do. So we are supposed to have some relatively small DRAM and large size of NVDIMM installed into a server. This raises the questions: # Where are your going to keep bucket cache? Not in DRAM definitely, hence in NVDIMM (PMEM)? # If you keep data in PMEM and use extended FileMmapIOEngine, where do yo you mmap it into? into DRAM? That is strange # My question, regarding file system required on top PMEM has remained unanswered. What I see in this patch (may be I am wrong) # You rely on file system on top of PMEM # You mmap PMEM resided file into DRAM Am I right? > Bucket cache on Persistent memory > - > > Key: HBASE-21874 > URL: https://issues.apache.org/jira/browse/HBASE-21874 > Project: HBase > Issue Type: New Feature > Components: BucketCache >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21874.patch, HBASE-21874.patch, > HBASE-21874_V2.patch, Pmem_BC.png > > > Non volatile persistent memory devices are byte addressable like DRAM (for > eg. Intel DCPMM). Bucket cache implementation can take advantage of this new > memory type and can make use of the existing offheap data structures to serve > data directly from this memory area without having to bring the data to > onheap. > The patch is a new IOEngine implementation that works with the persistent > memory. > Note : Here we don't make use of the persistence nature of the device and > just make use of the big memory it provides. > Performance numbers to follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21874) Bucket cache on Persistent memory
[ https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769571#comment-16769571 ] Vladimir Rodionov edited comment on HBASE-21874 at 2/15/19 6:27 PM: This patch requires good write up, it is not clear how this thing work, [~ram_krish]. From what I see, you extends FileMmapEngine, which works with persistent cache by mmapping it into RAM, so your new engine does exactly the same thing? As far as I understand, the major selling point of PMEM is providing RAM-like capacity beyond what DRAM can do. So we are supposed to have some relatively small DRAM and large size of NVDIMM installed into a server. This raises the questions: # Where are your going to keep bucket cache? Not in DRAM definitely, hence in NVDIMM (PMEM)? # If you keep data in PMEM and use extended FileMmapIOEngine, where do yo you mmap it into? into DRAM? That is strange # My question, regarding file system required on top PMEM has remained unanswered. What I see in this patch (may be I am wrong) # You rely on file system on top of PMEM # You mmap PMEM resided file into DRAM This does not look right to me. was (Author: vrodionov): This patch requires good write up, it is not clear how this thing work, [~ram_krish]. From what I see, you extends FileMmapEngine, which works with persistent cache by mmapping it into RAM, so your new engine does exactly the same thing? As far as I understand, the major selling point of PMEM is providing RAM-like capacity beyond what DRAM can do. So we are supposed to have some relatively small DRAM and large size of NVDIMM installed into a server. This raises the questions: 1. Where are your going to keep bucket cache? Not in DRAM definitely, hence in NVDIMM (PMEM)? 2. If you keep data in PMEM and use extended FileMmapIOEngine, where do yo you mmap it into? into DRAM? That is strange 3. My question, regarding file system required on top PMEM has remained unanswered. What I see in this patch (may be I am wrong) 1. You rely on file system on top of PMEM 2. You mmap PMEM resided file into DRAM This does not look optimal to me. > Bucket cache on Persistent memory > - > > Key: HBASE-21874 > URL: https://issues.apache.org/jira/browse/HBASE-21874 > Project: HBase > Issue Type: New Feature > Components: BucketCache >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21874.patch, HBASE-21874.patch, > HBASE-21874_V2.patch, Pmem_BC.png > > > Non volatile persistent memory devices are byte addressable like DRAM (for > eg. Intel DCPMM). Bucket cache implementation can take advantage of this new > memory type and can make use of the existing offheap data structures to serve > data directly from this memory area without having to bring the data to > onheap. > The patch is a new IOEngine implementation that works with the persistent > memory. > Note : Here we don't make use of the persistence nature of the device and > just make use of the big memory it provides. > Performance numbers to follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[ https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769571#comment-16769571 ] Vladimir Rodionov commented on HBASE-21874: --- This patch requires good write up, it is not clear how this thing work, [~ram_krish]. From what I see, you extends FileMmapEngine, which works with persistent cache by mmapping it into RAM, so your new engine does exactly the same thing? As far as I understand, the major selling point of PMEM is providing RAM-like capacity beyond what DRAM can do. So we are supposed to have some relatively small DRAM and large size of NVDIMM installed into a server. This raises the questions: 1. Where are your going to keep bucket cache? Not in DRAM definitely, hence in NVDIMM (PMEM)? 2. If you keep data in PMEM and use extended FileMmapIOEngine, where do yo you mmap it into? into DRAM? That is strange 3. My question, regarding file system required on top PMEM has remained unanswered. What I see in this patch (may be I am wrong) 1. You rely on file system on top of PMEM 2. You mmap PMEM resided file into DRAM This does not look optimal to me. > Bucket cache on Persistent memory > - > > Key: HBASE-21874 > URL: https://issues.apache.org/jira/browse/HBASE-21874 > Project: HBase > Issue Type: New Feature > Components: BucketCache >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21874.patch, HBASE-21874.patch, > HBASE-21874_V2.patch, Pmem_BC.png > > > Non volatile persistent memory devices are byte addressable like DRAM (for > eg. Intel DCPMM). Bucket cache implementation can take advantage of this new > memory type and can make use of the existing offheap data structures to serve > data directly from this memory area without having to bring the data to > onheap. > The patch is a new IOEngine implementation that works with the persistent > memory. > Note : Here we don't make use of the persistence nature of the device and > just make use of the big memory it provides. > Performance numbers to follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21913) src/main/asciidoc/images link is incorrect
[ https://issues.apache.org/jira/browse/HBASE-21913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769566#comment-16769566 ] Josh Elser commented on HBASE-21913: +1 > src/main/asciidoc/images link is incorrect > -- > > Key: HBASE-21913 > URL: https://issues.apache.org/jira/browse/HBASE-21913 > Project: HBase > Issue Type: Bug > Components: website >Affects Versions: 2.0.4 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Trivial > Fix For: 2.0.5 > > Attachments: HBASE-21913.branch-2.0.001.patch > > > The link for src/main/asciidoc/images points to ../../site/resources/images/ > but the site is one folder up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21505) Several inconsistencies on information reported for Replication Sources by hbase shell status 'replication' command.
[
https://issues.apache.org/jira/browse/HBASE-21505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769564#comment-16769564
]
Hadoop QA commented on HBASE-21505:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
24s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} hbase-hadoop2-compat in master has 18 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch passed checkstyle in hbase-protocol-shaded
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch passed checkstyle in hbase-hadoop-compat
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch passed checkstyle in hbase-hadoop2-compat
{color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} hbase-client: The patch generated 1 new + 185
unchanged - 0 fixed = 186 total (was 185) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} hbase-server: The patch generated 0 new + 84
unchanged - 3 fixed = 84 total (was 87) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch passed checkstyle in hbase-shell {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
9s{color} | {color:red} The patch generated 4 new + 425 unchanged - 26 fixed =
429 total (was 451) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 4s{color} | {color:orange} The patch generated 3 new + 752 unchanged - 2
fixed = 755 total (was 754) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
57s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 40s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
2m 29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}
[jira] [Updated] (HBASE-21914) Set assert to check if exception is set in procedures
[
https://issues.apache.org/jira/browse/HBASE-21914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-21914:
-
Description:
Take CreateTableProcedure as an example, in executeFromState()
{code:java}
case CREATE_TABLE_PRE_OPERATION:
// Verify if we can create the table
boolean exists = !prepareCreate(env);
releaseSyncLatch();
if (exists) {
assert isFailed() : "the delete should have an exception here";
return Flow.NO_MORE_STATE;
}
{code}
The following assertion:
{code}
assert isFailed() : "the delete should have an exception here";
{code}
If I get the idea behind "the delete should have an exception here" correctly,
it is to make sure that when setting the state to FAILED, the exception must be
set. (or must call setFailure()). But the assertion only check isFailed() but
no "!hasException()"
was:
Take CreateTableProcedure as an example, in executeFromState()
{code:java}
case CREATE_TABLE_PRE_OPERATION:
// Verify if we can create the table
boolean exists = !prepareCreate(env);
releaseSyncLatch();
if (exists) {
assert isFailed() : "the delete should have an exception here";
return Flow.NO_MORE_STATE;
}
{code}
The following assertion:
{code}
assert isFailed() : "the delete should have an exception here";
{code}
If I get it correctly, the idea is to make sure that when setting the state to
FAILDED, exception must be set. (or must call setFailure()). But the assertion
only check isFailed() but no "!hasException()"
> Set assert to check if exception is set in procedures
> -
>
> Key: HBASE-21914
> URL: https://issues.apache.org/jira/browse/HBASE-21914
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Critical
>
> Take CreateTableProcedure as an example, in executeFromState()
> {code:java}
> case CREATE_TABLE_PRE_OPERATION:
> // Verify if we can create the table
> boolean exists = !prepareCreate(env);
> releaseSyncLatch();
>
> if (exists) {
> assert isFailed() : "the delete should have an exception here";
> return Flow.NO_MORE_STATE;
> }
> {code}
> The following assertion:
> {code}
> assert isFailed() : "the delete should have an exception here";
> {code}
> If I get the idea behind "the delete should have an exception here"
> correctly, it is to make sure that when setting the state to FAILED, the
> exception must be set. (or must call setFailure()). But the assertion only
> check isFailed() but no "!hasException()"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21914) Set assert to check if exception is set in procedures
[
https://issues.apache.org/jira/browse/HBASE-21914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-21914:
-
Description:
Take CreateTableProcedure as an example, in executeFromState()
{code:java}
case CREATE_TABLE_PRE_OPERATION:
// Verify if we can create the table
boolean exists = !prepareCreate(env);
releaseSyncLatch();
if (exists) {
assert isFailed() : "the delete should have an exception here";
return Flow.NO_MORE_STATE;
}
{code}
The following assertion:
{code}
assert isFailed() : "the delete should have an exception here";
{code}
If I get it correctly, the idea is to make sure that when setting the state to
FAILDED, exception must be set. (or must call setFailure()). But the assertion
only check isFailed() but no "!hasException()"
> Set assert to check if exception is set in procedures
> -
>
> Key: HBASE-21914
> URL: https://issues.apache.org/jira/browse/HBASE-21914
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Critical
>
> Take CreateTableProcedure as an example, in executeFromState()
> {code:java}
> case CREATE_TABLE_PRE_OPERATION:
> // Verify if we can create the table
> boolean exists = !prepareCreate(env);
> releaseSyncLatch();
>
> if (exists) {
> assert isFailed() : "the delete should have an exception here";
> return Flow.NO_MORE_STATE;
> }
> {code}
> The following assertion:
> {code}
> assert isFailed() : "the delete should have an exception here";
> {code}
> If I get it correctly, the idea is to make sure that when setting the state
> to FAILDED, exception must be set. (or must call setFailure()). But the
> assertion only check isFailed() but no "!hasException()"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21914) Set assert to check if exception is set in procedures
Xiang Li created HBASE-21914: Summary: Set assert to check if exception is set in procedures Key: HBASE-21914 URL: https://issues.apache.org/jira/browse/HBASE-21914 Project: HBase Issue Type: Bug Components: proc-v2 Reporter: Xiang Li Assignee: Xiang Li -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21894) Master doesn't update the meta state as soon as the meta server dies
[ https://issues.apache.org/jira/browse/HBASE-21894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769549#comment-16769549 ] Bahram Chehrazy commented on HBASE-21894: - [~sershe], - who does it for non-system regions? Good question. I was wondering about that too, but I had no strong evidence of user region state going inconsistent when the meta remains alive. The code path for user regions is different and their state gets updated into the meta. So, I limited this fix to the meta regions to minimize impact. But now that I think about it, the meta could crash when a user regions has just begun TRSP. Therefore, it might be a good idea to do the same for all regions. I need to think about further and test it. > Master doesn't update the meta state as soon as the meta server dies > > > Key: HBASE-21894 > URL: https://issues.apache.org/jira/browse/HBASE-21894 > Project: HBase > Issue Type: Bug > Components: master, meta >Affects Versions: 3.0.0 >Reporter: Bahram Chehrazy >Assignee: Bahram Chehrazy >Priority: Major > Attachments: Master-to-update-meta-state-on-ZK-asap.patch, > Update-master-in-memory-state-of-meta-asap.patch > > > > When the meta server dies, Master moves that server to the deadServers list > and submits a SCP, but it doesn't change the Meta region state (to CLOSING, > CLOSED or OFFLINE) until after SCP finishes. Only at that time the meta > region state changes from OPEN to OPENING, and then quickly back to OPEN. > > This could cause problems if some procedures try to update meta while master > is recovering the meta region, or even worse, if the master also dies in the > mean time. Other potential problem include servers trying to update the meta > which it's down, causing them to abort after several retries. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[ https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769543#comment-16769543 ] ramkrishna.s.vasudevan commented on HBASE-21874: bq.ByteBufferIOEngine basically offers the same functionality if we don't care about persistence and the Intel DCPMM comes in a DIMM form factor so shouldn't that _just work_? At a high level the basic idea is that the DCPMM address space and DRAM address space are different and we want our cache to be there in the DCPMM address space only. bq.SharedMemoryFileMmapEngine if it's not going to be persistent. This 'shared' feature works only because we are on Pmem device. Otherwise for a normal mmap case if the DRAM size is not enough we may have to copy the data to the RAM space which could happen due to page thrashing. > Bucket cache on Persistent memory > - > > Key: HBASE-21874 > URL: https://issues.apache.org/jira/browse/HBASE-21874 > Project: HBase > Issue Type: New Feature > Components: BucketCache >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21874.patch, HBASE-21874.patch, > HBASE-21874_V2.patch, Pmem_BC.png > > > Non volatile persistent memory devices are byte addressable like DRAM (for > eg. Intel DCPMM). Bucket cache implementation can take advantage of this new > memory type and can make use of the existing offheap data structures to serve > data directly from this memory area without having to bring the data to > onheap. > The patch is a new IOEngine implementation that works with the persistent > memory. > Note : Here we don't make use of the persistence nature of the device and > just make use of the big memory it provides. > Performance numbers to follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21866) Do not move the table to null rsgroup when creating an existing table
[
https://issues.apache.org/jira/browse/HBASE-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769546#comment-16769546
]
Xiang Li edited comment on HBASE-21866 at 2/15/19 5:53 PM:
---
Hi [~xucang] I uploaded patch v003 to give a check by #hasException() before
calling #getException().
But after I read the context, I think that might not be needed:
CreateTableProcedure#executeFromState() has the following code:
{code:java}
case CREATE_TABLE_PRE_OPERATION:
// Verify if we can create the table
boolean exists = !prepareCreate(env);
releaseSyncLatch();
if (exists) {
assert isFailed() : "the delete should have an exception here";
return Flow.NO_MORE_STATE;
}
{code}
The assert will fail if prepareCreate() returns false but does not call
setFailure() in it. So when doing roll-back, setFailure() has been called
before, and an Exception has been new-ed and set to exception. Or someone
bypasses setFailure() and set state to FAILED directly...
was (Author: water):
Hi [~xucang] I uploaded patch v003 to give a check by #hasException() before
calling #getException().
But after I read the context, I think that might not be needed:
CreateTableProcedure#executeFromState() has the following code:
{code:java}
case CREATE_TABLE_PRE_OPERATION:
// Verify if we can create the table
boolean exists = !prepareCreate(env);
releaseSyncLatch();
if (exists) {
assert isFailed() : "the delete should have an exception here";
return Flow.NO_MORE_STATE;
}
{code}
The assert will fail if prepareCreate() returns false but does not call
setFailure() in it. So when doing roll-back, setFailure() has been called
before, and an Exception has been new-ed and set to exception. Or someone
bypasses setFailure() and set state to FAILED directly...
> Do not move the table to null rsgroup when creating an existing table
> -
>
> Key: HBASE-21866
> URL: https://issues.apache.org/jira/browse/HBASE-21866
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Major
> Attachments: HBASE-21866.master.000.patch,
> HBASE-21866.master.001.patch, HBASE-21866.master.002.patch
>
>
> By using the latest HBase master branch, the bug could be re-produced as:
> # create 't1', 'cf1'
> # create 't1', 'cf1'
> The following message is logged into HMaster's log:
> {code}
> INFO [PEWorker-12] rsgroup.RSGroupAdminServer: Moving table t1 to RSGroup
> null
> {code}
> This is a wrong action and instead, we should keep t1 as where it originally
> is.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21866) Do not move the table to null rsgroup when creating an existing table
[
https://issues.apache.org/jira/browse/HBASE-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769546#comment-16769546
]
Xiang Li commented on HBASE-21866:
--
Hi [~xucang] I uploaded patch v003 to give a check by #hasException() before
calling #getException().
But after I read the context, I think that might not be needed:
CreateTableProcedure#executeFromState() has the following code:
{code:java}
case CREATE_TABLE_PRE_OPERATION:
// Verify if we can create the table
boolean exists = !prepareCreate(env);
releaseSyncLatch();
if (exists) {
assert isFailed() : "the delete should have an exception here";
return Flow.NO_MORE_STATE;
}
{code}
The assert will fail if prepareCreate() returns false but does not call
setFailure() in it. So when doing roll-back, setFailure() has been called
before, and an Exception has been new-ed and set to exception. Or someone
bypasses setFailure() and set state to FAILED directly...
> Do not move the table to null rsgroup when creating an existing table
> -
>
> Key: HBASE-21866
> URL: https://issues.apache.org/jira/browse/HBASE-21866
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Major
> Attachments: HBASE-21866.master.000.patch,
> HBASE-21866.master.001.patch, HBASE-21866.master.002.patch
>
>
> By using the latest HBase master branch, the bug could be re-produced as:
> # create 't1', 'cf1'
> # create 't1', 'cf1'
> The following message is logged into HMaster's log:
> {code}
> INFO [PEWorker-12] rsgroup.RSGroupAdminServer: Moving table t1 to RSGroup
> null
> {code}
> This is a wrong action and instead, we should keep t1 as where it originally
> is.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21667) Move to latest ASF Parent POM
[
https://issues.apache.org/jira/browse/HBASE-21667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769534#comment-16769534
]
Hadoop QA commented on HBASE-21667:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 14m
27s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 4m
55s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
10s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 24m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
6s{color} | {color:blue} patch has no errors when building the reference guide.
See footer for rendered docs, which you should manually inspect. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 1s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}191m
22s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}282m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21667 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12958881/HBASE-21667.master.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile refguide mvnsite |
| uname | Linux fab50e53ae52 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 81e1e2f943 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/15991/artifact/patchprocess/branch-site/book.html
|
| refguide
[jira] [Updated] (HBASE-21866) Do not move the table to null rsgroup when creating an existing table
[
https://issues.apache.org/jira/browse/HBASE-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-21866:
-
Attachment: HBASE-21866.master.002.patch
> Do not move the table to null rsgroup when creating an existing table
> -
>
> Key: HBASE-21866
> URL: https://issues.apache.org/jira/browse/HBASE-21866
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Major
> Attachments: HBASE-21866.master.000.patch,
> HBASE-21866.master.001.patch, HBASE-21866.master.002.patch
>
>
> By using the latest HBase master branch, the bug could be re-produced as:
> # create 't1', 'cf1'
> # create 't1', 'cf1'
> The following message is logged into HMaster's log:
> {code}
> INFO [PEWorker-12] rsgroup.RSGroupAdminServer: Moving table t1 to RSGroup
> null
> {code}
> This is a wrong action and instead, we should keep t1 as where it originally
> is.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21866) Do not move the table to null rsgroup when creating an existing table
[
https://issues.apache.org/jira/browse/HBASE-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-21866:
-
Status: Patch Available (was: Open)
> Do not move the table to null rsgroup when creating an existing table
> -
>
> Key: HBASE-21866
> URL: https://issues.apache.org/jira/browse/HBASE-21866
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Major
> Attachments: HBASE-21866.master.000.patch,
> HBASE-21866.master.001.patch, HBASE-21866.master.002.patch
>
>
> By using the latest HBase master branch, the bug could be re-produced as:
> # create 't1', 'cf1'
> # create 't1', 'cf1'
> The following message is logged into HMaster's log:
> {code}
> INFO [PEWorker-12] rsgroup.RSGroupAdminServer: Moving table t1 to RSGroup
> null
> {code}
> This is a wrong action and instead, we should keep t1 as where it originally
> is.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21866) Do not move the table to null rsgroup when creating an existing table
[
https://issues.apache.org/jira/browse/HBASE-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-21866:
-
Status: Open (was: Patch Available)
> Do not move the table to null rsgroup when creating an existing table
> -
>
> Key: HBASE-21866
> URL: https://issues.apache.org/jira/browse/HBASE-21866
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Major
> Attachments: HBASE-21866.master.000.patch,
> HBASE-21866.master.001.patch
>
>
> By using the latest HBase master branch, the bug could be re-produced as:
> # create 't1', 'cf1'
> # create 't1', 'cf1'
> The following message is logged into HMaster's log:
> {code}
> INFO [PEWorker-12] rsgroup.RSGroupAdminServer: Moving table t1 to RSGroup
> null
> {code}
> This is a wrong action and instead, we should keep t1 as where it originally
> is.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21872) Clean up getBytes() calls without charsets provided
[
https://issues.apache.org/jira/browse/HBASE-21872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769519#comment-16769519
]
Josh Elser commented on HBASE-21872:
Reverted and re-applied. Thanks for letting me know, Duo.
> Clean up getBytes() calls without charsets provided
> ---
>
> Key: HBASE-21872
> URL: https://issues.apache.org/jira/browse/HBASE-21872
> Project: HBase
> Issue Type: Task
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Trivial
> Fix For: 3.0.0
>
> Attachments: HBASE-21782.001.patch, HBASE-21782.002.patch,
> HBASE-21782.003.patch, HBASE-21872.004.patch
>
>
> As we saw over in HBASE-21201, the use of {{String.getBytes()}} without a
> Charset can result is some compiler warnings. Let's just get rid of these
> calls. There are only a handful anymore in master.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[ https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769508#comment-16769508 ] Jean-Daniel Cryans commented on HBASE-21874: Thank you Anoop and Ram for addressing my questions. Some follow-ups: bq. We can remove that part if it is distracting but it may be needed if you need to create bigger sized cache. It seems like a java restriction from mmapping buffers. We should document this limitation and add a check in the code for the moment. bq. So it becomes a direct replacement of the DRAM based IOEngine. ( you don't copy the buffers onheap) That's the part I don't get. ByteBufferIOEngine basically offers the same functionality if we don't care about persistence and the Intel DCPMM comes in a DIMM form factor so shouldn't that _just work_? > Bucket cache on Persistent memory > - > > Key: HBASE-21874 > URL: https://issues.apache.org/jira/browse/HBASE-21874 > Project: HBase > Issue Type: New Feature > Components: BucketCache >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21874.patch, HBASE-21874.patch, > HBASE-21874_V2.patch, Pmem_BC.png > > > Non volatile persistent memory devices are byte addressable like DRAM (for > eg. Intel DCPMM). Bucket cache implementation can take advantage of this new > memory type and can make use of the existing offheap data structures to serve > data directly from this memory area without having to bring the data to > onheap. > The patch is a new IOEngine implementation that works with the persistent > memory. > Note : Here we don't make use of the persistence nature of the device and > just make use of the big memory it provides. > Performance numbers to follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[ https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769510#comment-16769510 ] Jean-Daniel Cryans commented on HBASE-21874: Oh additionally maybe we shouldn't call this PmemIOEngine but something more generic like SharedMemoryFileMmapEngine if it's not going to be persistent. > Bucket cache on Persistent memory > - > > Key: HBASE-21874 > URL: https://issues.apache.org/jira/browse/HBASE-21874 > Project: HBase > Issue Type: New Feature > Components: BucketCache >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21874.patch, HBASE-21874.patch, > HBASE-21874_V2.patch, Pmem_BC.png > > > Non volatile persistent memory devices are byte addressable like DRAM (for > eg. Intel DCPMM). Bucket cache implementation can take advantage of this new > memory type and can make use of the existing offheap data structures to serve > data directly from this memory area without having to bring the data to > onheap. > The patch is a new IOEngine implementation that works with the persistent > memory. > Note : Here we don't make use of the persistence nature of the device and > just make use of the big memory it provides. > Performance numbers to follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-21686) Release 1.2.10
[ https://issues.apache.org/jira/browse/HBASE-21686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved HBASE-21686. - Resolution: Fixed * added 1.2.10 to the downloads page * triggered a website build > Release 1.2.10 > -- > > Key: HBASE-21686 > URL: https://issues.apache.org/jira/browse/HBASE-21686 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.2.10 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21910) The nonce implementation is wrong for AsyncTable
[
https://issues.apache.org/jira/browse/HBASE-21910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769489#comment-16769489
]
Hudson commented on HBASE-21910:
Results for branch branch-2.1
[build #868 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/868/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/868//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/868//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/868//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The nonce implementation is wrong for AsyncTable
>
>
> Key: HBASE-21910
> URL: https://issues.apache.org/jira/browse/HBASE-21910
> Project: HBase
> Issue Type: Bug
> Components: asyncclient, Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4
>
> Attachments: HBASE-21910.patch
>
>
> We recreate the nonce every time when retry, which is not correct.
> And it is strange that we even do not have a UT for this...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (HBASE-21686) Release 1.2.10
[ https://issues.apache.org/jira/browse/HBASE-21686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reopened HBASE-21686: - reopening as our Peter noticed I forgot to update the downloads page on the website (and I want to make sure all the steps are on this jira) > Release 1.2.10 > -- > > Key: HBASE-21686 > URL: https://issues.apache.org/jira/browse/HBASE-21686 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.2.10 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21898) Release 1.2.11
[ https://issues.apache.org/jira/browse/HBASE-21898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769482#comment-16769482 ] Sean Busbey commented on HBASE-21898: - [build #664 for nightly|https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.2/664/] looks green. flaky board still looks reasonable. (the test that failed build #663 isn't blowing up when in the flaky run job) Barring a new blocker showing up, I'll cut 1.2.11 RC0 from ca53d58f5b7abde0c189c9f78baf4246bddffac3 > Release 1.2.11 > -- > > Key: HBASE-21898 > URL: https://issues.apache.org/jira/browse/HBASE-21898 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.2.11 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.11 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20448) update ref guide to expressly use shaded clients for examples
[ https://issues.apache.org/jira/browse/HBASE-20448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769478#comment-16769478 ] Sean Busbey commented on HBASE-20448: - what's the approximate timeline for 2.2.0 at this point? I think this maybe needs to be blocked on being able to use HBTU with shaded client > update ref guide to expressly use shaded clients for examples > - > > Key: HBASE-20448 > URL: https://issues.apache.org/jira/browse/HBASE-20448 > Project: HBase > Issue Type: Improvement > Components: Client, documentation, mapreduce >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.2.0 > > > the whole mapreduce section, especially, should be using the shaded version. > Specifically include an example of running the ITBLL we ship -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21913) src/main/asciidoc/images link is incorrect
[ https://issues.apache.org/jira/browse/HBASE-21913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-21913: -- Attachment: HBASE-21913.branch-2.0.001.patch > src/main/asciidoc/images link is incorrect > -- > > Key: HBASE-21913 > URL: https://issues.apache.org/jira/browse/HBASE-21913 > Project: HBase > Issue Type: Bug > Components: website >Affects Versions: 2.0.4 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Trivial > Fix For: 2.0.5 > > Attachments: HBASE-21913.branch-2.0.001.patch > > > The link for src/main/asciidoc/images points to ../../site/resources/images/ > but the site is one folder up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21909) Validate the put instance before executing in AsyncTable.put method
[
https://issues.apache.org/jira/browse/HBASE-21909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769458#comment-16769458
]
Hadoop QA commented on HBASE-21909:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
38s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 50s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
12s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}128m
37s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}181m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21909 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12958885/HBASE-21909.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 5c2ca6c080f2 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 81e1e2f943 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs
[jira] [Commented] (HBASE-21913) src/main/asciidoc/images link is incorrect
[ https://issues.apache.org/jira/browse/HBASE-21913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769453#comment-16769453 ] Peter Somogyi commented on HBASE-21913: --- Trivial patch attached. > src/main/asciidoc/images link is incorrect > -- > > Key: HBASE-21913 > URL: https://issues.apache.org/jira/browse/HBASE-21913 > Project: HBase > Issue Type: Bug > Components: website >Affects Versions: 2.0.4 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Trivial > Fix For: 2.0.5 > > Attachments: HBASE-21913.branch-2.0.001.patch > > > The link for src/main/asciidoc/images points to ../../site/resources/images/ > but the site is one folder up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21875) Change the retry logic in RSProcedureDispatcher to 'retry by default, only if xxx'
[
https://issues.apache.org/jira/browse/HBASE-21875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769423#comment-16769423
]
Hudson commented on HBASE-21875:
Results for branch branch-2
[build #1686 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1686/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1686//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1686//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1686//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Change the retry logic in RSProcedureDispatcher to 'retry by default, only if
> xxx'
> --
>
> Key: HBASE-21875
> URL: https://issues.apache.org/jira/browse/HBASE-21875
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21875-branch-2.1.patch, HBASE-21875-v1.patch,
> HBASE-21875-v2.patch, HBASE-21875-v3.patch, HBASE-21875-v4.patch,
> HBASE-21875.patch
>
>
> For now it is not retry by default, only if xxx.
> In executeProcedures, we will only throw a fixed set of exception, so we
> should change to retry by default, and check for the exceptions which we do
> not need to retry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21913) src/main/asciidoc/images link is incorrect
Peter Somogyi created HBASE-21913: - Summary: src/main/asciidoc/images link is incorrect Key: HBASE-21913 URL: https://issues.apache.org/jira/browse/HBASE-21913 Project: HBase Issue Type: Bug Components: website Affects Versions: 2.0.4 Reporter: Peter Somogyi Assignee: Peter Somogyi Fix For: 2.0.5 The link for src/main/asciidoc/images points to ../../site/resources/images/ but the site is one folder up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] asfgit commented on issue #16: Dummy change in pom.xml for PR verification
asfgit commented on issue #16: Dummy change in pom.xml for PR verification URL: https://github.com/apache/hbase-connectors/pull/16#issuecomment-464089482 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-CONNECTORS-Build/14/ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] petersomogyi commented on issue #16: Dummy change in pom.xml for PR verification
petersomogyi commented on issue #16: Dummy change in pom.xml for PR verification URL: https://github.com/apache/hbase-connectors/pull/16#issuecomment-464085642 retest build This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-21875) Change the retry logic in RSProcedureDispatcher to 'retry by default, only if xxx'
[
https://issues.apache.org/jira/browse/HBASE-21875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769395#comment-16769395
]
Hudson commented on HBASE-21875:
Results for branch branch-2.2
[build #44 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/44/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/44//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/44//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/44//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Change the retry logic in RSProcedureDispatcher to 'retry by default, only if
> xxx'
> --
>
> Key: HBASE-21875
> URL: https://issues.apache.org/jira/browse/HBASE-21875
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21875-branch-2.1.patch, HBASE-21875-v1.patch,
> HBASE-21875-v2.patch, HBASE-21875-v3.patch, HBASE-21875-v4.patch,
> HBASE-21875.patch
>
>
> For now it is not retry by default, only if xxx.
> In executeProcedures, we will only throw a fixed set of exception, so we
> should change to retry by default, and check for the exceptions which we do
> not need to retry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
