[jira] [Commented] (HBASE-22208) Create auth manager and expose it in RS
[ https://issues.apache.org/jira/browse/HBASE-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815065#comment-16815065 ] Yi Mei commented on HBASE-22208: The review link: https://reviews.apache.org/r/70452/ > Create auth manager and expose it in RS > --- > > Key: HBASE-22208 > URL: https://issues.apache.org/jira/browse/HBASE-22208 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-22208.master.001.patch > > > In HBase access control service, auth manager cache all global, namespace and > table permissions, and performs authorization checks for a given user's > assigned permissions. > The auth manager instance is created when master, RS and region load > AccessController. Its cache is refreshed when acl znode changed. > We can create auth manager when master and RS start and expose it in order to > use procedure to refresh its cache rather than watch ZK. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-22208) Create auth manager and expose it in RS
[ https://issues.apache.org/jira/browse/HBASE-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei reassigned HBASE-22208: -- Assignee: Yi Mei > Create auth manager and expose it in RS > --- > > Key: HBASE-22208 > URL: https://issues.apache.org/jira/browse/HBASE-22208 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-22208.master.001.patch > > > In HBase access control service, auth manager cache all global, namespace and > table permissions, and performs authorization checks for a given user's > assigned permissions. > The auth manager instance is created when master, RS and region load > AccessController. Its cache is refreshed when acl znode changed. > We can create auth manager when master and RS start and expose it in order to > use procedure to refresh its cache rather than watch ZK. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-22208) Create auth manager and expose it in RS
[ https://issues.apache.org/jira/browse/HBASE-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-22208 started by Yi Mei. -- > Create auth manager and expose it in RS > --- > > Key: HBASE-22208 > URL: https://issues.apache.org/jira/browse/HBASE-22208 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-22208.master.001.patch > > > In HBase access control service, auth manager cache all global, namespace and > table permissions, and performs authorization checks for a given user's > assigned permissions. > The auth manager instance is created when master, RS and region load > AccessController. Its cache is refreshed when acl znode changed. > We can create auth manager when master and RS start and expose it in order to > use procedure to refresh its cache rather than watch ZK. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22208) Create auth manager and expose it in RS
[ https://issues.apache.org/jira/browse/HBASE-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-22208: --- Status: Patch Available (was: In Progress) > Create auth manager and expose it in RS > --- > > Key: HBASE-22208 > URL: https://issues.apache.org/jira/browse/HBASE-22208 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-22208.master.001.patch > > > In HBase access control service, auth manager cache all global, namespace and > table permissions, and performs authorization checks for a given user's > assigned permissions. > The auth manager instance is created when master, RS and region load > AccessController. Its cache is refreshed when acl znode changed. > We can create auth manager when master and RS start and expose it in order to > use procedure to refresh its cache rather than watch ZK. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22208) Create auth manager and expose it in RS
[ https://issues.apache.org/jira/browse/HBASE-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-22208: --- Attachment: HBASE-22208.master.001.patch > Create auth manager and expose it in RS > --- > > Key: HBASE-22208 > URL: https://issues.apache.org/jira/browse/HBASE-22208 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Priority: Major > Attachments: HBASE-22208.master.001.patch > > > In HBase access control service, auth manager cache all global, namespace and > table permissions, and performs authorization checks for a given user's > assigned permissions. > The auth manager instance is created when master, RS and region load > AccessController. Its cache is refreshed when acl znode changed. > We can create auth manager when master and RS start and expose it in order to > use procedure to refresh its cache rather than watch ZK. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22208) Create auth manager and expose it in RS
Yi Mei created HBASE-22208: -- Summary: Create auth manager and expose it in RS Key: HBASE-22208 URL: https://issues.apache.org/jira/browse/HBASE-22208 Project: HBase Issue Type: Sub-task Reporter: Yi Mei In HBase access control service, auth manager cache all global, namespace and table permissions, and performs authorization checks for a given user's assigned permissions. The auth manager instance is created when master, RS and region load AccessController. Its cache is refreshed when acl znode changed. We can create auth manager when master and RS start and expose it in order to use procedure to refresh its cache rather than watch ZK. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22207) Fix flakey TestAssignmentManager.testAssignSocketTimeout
Duo Zhang created HBASE-22207: - Summary: Fix flakey TestAssignmentManager.testAssignSocketTimeout Key: HBASE-22207 URL: https://issues.apache.org/jira/browse/HBASE-22207 Project: HBase Issue Type: Bug Components: test Reporter: Duo Zhang The problem is that we may kill the RS which holds meta so the assertion of the number of procedures maybe incorrect, as we may schedule another TRSP for assigning meta... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22127) Ensure that the block cached in the LRUBlockCache offheap is allocated from heap
[ https://issues.apache.org/jira/browse/HBASE-22127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815050#comment-16815050 ] Zheng Hu commented on HBASE-22127: -- bq. we don't have a version for the HBASE-21879 feature branch? is the plan to fixup jira versions once the branch merges? [~busbey], I plan to do that after all these patches are merged into master branch, we still don't know what's the version it will be in. Thanks for the reminding. > Ensure that the block cached in the LRUBlockCache offheap is allocated from > heap > > > Key: HBASE-22127 > URL: https://issues.apache.org/jira/browse/HBASE-22127 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-22127.HBASE-21879.v1.patch, > HBASE-22127.HBASE-21879.v2.patch, HBASE-22127.HBASE-21879.v3.patch, > HBASE-22127.HBASE-21879.v4.patch, HBASE-22127.HBASE-21879.v5.patch, > HBASE-22127.HBASE-21879.v6.patch, HBASE-22127.HBASE-21879.v7.patch, > HBASE-22127.HBASE-21879.v8.patch > > > In here [1], [~anoop.hbase] pointed out an crtial problem , I pasted here: > bq. So if we read from HDFS into a pooled BB and then give to LRU cache for > caching (ya mostly cache on read might be true) we will cache the block which > is backed by this pooled DBB? Unless the block is evicted , this BB wont go > back to pool. I think this is some thing we can not livw with !! For LRU > cache the sizing itself is based on what % of heap size we can grow. But here > in effect we are occupying the off heap space for the cached blocks. All the > sizing assumptions and calc going out of control ! > It's indeed an big problem here. so we can only make the block ref to an heap > area if we use LRUCache (both LruBlockCache and CombinedBlockCache case). Or > we can also make the lru cache offheap ? > I think we can introduce an switch indicate that whether the lru block cache > offheap or not, if heap, then coping those bytes from ByteBuff to heap. > https://reviews.apache.org/r/70153/diff/6?file=2133545#file2133545line398 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22020) upgrade to yetus 0.9.0
[
https://issues.apache.org/jira/browse/HBASE-22020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815040#comment-16815040
]
Hudson commented on HBASE-22020:
Results for branch HBASE-22020
[build #8 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22020/8/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22020/8//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22020/8//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22020/8//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> upgrade to yetus 0.9.0
> --
>
> Key: HBASE-22020
> URL: https://issues.apache.org/jira/browse/HBASE-22020
> Project: HBase
> Issue Type: Task
> Components: build, community
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-22020-branch-1.v1.patch, HBASE-22020.0.patch,
> HBASE-22020.1.patch
>
>
> branch-1/jdk7 checkstyle dtd xml parse complaint; "script engine for language
> js can not be found"
> See parent for some context. Checkstyle references dtds that were hosted on
> puppycrawl, then on sourceforge up until ten days ago. Nightlies are failing
> for among other reasons, complaint that there is bad xml in the build...
> notably, the unresolvable DTDs.
> I'd just update the DTDs but there is a need for a js engine some where and
> openjdk7 doesn't ship with one (openjdk8 does). That needs addressing and
> then we can backport the parent issue...
> See
> https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1/710/artifact/output-general/xml.txt
> ... which in case its rolled away, is filled with this message:
> "script engine for language js can not be found"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21048) Get LogLevel is not working from console in secure environment
[
https://issues.apache.org/jira/browse/HBASE-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815036#comment-16815036
]
Reid Chan commented on HBASE-21048:
---
+1.
nit,
fix check-style warning,
{{args[index+1]}}, please add whitespaces between symbol '+' (several places)
ping [~chandrasekharn], do you want to take a look as well?
> Get LogLevel is not working from console in secure environment
> --
>
> Key: HBASE-21048
> URL: https://issues.apache.org/jira/browse/HBASE-21048
> Project: HBase
> Issue Type: Bug
>Reporter: Chandra Sekhar
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-21048.001.patch, HBASE-21048.master.001.patch,
> HBASE-21048.master.002.patch, HBASE-21048.master.003.patch,
> HBASE-21048.master.004.patch, HBASE-21048.master.005.patch,
> HBASE-21048.master.006.patch, HBASE-21048.master.007.patch
>
>
> When we try to get log level of specific package in secure environment,
> getting SocketException.
> {code:java}
> hbase/master/bin# ./hbase org.apache.hadoop.hbase.http.log.LogLevel -getlevel
> host-:16010 org.apache.hadoop.hbase
> Connecting to http://host-:16010/logLevel?log=org.apache.hadoop.hbase
> java.net.SocketException: Unexpected end of file from server
> {code}
> It is trying to connect http instead of https
> code snippet that handling only http in *LogLevel.java*
> {code:java}
> public static void main(String[] args) {
> if (args.length == 3 && "-getlevel".equals(args[0])) {
> process("http://; + args[1] + "/logLevel?log=" + args[2]);
> return;
> }
> else if (args.length == 4 && "-setlevel".equals(args[0])) {
> process("http://; + args[1] + "/logLevel?log=" + args[2]
> + "=" + args[3]);
> return;
> }
> System.err.println(USAGES);
> System.exit(-1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21688) Address WAL filesystem issues
[ https://issues.apache.org/jira/browse/HBASE-21688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815035#comment-16815035 ] Vladimir Rodionov commented on HBASE-21688: --- I checked 1.3, 1.4, 1.5 and all of them looks OK, so there is no need for any back porting. > Address WAL filesystem issues > - > > Key: HBASE-21688 > URL: https://issues.apache.org/jira/browse/HBASE-21688 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Labels: s3 > Fix For: 3.0.0, 2.2.0, 2.0.6, 2.1.5 > > Attachments: HBASE-21688-branch-2.0-v1.patch, > HBASE-21688-branch-2.1-v2.patch, HBASE-21688-branch-2.2-v1.patch, > HBASE-21688-master-addendum.patch, HBASE-21688-v1.patch > > > Scan and fix code base to use new way of instantiating WAL File System. > https://issues.apache.org/jira/browse/HBASE-21457?focusedCommentId=16734688=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16734688 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-22205) Backport HBASE-21688 to 1.3+ branches
[ https://issues.apache.org/jira/browse/HBASE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov resolved HBASE-22205. --- Resolution: Not A Problem > Backport HBASE-21688 to 1.3+ branches > - > > Key: HBASE-22205 > URL: https://issues.apache.org/jira/browse/HBASE-22205 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > > We need to address WAL file system issues in 1.x as well (those branches, > which support separate file system for WAL - 1.3+) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22205) Backport HBASE-21688 to 1.3+ branches
[ https://issues.apache.org/jira/browse/HBASE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815026#comment-16815026 ] Vladimir Rodionov edited comment on HBASE-22205 at 4/11/19 2:56 AM: All 1.x branches are safe. I have checked the code and it seems that WAL file system separation has been done better in 1.x than in 2.x. All 1.x: 1.3, 1.4, 1.5 looks OK. was (Author: vrodionov): All 1.x branches are safe. I have checked the code and it seems that WAL file system separation has been done better in 1.x than in 2.x. > Backport HBASE-21688 to 1.3+ branches > - > > Key: HBASE-22205 > URL: https://issues.apache.org/jira/browse/HBASE-22205 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > > We need to address WAL file system issues in 1.x as well (those branches, > which support separate file system for WAL - 1.3+) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22149) HBOSS: A FileSystem implementation to provide HBase's required semantics
[ https://issues.apache.org/jira/browse/HBASE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815031#comment-16815031 ] Aaron Fabbri commented on HBASE-22149: -- Thanks for your work on this interesting patch [~mackrorysd]. In the process of reading it. A couple of initial thoughts–not really looking at overall correctness yet. * On path normalization. Do you need a fs.qualify(path) before calling into your lock manager to ensure you are always looking at an absolute path? What about multi-bucket support? You may need to preserve the authority/host once you look at supporting that. S3guard had these issues as it also used path as a lookup key for stuff. * S3Mock sounds interesting. Would be nice to be able to work on S3A some without paying for AWS usage (cost has been limiting my involvement). * For lockListing(), why is a shared lock on the path being listed not sufficient? * Deadlock detection and debuggability. You might want the concepts of waiters / owners and wait-for graphs at some point to be able to avoid deadlock. Assuming you keep going down this route, and we cannot convince our selves that applications (HBASE) will not hold and wait. Probably a bit early to go this deep though. I don't have much experience with Curator but have heard of it. > HBOSS: A FileSystem implementation to provide HBase's required semantics > > > Key: HBASE-22149 > URL: https://issues.apache.org/jira/browse/HBASE-22149 > Project: HBase > Issue Type: New Feature > Components: Filesystem Integration >Reporter: Sean Mackrory >Assignee: Sean Mackrory >Priority: Critical > Attachments: HBASE-22149-hadoop.patch, HBASE-22149-hbase-2.patch, > HBASE-22149-hbase-3.patch, HBASE-22149-hbase.patch > > > (Have been using the name HBOSS for HBase / Object Store Semantics) > I've had some thoughts about how to solve the problem of running HBase on > object stores. There has been some thought in the past about adding the > required semantics to S3Guard, but I have some concerns about that. First, > it's mixing complicated solutions to different problems (bridging the gap > between a flat namespace and a hierarchical namespace vs. solving > inconsistency). Second, it's S3-specific, whereas other objects stores could > use virtually identical solutions. And third, we can't do things like atomic > renames in a true sense. There would have to be some trade-offs specific to > HBase's needs and it's better if we can solve that in an HBase-specific > module without mixing all that logic in with the rest of S3A. > Ideas to solve this above the FileSystem layer have been proposed and > considered (HBASE-20431, for one), and maybe that's the right way forward > long-term, but it certainly seems to be a hard problem and hasn't been done > yet. But I don't know enough of all the internal considerations to make much > of a judgment on that myself. > I propose a FileSystem implementation that wraps another FileSystem instance > and provides locking of FileSystem operations to ensure correct semantics. > Locking could quite possibly be done on the same ZooKeeper ensemble as an > HBase cluster already uses (I'm sure there are some performance > considerations here that deserve more attention). I've put together a > proof-of-concept on which I've tested some aspects of atomic renames and > atomic file creates. Both of these tests fail reliably on a naked s3a > instance. I've also done a small YCSB run against a small cluster to sanity > check other functionality and was successful. I will post the patch, and my > laundry list of things that still need work. The WAL is still placed on HDFS, > but the HBase root directory is otherwise on S3. > Note that my prototype is built on Hadoop's source tree right now. That's > purely for my convenience in putting it together quickly, as that's where I > mostly work. I actually think long-term, if this is accepted as a good > solution, it makes sense to live in HBase (or it's own repository). It only > depends on stable, public APIs in Hadoop and is targeted entirely at HBase's > needs, so it should be able to iterate on the HBase community's terms alone. > Another idea [~ste...@apache.org] proposed to me is that of an inode-based > FileSystem that keeps hierarchical metadata in a more appropriate store that > would allow the required transactions (maybe a special table in HBase could > provide that store itself for other tables), and stores the underlying files > with unique identifiers on S3. This allows renames to actually become fast > instead of just large atomic operations. It does however place a strong > dependency on the metadata store. I have not explored this idea much. My > current
[jira] [Commented] (HBASE-22198) Fix flakey TestAsyncTableGetMultiThreaded
[
https://issues.apache.org/jira/browse/HBASE-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815029#comment-16815029
]
Hudson commented on HBASE-22198:
Results for branch master
[build #923 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/923/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/923//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/923//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/923//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix flakey TestAsyncTableGetMultiThreaded
> -
>
> Key: HBASE-22198
> URL: https://issues.apache.org/jira/browse/HBASE-22198
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0, 2.1.5
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/2959/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncTableGetMultiThreaded/test/
> The error is thrown from an admin method, where we do not have any retries if
> the region is not online yet. Should be a test issue, let me fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22205) Backport HBASE-21688 to 1.3+ branches
[ https://issues.apache.org/jira/browse/HBASE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815026#comment-16815026 ] Vladimir Rodionov commented on HBASE-22205: --- All 1.x branches are safe. I have checked the code and it seems that WAL file system separation has been done better in 1.x than in 2.x. > Backport HBASE-21688 to 1.3+ branches > - > > Key: HBASE-22205 > URL: https://issues.apache.org/jira/browse/HBASE-22205 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > > We need to address WAL file system issues in 1.x as well (those branches, > which support separate file system for WAL - 1.3+) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-14850) C++ client implementation
[ https://issues.apache.org/jira/browse/HBASE-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815011#comment-16815011 ] Guanghao Zhang commented on HBASE-14850: And the new hbase-native-client repo need a new precommit job for github pr. > C++ client implementation > - > > Key: HBASE-14850 > URL: https://issues.apache.org/jira/browse/HBASE-14850 > Project: HBase > Issue Type: Task >Reporter: Elliott Clark >Priority: Major > > It's happening. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22084) Rename AccessControlLists to PermissionStorage
[ https://issues.apache.org/jira/browse/HBASE-22084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-22084: --- Attachment: HBASE-22084.branch-2.2.001.patch > Rename AccessControlLists to PermissionStorage > -- > > Key: HBASE-22084 > URL: https://issues.apache.org/jira/browse/HBASE-22084 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-22084.branch-2.2.001.patch, > HBASE-22084.master.001.patch, HBASE-22084.master.002.patch > > > AccessControlLists is a utility class which deal with get/put/delete > operations with hbase acl table. The name of the class is confusing, so shall > we rename it to PermissionStorage? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-14850) C++ client implementation
[ https://issues.apache.org/jira/browse/HBASE-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815003#comment-16815003 ] Guanghao Zhang commented on HBASE-14850: The last nightly build is [https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/HBASE-14850/5/]. But it is not very useful for hbase-native-client, as it didn't test the stuffs about hbase-native-client. We should introduce a new precommit job for hbase-native-client repo and add some tests for it. > C++ client implementation > - > > Key: HBASE-14850 > URL: https://issues.apache.org/jira/browse/HBASE-14850 > Project: HBase > Issue Type: Task >Reporter: Elliott Clark >Priority: Major > > It's happening. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22106) Log cause of the failure when coprocessor specification parsing fails.
[
https://issues.apache.org/jira/browse/HBASE-22106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815002#comment-16815002
]
Hudson commented on HBASE-22106:
Results for branch branch-1
[build #765 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/765/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/765//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/765//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/765//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Log cause of the failure when coprocessor specification parsing fails.
> --
>
> Key: HBASE-22106
> URL: https://issues.apache.org/jira/browse/HBASE-22106
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 1.4.9
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Minor
> Fix For: 1.4.10, 1.3.4, 1.5.1
>
> Attachments: HBASE-22106.branch-1.4.001.patch
>
>
> {code}
> 2019-02-15 22:56:33,008 ERROR [RS_OPEN_REGION-n79:16020-16]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$2, spec:
> |org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-14]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-19]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> {code}
> I checked the same coprocessor specification(logged in exception) with the
> code and it is parsed successfully. And even after the restart , regionserver
> also didn't complain about it.
> I think we should be logging the cause of the exception while parsing table
> descriptor for the coprocessor for better debugging.
> https://github.com/apache/hbase/blob/branch-1.4/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L318
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22086) space quota issue: deleting snapshot doesn't update the usage of table
[
https://issues.apache.org/jira/browse/HBASE-22086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815001#comment-16815001
]
HBase QA commented on HBASE-22086:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
26s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
47s{color} | {color:blue} hbase-server in master has 11 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
16s{color} | {color:red} hbase-server: The patch generated 19 new + 1 unchanged
- 0 fixed = 20 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
13m 26s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
3s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}146m 33s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}206m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/59/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22086 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12965522/hbase-22086.master.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 5e0e64e41e25 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ffede2edfa |
| maven | version: Apache Maven 3.5.4
[jira] [Updated] (HBASE-22196) Split TestRestartCluster
[
https://issues.apache.org/jira/browse/HBASE-22196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-22196:
--
Fix Version/s: 2.3.0
2.2.0
3.0.0
> Split TestRestartCluster
>
>
> Key: HBASE-22196
> URL: https://issues.apache.org/jira/browse/HBASE-22196
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
>
> The logs for later tests are messed up with error messages, like
> {noformat}
> 2019-04-09 09:41:11,717 WARN [LeaseRenewer:jenkins.hfs.12@localhost:41108]
> hdfs.LeaseRenewer(468): Failed to renew lease for
> [DFSClient_NONMAPREDUCE_400481390_21] for 55 seconds. Will retry shortly ...
> java.net.ConnectException: Call From asf918.gq1.ygridcore.net/67.195.81.138
> to localhost:41108 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.GeneratedConstructorAccessor79.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
> at org.apache.hadoop.ipc.Client.call(Client.java:1480)
> at org.apache.hadoop.ipc.Client.call(Client.java:1413)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
> at com.sun.proxy.$Proxy30.renewLease(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.renewLease(ClientNamenodeProtocolTranslatorPB.java:595)
> at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy33.renewLease(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
> at com.sun.proxy.$Proxy34.renewLease(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
> at com.sun.proxy.$Proxy34.renewLease(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:901)
> at org.apache.hadoop.hdfs.LeaseRenewer.renew(LeaseRenewer.java:423)
> at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:448)
> at org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71)
> at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:304)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
> at
> org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:615)
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:713)
> at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:376)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1529)
> at org.apache.hadoop.ipc.Client.call(Client.java:1452)
> ... 26 more
> 2019-04-09 09:41:11,949 WARN [RS_OPEN_REGION-regionserver/asf918:33671-1]
> regionserver.HStore(1062): Failed flushing store file, retrying num=8
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:817)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2114)
> at
>
[jira] [Assigned] (HBASE-22196) Split TestRestartCluster
[
https://issues.apache.org/jira/browse/HBASE-22196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang reassigned HBASE-22196:
-
Assignee: Duo Zhang
> Split TestRestartCluster
>
>
> Key: HBASE-22196
> URL: https://issues.apache.org/jira/browse/HBASE-22196
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> The logs for later tests are messed up with error messages, like
> {noformat}
> 2019-04-09 09:41:11,717 WARN [LeaseRenewer:jenkins.hfs.12@localhost:41108]
> hdfs.LeaseRenewer(468): Failed to renew lease for
> [DFSClient_NONMAPREDUCE_400481390_21] for 55 seconds. Will retry shortly ...
> java.net.ConnectException: Call From asf918.gq1.ygridcore.net/67.195.81.138
> to localhost:41108 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.GeneratedConstructorAccessor79.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
> at org.apache.hadoop.ipc.Client.call(Client.java:1480)
> at org.apache.hadoop.ipc.Client.call(Client.java:1413)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
> at com.sun.proxy.$Proxy30.renewLease(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.renewLease(ClientNamenodeProtocolTranslatorPB.java:595)
> at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy33.renewLease(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
> at com.sun.proxy.$Proxy34.renewLease(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
> at com.sun.proxy.$Proxy34.renewLease(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:901)
> at org.apache.hadoop.hdfs.LeaseRenewer.renew(LeaseRenewer.java:423)
> at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:448)
> at org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71)
> at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:304)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
> at
> org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:615)
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:713)
> at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:376)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1529)
> at org.apache.hadoop.ipc.Client.call(Client.java:1452)
> ... 26 more
> 2019-04-09 09:41:11,949 WARN [RS_OPEN_REGION-regionserver/asf918:33671-1]
> regionserver.HStore(1062): Failed flushing store file, retrying num=8
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:817)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2114)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)
> at
>
[GitHub] [hbase] Apache9 commented on issue #137: HBASE-22203 Reformatted DemoClient.java
Apache9 commented on issue #137: HBASE-22203 Reformatted DemoClient.java URL: https://github.com/apache/hbase/pull/137#issuecomment-481930840 Hey @HorizonNet , Please resolve some of the PRs before opening new one? I see lots of your PRs have already been approved so please merge them and cherry-pick to relevant branches? Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 merged pull request #134: HBASE-22196 Split TestRestartCluster
Apache9 merged pull request #134: HBASE-22196 Split TestRestartCluster URL: https://github.com/apache/hbase/pull/134 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (HBASE-22144) MultiRowRangeFilter does not work with reversed scans
[
https://issues.apache.org/jira/browse/HBASE-22144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814998#comment-16814998
]
Toshihiro Suzuki edited comment on HBASE-22144 at 4/11/19 1:29 AM:
---
{quote}
but I don't want to convolute this bug fix with changing how the implementation
works, especially when this is public API (and we have to be certain that we
aren't breaking user-code).
{quote}
Okay I see. Then +1.
Will create another Jira ticket for what I mentioned in my last comment maybe
for 3.0 as it's an incompatible change.
was (Author: brfrn169):
{quote}
but I don't want to convolute this bug fix with changing how the implementation
works, especially when this is public API (and we have to be certain that we
aren't breaking user-code).
{quote}
Okay I see. Then +1.
Will create another Jira ticket for what I mentioned in my last comment maybe
for 3.0 as it's incompatible change.
> MultiRowRangeFilter does not work with reversed scans
> -
>
> Key: HBASE-22144
> URL: https://issues.apache.org/jira/browse/HBASE-22144
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Attachments: HBASE-22144.001.patch, HBASE-22144.002.patch,
> HBASE-22144.002.patch
>
>
> It appears that MultiRowRangeFilter was never written to function with
> reverse scans. There is too much logic that operates with the assumption that
> we are always moving "forward" through increasing ranges. It needs to be
> rewritten to "traverse" forward or backward, given how the context of the
> scan being used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22144) MultiRowRangeFilter does not work with reversed scans
[
https://issues.apache.org/jira/browse/HBASE-22144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814998#comment-16814998
]
Toshihiro Suzuki edited comment on HBASE-22144 at 4/11/19 1:28 AM:
---
{quote}
but I don't want to convolute this bug fix with changing how the implementation
works, especially when this is public API (and we have to be certain that we
aren't breaking user-code).
{quote}
Okay I see. Then +1.
Will create another Jira ticket for what I mentioned in my last comment maybe
for 3.0 as it's incompatible change.
was (Author: brfrn169):
{quote}
but I don't want to convolute this bug fix with changing how the implementation
works, especially when this is public API (and we have to be certain that we
aren't breaking user-code).
{quote}
Okay I see. Then +1.
Will create another Jira ticket for what I mentioned in my last comment.
> MultiRowRangeFilter does not work with reversed scans
> -
>
> Key: HBASE-22144
> URL: https://issues.apache.org/jira/browse/HBASE-22144
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Attachments: HBASE-22144.001.patch, HBASE-22144.002.patch,
> HBASE-22144.002.patch
>
>
> It appears that MultiRowRangeFilter was never written to function with
> reverse scans. There is too much logic that operates with the assumption that
> we are always moving "forward" through increasing ranges. It needs to be
> rewritten to "traverse" forward or backward, given how the context of the
> scan being used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22144) MultiRowRangeFilter does not work with reversed scans
[
https://issues.apache.org/jira/browse/HBASE-22144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814998#comment-16814998
]
Toshihiro Suzuki commented on HBASE-22144:
--
{quote}
but I don't want to convolute this bug fix with changing how the implementation
works, especially when this is public API (and we have to be certain that we
aren't breaking user-code).
{quote}
Okay I see. Then +1.
Will create another Jira ticket for what I mentioned in my last comment.
> MultiRowRangeFilter does not work with reversed scans
> -
>
> Key: HBASE-22144
> URL: https://issues.apache.org/jira/browse/HBASE-22144
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Attachments: HBASE-22144.001.patch, HBASE-22144.002.patch,
> HBASE-22144.002.patch
>
>
> It appears that MultiRowRangeFilter was never written to function with
> reverse scans. There is too much logic that operates with the assumption that
> we are always moving "forward" through increasing ranges. It needs to be
> rewritten to "traverse" forward or backward, given how the context of the
> scan being used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22084) Rename AccessControlLists to PermissionStorage
[ https://issues.apache.org/jira/browse/HBASE-22084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-22084: --- Attachment: HBASE-22084.master.002.patch > Rename AccessControlLists to PermissionStorage > -- > > Key: HBASE-22084 > URL: https://issues.apache.org/jira/browse/HBASE-22084 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-22084.master.001.patch, > HBASE-22084.master.002.patch > > > AccessControlLists is a utility class which deal with get/put/delete > operations with hbase acl table. The name of the class is confusing, so shall > we rename it to PermissionStorage? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22084) Rename AccessControlLists to PermissionStorage
[ https://issues.apache.org/jira/browse/HBASE-22084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-22084: --- Attachment: (was: HBASE-22084.master.002.patch) > Rename AccessControlLists to PermissionStorage > -- > > Key: HBASE-22084 > URL: https://issues.apache.org/jira/browse/HBASE-22084 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-22084.master.001.patch, > HBASE-22084.master.002.patch > > > AccessControlLists is a utility class which deal with get/put/delete > operations with hbase acl table. The name of the class is confusing, so shall > we rename it to PermissionStorage? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22057) Impose upper-bound on size of ZK ops sent in a single multi()
[
https://issues.apache.org/jira/browse/HBASE-22057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-22057:
---
Attachment: HBASE-22057-branch-1.patch
> Impose upper-bound on size of ZK ops sent in a single multi()
> -
>
> Key: HBASE-22057
> URL: https://issues.apache.org/jira/browse/HBASE-22057
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-22057-branch-1.patch, HBASE-22057.001.patch,
> HBASE-22057.002.patch, HBASE-22057.003.patch, HBASE-22057.004.patch
>
>
> In {{ZKUtil#multiOrSequential}}, we accept a list of {{ZKUtilOp}}'s to pass
> down to the {{ZooKeeper#multi(Iterable)}} method.
> One problem with this approach is that we may generate a large list of ZNodes
> to mutate in one batch which exceeds the allowable client package length,
> specified by {{jute.maxbuffer}}.
> This problem can manifest when we have a large number of WALs to replicate,
> queued in ZooKeeper, from a disabled peer. When that peer is dropped, the RS
> would submit deletes of those queued WALs. The RS will see ConnectionLoss for
> the resulting {{multi()}} calls it tries to make, because we are sending too
> large of a client message (because we're trying to delete too many WALs at
> once). The result (at least in branch-1 ish versions) is that the RS aborts
> after exceeding the ZK retries (as this operation will never succeed).
> A simple fix would be to impose a maximum number of Ops to run in a single
> batch inside ZKUtil, and split apart the caller-submitted batch into smaller
> chunks. Before we make such a change, I do need to make sure that we don't
> have any expectations on atomicity of the operations. I'm not sure what ZK
> provides here -- for the above example, splitting up batches of deletes is
> not an issue, but there could be issues with batches of creates where we only
> apply some.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22057) Impose upper-bound on size of ZK ops sent in a single multi()
[
https://issues.apache.org/jira/browse/HBASE-22057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-22057:
---
Fix Version/s: 1.6.0
> Impose upper-bound on size of ZK ops sent in a single multi()
> -
>
> Key: HBASE-22057
> URL: https://issues.apache.org/jira/browse/HBASE-22057
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 1.6.0, 2.2.0
>
> Attachments: HBASE-22057-branch-1.patch, HBASE-22057.001.patch,
> HBASE-22057.002.patch, HBASE-22057.003.patch, HBASE-22057.004.patch
>
>
> In {{ZKUtil#multiOrSequential}}, we accept a list of {{ZKUtilOp}}'s to pass
> down to the {{ZooKeeper#multi(Iterable)}} method.
> One problem with this approach is that we may generate a large list of ZNodes
> to mutate in one batch which exceeds the allowable client package length,
> specified by {{jute.maxbuffer}}.
> This problem can manifest when we have a large number of WALs to replicate,
> queued in ZooKeeper, from a disabled peer. When that peer is dropped, the RS
> would submit deletes of those queued WALs. The RS will see ConnectionLoss for
> the resulting {{multi()}} calls it tries to make, because we are sending too
> large of a client message (because we're trying to delete too many WALs at
> once). The result (at least in branch-1 ish versions) is that the RS aborts
> after exceeding the ZK retries (as this operation will never succeed).
> A simple fix would be to impose a maximum number of Ops to run in a single
> batch inside ZKUtil, and split apart the caller-submitted batch into smaller
> chunks. Before we make such a change, I do need to make sure that we don't
> have any expectations on atomicity of the operations. I'm not sure what ZK
> provides here -- for the above example, splitting up batches of deletes is
> not an issue, but there could be issues with batches of creates where we only
> apply some.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-14850) C++ client implementation
[ https://issues.apache.org/jira/browse/HBASE-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-14850: --- Release Note: The hbase-native-client module has been moved to new repo https://github.com/apache/hbase-native-client. The old feature branch HBASE-14850 was removed from hbase repo. (was: The hbase-native-client module has been moved to new repo https://github.com/apache/hbase-native-client.) > C++ client implementation > - > > Key: HBASE-14850 > URL: https://issues.apache.org/jira/browse/HBASE-14850 > Project: HBase > Issue Type: Task >Reporter: Elliott Clark >Priority: Major > > It's happening. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-14850) C++ client implementation
[ https://issues.apache.org/jira/browse/HBASE-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-14850: --- Release Note: The hbase-native-client module has been moved to new repo https://github.com/apache/hbase-native-client. (was: All hbase-native-client stuff was moved to new repo https://github.com/apache/hbase-native-client.) > C++ client implementation > - > > Key: HBASE-14850 > URL: https://issues.apache.org/jira/browse/HBASE-14850 > Project: HBase > Issue Type: Task >Reporter: Elliott Clark >Priority: Major > > It's happening. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-14850) C++ client implementation
[ https://issues.apache.org/jira/browse/HBASE-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-14850: --- Release Note: All hbase-native-client stuff was moved to new repo https://github.com/apache/hbase-native-client. > C++ client implementation > - > > Key: HBASE-14850 > URL: https://issues.apache.org/jira/browse/HBASE-14850 > Project: HBase > Issue Type: Task >Reporter: Elliott Clark >Priority: Major > > It's happening. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-14850) C++ client implementation
[ https://issues.apache.org/jira/browse/HBASE-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814960#comment-16814960 ] Guanghao Zhang commented on HBASE-14850: Open a new repo [https://github.com/apache/hbase-native-client] for hbase c++ client. > C++ client implementation > - > > Key: HBASE-14850 > URL: https://issues.apache.org/jira/browse/HBASE-14850 > Project: HBase > Issue Type: Task >Reporter: Elliott Clark >Priority: Major > > It's happening. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22106) Log cause of the failure when coprocessor specification parsing fails.
[
https://issues.apache.org/jira/browse/HBASE-22106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814958#comment-16814958
]
Hudson commented on HBASE-22106:
Results for branch branch-1.4
[build #741 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/741/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/741//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/741//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/741//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Log cause of the failure when coprocessor specification parsing fails.
> --
>
> Key: HBASE-22106
> URL: https://issues.apache.org/jira/browse/HBASE-22106
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 1.4.9
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Minor
> Fix For: 1.4.10, 1.3.4, 1.5.1
>
> Attachments: HBASE-22106.branch-1.4.001.patch
>
>
> {code}
> 2019-02-15 22:56:33,008 ERROR [RS_OPEN_REGION-n79:16020-16]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$2, spec:
> |org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-14]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-19]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> {code}
> I checked the same coprocessor specification(logged in exception) with the
> code and it is parsed successfully. And even after the restart , regionserver
> also didn't complain about it.
> I think we should be logging the cause of the exception while parsing table
> descriptor for the coprocessor for better debugging.
> https://github.com/apache/hbase/blob/branch-1.4/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L318
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[
https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814947#comment-16814947
]
HBase QA commented on HBASE-16488:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m
7s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 11 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
32s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} branch-1 passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} branch-1 passed with JDK v1.7.0_211 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
46s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
49s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} branch-1 passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} branch-1 passed with JDK v1.7.0_211 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed with JDK v1.7.0_211 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
28s{color} | {color:red} hbase-server: The patch generated 4 new + 762
unchanged - 3 fixed = 766 total (was 765) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 5 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
48s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.7.0_211 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}116m 1s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}155m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase |
| | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/57/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-16488 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12965515/HBASE-16488.branch-1.012.patch
|
| Optional
[jira] [Commented] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814933#comment-16814933 ] Xu Cang commented on HBASE-16488: - Yes, will do. [~apurtell] I am also waiting Hadoop-qa picks up patch 012 to run. > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 1.3.0, 1.0.3, 1.4.0, 1.1.5, 1.2.2, 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-16488.branch-1.012.patch, > HBASE-16488.branch-1.012.patch, HBASE-16488.revisit.v11-branch-1.patch, > HBASE-16488.v1-branch-1.patch, HBASE-16488.v1-master.patch, > HBASE-16488.v10-branch-1.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v2-branch-1.patch, HBASE-16488.v3-branch-1.patch, > HBASE-16488.v3-branch-1.patch, HBASE-16488.v4-branch-1.patch, > HBASE-16488.v5-branch-1.patch, HBASE-16488.v6-branch-1.patch, > HBASE-16488.v7-branch-1.patch, HBASE-16488.v8-branch-1.patch, > HBASE-16488.v9-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814932#comment-16814932 ] Andrew Purtell commented on HBASE-16488: Tests looked good running locally. [~xucang] if you have some time in the interest, it would be good if we could have a unit test for this change where: * Case 1: hbase.master.start.wait.for.namespacemanager=true . Set the namespace init timeout low. Inject a stall to keep the master from starting up in time. Expect failure (master shutdown) * Case 2: hbase.master.start.wait.for.namespacemanager=true . Same timeout and injected delay but master should successfully initialize. > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 1.3.0, 1.0.3, 1.4.0, 1.1.5, 1.2.2, 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-16488.branch-1.012.patch, > HBASE-16488.branch-1.012.patch, HBASE-16488.revisit.v11-branch-1.patch, > HBASE-16488.v1-branch-1.patch, HBASE-16488.v1-master.patch, > HBASE-16488.v10-branch-1.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v2-branch-1.patch, HBASE-16488.v3-branch-1.patch, > HBASE-16488.v3-branch-1.patch, HBASE-16488.v4-branch-1.patch, > HBASE-16488.v5-branch-1.patch, HBASE-16488.v6-branch-1.patch, > HBASE-16488.v7-branch-1.patch, HBASE-16488.v8-branch-1.patch, > HBASE-16488.v9-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814932#comment-16814932 ] Andrew Purtell edited comment on HBASE-16488 at 4/10/19 11:18 PM: -- Tests looked good running locally. [~xucang] if you have some time and the interest, it would be good if we could have a unit test for this change where: * Case 1: hbase.master.start.wait.for.namespacemanager=true . Set the namespace init timeout low. Inject a stall to keep the master from starting up in time. Expect failure (master shutdown) * Case 2: hbase.master.start.wait.for.namespacemanager=false. Same timeout and injected delay but master should successfully initialize. was (Author: apurtell): Tests looked good running locally. [~xucang] if you have some time and the interest, it would be good if we could have a unit test for this change where: * Case 1: hbase.master.start.wait.for.namespacemanager=true . Set the namespace init timeout low. Inject a stall to keep the master from starting up in time. Expect failure (master shutdown) * Case 2: hbase.master.start.wait.for.namespacemanager=true . Same timeout and injected delay but master should successfully initialize. > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 1.3.0, 1.0.3, 1.4.0, 1.1.5, 1.2.2, 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-16488.branch-1.012.patch, > HBASE-16488.branch-1.012.patch, HBASE-16488.revisit.v11-branch-1.patch, > HBASE-16488.v1-branch-1.patch, HBASE-16488.v1-master.patch, > HBASE-16488.v10-branch-1.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v2-branch-1.patch, HBASE-16488.v3-branch-1.patch, > HBASE-16488.v3-branch-1.patch, HBASE-16488.v4-branch-1.patch, > HBASE-16488.v5-branch-1.patch, HBASE-16488.v6-branch-1.patch, > HBASE-16488.v7-branch-1.patch, HBASE-16488.v8-branch-1.patch, > HBASE-16488.v9-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814932#comment-16814932 ] Andrew Purtell edited comment on HBASE-16488 at 4/10/19 11:18 PM: -- Tests looked good running locally. [~xucang] if you have some time and the interest, it would be good if we could have a unit test for this change where: * Case 1: hbase.master.start.wait.for.namespacemanager=true . Set the namespace init timeout low. Inject a stall to keep the master from starting up in time. Expect failure (master shutdown) * Case 2: hbase.master.start.wait.for.namespacemanager=true . Same timeout and injected delay but master should successfully initialize. was (Author: apurtell): Tests looked good running locally. [~xucang] if you have some time in the interest, it would be good if we could have a unit test for this change where: * Case 1: hbase.master.start.wait.for.namespacemanager=true . Set the namespace init timeout low. Inject a stall to keep the master from starting up in time. Expect failure (master shutdown) * Case 2: hbase.master.start.wait.for.namespacemanager=true . Same timeout and injected delay but master should successfully initialize. > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 1.3.0, 1.0.3, 1.4.0, 1.1.5, 1.2.2, 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-16488.branch-1.012.patch, > HBASE-16488.branch-1.012.patch, HBASE-16488.revisit.v11-branch-1.patch, > HBASE-16488.v1-branch-1.patch, HBASE-16488.v1-master.patch, > HBASE-16488.v10-branch-1.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v2-branch-1.patch, HBASE-16488.v3-branch-1.patch, > HBASE-16488.v3-branch-1.patch, HBASE-16488.v4-branch-1.patch, > HBASE-16488.v5-branch-1.patch, HBASE-16488.v6-branch-1.patch, > HBASE-16488.v7-branch-1.patch, HBASE-16488.v8-branch-1.patch, > HBASE-16488.v9-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22106) Log cause of the failure when coprocessor specification parsing fails.
[
https://issues.apache.org/jira/browse/HBASE-22106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814924#comment-16814924
]
Hudson commented on HBASE-22106:
Results for branch branch-1.3
[build #720 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/720/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/720//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/720//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/720//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Log cause of the failure when coprocessor specification parsing fails.
> --
>
> Key: HBASE-22106
> URL: https://issues.apache.org/jira/browse/HBASE-22106
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 1.4.9
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Minor
> Fix For: 1.4.10, 1.3.4, 1.5.1
>
> Attachments: HBASE-22106.branch-1.4.001.patch
>
>
> {code}
> 2019-02-15 22:56:33,008 ERROR [RS_OPEN_REGION-n79:16020-16]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$2, spec:
> |org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-14]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-19]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> {code}
> I checked the same coprocessor specification(logged in exception) with the
> code and it is parsed successfully. And even after the restart , regionserver
> also didn't complain about it.
> I think we should be logging the cause of the exception while parsing table
> descriptor for the coprocessor for better debugging.
> https://github.com/apache/hbase/blob/branch-1.4/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L318
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22206) dist.apache.org must not be used for public downloads
[
https://issues.apache.org/jira/browse/HBASE-22206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814919#comment-16814919
]
HBase QA commented on HBASE-22206:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 8m
18s{color} | {color:red} root in master failed. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 8m
11s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/60/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22206 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12965524/HBASE-22206.master.001.patch
|
| Optional Tests | dupname asflicense mvnsite xml |
| uname | Linux 7e156a3a5429 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb
13 15:00:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ffede2edfa |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| mvnsite |
https://builds.apache.org/job/PreCommit-HBASE-Build/60/artifact/patchprocess/branch-mvnsite-root.txt
|
| mvnsite |
https://builds.apache.org/job/PreCommit-HBASE-Build/60/artifact/patchprocess/patch-mvnsite-root.txt
|
| Max. process+thread count | 85 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/60/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |
This message was automatically generated.
> dist.apache.org must not be used for public downloads
> -
>
> Key: HBASE-22206
> URL: https://issues.apache.org/jira/browse/HBASE-22206
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sebb
>Assignee: Dima Spivak
>Priority: Major
> Attachments: HBASE-22206.master.001.patch
>
>
> The dist.apache.org server is only intended for use by developers in staging
> releases.
> It must not be used on public download pages.
> Please use www.apache.org/dist (for KEYS, hashes and sigs) and the mirror
> system instead.
> The current download page has lots of references to dist.a.o; please replace
> thes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22172) Suppress Java 11 reflective access warnings
[
https://issues.apache.org/jira/browse/HBASE-22172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814914#comment-16814914
]
Josh Elser commented on HBASE-22172:
{quote}I see that error before the patch but not after. Could you please share
the steps you followed? Was it just after the launch or while some operations
on the shell?
{quote}
Just launching {{hbase shell}}
{code:java}
% ./bin/hbase shell
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in
version 9.0 and will likely be removed in a future release.
unsupported Java version "11", defaulting to 1.7
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.util.io.FilenoUtil
(file:/Users/jelser/projects/hbase-copy.git/hbase-assembly/target/hbase-3.0.0-SNAPSHOT/lib/ruby/jruby-complete-9.1.13.0.jar)
to method sun.nio.ch.SelChImpl.getFD()
WARNING: Please consider reporting this to the maintainers of
org.jruby.util.io.FilenoUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal
reflective access operations
WARNING: All illegal access operations will be denied in a future release
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/lib/hadoop-3.1.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/Users/jelser/projects/hbase-copy.git/hbase-assembly/target/hbase-3.0.0-SNAPSHOT/lib/client-facing-thirdparty/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/book.html#shell
Version 3.0.0-SNAPSHOT, rffede2edfa32e1f6364a13f9a0dd534ad5696a38, Wed Apr 10
16:16:28 EDT 2019
Took 0.0038 seconds
hbase(main):001:0>{code}
{code:java}
% tail -n 24 bin/hbase-config.sh
EOF
exit 1
fi
JAVA=$JAVA_HOME/bin/java
version=$(JAVA -version 2>&1 | awk -F '"' '/version/ {print $2}')
if [[ "$version" > "11" ]]; then
# Uncomment the following line to enable warnings of further illegal reflective
access operations
# HBASE_OPTS="$HBASE_OPTS --illegal-access=warn"
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/java.nio=ALL-UNNAMED"
HBASE_OPTS="$HBASE_OPTS --add-opens
java.security.jgss/sun.security.krb5=ALL-UNNAMED"
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/sun.nio.ch=ALL-UNNAMED"
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/sun.nio.cs=ALL-UNNAMED"
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/java.lang=ALL-UNNAMED"
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/java.lang.reflect=ALL-UNNAMED"
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/java.util.regex=ALL-UNNAMED"
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/java.util.zip=ALL-UNNAMED"
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/java.util=ALL-UNNAMED"
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/java.net=ALL-UNNAMED"
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/java.io=ALL-UNNAMED"
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/java.security=ALL-UNNAMED"
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/javax.crypto=ALL-UNNAMED"
fi{code}
{code:java}
% $(/usr/libexec/java_home -v11)/bin/java -version
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode){code}
> Suppress Java 11 reflective access warnings
> ---
>
> Key: HBASE-22172
> URL: https://issues.apache.org/jira/browse/HBASE-22172
> Project: HBase
> Issue Type: Task
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Labels: jdk11
> Attachments: hbase-22172.master.001.patch
>
>
> While running a Java 8 compiled hbase on Java 11 system, I found the
> following warnings being thrown. I think we can add the "--add-opens" flag to
> HBASE_OPTS (if the jdk version is 11) to suppress this warning.
> {code:java}
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> (file:/Users/jatsakthi/test/HBASE_TEST_AREA/hbase-3.0.0-SNAPSHOT/lib/hbase-common-3.0.0-SNAPSHOT.jar)
> to method java.nio.Bits.unaligned()
> WARNING: Please consider reporting this to the maintainers of
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22172) Suppress Java 11 reflective access warnings
[
https://issues.apache.org/jira/browse/HBASE-22172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814910#comment-16814910
]
Sakthi edited comment on HBASE-22172 at 4/10/19 10:28 PM:
--
{quote}I did see a warning when trying to launch the hbase shell.
{quote}
Was this after the patch applied [~elserj]. I see that error before the patch
but not after. Could you please share the steps you followed? Was it just after
the launch or while some operations on the shell?
{quote}Where does the prefix java.base come from? Is it maybe wrong (as I feel
like your suppression here should have caught the above warning)?
{quote}
It follows this particular syntax: _--add-opens
module/package=target-module(,target-module)\*_ . So looks like
[java.base|https://docs.oracle.com/en/java/javase/11/docs/api/java.base/module-summary.html]
contains the foundational APIs. Also I followed the *Understanding Runtime
Access Warnings* from this [migration
doc|https://docs.oracle.com/en/java/javase/11/migrate/migration-guide.pdf].
{quote}something else to fix
{quote}
Was planning to file a JIRA for that. Thanks Josh.
was (Author: jatsakthi):
{quote}I did see a warning when trying to launch the hbase shell.
{quote}
Was this after the patch applied [~elserj]. I see that error before the patch
but not after. Could you please share the steps you followed? Was it just after
the launch or while some operations on the shell?
{quote}Where does the prefix java.base come from? Is it maybe wrong (as I feel
like your suppression here should have caught the above warning)?
{quote}
It follows this particular syntax: _--add-opens
module/package=target-module(,target-module)*_ . So looks like
[java.base|https://docs.oracle.com/en/java/javase/11/docs/api/java.base/module-summary.html]
contains the foundational APIs. Also I followed the *Understanding Runtime
Access Warnings* from this [migration
doc|https://docs.oracle.com/en/java/javase/11/migrate/migration-guide.pdf].
{quote}something else to fix
{quote}
Was planning to file a JIRA for that. Thanks Josh.
> Suppress Java 11 reflective access warnings
> ---
>
> Key: HBASE-22172
> URL: https://issues.apache.org/jira/browse/HBASE-22172
> Project: HBase
> Issue Type: Task
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Labels: jdk11
> Attachments: hbase-22172.master.001.patch
>
>
> While running a Java 8 compiled hbase on Java 11 system, I found the
> following warnings being thrown. I think we can add the "--add-opens" flag to
> HBASE_OPTS (if the jdk version is 11) to suppress this warning.
> {code:java}
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> (file:/Users/jatsakthi/test/HBASE_TEST_AREA/hbase-3.0.0-SNAPSHOT/lib/hbase-common-3.0.0-SNAPSHOT.jar)
> to method java.nio.Bits.unaligned()
> WARNING: Please consider reporting this to the maintainers of
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22172) Suppress Java 11 reflective access warnings
[
https://issues.apache.org/jira/browse/HBASE-22172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814910#comment-16814910
]
Sakthi commented on HBASE-22172:
{quote}I did see a warning when trying to launch the hbase shell.
{quote}
Was this after the patch applied [~elserj]. I see that error before the patch
but not after. Could you please share the steps you followed? Was it just after
the launch or while some operations on the shell?
{quote}Where does the prefix java.base come from? Is it maybe wrong (as I feel
like your suppression here should have caught the above warning)?
{quote}
It follows this particular syntax: _--add-opens
module/package=target-module(,target-module)*_ . So looks like
[java.base|https://docs.oracle.com/en/java/javase/11/docs/api/java.base/module-summary.html]
contains the foundational APIs. Also I followed the *Understanding Runtime
Access Warnings* from this [migration
doc|https://docs.oracle.com/en/java/javase/11/migrate/migration-guide.pdf].
{quote}something else to fix
{quote}
Was planning to file a JIRA for that. Thanks Josh.
> Suppress Java 11 reflective access warnings
> ---
>
> Key: HBASE-22172
> URL: https://issues.apache.org/jira/browse/HBASE-22172
> Project: HBase
> Issue Type: Task
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Labels: jdk11
> Attachments: hbase-22172.master.001.patch
>
>
> While running a Java 8 compiled hbase on Java 11 system, I found the
> following warnings being thrown. I think we can add the "--add-opens" flag to
> HBASE_OPTS (if the jdk version is 11) to suppress this warning.
> {code:java}
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> (file:/Users/jatsakthi/test/HBASE_TEST_AREA/hbase-3.0.0-SNAPSHOT/lib/hbase-common-3.0.0-SNAPSHOT.jar)
> to method java.nio.Bits.unaligned()
> WARNING: Please consider reporting this to the maintainers of
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21048) Get LogLevel is not working from console in secure environment
[
https://issues.apache.org/jira/browse/HBASE-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814905#comment-16814905
]
HBase QA commented on HBASE-21048:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} hbase-http: The patch generated 8 new + 4 unchanged -
4 fixed = 12 total (was 8) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
40s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 57s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
56s{color} | {color:green} hbase-http in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/58/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-21048 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12965518/HBASE-21048.master.007.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile findbugs hbaseanti checkstyle |
| uname | Linux 2da197c54467 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb
13 15:00:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ffede2edfa |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/58/artifact/patchprocess/diff-checkstyle-hbase-http.txt
|
| Test Results |
[jira] [Updated] (HBASE-22206) dist.apache.org must not be used for public downloads
[ https://issues.apache.org/jira/browse/HBASE-22206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-22206: Status: Patch Available (was: Open) > dist.apache.org must not be used for public downloads > - > > Key: HBASE-22206 > URL: https://issues.apache.org/jira/browse/HBASE-22206 > Project: HBase > Issue Type: Bug > Components: website >Reporter: Sebb >Assignee: Dima Spivak >Priority: Major > Attachments: HBASE-22206.master.001.patch > > > The dist.apache.org server is only intended for use by developers in staging > releases. > It must not be used on public download pages. > Please use www.apache.org/dist (for KEYS, hashes and sigs) and the mirror > system instead. > The current download page has lots of references to dist.a.o; please replace > thes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22206) dist.apache.org must not be used for public downloads
[ https://issues.apache.org/jira/browse/HBASE-22206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-22206: Attachment: HBASE-22206.master.001.patch > dist.apache.org must not be used for public downloads > - > > Key: HBASE-22206 > URL: https://issues.apache.org/jira/browse/HBASE-22206 > Project: HBase > Issue Type: Bug > Components: website >Reporter: Sebb >Assignee: Dima Spivak >Priority: Major > Attachments: HBASE-22206.master.001.patch > > > The dist.apache.org server is only intended for use by developers in staging > releases. > It must not be used on public download pages. > Please use www.apache.org/dist (for KEYS, hashes and sigs) and the mirror > system instead. > The current download page has lots of references to dist.a.o; please replace > thes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22086) space quota issue: deleting snapshot doesn't update the usage of table
[ https://issues.apache.org/jira/browse/HBASE-22086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi updated HBASE-22086: --- Attachment: hbase-22086.master.003.patch > space quota issue: deleting snapshot doesn't update the usage of table > -- > > Key: HBASE-22086 > URL: https://issues.apache.org/jira/browse/HBASE-22086 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Assignee: Sakthi >Priority: Minor > Attachments: hbase-22086.master.001.patch, > hbase-22086.master.002.patch, hbase-22086.master.003.patch > > > space quota issue: deleting snapshot doesn't update the usage of table > Steps: 1: > set_quota TYPE => SPACE, TABLE => 'bugatti', LIMIT => '7M', POLICY => > NO_WRITES_COMPACTIONS > 2: ./hbase pe --table="bugatti" --nomapred --rows=200 sequentialWrite 10 > 3: ./hbase pe --table="bugatti" --nomapred --rows=200 sequentialWrite 10 > 4: snapshot 'bugatti','bugatti_snapshot' > 5: ./hbase pe --table="bugatti" --nomapred --rows=200 sequentialWrite 10 > 6: major_compact 'bugatti' > 7: delete_snapshot 'bugatti_snapshot' > now check the usage and observe that it is not getting updated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-22206) dist.apache.org must not be used for public downloads
[ https://issues.apache.org/jira/browse/HBASE-22206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak reassigned HBASE-22206: --- Assignee: Dima Spivak Let's see if I still remember how to do this... > dist.apache.org must not be used for public downloads > - > > Key: HBASE-22206 > URL: https://issues.apache.org/jira/browse/HBASE-22206 > Project: HBase > Issue Type: Bug > Components: website >Reporter: Sebb >Assignee: Dima Spivak >Priority: Major > > The dist.apache.org server is only intended for use by developers in staging > releases. > It must not be used on public download pages. > Please use www.apache.org/dist (for KEYS, hashes and sigs) and the mirror > system instead. > The current download page has lots of references to dist.a.o; please replace > thes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
[
https://issues.apache.org/jira/browse/HBASE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814902#comment-16814902
]
HBase QA commented on HBASE-22200:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
48s{color} | {color:blue} hbase-server in master has 11 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 12s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}146m
5s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}182m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/56/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22200 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12965504/HBASE-22200-master-002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f3176d6fb5b5 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ffede2edfa |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/56/testReport/ |
| Max. process+thread count | 4895 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/56/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Comment Edited] (HBASE-22206) dist.apache.org must not be used for public downloads
[
https://issues.apache.org/jira/browse/HBASE-22206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814901#comment-16814901
]
Dima Spivak edited comment on HBASE-22206 at 4/10/19 10:02 PM:
---
The 2.0.5 listing in {{downloads.xml}} links to d.a.o. The rest of the releases
are good.
was (Author: dimaspivak):
The 2.0.5 listing is {{downloads.xml}} links to d.a.o. The rest of the releases
are good.
> dist.apache.org must not be used for public downloads
> -
>
> Key: HBASE-22206
> URL: https://issues.apache.org/jira/browse/HBASE-22206
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sebb
>Priority: Major
>
> The dist.apache.org server is only intended for use by developers in staging
> releases.
> It must not be used on public download pages.
> Please use www.apache.org/dist (for KEYS, hashes and sigs) and the mirror
> system instead.
> The current download page has lots of references to dist.a.o; please replace
> thes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22206) dist.apache.org must not be used for public downloads
[
https://issues.apache.org/jira/browse/HBASE-22206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814901#comment-16814901
]
Dima Spivak commented on HBASE-22206:
-
The 2.0.5 listing is {{downloads.xml}} links to d.a.o. The rest of the releases
are good.
> dist.apache.org must not be used for public downloads
> -
>
> Key: HBASE-22206
> URL: https://issues.apache.org/jira/browse/HBASE-22206
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sebb
>Priority: Major
>
> The dist.apache.org server is only intended for use by developers in staging
> releases.
> It must not be used on public download pages.
> Please use www.apache.org/dist (for KEYS, hashes and sigs) and the mirror
> system instead.
> The current download page has lots of references to dist.a.o; please replace
> thes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22206) dist.apache.org must not be used for public downloads
[ https://issues.apache.org/jira/browse/HBASE-22206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814893#comment-16814893 ] Sean Busbey commented on HBASE-22206: - or attach a patch. > dist.apache.org must not be used for public downloads > - > > Key: HBASE-22206 > URL: https://issues.apache.org/jira/browse/HBASE-22206 > Project: HBase > Issue Type: Bug > Components: website >Reporter: Sebb >Priority: Major > > The dist.apache.org server is only intended for use by developers in staging > releases. > It must not be used on public download pages. > Please use www.apache.org/dist (for KEYS, hashes and sigs) and the mirror > system instead. > The current download page has lots of references to dist.a.o; please replace > thes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22206) dist.apache.org must not be used for public downloads
[ https://issues.apache.org/jira/browse/HBASE-22206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814892#comment-16814892 ] Sean Busbey commented on HBASE-22206: - please be specific on where you see a use of dist.a.o. > dist.apache.org must not be used for public downloads > - > > Key: HBASE-22206 > URL: https://issues.apache.org/jira/browse/HBASE-22206 > Project: HBase > Issue Type: Bug > Components: website >Reporter: Sebb >Priority: Major > > The dist.apache.org server is only intended for use by developers in staging > releases. > It must not be used on public download pages. > Please use www.apache.org/dist (for KEYS, hashes and sigs) and the mirror > system instead. > The current download page has lots of references to dist.a.o; please replace > thes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22206) dist.apache.org must not be used for public downloads
Sebb created HBASE-22206: Summary: dist.apache.org must not be used for public downloads Key: HBASE-22206 URL: https://issues.apache.org/jira/browse/HBASE-22206 Project: HBase Issue Type: Bug Components: website Reporter: Sebb The dist.apache.org server is only intended for use by developers in staging releases. It must not be used on public download pages. Please use www.apache.org/dist (for KEYS, hashes and sigs) and the mirror system instead. The current download page has lots of references to dist.a.o; please replace thes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21048) Get LogLevel is not working from console in secure environment
[
https://issues.apache.org/jira/browse/HBASE-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HBASE-21048:
Attachment: HBASE-21048.master.007.patch
> Get LogLevel is not working from console in secure environment
> --
>
> Key: HBASE-21048
> URL: https://issues.apache.org/jira/browse/HBASE-21048
> Project: HBase
> Issue Type: Bug
>Reporter: Chandra Sekhar
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-21048.001.patch, HBASE-21048.master.001.patch,
> HBASE-21048.master.002.patch, HBASE-21048.master.003.patch,
> HBASE-21048.master.004.patch, HBASE-21048.master.005.patch,
> HBASE-21048.master.006.patch, HBASE-21048.master.007.patch
>
>
> When we try to get log level of specific package in secure environment,
> getting SocketException.
> {code:java}
> hbase/master/bin# ./hbase org.apache.hadoop.hbase.http.log.LogLevel -getlevel
> host-:16010 org.apache.hadoop.hbase
> Connecting to http://host-:16010/logLevel?log=org.apache.hadoop.hbase
> java.net.SocketException: Unexpected end of file from server
> {code}
> It is trying to connect http instead of https
> code snippet that handling only http in *LogLevel.java*
> {code:java}
> public static void main(String[] args) {
> if (args.length == 3 && "-getlevel".equals(args[0])) {
> process("http://; + args[1] + "/logLevel?log=" + args[2]);
> return;
> }
> else if (args.length == 4 && "-setlevel".equals(args[0])) {
> process("http://; + args[1] + "/logLevel?log=" + args[2]
> + "=" + args[3]);
> return;
> }
> System.err.println(USAGES);
> System.exit(-1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21048) Get LogLevel is not working from console in secure environment
[
https://issues.apache.org/jira/browse/HBASE-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HBASE-21048:
Attachment: HBASE-21048.master.006.patch
> Get LogLevel is not working from console in secure environment
> --
>
> Key: HBASE-21048
> URL: https://issues.apache.org/jira/browse/HBASE-21048
> Project: HBase
> Issue Type: Bug
>Reporter: Chandra Sekhar
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-21048.001.patch, HBASE-21048.master.001.patch,
> HBASE-21048.master.002.patch, HBASE-21048.master.003.patch,
> HBASE-21048.master.004.patch, HBASE-21048.master.005.patch,
> HBASE-21048.master.006.patch
>
>
> When we try to get log level of specific package in secure environment,
> getting SocketException.
> {code:java}
> hbase/master/bin# ./hbase org.apache.hadoop.hbase.http.log.LogLevel -getlevel
> host-:16010 org.apache.hadoop.hbase
> Connecting to http://host-:16010/logLevel?log=org.apache.hadoop.hbase
> java.net.SocketException: Unexpected end of file from server
> {code}
> It is trying to connect http instead of https
> code snippet that handling only http in *LogLevel.java*
> {code:java}
> public static void main(String[] args) {
> if (args.length == 3 && "-getlevel".equals(args[0])) {
> process("http://; + args[1] + "/logLevel?log=" + args[2]);
> return;
> }
> else if (args.length == 4 && "-setlevel".equals(args[0])) {
> process("http://; + args[1] + "/logLevel?log=" + args[2]
> + "=" + args[3]);
> return;
> }
> System.err.println(USAGES);
> System.exit(-1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
[
https://issues.apache.org/jira/browse/HBASE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814875#comment-16814875
]
HBase QA commented on HBASE-22200:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
30s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
29s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
17s{color} | {color:red} hbase-server: The patch generated 1 new + 60 unchanged
- 0 fixed = 61 total (was 60) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 37s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}128m
57s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}166m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/55/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22200 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12965498/HBASE-22200-branch-2.1-002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 9704ec6c7281 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb
13 15:00:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.1 / 4ceffc83fe |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/55/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/55/testReport/ |
| Max. process+thread count | 5048 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
|
[jira] [Updated] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-16488: Attachment: HBASE-16488.branch-1.012.patch Status: Patch Available (was: In Progress) > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 2.0.0, 1.2.2, 1.1.5, 1.4.0, 1.0.3, 1.3.0 >Reporter: Stephen Yuan Jiang >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-16488.branch-1.012.patch, > HBASE-16488.branch-1.012.patch, HBASE-16488.revisit.v11-branch-1.patch, > HBASE-16488.v1-branch-1.patch, HBASE-16488.v1-master.patch, > HBASE-16488.v10-branch-1.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v2-branch-1.patch, HBASE-16488.v3-branch-1.patch, > HBASE-16488.v3-branch-1.patch, HBASE-16488.v4-branch-1.patch, > HBASE-16488.v5-branch-1.patch, HBASE-16488.v6-branch-1.patch, > HBASE-16488.v7-branch-1.patch, HBASE-16488.v8-branch-1.patch, > HBASE-16488.v9-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-16488 started by Xu Cang. --- > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 1.3.0, 1.0.3, 1.4.0, 1.1.5, 1.2.2, 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-16488.branch-1.012.patch, > HBASE-16488.branch-1.012.patch, HBASE-16488.revisit.v11-branch-1.patch, > HBASE-16488.v1-branch-1.patch, HBASE-16488.v1-master.patch, > HBASE-16488.v10-branch-1.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v2-branch-1.patch, HBASE-16488.v3-branch-1.patch, > HBASE-16488.v3-branch-1.patch, HBASE-16488.v4-branch-1.patch, > HBASE-16488.v5-branch-1.patch, HBASE-16488.v6-branch-1.patch, > HBASE-16488.v7-branch-1.patch, HBASE-16488.v8-branch-1.patch, > HBASE-16488.v9-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22172) Suppress Java 11 reflective access warnings
[
https://issues.apache.org/jira/browse/HBASE-22172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814861#comment-16814861
]
Josh Elser commented on HBASE-22172:
I can confirm your patch helps, Sakthi!
I did see a warning when tryign to launch the hbase shell.
{noformat}
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.util.io.FilenoUtil
(file:/Users/jelser/projects/hbase-copy.git/hbase-assembly/target/hbase-3.0.0-SNAPSHOT/lib/ruby/jruby-complete-9.1.13.0.jar)
to method sun.nio.ch.SelChImpl.getFD()
WARNING: Please consider reporting this to the maintainers of
org.jruby.util.io.FilenoUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal
reflective access operations
WARNING: All illegal access operations will be denied in a future
release{noformat}
I feel like trying to get all of these warnings suppressed is chasing your tail
for little good :(
However, for this specific case, you do have the following:
{code:java}
HBASE_OPTS="$HBASE_OPTS --add-opens java.base/sun.nio.ch=ALL-UNNAMED"{code}
Where does the prefix {{java.base}} come from? Is it maybe wrong (as I feel
like your suppression here should have caught the above warning)? I wonder if
we have any contacts with folks who have jumped early onto the jdk11 train or
might do/have done work in JDK to tell us more about how this is supposed to
work?
Related, I also see this:
{noformat}
unsupported Java version "11", defaulting to 1.7{noformat}
Busbey mentioned that this is HBase expecting a "1.x" version from the JDK and
getting confused about the "11" it gets back. So, something else to fix :)
> Suppress Java 11 reflective access warnings
> ---
>
> Key: HBASE-22172
> URL: https://issues.apache.org/jira/browse/HBASE-22172
> Project: HBase
> Issue Type: Task
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Labels: jdk11
> Attachments: hbase-22172.master.001.patch
>
>
> While running a Java 8 compiled hbase on Java 11 system, I found the
> following warnings being thrown. I think we can add the "--add-opens" flag to
> HBASE_OPTS (if the jdk version is 11) to suppress this warning.
> {code:java}
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> (file:/Users/jatsakthi/test/HBASE_TEST_AREA/hbase-3.0.0-SNAPSHOT/lib/hbase-common-3.0.0-SNAPSHOT.jar)
> to method java.nio.Bits.unaligned()
> WARNING: Please consider reporting this to the maintainers of
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [hbase] Apache-HBase commented on issue #126: HBASE-21718 Implement Admin based on AsyncAdmin
Apache-HBase commented on issue #126: HBASE-21718 Implement Admin based on AsyncAdmin URL: https://github.com/apache/hbase/pull/126#issuecomment-481843343 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 295 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 44 new or modified test files. | ||| _ HBASE-21512 Compile Tests _ | | 0 | mvndep | 7 | Maven dependency ordering for branch | | +1 | mvninstall | 239 | HBASE-21512 passed | | +1 | compile | 148 | HBASE-21512 passed | | +1 | checkstyle | 166 | HBASE-21512 passed | | +1 | shadedjars | 254 | branch has no errors when building our shaded downstream artifacts. | | +1 | findbugs | 354 | HBASE-21512 passed | | +1 | javadoc | 109 | HBASE-21512 passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for patch | | +1 | mvninstall | 249 | the patch passed | | +1 | compile | 157 | the patch passed | | +1 | javac | 157 | the patch passed | | +1 | checkstyle | 31 | The patch passed checkstyle in hbase-client | | +1 | checkstyle | 71 | hbase-server: The patch generated 0 new + 412 unchanged - 4 fixed = 412 total (was 416) | | +1 | checkstyle | 16 | The patch passed checkstyle in hbase-mapreduce | | +1 | checkstyle | 27 | The patch passed checkstyle in hbase-thrift | | +1 | checkstyle | 12 | The patch passed checkstyle in hbase-backup | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 265 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 538 | Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. | | +1 | findbugs | 399 | the patch passed | | +1 | javadoc | 107 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 191 | hbase-client in the patch passed. | | -1 | unit | 14339 | hbase-server in the patch failed. | | -1 | unit | 1569 | hbase-mapreduce in the patch failed. | | +1 | unit | 411 | hbase-thrift in the patch passed. | | +1 | unit | 1147 | hbase-backup in the patch passed. | | +1 | asflicense | 172 | The patch does not generate ASF License warnings. | | | | 21450 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.procedure.TestSCPWithReplicas | | | hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL | | | hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster | | | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated | | | hadoop.hbase.client.TestAdmin1 | | | hadoop.hbase.master.TestSplitWALManager | | | hadoop.hbase.master.procedure.TestProcedurePriority | | | hadoop.hbase.snapshot.TestExportSnapshotNoCluster | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-126/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/126 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux dc14f28d6c1c 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | HBASE-21512 / ee3774040f | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-126/2/artifact/out/patch-unit-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-126/2/artifact/out/patch-unit-hbase-mapreduce.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-126/2/testReport/ | | Max. process+thread count | 5479 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server hbase-mapreduce hbase-thrift hbase-backup U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-126/2/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache
[jira] [Commented] (HBASE-22012) SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable table
[ https://issues.apache.org/jira/browse/HBASE-22012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814835#comment-16814835 ] Shardul Singh commented on HBASE-22012: --- [~elserj] Hey thanks for replying...no I just tried your suggestion and it was not working so came up with an alternative. [~nihaljain.cs] Anything is fine with me..i can also work on it..but if you want to continue no problem..thanks :) > SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable > table > > > Key: HBASE-22012 > URL: https://issues.apache.org/jira/browse/HBASE-22012 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.2.1 >Reporter: Ajeet Rai >Assignee: Nihal Jain >Priority: Major > > Space Quota: Policy state is getting changed from disable to Observance after > sometime automatically. > Steps: > 1: Create a table with space quota policy as Disable > 2: Put some data so that table state is in space quota violation > 3: So observe that table state is in violation > 4: Now wait for some time > 5: Observe that after some time table state is changing to to Observance > however table is still disabled > edit (elserj): The table is automatically moved back from the violation state > because of the code added that tried to ride over RITs. When a Region is not > online (whether normally or abnormally), the RegionSizeReports are not sent > from RS to Master. Eventually, enough Regions are not reported which dips > below the acceptable threshold and we automatically move the table back to > the "acceptable" space quota state (not in violation). We could skip this > failsafe when we're checking for a quota that has the DisableTable violation > policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase] Apache-HBase commented on issue #136: HBASE-22199 Replaced UTF-8 String with StandardCharsets.UTF_8
Apache-HBase commented on issue #136: HBASE-22199 Replaced UTF-8 String with StandardCharsets.UTF_8 URL: https://github.com/apache/hbase/pull/136#issuecomment-481830980 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 301 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 4 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 32 | Maven dependency ordering for branch | | +1 | mvninstall | 346 | master passed | | +1 | compile | 165 | master passed | | +1 | checkstyle | 151 | master passed | | +1 | shadedjars | 353 | branch has no errors when building our shaded downstream artifacts. | | -1 | findbugs | 266 | hbase-server in master has 11 extant Findbugs warnings. | | +1 | javadoc | 106 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for patch | | +1 | mvninstall | 313 | the patch passed | | +1 | compile | 161 | the patch passed | | +1 | javac | 161 | the patch passed | | -1 | checkstyle | 23 | hbase-mapreduce: The patch generated 1 new + 45 unchanged - 0 fixed = 46 total (was 45) | | -1 | checkstyle | 21 | hbase-examples: The patch generated 3 new + 245 unchanged - 0 fixed = 248 total (was 245) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 333 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 712 | Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. | | +1 | findbugs | 423 | the patch passed | | +1 | javadoc | 106 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 18455 | hbase-server in the patch failed. | | +1 | unit | 1606 | hbase-mapreduce in the patch passed. | | +1 | unit | 536 | hbase-rest in the patch passed. | | +1 | unit | 146 | hbase-examples in the patch passed. | | +1 | asflicense | 147 | The patch does not generate ASF License warnings. | | | | 25110 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.util.TestFromClientSide3WoUnsafe | | | hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL | | | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated | | | hadoop.hbase.replication.TestReplicationDisableInactivePeer | | | hadoop.hbase.replication.TestReplicationSmallTestsSync | | | hadoop.hbase.quotas.TestSpaceQuotas | | | hadoop.hbase.master.procedure.TestSCPWithReplicas | | | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.client.TestAdmin1 | | | hadoop.hbase.client.TestFromClientSide | | | hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL | | | hadoop.hbase.client.TestFromClientSideWithCoprocessor | | | hadoop.hbase.client.TestFromClientSide3 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-136/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/136 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux a57bb368ab59 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 494a8efe5f | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-136/1/artifact/out/branch-findbugs-hbase-server-warnings.html | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-136/1/artifact/out/diff-checkstyle-hbase-mapreduce.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-136/1/artifact/out/diff-checkstyle-hbase-examples.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-136/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-136/1/testReport/ | | Max. process+thread count | 5600 (vs. ulimit of 1) | | modules | C: hbase-server hbase-mapreduce hbase-rest hbase-examples U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-136/1/console | |
[jira] [Commented] (HBASE-22201) result-test.cc fails on EstimatedSize assertion
[
https://issues.apache.org/jira/browse/HBASE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814816#comment-16814816
]
HBase QA commented on HBASE-22201:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
58s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-14850 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
15s{color} | {color:green} HBASE-14850 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
32s{color} | {color:green} HBASE-14850 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 7m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}280m 11s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}309m 23s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.quotas.TestClusterScopeQuotaThrottle |
| | hadoop.hbase.client.TestAdmin1 |
| | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas |
| | hadoop.hbase.regionserver.TestSplitTransactionOnCluster |
| | hadoop.hbase.client.TestFromClientSide |
| | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/53/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22201 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12965460/HBASE-22201-HBASE-14850.v0.patch
|
| Optional Tests | dupname asflicense cc unit compile |
| uname | Linux 6c6e2f0e3575 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-14850 / 2eb1b29e8b |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/53/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/53/testReport/ |
| Max. process+thread count | 5913 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/53/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |
This message was automatically generated.
> result-test.cc fails on EstimatedSize assertion
> ---
>
> Key: HBASE-22201
> URL: https://issues.apache.org/jira/browse/HBASE-22201
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: HBASE-14850
>Reporter: Ian Buss
>Assignee: Ian Buss
>Priority: Major
> Attachments: HBASE-22201-HBASE-14850.v0.patch
>
>
> The test assumes {{sizeof(Result)}} will be the same as the estimated size of
> an empty Result which is not the case because estimated size also accounts
> for current capacity of its {{row_}} string member.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814806#comment-16814806 ] Andrew Purtell commented on HBASE-16488: Skimmed the latest patch. Trying it out. > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 1.3.0, 1.0.3, 1.4.0, 1.1.5, 1.2.2, 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-16488.branch-1.012.patch, > HBASE-16488.revisit.v11-branch-1.patch, HBASE-16488.v1-branch-1.patch, > HBASE-16488.v1-master.patch, HBASE-16488.v10-branch-1.patch, > HBASE-16488.v2-branch-1.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v3-branch-1.patch, HBASE-16488.v3-branch-1.patch, > HBASE-16488.v4-branch-1.patch, HBASE-16488.v5-branch-1.patch, > HBASE-16488.v6-branch-1.patch, HBASE-16488.v7-branch-1.patch, > HBASE-16488.v8-branch-1.patch, HBASE-16488.v9-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
[
https://issues.apache.org/jira/browse/HBASE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814798#comment-16814798
]
Wellington Chevreuil commented on HBASE-22200:
--
Submitted two new patch versions with the suggestion and javadoc issue fixed.
Patch for master should apply for branch-2, branch-2.2 and master branches,
whilst branch-2.1 would need its own patch provided here.
> WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
> -
>
> Key: HBASE-22200
> URL: https://issues.apache.org/jira/browse/HBASE-22200
> Project: HBase
> Issue Type: Bug
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Labels: S3, WAL
> Attachments: HBASE-22200-branch-2.1-001.patch,
> HBASE-22200-branch-2.1-002.patch, HBASE-22200-master-001.patch,
> HBASE-22200-master-002.patch
>
>
> *WALSplitter.hasRecoveredEdits* should use same FS instance from WAL region
> dir when checking for recovered.edits files, instead of taking FS instance as
> additional method parameter. When specifying different file systems for *wal
> dir* and *root dir*, *WALSplitter.hasRecoveredEdits* current implementation
> will crash or give wrong results. As of now, it's being used indirectly by
> *SplitTableRegionProcedure*. When running tests with *WAL dir* on HDFS and
> *root dir* on S3, for example, noticed region split failing with below error:
> {noformat}
> 2019-04-08 13:53:58,064 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=98,
> state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true;
> SplitTableRegionProcedure table=test-tbl,
> parent=4c5db01611e97e3abbe02e781e867212,
> daughterA=28a0a5e4ef7618899f6bd6dfb5335fe7,
> daughterB=05fa26feaf03ebf9e87e099cbd1eabac
> java.lang.IllegalArgumentException: Path
> hdfs://host-1.example.com:8020/wal_dir/default/test-tbl/4c5db01611e97e3abbe02e781e867212/recovered.edits
> scheme must be s3a
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.checkPath(DynamoDBMetadataStore.java:1127)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:437)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2110)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2088)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1668)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.getSplitEditFilesSorted(WALSplitter.java:576)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.hasRecoveredEdits(WALSplitter.java:558)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.hasRecoveredEdits(SplitTableRegionProcedure.java:148)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:255)
> {noformat}
> Since *WALSplitter.hasRecoveredEdits* already resolves the proper WAL dir for
> the region, we can simply re-use FS instance from the path instance for the
> WAL dir region, when searching for recovered.edits.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
[
https://issues.apache.org/jira/browse/HBASE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil updated HBASE-22200:
-
Attachment: HBASE-22200-master-002.patch
> WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
> -
>
> Key: HBASE-22200
> URL: https://issues.apache.org/jira/browse/HBASE-22200
> Project: HBase
> Issue Type: Bug
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Labels: S3, WAL
> Attachments: HBASE-22200-branch-2.1-001.patch,
> HBASE-22200-branch-2.1-002.patch, HBASE-22200-master-001.patch,
> HBASE-22200-master-002.patch
>
>
> *WALSplitter.hasRecoveredEdits* should use same FS instance from WAL region
> dir when checking for recovered.edits files, instead of taking FS instance as
> additional method parameter. When specifying different file systems for *wal
> dir* and *root dir*, *WALSplitter.hasRecoveredEdits* current implementation
> will crash or give wrong results. As of now, it's being used indirectly by
> *SplitTableRegionProcedure*. When running tests with *WAL dir* on HDFS and
> *root dir* on S3, for example, noticed region split failing with below error:
> {noformat}
> 2019-04-08 13:53:58,064 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=98,
> state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true;
> SplitTableRegionProcedure table=test-tbl,
> parent=4c5db01611e97e3abbe02e781e867212,
> daughterA=28a0a5e4ef7618899f6bd6dfb5335fe7,
> daughterB=05fa26feaf03ebf9e87e099cbd1eabac
> java.lang.IllegalArgumentException: Path
> hdfs://host-1.example.com:8020/wal_dir/default/test-tbl/4c5db01611e97e3abbe02e781e867212/recovered.edits
> scheme must be s3a
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.checkPath(DynamoDBMetadataStore.java:1127)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:437)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2110)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2088)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1668)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.getSplitEditFilesSorted(WALSplitter.java:576)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.hasRecoveredEdits(WALSplitter.java:558)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.hasRecoveredEdits(SplitTableRegionProcedure.java:148)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:255)
> {noformat}
> Since *WALSplitter.hasRecoveredEdits* already resolves the proper WAL dir for
> the region, we can simply re-use FS instance from the path instance for the
> WAL dir region, when searching for recovered.edits.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22172) Suppress Java 11 reflective access warnings
[
https://issues.apache.org/jira/browse/HBASE-22172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814793#comment-16814793
]
Sakthi commented on HBASE-22172:
Yes [~elserj] you are right. I am not sure if there is any way we can do this
testing currently via Jenkins. Also I'm not sure if these are the complete of
list of warnings that we might see. Some other type of invocations
(pe,ltt,itbll, other mr jobs, etc) might list out other set of warnings. The
above 250 were hit with hbase shell's mere invocation alone. So this is bad if
we have to try all other possible ways as well to access[thrift, rest] (to list
out all such warnings).
> Suppress Java 11 reflective access warnings
> ---
>
> Key: HBASE-22172
> URL: https://issues.apache.org/jira/browse/HBASE-22172
> Project: HBase
> Issue Type: Task
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Labels: jdk11
> Attachments: hbase-22172.master.001.patch
>
>
> While running a Java 8 compiled hbase on Java 11 system, I found the
> following warnings being thrown. I think we can add the "--add-opens" flag to
> HBASE_OPTS (if the jdk version is 11) to suppress this warning.
> {code:java}
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> (file:/Users/jatsakthi/test/HBASE_TEST_AREA/hbase-3.0.0-SNAPSHOT/lib/hbase-common-3.0.0-SNAPSHOT.jar)
> to method java.nio.Bits.unaligned()
> WARNING: Please consider reporting this to the maintainers of
> org.apache.hadoop.hbase.util.UnsafeAvailChecker
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22106) Log cause of the failure when coprocessor specification parsing fails.
[
https://issues.apache.org/jira/browse/HBASE-22106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814792#comment-16814792
]
Hudson commented on HBASE-22106:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #541 (See
[https://builds.apache.org/job/HBase-1.3-IT/541/])
HBASE-22106 Better log message when failing to load coprocessor (elserj:
[https://github.com/apache/hbase/commit/085326a1353ca036bb01ae787728300d3d32b7ae])
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
> Log cause of the failure when coprocessor specification parsing fails.
> --
>
> Key: HBASE-22106
> URL: https://issues.apache.org/jira/browse/HBASE-22106
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 1.4.9
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Minor
> Fix For: 1.4.10, 1.3.4, 1.5.1
>
> Attachments: HBASE-22106.branch-1.4.001.patch
>
>
> {code}
> 2019-02-15 22:56:33,008 ERROR [RS_OPEN_REGION-n79:16020-16]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$2, spec:
> |org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-14]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-19]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> {code}
> I checked the same coprocessor specification(logged in exception) with the
> code and it is parsed successfully. And even after the restart , regionserver
> also didn't complain about it.
> I think we should be logging the cause of the exception while parsing table
> descriptor for the coprocessor for better debugging.
> https://github.com/apache/hbase/blob/branch-1.4/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L318
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-22120) remove HTrace
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HBASE-22120: Assignee: Wei-Chiu Chuang > remove HTrace > - > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Assignee: Wei-Chiu Chuang >Priority: Major > > Suggested in HBASE-22115 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22120) remove HTrace
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814778#comment-16814778 ] Sergey Shelukhin commented on HBASE-22120: -- That sounds good to me... There seems to be broad agreement on removing it, and on aligning with Hadoop. Feel free to create a DISCUSS thread, but I don't recall how it's done in HBase. IIRC in Hive you just need 3 +1s for branch merges/major dependency changes - smth similar might be enough > remove HTrace > - > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Priority: Major > > Suggested in HBASE-22115 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase] Apache-HBase commented on issue #134: HBASE-22196 Split TestRestartCluster
Apache-HBase commented on issue #134: HBASE-22196 Split TestRestartCluster URL: https://github.com/apache/hbase/pull/134#issuecomment-481814991 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 43 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 9 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 278 | master passed | | +1 | compile | 55 | master passed | | +1 | checkstyle | 68 | master passed | | +1 | shadedjars | 275 | branch has no errors when building our shaded downstream artifacts. | | -1 | findbugs | 174 | hbase-server in master has 11 extant Findbugs warnings. | | +1 | javadoc | 35 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 279 | the patch passed | | +1 | compile | 53 | the patch passed | | +1 | javac | 53 | the patch passed | | +1 | checkstyle | 70 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 277 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 571 | Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. | | +1 | findbugs | 173 | the patch passed | | +1 | javadoc | 32 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 17249 | hbase-server in the patch failed. | | +1 | asflicense | 30 | The patch does not generate ASF License warnings. | | | | 19739 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.quotas.TestSpaceQuotas | | | hadoop.hbase.util.TestFromClientSide3WoUnsafe | | | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | | hadoop.hbase.master.procedure.TestSCPWithReplicas | | | hadoop.hbase.client.TestFromClientSideWithCoprocessor | | | hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL | | | hadoop.hbase.client.TestSnapshotTemporaryDirectory | | | hadoop.hbase.client.TestFromClientSide | | | hadoop.hbase.client.TestFromClientSide3 | | | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.client.TestAsyncTableBatch | | | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated | | | hadoop.hbase.client.TestAdmin1 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-134/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/134 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 644fb89aa0e3 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 494a8efe5f | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-134/5/artifact/out/branch-findbugs-hbase-server-warnings.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-134/5/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-134/5/testReport/ | | Max. process+thread count | 4808 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-134/5/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-20910) Fix dev-support/submit-patch.py by opening file with 'b' mode
[
https://issues.apache.org/jira/browse/HBASE-20910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814773#comment-16814773
]
Josh Elser commented on HBASE-20910:
Looks to me like someone has already made this change.
> Fix dev-support/submit-patch.py by opening file with 'b' mode
> -
>
> Key: HBASE-20910
> URL: https://issues.apache.org/jira/browse/HBASE-20910
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Attachments: HBASE-20910.patch, HBASE-20910_v1.patch
>
>
> {code:python}
> hbase$ python3.6 dev-support/submit-patch.py
> INFO:submit-patch: Active branch: master
> INFO:submit-patch: Using tracking branch as base branch
> INFO:submit-patch: Base branch: origin/master
> INFO:submit-patch: Patch directory: /Users/asinghal/patches
> INFO:submit-patch: Patch name: master.patch
> Traceback (most recent call last):
> File "dev-support/submit-patch.py", line 253, in
> f.write(diff.encode('utf8'))
> TypeError: write() argument must be str, not bytes
> {code}
> [~te...@apache.org], FYI
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
[
https://issues.apache.org/jira/browse/HBASE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814749#comment-16814749
]
Wellington Chevreuil commented on HBASE-22200:
--
Thanks for reviewing it [~zyork]. Was about to submit new patch version for
both branches addressing javadoc issues, will also include your suggestion. As
for the failed branch-2.1 test, this should be unrelated, as it's passing on my
local build:
{noformat}
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.hadoop.hbase.master.procedure.TestProcedurePriority
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 76.19 s
- in org.apache.hadoop.hbase.master.procedure.TestProcedurePriority
{noformat}
> WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
> -
>
> Key: HBASE-22200
> URL: https://issues.apache.org/jira/browse/HBASE-22200
> Project: HBase
> Issue Type: Bug
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Labels: S3, WAL
> Attachments: HBASE-22200-branch-2.1-001.patch,
> HBASE-22200-branch-2.1-002.patch, HBASE-22200-master-001.patch
>
>
> *WALSplitter.hasRecoveredEdits* should use same FS instance from WAL region
> dir when checking for recovered.edits files, instead of taking FS instance as
> additional method parameter. When specifying different file systems for *wal
> dir* and *root dir*, *WALSplitter.hasRecoveredEdits* current implementation
> will crash or give wrong results. As of now, it's being used indirectly by
> *SplitTableRegionProcedure*. When running tests with *WAL dir* on HDFS and
> *root dir* on S3, for example, noticed region split failing with below error:
> {noformat}
> 2019-04-08 13:53:58,064 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=98,
> state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true;
> SplitTableRegionProcedure table=test-tbl,
> parent=4c5db01611e97e3abbe02e781e867212,
> daughterA=28a0a5e4ef7618899f6bd6dfb5335fe7,
> daughterB=05fa26feaf03ebf9e87e099cbd1eabac
> java.lang.IllegalArgumentException: Path
> hdfs://host-1.example.com:8020/wal_dir/default/test-tbl/4c5db01611e97e3abbe02e781e867212/recovered.edits
> scheme must be s3a
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.checkPath(DynamoDBMetadataStore.java:1127)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:437)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2110)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2088)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1668)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.getSplitEditFilesSorted(WALSplitter.java:576)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.hasRecoveredEdits(WALSplitter.java:558)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.hasRecoveredEdits(SplitTableRegionProcedure.java:148)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:255)
> {noformat}
> Since *WALSplitter.hasRecoveredEdits* already resolves the proper WAL dir for
> the region, we can simply re-use FS instance from the path instance for the
> WAL dir region, when searching for recovered.edits.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
[
https://issues.apache.org/jira/browse/HBASE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil updated HBASE-22200:
-
Attachment: HBASE-22200-branch-2.1-002.patch
> WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
> -
>
> Key: HBASE-22200
> URL: https://issues.apache.org/jira/browse/HBASE-22200
> Project: HBase
> Issue Type: Bug
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Labels: S3, WAL
> Attachments: HBASE-22200-branch-2.1-001.patch,
> HBASE-22200-branch-2.1-002.patch, HBASE-22200-master-001.patch
>
>
> *WALSplitter.hasRecoveredEdits* should use same FS instance from WAL region
> dir when checking for recovered.edits files, instead of taking FS instance as
> additional method parameter. When specifying different file systems for *wal
> dir* and *root dir*, *WALSplitter.hasRecoveredEdits* current implementation
> will crash or give wrong results. As of now, it's being used indirectly by
> *SplitTableRegionProcedure*. When running tests with *WAL dir* on HDFS and
> *root dir* on S3, for example, noticed region split failing with below error:
> {noformat}
> 2019-04-08 13:53:58,064 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=98,
> state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true;
> SplitTableRegionProcedure table=test-tbl,
> parent=4c5db01611e97e3abbe02e781e867212,
> daughterA=28a0a5e4ef7618899f6bd6dfb5335fe7,
> daughterB=05fa26feaf03ebf9e87e099cbd1eabac
> java.lang.IllegalArgumentException: Path
> hdfs://host-1.example.com:8020/wal_dir/default/test-tbl/4c5db01611e97e3abbe02e781e867212/recovered.edits
> scheme must be s3a
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.checkPath(DynamoDBMetadataStore.java:1127)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:437)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2110)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2088)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1668)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.getSplitEditFilesSorted(WALSplitter.java:576)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.hasRecoveredEdits(WALSplitter.java:558)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.hasRecoveredEdits(SplitTableRegionProcedure.java:148)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:255)
> {noformat}
> Since *WALSplitter.hasRecoveredEdits* already resolves the proper WAL dir for
> the region, we can simply re-use FS instance from the path instance for the
> WAL dir region, when searching for recovered.edits.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reassigned HBASE-16488: -- Assignee: Xu Cang (was: Stephen Yuan Jiang) > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 1.3.0, 1.0.3, 1.4.0, 1.1.5, 1.2.2, 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-16488.branch-1.012.patch, > HBASE-16488.revisit.v11-branch-1.patch, HBASE-16488.v1-branch-1.patch, > HBASE-16488.v1-master.patch, HBASE-16488.v10-branch-1.patch, > HBASE-16488.v2-branch-1.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v3-branch-1.patch, HBASE-16488.v3-branch-1.patch, > HBASE-16488.v4-branch-1.patch, HBASE-16488.v5-branch-1.patch, > HBASE-16488.v6-branch-1.patch, HBASE-16488.v7-branch-1.patch, > HBASE-16488.v8-branch-1.patch, HBASE-16488.v9-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
[
https://issues.apache.org/jira/browse/HBASE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814724#comment-16814724
]
Zach York commented on HBASE-22200:
---
Ah I see what is changed in the master branch. It looks like this was
refactored around the time that this went in so likely it was missed. I think
moving the FS into the hasRecoveredEdits makes sense.
Can we please use FSUtils.getWALFileSystem though?
> WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
> -
>
> Key: HBASE-22200
> URL: https://issues.apache.org/jira/browse/HBASE-22200
> Project: HBase
> Issue Type: Bug
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Labels: S3, WAL
> Attachments: HBASE-22200-branch-2.1-001.patch,
> HBASE-22200-master-001.patch
>
>
> *WALSplitter.hasRecoveredEdits* should use same FS instance from WAL region
> dir when checking for recovered.edits files, instead of taking FS instance as
> additional method parameter. When specifying different file systems for *wal
> dir* and *root dir*, *WALSplitter.hasRecoveredEdits* current implementation
> will crash or give wrong results. As of now, it's being used indirectly by
> *SplitTableRegionProcedure*. When running tests with *WAL dir* on HDFS and
> *root dir* on S3, for example, noticed region split failing with below error:
> {noformat}
> 2019-04-08 13:53:58,064 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=98,
> state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true;
> SplitTableRegionProcedure table=test-tbl,
> parent=4c5db01611e97e3abbe02e781e867212,
> daughterA=28a0a5e4ef7618899f6bd6dfb5335fe7,
> daughterB=05fa26feaf03ebf9e87e099cbd1eabac
> java.lang.IllegalArgumentException: Path
> hdfs://host-1.example.com:8020/wal_dir/default/test-tbl/4c5db01611e97e3abbe02e781e867212/recovered.edits
> scheme must be s3a
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.checkPath(DynamoDBMetadataStore.java:1127)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:437)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2110)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2088)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1668)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.getSplitEditFilesSorted(WALSplitter.java:576)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.hasRecoveredEdits(WALSplitter.java:558)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.hasRecoveredEdits(SplitTableRegionProcedure.java:148)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:255)
> {noformat}
> Since *WALSplitter.hasRecoveredEdits* already resolves the proper WAL dir for
> the region, we can simply re-use FS instance from the path instance for the
> WAL dir region, when searching for recovered.edits.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
[
https://issues.apache.org/jira/browse/HBASE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814721#comment-16814721
]
Zach York commented on HBASE-22200:
---
[~wchevreuil] I don't see this code path in the master branch. Which version
are you using?
> WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
> -
>
> Key: HBASE-22200
> URL: https://issues.apache.org/jira/browse/HBASE-22200
> Project: HBase
> Issue Type: Bug
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Labels: S3, WAL
> Attachments: HBASE-22200-branch-2.1-001.patch,
> HBASE-22200-master-001.patch
>
>
> *WALSplitter.hasRecoveredEdits* should use same FS instance from WAL region
> dir when checking for recovered.edits files, instead of taking FS instance as
> additional method parameter. When specifying different file systems for *wal
> dir* and *root dir*, *WALSplitter.hasRecoveredEdits* current implementation
> will crash or give wrong results. As of now, it's being used indirectly by
> *SplitTableRegionProcedure*. When running tests with *WAL dir* on HDFS and
> *root dir* on S3, for example, noticed region split failing with below error:
> {noformat}
> 2019-04-08 13:53:58,064 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=98,
> state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true;
> SplitTableRegionProcedure table=test-tbl,
> parent=4c5db01611e97e3abbe02e781e867212,
> daughterA=28a0a5e4ef7618899f6bd6dfb5335fe7,
> daughterB=05fa26feaf03ebf9e87e099cbd1eabac
> java.lang.IllegalArgumentException: Path
> hdfs://host-1.example.com:8020/wal_dir/default/test-tbl/4c5db01611e97e3abbe02e781e867212/recovered.edits
> scheme must be s3a
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.checkPath(DynamoDBMetadataStore.java:1127)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:437)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2110)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2088)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1668)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.getSplitEditFilesSorted(WALSplitter.java:576)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.hasRecoveredEdits(WALSplitter.java:558)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.hasRecoveredEdits(SplitTableRegionProcedure.java:148)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:255)
> {noformat}
> Since *WALSplitter.hasRecoveredEdits* already resolves the proper WAL dir for
> the region, we can simply re-use FS instance from the path instance for the
> WAL dir region, when searching for recovered.edits.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22104) Remove Hadoop 2.7 from next minor releases
[ https://issues.apache.org/jira/browse/HBASE-22104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814716#comment-16814716 ] Sean Busbey commented on HBASE-22104: - off the top of my head: * ref guide * default version in hadoop 2 profile > Remove Hadoop 2.7 from next minor releases > -- > > Key: HBASE-22104 > URL: https://issues.apache.org/jira/browse/HBASE-22104 > Project: HBase > Issue Type: Task > Components: hadoop2 >Affects Versions: 3.0.0, 1.6.0, 2.3.0 >Reporter: Sean Busbey >Assignee: Josh Elser >Priority: Critical > > Hadoop 2.7 is now EOM ([common-dev@hadoop "\[DISCUSS\] branch 2.7 > EoL"|https://lists.apache.org/thread.html/d1f98c2c386f2f4b980489b543db3d0bb7bdb94ea12f8fc5a90f527b@%3Ccommon-dev.hadoop.apache.org%3E]) > and has an active licensing issue (HADOOP-13794) > Let's go ahead and axe it from the next minor releases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22106) Log cause of the failure when coprocessor specification parsing fails.
[
https://issues.apache.org/jira/browse/HBASE-22106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-22106:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Ankit!
> Log cause of the failure when coprocessor specification parsing fails.
> --
>
> Key: HBASE-22106
> URL: https://issues.apache.org/jira/browse/HBASE-22106
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 1.4.9
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Minor
> Fix For: 1.4.10, 1.3.4, 1.5.1
>
> Attachments: HBASE-22106.branch-1.4.001.patch
>
>
> {code}
> 2019-02-15 22:56:33,008 ERROR [RS_OPEN_REGION-n79:16020-16]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$2, spec:
> |org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-14]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-19]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> {code}
> I checked the same coprocessor specification(logged in exception) with the
> code and it is parsed successfully. And even after the restart , regionserver
> also didn't complain about it.
> I think we should be logging the cause of the exception while parsing table
> descriptor for the coprocessor for better debugging.
> https://github.com/apache/hbase/blob/branch-1.4/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L318
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-16488: Attachment: HBASE-16488.branch-1.012.patch > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 1.3.0, 1.0.3, 1.4.0, 1.1.5, 1.2.2, 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Major > Attachments: HBASE-16488.branch-1.012.patch, > HBASE-16488.revisit.v11-branch-1.patch, HBASE-16488.v1-branch-1.patch, > HBASE-16488.v1-master.patch, HBASE-16488.v10-branch-1.patch, > HBASE-16488.v2-branch-1.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v3-branch-1.patch, HBASE-16488.v3-branch-1.patch, > HBASE-16488.v4-branch-1.patch, HBASE-16488.v5-branch-1.patch, > HBASE-16488.v6-branch-1.patch, HBASE-16488.v7-branch-1.patch, > HBASE-16488.v8-branch-1.patch, HBASE-16488.v9-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-15560) TinyLFU-based BlockCache
[ https://issues.apache.org/jira/browse/HBASE-15560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814706#comment-16814706 ] Andrew Purtell commented on HBASE-15560: After the downgrade this looks better. Any concerns for commit to branch-2s and master now? /cc [~busbey] > TinyLFU-based BlockCache > > > Key: HBASE-15560 > URL: https://issues.apache.org/jira/browse/HBASE-15560 > Project: HBase > Issue Type: Improvement > Components: BlockCache >Affects Versions: 2.0.0 >Reporter: Ben Manes >Assignee: Ben Manes >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-15560.patch, HBASE-15560.patch, HBASE-15560.patch, > HBASE-15560.patch, HBASE-15560.patch, HBASE-15560.patch, HBASE-15560.patch, > HBASE-15560.patch, HBASE-15560.patch, HBASE-15560.patch, HBASE-15560.patch, > HBASE-15560.patch, bc.hit.count, bc.miss.count, branch-1.tinylfu.txt, gets, > run_ycsb_c.sh, run_ycsb_loading.sh, tinylfu.patch > > > LruBlockCache uses the Segmented LRU (SLRU) policy to capture frequency and > recency of the working set. It achieves concurrency by using an O( n ) > background thread to prioritize the entries and evict. Accessing an entry is > O(1) by a hash table lookup, recording its logical access time, and setting a > frequency flag. A write is performed in O(1) time by updating the hash table > and triggering an async eviction thread. This provides ideal concurrency and > minimizes the latencies by penalizing the thread instead of the caller. > However the policy does not age the frequencies and may not be resilient to > various workload patterns. > W-TinyLFU ([research paper|http://arxiv.org/pdf/1512.00727.pdf]) records the > frequency in a counting sketch, ages periodically by halving the counters, > and orders entries by SLRU. An entry is discarded by comparing the frequency > of the new arrival (candidate) to the SLRU's victim, and keeping the one with > the highest frequency. This allows the operations to be performed in O(1) > time and, though the use of a compact sketch, a much larger history is > retained beyond the current working set. In a variety of real world traces > the policy had [near optimal hit > rates|https://github.com/ben-manes/caffeine/wiki/Efficiency]. > Concurrency is achieved by buffering and replaying the operations, similar to > a write-ahead log. A read is recorded into a striped ring buffer and writes > to a queue. The operations are applied in batches under a try-lock by an > asynchronous thread, thereby track the usage pattern without incurring high > latencies > ([benchmarks|https://github.com/ben-manes/caffeine/wiki/Benchmarks#server-class]). > In YCSB benchmarks the results were inconclusive. For a large cache (99% hit > rates) the two caches have near identical throughput and latencies with > LruBlockCache narrowly winning. At medium and small caches, TinyLFU had a > 1-4% hit rate improvement and therefore lower latencies. The lack luster > result is because a synthetic Zipfian distribution is used, which SLRU > performs optimally. In a more varied, real-world workload we'd expect to see > improvements by being able to make smarter predictions. > The provided patch implements BlockCache using the > [Caffeine|https://github.com/ben-manes/caffeine] caching library (see > HighScalability > [article|http://highscalability.com/blog/2016/1/25/design-of-a-modern-cache.html]). > Edward Bortnikov and Eshcar Hillel have graciously provided guidance for > evaluating this patch ([github > branch|https://github.com/ben-manes/hbase/tree/tinylfu]). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
[
https://issues.apache.org/jira/browse/HBASE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814699#comment-16814699
]
Wellington Chevreuil commented on HBASE-22200:
--
[~busbey], [~elserj],
Yes, the problem here is that although recovered.edits are indeed in the same
FS from WAL dir, current version of *WALSplitter.hasRecoveredEdits* receives an
FS instance as parameter. Then, *SplitTableRegionProcedure* is passing its own
FS instance (which is not the one from WAL dir, but from StoreFiles) to
*WALSplitter.hasRecoveredEdits*. Since we do have the recovered.edits on WAL
FS, it does not make sense for *WALSplitter.hasRecoveredEdits* to receive FS
instance as param, it should always reuse the one from the resolved Path for
the recovered edits dir.
> WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
> -
>
> Key: HBASE-22200
> URL: https://issues.apache.org/jira/browse/HBASE-22200
> Project: HBase
> Issue Type: Bug
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Labels: S3, WAL
> Attachments: HBASE-22200-branch-2.1-001.patch,
> HBASE-22200-master-001.patch
>
>
> *WALSplitter.hasRecoveredEdits* should use same FS instance from WAL region
> dir when checking for recovered.edits files, instead of taking FS instance as
> additional method parameter. When specifying different file systems for *wal
> dir* and *root dir*, *WALSplitter.hasRecoveredEdits* current implementation
> will crash or give wrong results. As of now, it's being used indirectly by
> *SplitTableRegionProcedure*. When running tests with *WAL dir* on HDFS and
> *root dir* on S3, for example, noticed region split failing with below error:
> {noformat}
> 2019-04-08 13:53:58,064 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=98,
> state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true;
> SplitTableRegionProcedure table=test-tbl,
> parent=4c5db01611e97e3abbe02e781e867212,
> daughterA=28a0a5e4ef7618899f6bd6dfb5335fe7,
> daughterB=05fa26feaf03ebf9e87e099cbd1eabac
> java.lang.IllegalArgumentException: Path
> hdfs://host-1.example.com:8020/wal_dir/default/test-tbl/4c5db01611e97e3abbe02e781e867212/recovered.edits
> scheme must be s3a
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.checkPath(DynamoDBMetadataStore.java:1127)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:437)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2110)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2088)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1668)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.getSplitEditFilesSorted(WALSplitter.java:576)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.hasRecoveredEdits(WALSplitter.java:558)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.hasRecoveredEdits(SplitTableRegionProcedure.java:148)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:255)
> {noformat}
> Since *WALSplitter.hasRecoveredEdits* already resolves the proper WAL dir for
> the region, we can simply re-use FS instance from the path instance for the
> WAL dir region, when searching for recovered.edits.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22104) Remove Hadoop 2.7 from next minor releases
[ https://issues.apache.org/jira/browse/HBASE-22104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814698#comment-16814698 ] Josh Elser commented on HBASE-22104: I can look at an update to precommit for this. Do you have a list of other things that need to happen, [~busbey]? > Remove Hadoop 2.7 from next minor releases > -- > > Key: HBASE-22104 > URL: https://issues.apache.org/jira/browse/HBASE-22104 > Project: HBase > Issue Type: Task > Components: hadoop2 >Affects Versions: 3.0.0, 1.6.0, 2.3.0 >Reporter: Sean Busbey >Assignee: Josh Elser >Priority: Critical > > Hadoop 2.7 is now EOM ([common-dev@hadoop "\[DISCUSS\] branch 2.7 > EoL"|https://lists.apache.org/thread.html/d1f98c2c386f2f4b980489b543db3d0bb7bdb94ea12f8fc5a90f527b@%3Ccommon-dev.hadoop.apache.org%3E]) > and has an active licensing issue (HADOOP-13794) > Let's go ahead and axe it from the next minor releases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-22104) Remove Hadoop 2.7 from next minor releases
[ https://issues.apache.org/jira/browse/HBASE-22104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser reassigned HBASE-22104: -- Assignee: Josh Elser > Remove Hadoop 2.7 from next minor releases > -- > > Key: HBASE-22104 > URL: https://issues.apache.org/jira/browse/HBASE-22104 > Project: HBase > Issue Type: Task > Components: hadoop2 >Affects Versions: 3.0.0, 1.6.0, 2.3.0 >Reporter: Sean Busbey >Assignee: Josh Elser >Priority: Critical > > Hadoop 2.7 is now EOM ([common-dev@hadoop "\[DISCUSS\] branch 2.7 > EoL"|https://lists.apache.org/thread.html/d1f98c2c386f2f4b980489b543db3d0bb7bdb94ea12f8fc5a90f527b@%3Ccommon-dev.hadoop.apache.org%3E]) > and has an active licensing issue (HADOOP-13794) > Let's go ahead and axe it from the next minor releases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
[
https://issues.apache.org/jira/browse/HBASE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814690#comment-16814690
]
HBase QA commented on HBASE-22200:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
34s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
56s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
18s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 39s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
28s{color} | {color:red} hbase-server generated 1 new + 1 unchanged - 0 fixed =
2 total (was 1) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}211m
23s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}251m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/51/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22200 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12965452/HBASE-22200-branch-2.1-001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 059c383f0b05 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.1 / 4ceffc83fe |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| javadoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/51/artifact/patchprocess/diff-javadoc-javadoc-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/51/testReport/ |
| Max. process+thread count | 4788 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-22106) Log cause of the failure when coprocessor specification parsing fails.
[
https://issues.apache.org/jira/browse/HBASE-22106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814691#comment-16814691
]
Josh Elser commented on HBASE-22106:
Pushing this around to branch-1's. Thanks Ankit.
FYI, I also spun out a ML post about the -1's that precommit recorded. Not your
fault (actually, mine).
> Log cause of the failure when coprocessor specification parsing fails.
> --
>
> Key: HBASE-22106
> URL: https://issues.apache.org/jira/browse/HBASE-22106
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 1.4.9
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Minor
> Fix For: 1.4.10, 1.3.4, 1.5.1
>
> Attachments: HBASE-22106.branch-1.4.001.patch
>
>
> {code}
> 2019-02-15 22:56:33,008 ERROR [RS_OPEN_REGION-n79:16020-16]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$2, spec:
> |org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-14]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-19]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> {code}
> I checked the same coprocessor specification(logged in exception) with the
> code and it is parsed successfully. And even after the restart , regionserver
> also didn't complain about it.
> I think we should be logging the cause of the exception while parsing table
> descriptor for the coprocessor for better debugging.
> https://github.com/apache/hbase/blob/branch-1.4/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L318
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22012) SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable table
[ https://issues.apache.org/jira/browse/HBASE-22012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814689#comment-16814689 ] Nihal Jain commented on HBASE-22012: [~shardulsingh] I may not be able to submit a patch until next week. please feel free to assign it to yourself, if you wish to submit a fix. > SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable > table > > > Key: HBASE-22012 > URL: https://issues.apache.org/jira/browse/HBASE-22012 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.2.1 >Reporter: Ajeet Rai >Assignee: Nihal Jain >Priority: Major > > Space Quota: Policy state is getting changed from disable to Observance after > sometime automatically. > Steps: > 1: Create a table with space quota policy as Disable > 2: Put some data so that table state is in space quota violation > 3: So observe that table state is in violation > 4: Now wait for some time > 5: Observe that after some time table state is changing to to Observance > however table is still disabled > edit (elserj): The table is automatically moved back from the violation state > because of the code added that tried to ride over RITs. When a Region is not > online (whether normally or abnormally), the RegionSizeReports are not sent > from RS to Master. Eventually, enough Regions are not reported which dips > below the acceptable threshold and we automatically move the table back to > the "acceptable" space quota state (not in violation). We could skip this > failsafe when we're checking for a quota that has the DisableTable violation > policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
[
https://issues.apache.org/jira/browse/HBASE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814686#comment-16814686
]
Josh Elser commented on HBASE-22200:
bq. I thought we fixed this in HBASE-20734. what went wrong?
My take is that we missed some spots, unexpecting uses of WALSplitter by the
split and merge code paths. So, same bug, but under different conditions.
> WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
> -
>
> Key: HBASE-22200
> URL: https://issues.apache.org/jira/browse/HBASE-22200
> Project: HBase
> Issue Type: Bug
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Labels: S3, WAL
> Attachments: HBASE-22200-branch-2.1-001.patch,
> HBASE-22200-master-001.patch
>
>
> *WALSplitter.hasRecoveredEdits* should use same FS instance from WAL region
> dir when checking for recovered.edits files, instead of taking FS instance as
> additional method parameter. When specifying different file systems for *wal
> dir* and *root dir*, *WALSplitter.hasRecoveredEdits* current implementation
> will crash or give wrong results. As of now, it's being used indirectly by
> *SplitTableRegionProcedure*. When running tests with *WAL dir* on HDFS and
> *root dir* on S3, for example, noticed region split failing with below error:
> {noformat}
> 2019-04-08 13:53:58,064 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=98,
> state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true;
> SplitTableRegionProcedure table=test-tbl,
> parent=4c5db01611e97e3abbe02e781e867212,
> daughterA=28a0a5e4ef7618899f6bd6dfb5335fe7,
> daughterB=05fa26feaf03ebf9e87e099cbd1eabac
> java.lang.IllegalArgumentException: Path
> hdfs://host-1.example.com:8020/wal_dir/default/test-tbl/4c5db01611e97e3abbe02e781e867212/recovered.edits
> scheme must be s3a
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.checkPath(DynamoDBMetadataStore.java:1127)
> at
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:437)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2110)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2088)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1668)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.getSplitEditFilesSorted(WALSplitter.java:576)
> at
> org.apache.hadoop.hbase.wal.WALSplitter.hasRecoveredEdits(WALSplitter.java:558)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.hasRecoveredEdits(SplitTableRegionProcedure.java:148)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:255)
> {noformat}
> Since *WALSplitter.hasRecoveredEdits* already resolves the proper WAL dir for
> the region, we can simply re-use FS instance from the path instance for the
> WAL dir region, when searching for recovered.edits.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
[
https://issues.apache.org/jira/browse/HBASE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814676#comment-16814676
]
HBase QA commented on HBASE-22200:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
52s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 20s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
29s{color} | {color:red} hbase-server generated 1 new + 1 unchanged - 0 fixed =
2 total (was 1) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}174m 6s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}208m 8s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.procedure.TestProcedurePriority |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HBASE-Build/52/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22200 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12965452/HBASE-22200-branch-2.1-001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b535613d113d 4.4.0-131-generic #157~14.04.1-Ubuntu SMP Fri Jul
13 08:53:17 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.1 / 4ceffc83fe |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| javadoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/52/artifact/patchprocess/diff-javadoc-javadoc-hbase-server.txt
|
| unit |
[jira] [Commented] (HBASE-21688) Address WAL filesystem issues
[ https://issues.apache.org/jira/browse/HBASE-21688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814674#comment-16814674 ] Sean Busbey commented on HBASE-21688: - thanks! > Address WAL filesystem issues > - > > Key: HBASE-21688 > URL: https://issues.apache.org/jira/browse/HBASE-21688 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Labels: s3 > Fix For: 3.0.0, 2.2.0, 2.0.6, 2.1.5 > > Attachments: HBASE-21688-branch-2.0-v1.patch, > HBASE-21688-branch-2.1-v2.patch, HBASE-21688-branch-2.2-v1.patch, > HBASE-21688-master-addendum.patch, HBASE-21688-v1.patch > > > Scan and fix code base to use new way of instantiating WAL File System. > https://issues.apache.org/jira/browse/HBASE-21457?focusedCommentId=16734688=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16734688 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22106) Log cause of the failure when coprocessor specification parsing fails.
[
https://issues.apache.org/jira/browse/HBASE-22106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814635#comment-16814635
]
Ankit Singhal edited comment on HBASE-22106 at 4/10/19 5:09 PM:
{quote}Should this go to all active branch-1 releases, Ankit? No need on
branch-2 releases/master?
{quote}
It seems to be applicable on branch-1 only. As in branch-2 , we are simply
ignoring any malformed coprocessor while loading.(Not sure if ignoring them
without warning is a good idea)
was (Author: an...@apache.org):
bq. Should this go to all active branch-1 releases, Ankit? No need on branch-2
releases/master?
It seems to be applicable on branch-1 only. As in branch-2 , we are simply
ignoring any malformed coprocessor while loading.(Not sure if ignoring them is
a good idea)
> Log cause of the failure when coprocessor specification parsing fails.
> --
>
> Key: HBASE-22106
> URL: https://issues.apache.org/jira/browse/HBASE-22106
> Project: HBase
> Issue Type: Bug
> Components: logging
>Affects Versions: 1.4.9
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Minor
> Fix For: 1.4.10, 1.3.4, 1.5.1
>
> Attachments: HBASE-22106.branch-1.4.001.patch
>
>
> {code}
> 2019-02-15 22:56:33,008 ERROR [RS_OPEN_REGION-n79:16020-16]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$2, spec:
> |org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-14]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> 2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-19]
> regionserver.RegionCoprocessorHost: Malformed table coprocessor
> specification: key=coprocessor$1, spec:
> |org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
> {code}
> I checked the same coprocessor specification(logged in exception) with the
> code and it is parsed successfully. And even after the restart , regionserver
> also didn't complain about it.
> I think we should be logging the cause of the exception while parsing table
> descriptor for the coprocessor for better debugging.
> https://github.com/apache/hbase/blob/branch-1.4/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L318
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22199) Replace "UTF-8" with StandardCharsets.UTF_8 where possible
[ https://issues.apache.org/jira/browse/HBASE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814668#comment-16814668 ] Jan Hentschel commented on HBASE-22199: --- [~busbey] Good to know. Thanks for looking into it. > Replace "UTF-8" with StandardCharsets.UTF_8 where possible > -- > > Key: HBASE-22199 > URL: https://issues.apache.org/jira/browse/HBASE-22199 > Project: HBase > Issue Type: Task >Reporter: Jan Hentschel >Assignee: Jan Hentschel >Priority: Trivial > > Currently the String "UTF-8" is used in some places where > StandardCharsets.UTF_8 could be used. To make it easier to maintain, the > current usages of "UTF-8" as a String should be replaced with > StandardCharsets.UTF_8. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22199) Replace "UTF-8" with StandardCharsets.UTF_8 where possible
[ https://issues.apache.org/jira/browse/HBASE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814667#comment-16814667 ] Sean Busbey commented on HBASE-22199: - not a bug, but JDK6 behavior on how toBytes handles encoders when given a charset vs a charset name (AVRO-1348). none of these changes look like hot path anyways. so carry on :) > Replace "UTF-8" with StandardCharsets.UTF_8 where possible > -- > > Key: HBASE-22199 > URL: https://issues.apache.org/jira/browse/HBASE-22199 > Project: HBase > Issue Type: Task >Reporter: Jan Hentschel >Assignee: Jan Hentschel >Priority: Trivial > > Currently the String "UTF-8" is used in some places where > StandardCharsets.UTF_8 could be used. To make it easier to maintain, the > current usages of "UTF-8" as a String should be replaced with > StandardCharsets.UTF_8. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22106) Log cause of the failure when coprocessor specification parsing fails.
[
https://issues.apache.org/jira/browse/HBASE-22106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814663#comment-16814663
]
HBase QA commented on HBASE-22106:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1.4 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
36s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_211 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
52s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_211 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed with JDK v1.7.0_211 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
48s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
28s{color} | {color:red} The patch causes 26 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
54s{color} | {color:red} The patch causes 26 errors with Hadoop v2.5.2. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed with JDK v1.7.0_211 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 41m 54s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 42s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestHColumnDescriptorDefaultVersions |
| | hadoop.hbase.regionserver.TestRegionOpen |
| | hadoop.hbase.TestInfoServers |
| | hadoop.hbase.regionserver.TestRegionFavoredNodes |
| |
[jira] [Commented] (HBASE-21688) Address WAL filesystem issues
[ https://issues.apache.org/jira/browse/HBASE-21688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814657#comment-16814657 ] Vladimir Rodionov commented on HBASE-21688: --- I just created and linked new jira, [~busbey] > Address WAL filesystem issues > - > > Key: HBASE-21688 > URL: https://issues.apache.org/jira/browse/HBASE-21688 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Labels: s3 > Fix For: 3.0.0, 2.2.0, 2.0.6, 2.1.5 > > Attachments: HBASE-21688-branch-2.0-v1.patch, > HBASE-21688-branch-2.1-v2.patch, HBASE-21688-branch-2.2-v1.patch, > HBASE-21688-master-addendum.patch, HBASE-21688-v1.patch > > > Scan and fix code base to use new way of instantiating WAL File System. > https://issues.apache.org/jira/browse/HBASE-21457?focusedCommentId=16734688=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16734688 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
