[jira] [Updated] (HBASE-21794) Update the Coprocessor observer example given in section 111.1 of the ref guide.
[ https://issues.apache.org/jira/browse/HBASE-21794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi updated HBASE-21794: --- Attachment: hbase-21794.master.003.patch > Update the Coprocessor observer example given in section 111.1 of the ref > guide. > > > Key: HBASE-21794 > URL: https://issues.apache.org/jira/browse/HBASE-21794 > Project: HBase > Issue Type: Bug > Components: Coprocessors, documentation >Reporter: Sakthi >Assignee: Sakthi >Priority: Minor > Attachments: hbase-21794.master.001.patch, > hbase-21794.master.002.patch, hbase-21794.master.003.patch > > > The given example should be changed after the CP changes (HBASE-17732) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21739) Move grant/revoke from regionserver to master
[ https://issues.apache.org/jira/browse/HBASE-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21739: --- Release Note: To implement user permission control in Precedure V2, move grant and revoke method from AccessController to master firstly. Mark AccessController#grant and AccessController#revoke as deprecated and please use Admin#grant and Admin#revoke instead. > Move grant/revoke from regionserver to master > - > > Key: HBASE-21739 > URL: https://issues.apache.org/jira/browse/HBASE-21739 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21739.branch-2.001.patch, > HBASE-21739.master.001.patch, HBASE-21739.master.002.patch, > HBASE-21739.master.003.patch, HBASE-21739.master.003.patch, > HBASE-21739.master.004.patch, HBASE-21739.master.005.patch > > > Create a sub-task to move grant/revoke from regionserver to master. Other > access control operations(getUserPermissions/ checkPermissions/ > hasPermission) will be moved in another sub-task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21794) Update the Coprocessor observer example given in section 111.1 of the ref guide.
[ https://issues.apache.org/jira/browse/HBASE-21794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi updated HBASE-21794: --- Attachment: hbase-21794.master.002.patch > Update the Coprocessor observer example given in section 111.1 of the ref > guide. > > > Key: HBASE-21794 > URL: https://issues.apache.org/jira/browse/HBASE-21794 > Project: HBase > Issue Type: Bug > Components: Coprocessors, documentation >Reporter: Sakthi >Assignee: Sakthi >Priority: Minor > Attachments: hbase-21794.master.001.patch, > hbase-21794.master.002.patch > > > The given example should be changed after the CP changes (HBASE-17732) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21519) Namespace region is never assigned in a HM failover scenario when multiwal is enabled
[
https://issues.apache.org/jira/browse/HBASE-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-21519:
---
Fix Version/s: (was: 2.2.0)
> Namespace region is never assigned in a HM failover scenario when multiwal is
> enabled
> -
>
> Key: HBASE-21519
> URL: https://issues.apache.org/jira/browse/HBASE-21519
> Project: HBase
> Issue Type: Bug
> Components: master, wal
>Affects Versions: 2.1.1
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Attachments: HBASE-21519.branch-2.patch, providerId_value.png
>
>
> In our test env we found that namespace region is never be assigned on HM
> failover scenario when multiwal feature is enabled,
> {noformat}
> 2018-11-28 01:38:28,085 WARN [master/HM-1:16000:becomeActiveMaster]
> master.HMaster:
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15. is NOT
> online; state=\{31f6d3383af09e18e1e81ca02a93de15 state=OPEN,
> ts=1543340156928, server=RS-2,16020,1543339824397};
> ServerCrashProcedures=false. Master startup cannot progress, in
> holding-pattern until region onlined.
> {noformat}
> And finally HM abort with following error,
> {noformat}
> 2018-11-28 01:39:16,858 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Master
> failed to complete initialization after 24ms. Please consider submitting
> a bug report including a thread dump of this process.
> 2018-11-28 01:39:18,980 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Zombie
> Master exiting. Thread dump to stdout
> {noformat}
> Stack trace:
> {noformat}
> Thread 102 (master/HM-1:16000:becomeActiveMaster):
> State: TIMED_WAITING
> Blocked count: 100
> Waited count: 246
> Stack:
> java.lang.Thread.sleep(Native Method)
> org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:148)
> org.apache.hadoop.hbase.master.HMaster.isRegionOnline(HMaster.java:1166)
>
> org.apache.hadoop.hbase.master.HMaster.waitForNamespaceOnline(HMaster.java:1187)
>
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1044)
>
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2285)
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:590)
> org.apache.hadoop.hbase.master.HMaster$$Lambda$40/1078246575.run(Unknown
> Source)
> java.lang.Thread.run(Thread.java:745)
> {noformat}
>
> Step to reproduce:
> 1) Setup a HBase cluster with 1/2 HM (say HM-1) and 2 RS(say RS-1 & RS-2)
> 2) Enable multiwal feature with following configuration setting and start
> the cluster,
> {noformat}
>
> hbase.wal.provider
> multiwal
>
>
> hbase.wal.regiongrouping.strategy
> identity
>
> {noformat}
> 3) Make sure meta and namespace regions are assigned on different RS, suppose
> RS-1 & RS-2 respectively.
> 4) Create table 't1'
> 5) Flush the meta table explicitly
> 6) Kill the RS-2, so during RS-2 SCP all regions including namespace region
> will be assigned to RS-1.
> 7) Now Kill RS-1 before meta flush happen. Here both RS-2 & RS-1 are
> shutdown now.
> 8) Stop the HM and start RS-1 & RS-2.
> 9) Now start the HM.
> Meta region is assigned successfully but HM is keep waiting for the namespace
> region onlline (Master startup cannot progress, in holding-pattern until
> region onlined) and abort with timeout.
> Observation:
> 1) After step-3 namespace region was assigned to RS-2 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> 2) After step-6 namespace region was assigned to RS-1 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339880920, value=RS-1:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339880920, value=1543339829288
> {noformat}
> 3) After Step-9, meta entry for namespace region was as follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> During SCP we do meta log split based on filter,
> {noformat}
> /**
> * Specialized method to handle the splitting for meta WAL
> * @param serverNames logs belonging to these servers will be split
>
[jira] [Updated] (HBASE-21535) Zombie Master detector is not working
[
https://issues.apache.org/jira/browse/HBASE-21535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-21535:
---
Resolution: Fixed
Fix Version/s: 2.3.0
2.0.5
2.1.3
Status: Resolved (was: Patch Available)
Pushed to branch-2+. Thanks [~pankaj2461] for contributing.
> Zombie Master detector is not working
> -
>
> Key: HBASE-21535
> URL: https://issues.apache.org/jira/browse/HBASE-21535
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0
>
> Attachments: HBASE-21535.branch-2.patch, HBASE-21535.branch-2.patch,
> HBASE-21535.patch, HBASE-21535.v2.patch
>
>
> We have InitializationMonitor thread in HMaster which detects Zombie Hmaster
> based on _hbase.master.initializationmonitor.timeout _and halts if
> _hbase.master.initializationmonitor.haltontimeout_ set _true_.
> After HBASE-19694, HMaster initialization order was correted. Hmaster is set
> active after Initializing ZK system trackers as follows,
> {noformat}
> status.setStatus("Initializing ZK system trackers");
> initializeZKBasedSystemTrackers();
> status.setStatus("Loading last flushed sequence id of regions");
> try {
> this.serverManager.loadLastFlushedSequenceIds();
> } catch (IOException e) {
> LOG.debug("Failed to load last flushed sequence id of regions"
> + " from file system", e);
> }
> // Set ourselves as active Master now our claim has succeeded up in zk.
> this.activeMaster = true;
> {noformat}
> But Zombie detector thread is started at the begining phase of
> finishActiveMasterInitialization(),
> {noformat}
> private void finishActiveMasterInitialization(MonitoredTask status) throws
> IOException,
> InterruptedException, KeeperException, ReplicationException {
> Thread zombieDetector = new Thread(new InitializationMonitor(this),
> "ActiveMasterInitializationMonitor-" + System.currentTimeMillis());
> zombieDetector.setDaemon(true);
> zombieDetector.start();
> {noformat}
> During zombieDetector execution "master.isActiveMaster()" will be false, so
> it won't wait and cant detect zombie master.
> {noformat}
> @Override
> public void run() {
> try {
> while (!master.isStopped() && master.isActiveMaster()) {
> Thread.sleep(timeout);
> if (master.isInitialized()) {
> LOG.debug("Initialization completed within allotted tolerance. Monitor
> exiting.");
> } else {
> LOG.error("Master failed to complete initialization after " + timeout + "ms.
> Please"
> + " consider submitting a bug report including a thread dump of this
> process.");
> if (haltOnTimeout) {
> LOG.error("Zombie Master exiting. Thread dump to stdout");
> Threads.printThreadInfo(System.out, "Zombie HMaster");
> System.exit(-1);
> }
> }
> }
> } catch (InterruptedException ie) {
> LOG.trace("InitMonitor thread interrupted. Existing.");
> }
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21795) Client application may get stuck (time bound) if a table modify op is called immediately after split op
[ https://issues.apache.org/jira/browse/HBASE-21795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755779#comment-16755779 ] Bhupendra Kumar Jain commented on HBASE-21795: -- I mean, clear parent region state from AssignmentMaanger's regionStates memory (Similar as Merge region procedure flow).. Not clearing from Meta table... Meta table clean up will be taken care by CatalogJanitor itself ... > Client application may get stuck (time bound) if a table modify op is called > immediately after split op > --- > > Key: HBASE-21795 > URL: https://issues.apache.org/jira/browse/HBASE-21795 > Project: HBase > Issue Type: Bug >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Critical > Attachments: HBASE-21795.master.001.patch > > > *Steps:* > * Create a table > * Split the table > * Modify the table immediately after splitting > *Expected*: > The modify table procedure completes and control returns back to client > *Actual:* > The modify table procedure completes and control does not return back to > client, until catalog janitor runs and deletes parent or future timeout occurs -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21535) Zombie Master detector is not working
[
https://issues.apache.org/jira/browse/HBASE-21535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755775#comment-16755775
]
Guanghao Zhang commented on HBASE-21535:
+1
> Zombie Master detector is not working
> -
>
> Key: HBASE-21535
> URL: https://issues.apache.org/jira/browse/HBASE-21535
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21535.branch-2.patch, HBASE-21535.branch-2.patch,
> HBASE-21535.patch, HBASE-21535.v2.patch
>
>
> We have InitializationMonitor thread in HMaster which detects Zombie Hmaster
> based on _hbase.master.initializationmonitor.timeout _and halts if
> _hbase.master.initializationmonitor.haltontimeout_ set _true_.
> After HBASE-19694, HMaster initialization order was correted. Hmaster is set
> active after Initializing ZK system trackers as follows,
> {noformat}
> status.setStatus("Initializing ZK system trackers");
> initializeZKBasedSystemTrackers();
> status.setStatus("Loading last flushed sequence id of regions");
> try {
> this.serverManager.loadLastFlushedSequenceIds();
> } catch (IOException e) {
> LOG.debug("Failed to load last flushed sequence id of regions"
> + " from file system", e);
> }
> // Set ourselves as active Master now our claim has succeeded up in zk.
> this.activeMaster = true;
> {noformat}
> But Zombie detector thread is started at the begining phase of
> finishActiveMasterInitialization(),
> {noformat}
> private void finishActiveMasterInitialization(MonitoredTask status) throws
> IOException,
> InterruptedException, KeeperException, ReplicationException {
> Thread zombieDetector = new Thread(new InitializationMonitor(this),
> "ActiveMasterInitializationMonitor-" + System.currentTimeMillis());
> zombieDetector.setDaemon(true);
> zombieDetector.start();
> {noformat}
> During zombieDetector execution "master.isActiveMaster()" will be false, so
> it won't wait and cant detect zombie master.
> {noformat}
> @Override
> public void run() {
> try {
> while (!master.isStopped() && master.isActiveMaster()) {
> Thread.sleep(timeout);
> if (master.isInitialized()) {
> LOG.debug("Initialization completed within allotted tolerance. Monitor
> exiting.");
> } else {
> LOG.error("Master failed to complete initialization after " + timeout + "ms.
> Please"
> + " consider submitting a bug report including a thread dump of this
> process.");
> if (haltOnTimeout) {
> LOG.error("Zombie Master exiting. Thread dump to stdout");
> Threads.printThreadInfo(System.out, "Zombie HMaster");
> System.exit(-1);
> }
> }
> }
> } catch (InterruptedException ie) {
> LOG.trace("InitMonitor thread interrupted. Existing.");
> }
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21795) Client application may get stuck (time bound) if a table modify op is called immediately after split op
[
https://issues.apache.org/jira/browse/HBASE-21795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755768#comment-16755768
]
Duo Zhang commented on HBASE-21795:
---
{quote}
Can we clear the parent region state during split ?
{quote}
Maybe not. IIRC, the actual HFiles are still placed under the directory for the
parent issue, so if we need the region state from meta, the HFiles will be
deleted... The CatalogJaniter will delete the region state when we find that
there are no references to the HFiles.
> Client application may get stuck (time bound) if a table modify op is called
> immediately after split op
> ---
>
> Key: HBASE-21795
> URL: https://issues.apache.org/jira/browse/HBASE-21795
> Project: HBase
> Issue Type: Bug
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Critical
> Attachments: HBASE-21795.master.001.patch
>
>
> *Steps:*
> * Create a table
> * Split the table
> * Modify the table immediately after splitting
> *Expected*:
> The modify table procedure completes and control returns back to client
> *Actual:*
> The modify table procedure completes and control does not return back to
> client, until catalog janitor runs and deletes parent or future timeout occurs
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21699) Create table failed when using SPLITS_FILE => 'splits.txt'
[
https://issues.apache.org/jira/browse/HBASE-21699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755772#comment-16755772
]
Hadoop QA commented on HBASE-21699:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} rubocop {color} | {color:green} 0m
9s{color} | {color:green} The patch generated 0 new + 270 unchanged - 18 fixed
= 270 total (was 288) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 13s{color} | {color:orange} The patch generated 8 new + 395 unchanged - 1
fixed = 403 total (was 396) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 39s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
54s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}138m
55s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
9s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}198m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21699 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956836/HBASE-21699.master.006.patch
|
| Optional Tests | dupname
[jira] [Updated] (HBASE-21739) Move grant/revoke from regionserver to master
[ https://issues.apache.org/jira/browse/HBASE-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21739: --- Attachment: HBASE-21739.branch-2.001.patch > Move grant/revoke from regionserver to master > - > > Key: HBASE-21739 > URL: https://issues.apache.org/jira/browse/HBASE-21739 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21739.branch-2.001.patch, > HBASE-21739.master.001.patch, HBASE-21739.master.002.patch, > HBASE-21739.master.003.patch, HBASE-21739.master.003.patch, > HBASE-21739.master.004.patch, HBASE-21739.master.005.patch > > > Create a sub-task to move grant/revoke from regionserver to master. Other > access control operations(getUserPermissions/ checkPermissions/ > hasPermission) will be moved in another sub-task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21809) Add retry thrift client for ThriftTable/Admin
[
https://issues.apache.org/jira/browse/HBASE-21809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755761#comment-16755761
]
Hadoop QA commented on HBASE-21809:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 38s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
38s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21809 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956848/HBASE-21809.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6fd61d9c77c4 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f997252344 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15781/testReport/ |
| Max. process+thread count | 2223 (vs. ulimit of 1) |
| modules | C: hbase-thrift U: hbase-thrift |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15781/console |
|
[jira] [Commented] (HBASE-21764) Size of in-memory compaction thread pool shoud be configurable
[
https://issues.apache.org/jira/browse/HBASE-21764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755747#comment-16755747
]
Guanghao Zhang commented on HBASE-21764:
Every region has an separate in-memory compaction pool?
> Size of in-memory compaction thread pool shoud be configurable
> --
>
> Key: HBASE-21764
> URL: https://issues.apache.org/jira/browse/HBASE-21764
> Project: HBase
> Issue Type: Sub-task
> Components: in-memory-compaction
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21764.v1.patch, HBASE-21764.v2.patch,
> HBASE-21764.v3.patch
>
>
> In RegionServicesForStores, we can see :
> {code}
> private static final int POOL_SIZE = 10;
> private static final ThreadPoolExecutor INMEMORY_COMPACTION_POOL =
> new ThreadPoolExecutor(POOL_SIZE, POOL_SIZE, 60, TimeUnit.SECONDS,
> new LinkedBlockingQueue<>(),
> new ThreadFactory() {
> @Override
> public Thread newThread(Runnable r) {
> String name = Thread.currentThread().getName() +
> "-inmemoryCompactions-" +
> System.currentTimeMillis();
> return new Thread(r, name);
> }
> });
> {code}
> The pool size should be configurable, because if many regions on a rs, the
> default 10 threads will be not enough.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21795) Client application may get stuck (time bound) if a table modify op is called immediately after split op
[
https://issues.apache.org/jira/browse/HBASE-21795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755755#comment-16755755
]
Bhupendra Kumar Jain commented on HBASE-21795:
--
{quote}Merge does not have this problem.{quote}
MergeTableRegionsProcedure clears the merged region entries from regionStates
by invoking markRegionAsMerged() on MERGE_TABLE_REGIONS_UPDATE_META state but
SplitTableRegionProcedure doesn't clears the parent region from regionStates
and thus its counted in rit later
{quote}Submitted a patch which reproduces the bug and proposes a fix.{quote}
Can we clear the parent region state during split ?
> Client application may get stuck (time bound) if a table modify op is called
> immediately after split op
> ---
>
> Key: HBASE-21795
> URL: https://issues.apache.org/jira/browse/HBASE-21795
> Project: HBase
> Issue Type: Bug
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Critical
> Attachments: HBASE-21795.master.001.patch
>
>
> *Steps:*
> * Create a table
> * Split the table
> * Modify the table immediately after splitting
> *Expected*:
> The modify table procedure completes and control returns back to client
> *Actual:*
> The modify table procedure completes and control does not return back to
> client, until catalog janitor runs and deletes parent or future timeout occurs
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755754#comment-16755754
]
Reid Chan commented on HBASE-21481:
---
ping [~zghaobac], It is planned to go in branch-2.2, as RM, would you mind
taking time to review?
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21481.master.001.patch,
> HBASE-21481.master.002.patch, HBASE-21481.master.003.patch,
> HBASE-21481.master.004.patch, HBASE-21481.master.005.patch,
> HBASE-21481.master.006.patch, HBASE-21481.master.007.patch,
> HBASE-21481.master.008.patch, HBASE-21481.master.009.patch,
> HBASE-21481.master.010.patch
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[
https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755752#comment-16755752
]
Duo Zhang commented on HBASE-21808:
---
{quote}
I've got it working locally and want to get things in place so I can automate
keeping us ready for when Hadoop has a version that works for runtime.
{quote}
Great.
> Ensure we can build with JDK11 targetting JDK8
> --
>
> Key: HBASE-21808
> URL: https://issues.apache.org/jira/browse/HBASE-21808
> Project: HBase
> Issue Type: Improvement
> Components: build, community, java
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> When folks "install a JDK" wether they get JDK11 now will depend on the
> platform (e.g. from homebrew you will get jdk11). For new contributors this
> results in confusing errors on first run.
> Make it so that a non-release build using JDK11 and using our default
> compiler source/target versions of JDK8 can complete.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[ https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755744#comment-16755744 ] Sean Busbey commented on HBASE-21808: - also I have already run into the case where I was getting a new contributor set up and their use of homebrew meant that we broke right away. So if I've missed something and we can't compile with JDK11-targetting-JDK8 then we need a section in the contributor guide to specifically call out how to install jdk8. I would rather things just work. > Ensure we can build with JDK11 targetting JDK8 > -- > > Key: HBASE-21808 > URL: https://issues.apache.org/jira/browse/HBASE-21808 > Project: HBase > Issue Type: Improvement > Components: build, community, java >Affects Versions: 3.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > When folks "install a JDK" wether they get JDK11 now will depend on the > platform (e.g. from homebrew you will get jdk11). For new contributors this > results in confusing errors on first run. > Make it so that a non-release build using JDK11 and using our default > compiler source/target versions of JDK8 can complete. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[ https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755742#comment-16755742 ] Sean Busbey commented on HBASE-21808: - yes just compiling. I've got it working locally and want to get things in place so I can automate keeping us ready for when Hadoop has a version that works for runtime. > Ensure we can build with JDK11 targetting JDK8 > -- > > Key: HBASE-21808 > URL: https://issues.apache.org/jira/browse/HBASE-21808 > Project: HBase > Issue Type: Improvement > Components: build, community, java >Affects Versions: 3.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > When folks "install a JDK" wether they get JDK11 now will depend on the > platform (e.g. from homebrew you will get jdk11). For new contributors this > results in confusing errors on first run. > Make it so that a non-release build using JDK11 and using our default > compiler source/target versions of JDK8 can complete. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[ https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755737#comment-16755737 ] Duo Zhang commented on HBASE-21808: --- So the goal here is just for compiling, not for running with JDK11? I've tried this before and then I notice that I need to wait for hadoop to be compatible with JDK11 so I gave up... Lots of the problems are that we use lots of JavaEE related annotations and classes, which are removed in JDK11. And maybe the Unsafe stuffs are also a problem, maybe we could use the PlatformDependent class in our shaded netty... > Ensure we can build with JDK11 targetting JDK8 > -- > > Key: HBASE-21808 > URL: https://issues.apache.org/jira/browse/HBASE-21808 > Project: HBase > Issue Type: Improvement > Components: build, community, java >Affects Versions: 3.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > When folks "install a JDK" wether they get JDK11 now will depend on the > platform (e.g. from homebrew you will get jdk11). For new contributors this > results in confusing errors on first run. > Make it so that a non-release build using JDK11 and using our default > compiler source/target versions of JDK8 can complete. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21809) Add retry thrift client for ThriftTable/Admin
[ https://issues.apache.org/jira/browse/HBASE-21809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allan Yang updated HBASE-21809: --- Attachment: HBASE-21809.patch > Add retry thrift client for ThriftTable/Admin > - > > Key: HBASE-21809 > URL: https://issues.apache.org/jira/browse/HBASE-21809 > Project: HBase > Issue Type: Sub-task >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21809.patch > > > This is for ThriftTable/Admin to handle exceptions like connection loss. > And only available for http thrift client. For client using TSocket, it is > not so easy to implement a retry client, maybe later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21809) Add retry thrift client for ThriftTable/Admin
[ https://issues.apache.org/jira/browse/HBASE-21809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allan Yang updated HBASE-21809: --- Status: Patch Available (was: Open) > Add retry thrift client for ThriftTable/Admin > - > > Key: HBASE-21809 > URL: https://issues.apache.org/jira/browse/HBASE-21809 > Project: HBase > Issue Type: Sub-task >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > This is for ThriftTable/Admin to handle exceptions like connection loss. > And only available for http thrift client. For client using TSocket, it is > not so easy to implement a retry client, maybe later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21809) Add retry thrift client for ThriftTable/Admin
Allan Yang created HBASE-21809: -- Summary: Add retry thrift client for ThriftTable/Admin Key: HBASE-21809 URL: https://issues.apache.org/jira/browse/HBASE-21809 Project: HBase Issue Type: Sub-task Reporter: Allan Yang Assignee: Allan Yang Fix For: 3.0.0, 2.2.0 This is for ThriftTable/Admin to handle exceptions like connection loss. And only available for http thrift client. For client using TSocket, it is not so easy to implement a retry client, maybe later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[
https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755700#comment-16755700
]
Sean Busbey commented on HBASE-21808:
-
another example for the search engines:
{code}
[busbey@busbey-centos76 hbase]$ mvn -version
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3;
2018-10-24T11:41:47-07:00)
Maven home: /usr/local/share/applications/apache-maven-3.6.0
Java version: 11.0.1, vendor: Oracle Corporation, runtime:
/usr/lib/jvm/java-11-openjdk-11.0.1.13-3.el7_6.x86_64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-957.el7.x86_64", arch: "amd64", family:
"unix"
[busbey@busbey-centos76 hbase]$ mvn -DskipTests --batch-mode install
[INFO] Scanning for projects...
...
[INFO] -
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR]
/home/busbey/projects/hbase/hbase-it/src/test/java/org/apache/hadoop/hbase/RESTApiClusterManager.java:[39,25]
package javax.xml.ws.http does not exist
[ERROR]
/home/busbey/projects/hbase/hbase-it/src/test/java/org/apache/hadoop/hbase/RESTApiClusterManager.java:[204,17]
cannot find symbol
symbol: class HTTPException
location: class org.apache.hadoop.hbase.RESTApiClusterManager
[ERROR]
/home/busbey/projects/hbase/hbase-it/src/test/java/org/apache/hadoop/hbase/RESTApiClusterManager.java:[247,17]
cannot find symbol
symbol: class HTTPException
location: class org.apache.hadoop.hbase.RESTApiClusterManager
[INFO] 3 errors
...
{code}
> Ensure we can build with JDK11 targetting JDK8
> --
>
> Key: HBASE-21808
> URL: https://issues.apache.org/jira/browse/HBASE-21808
> Project: HBase
> Issue Type: Improvement
> Components: build, community, java
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> When folks "install a JDK" wether they get JDK11 now will depend on the
> platform (e.g. from homebrew you will get jdk11). For new contributors this
> results in confusing errors on first run.
> Make it so that a non-release build using JDK11 and using our default
> compiler source/target versions of JDK8 can complete.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[
https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755696#comment-16755696
]
Sean Busbey commented on HBASE-21808:
-
example error for the search engines:
{code}[busbey@busbey-centos76 hbase]$ mvn -version
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3;
2018-10-24T11:41:47-07:00)
Maven home: /usr/local/share/applications/apache-maven-3.6.0
Java version: 11.0.1, vendor: Oracle Corporation, runtime:
/usr/lib/jvm/java-11-openjdk-11.0.1.13-3.el7_6.x86_64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-957.el7.x86_64", arch: "amd64", family:
"unix"
[busbey@busbey-centos76 hbase]$ mvn -DskipTests --batch-mode install
[INFO] Scanning for projects...
...
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR]
/home/busbey/projects/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java/org/apache/hadoop/hbase/shaded/protobuf/generated/SnapshotProtos.java:[6,18]
package javax.annotation does not exist
[ERROR]
/home/busbey/projects/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java/org/apache/hadoop/hbase/shaded/protobuf/generated/SnapshotProtos.java:[123,20]
package javax.annotation does not exist
[ERROR]
/home/busbey/projects/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java/org/apache/hadoop/hbase/shaded/protobuf/generated/AccessControlProtos.java:[6,18]
package javax.annotation does not exist
... (repeat for like 100 lines)
{code}
> Ensure we can build with JDK11 targetting JDK8
> --
>
> Key: HBASE-21808
> URL: https://issues.apache.org/jira/browse/HBASE-21808
> Project: HBase
> Issue Type: Improvement
> Components: build, community, java
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> When folks "install a JDK" wether they get JDK11 now will depend on the
> platform (e.g. from homebrew you will get jdk11). For new contributors this
> results in confusing errors on first run.
> Make it so that a non-release build using JDK11 and using our default
> compiler source/target versions of JDK8 can complete.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755681#comment-16755681
]
Reid Chan commented on HBASE-21481:
---
v10 rebased master branch and fixed conflicts introduced by HBASE-21739.
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21481.master.001.patch,
> HBASE-21481.master.002.patch, HBASE-21481.master.003.patch,
> HBASE-21481.master.004.patch, HBASE-21481.master.005.patch,
> HBASE-21481.master.006.patch, HBASE-21481.master.007.patch,
> HBASE-21481.master.008.patch, HBASE-21481.master.009.patch,
> HBASE-21481.master.010.patch
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-21481:
--
Attachment: HBASE-21481.master.010.patch
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21481.master.001.patch,
> HBASE-21481.master.002.patch, HBASE-21481.master.003.patch,
> HBASE-21481.master.004.patch, HBASE-21481.master.005.patch,
> HBASE-21481.master.006.patch, HBASE-21481.master.007.patch,
> HBASE-21481.master.008.patch, HBASE-21481.master.009.patch,
> HBASE-21481.master.010.patch
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21806) add an option to roll WAL on very slow syncs
[
https://issues.apache.org/jira/browse/HBASE-21806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755676#comment-16755676
]
Hadoop QA commented on HBASE-21806:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 5m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
57s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 11s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}251m 51s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}300m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas |
| | hadoop.hbase.master.TestAssignmentManagerMetrics |
| | hadoop.hbase.client.TestAdmin1 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21806 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956806/HBASE-21806.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 25fcf1232e58 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d82c1a6c2b |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| unit |
[jira] [Work started] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[ https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-21808 started by Sean Busbey. --- > Ensure we can build with JDK11 targetting JDK8 > -- > > Key: HBASE-21808 > URL: https://issues.apache.org/jira/browse/HBASE-21808 > Project: HBase > Issue Type: Improvement > Components: build, community, java >Affects Versions: 3.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > When folks "install a JDK" they're likely to get JDK11 now (e.g. from > homebrew, in RHEL/CentOS 7.6). For new contributors this results in confusing > errors on first run. > Make it so that a non-release build using JDK11 and using our default > compiler source/target versions of JDK8 can complete. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[ https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21808: Description: When folks "install a JDK" wether they get JDK11 now will depend on the platform (e.g. from homebrew you will get jdk11). For new contributors this results in confusing errors on first run. Make it so that a non-release build using JDK11 and using our default compiler source/target versions of JDK8 can complete. was: When folks "install a JDK" they're likely to get JDK11 now (e.g. from homebrew). For new contributors this results in confusing errors on first run. Make it so that a non-release build using JDK11 and using our default compiler source/target versions of JDK8 can complete. > Ensure we can build with JDK11 targetting JDK8 > -- > > Key: HBASE-21808 > URL: https://issues.apache.org/jira/browse/HBASE-21808 > Project: HBase > Issue Type: Improvement > Components: build, community, java >Affects Versions: 3.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > When folks "install a JDK" wether they get JDK11 now will depend on the > platform (e.g. from homebrew you will get jdk11). For new contributors this > results in confusing errors on first run. > Make it so that a non-release build using JDK11 and using our default > compiler source/target versions of JDK8 can complete. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[
https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755673#comment-16755673
]
Sean Busbey commented on HBASE-21808:
-
on RHEL7.6 {{yum install java-devel}} currently still gets 1.8u191. to get
jdk11 you have expressly ask via e.g. {{yum install java-11-openjdk-devel}}.
> Ensure we can build with JDK11 targetting JDK8
> --
>
> Key: HBASE-21808
> URL: https://issues.apache.org/jira/browse/HBASE-21808
> Project: HBase
> Issue Type: Improvement
> Components: build, community, java
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> When folks "install a JDK" they're likely to get JDK11 now (e.g. from
> homebrew). For new contributors this results in confusing errors on first run.
> Make it so that a non-release build using JDK11 and using our default
> compiler source/target versions of JDK8 can complete.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[ https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755670#comment-16755670 ] Sean Busbey commented on HBASE-21808: - Edited the description to remove RHEL7.6, as my retesting of assumptions got me jdk1.8 > Ensure we can build with JDK11 targetting JDK8 > -- > > Key: HBASE-21808 > URL: https://issues.apache.org/jira/browse/HBASE-21808 > Project: HBase > Issue Type: Improvement > Components: build, community, java >Affects Versions: 3.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > When folks "install a JDK" they're likely to get JDK11 now (e.g. from > homebrew). For new contributors this results in confusing errors on first run. > Make it so that a non-release build using JDK11 and using our default > compiler source/target versions of JDK8 can complete. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[ https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21808: Description: When folks "install a JDK" they're likely to get JDK11 now (e.g. from homebrew). For new contributors this results in confusing errors on first run. Make it so that a non-release build using JDK11 and using our default compiler source/target versions of JDK8 can complete. was: When folks "install a JDK" they're likely to get JDK11 now (e.g. from homebrew, in RHEL/CentOS 7.6). For new contributors this results in confusing errors on first run. Make it so that a non-release build using JDK11 and using our default compiler source/target versions of JDK8 can complete. > Ensure we can build with JDK11 targetting JDK8 > -- > > Key: HBASE-21808 > URL: https://issues.apache.org/jira/browse/HBASE-21808 > Project: HBase > Issue Type: Improvement > Components: build, community, java >Affects Versions: 3.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > When folks "install a JDK" they're likely to get JDK11 now (e.g. from > homebrew). For new contributors this results in confusing errors on first run. > Make it so that a non-release build using JDK11 and using our default > compiler source/target versions of JDK8 can complete. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
Sean Busbey created HBASE-21808: --- Summary: Ensure we can build with JDK11 targetting JDK8 Key: HBASE-21808 URL: https://issues.apache.org/jira/browse/HBASE-21808 Project: HBase Issue Type: Improvement Components: build, community, java Affects Versions: 3.0.0 Reporter: Sean Busbey Assignee: Sean Busbey When folks "install a JDK" they're likely to get JDK11 now (e.g. from homebrew, in RHEL/CentOS 7.6). For new contributors this results in confusing errors on first run. Make it so that a non-release build using JDK11 and using our default compiler source/target versions of JDK8 can complete. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21699) Create table failed when using SPLITS_FILE => 'splits.txt'
[
https://issues.apache.org/jira/browse/HBASE-21699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-21699:
---
Attachment: HBASE-21699.master.006.patch
> Create table failed when using SPLITS_FILE => 'splits.txt'
> ---
>
> Key: HBASE-21699
> URL: https://issues.apache.org/jira/browse/HBASE-21699
> Project: HBase
> Issue Type: Bug
> Components: Client, shell
>Affects Versions: 2.0.0, 2.0.1, 2.1.1, 2.0.2, 2.0.3, 2.1.2, 2.0.4
>Reporter: huan
>Priority: Blocker
> Attachments: HBASE-21699.master.006.patch, HBase-21699.v2.patch,
> HBase-21699.v3.patch, HBase-21699.v4.patch, HBase-21699.v5.patch,
> hbase-21699.001.patch
>
>
> Hi all:
> When I ran
> {code:java}
> create 't1', 'f1', SPLITS_FILE => 'splits.txt'
> {code}
> on HBase2.0.0, it failed, and no detailed error info, just like below:
> {code:java}
> ERROR:
> Creates a table. Pass a table name, and a set of column family
> specifications (at least one), and, optionally, table configuration.
> Column specification can be a simple string (name), or a dictionary
> (dictionaries are described below in main help output), necessarily
> including NAME attribute.
> Examples:
> {code}
> So I opened the debug:
> {code:java}
> hbase shell -d
> {code}
> and
> {code:java}
> ERROR:
> Backtrace:
> org.apache.hadoop.hbase.util.Bytes.toBytes(org/apache/hadoop/hbase/util/Bytes.java:732)
> org.apache.hadoop.hbase.HTableDescriptor.setValue(org/apache/hadoop/hbase/HTableDescriptor.java:190){code}
> But it works on branch 1.2.0.
> so I view the source code, I find the issue is because the below code:
> {code:java}
> // admin.rb
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE])
> end
> {code}
> {code:java}
> // HTableDescriptor part
> public HTableDescriptor setValue(String key, String value) {
> getDelegateeForModification().setValue(Bytes.toBytes(key),
> Bytes.toBytes(value));
> return this;
> }
> {code}
> {code:java}
> // Bytes part
> public static byte[] toBytes(String s) {
> try {
> return s.getBytes(UTF8_CSN);
> } catch (UnsupportedEncodingException e) {
> // should never happen!
> throw new IllegalArgumentException("UTF8 decoding is not supported", e);
> }
> }
> {code}
> Call flow is:
> {code:java}
> admin.rb ---> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE]) --->
> Bytes.toBytes(key) && Bytes.toBytes(value) {code}
> from Bytes.toBytes, if s is null, the function will throw
> NullPointerException, but HTableDescriptor.setValue(String key, String value)
> does not check key and value.
> in admin.rb, it use arg.delete(SPLITS_FILE) to get the value, but this means,
> after using arg.delete(SPLITS_FILE), arg[SPLITS_FILE] will return nil. so
> HTableDescriptor.setValue(String key, String value) does not check key and
> value is root cause.
> why branch below 2.0.0 works fine, because old code is :
> {code:java}
> public HTableDescriptor setValue(String key, String value) {
> if (value == null) {
> remove(key);
> } else {
> setValue(Bytes.toBytes(key), Bytes.toBytes(value));
> }
> return this;
> }
> {code}
> it check the value.
> since branch 2.0.0, HBase add new function called 'TableDescriptorBuilder'
> it included:
> {code:java}
> public ModifyableTableDescriptor setValue(String key, String value) {
> return setValue(toBytesOrNull(key, Bytes::toBytes),
> toBytesOrNull(value, Bytes::toBytes));
> }
> {code}
> it checked key and value, but HTableDescriptor.setValue(String key, String
> value) does not call it, so
> just change HTableDescriptor.setValue(String key, String value) to call it,
> it will works
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21699) Create table failed when using SPLITS_FILE => 'splits.txt'
[
https://issues.apache.org/jira/browse/HBASE-21699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-21699:
---
Fix Version/s: 2.3.0
2.0.5
2.1.3
2.2.0
3.0.0
> Create table failed when using SPLITS_FILE => 'splits.txt'
> ---
>
> Key: HBASE-21699
> URL: https://issues.apache.org/jira/browse/HBASE-21699
> Project: HBase
> Issue Type: Bug
> Components: Client, shell
>Affects Versions: 2.0.0, 2.0.1, 2.1.1, 2.0.2, 2.0.3, 2.1.2, 2.0.4
>Reporter: huan
>Assignee: huan
>Priority: Blocker
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0
>
> Attachments: HBASE-21699.master.006.patch, HBase-21699.v2.patch,
> HBase-21699.v3.patch, HBase-21699.v4.patch, HBase-21699.v5.patch,
> hbase-21699.001.patch
>
>
> Hi all:
> When I ran
> {code:java}
> create 't1', 'f1', SPLITS_FILE => 'splits.txt'
> {code}
> on HBase2.0.0, it failed, and no detailed error info, just like below:
> {code:java}
> ERROR:
> Creates a table. Pass a table name, and a set of column family
> specifications (at least one), and, optionally, table configuration.
> Column specification can be a simple string (name), or a dictionary
> (dictionaries are described below in main help output), necessarily
> including NAME attribute.
> Examples:
> {code}
> So I opened the debug:
> {code:java}
> hbase shell -d
> {code}
> and
> {code:java}
> ERROR:
> Backtrace:
> org.apache.hadoop.hbase.util.Bytes.toBytes(org/apache/hadoop/hbase/util/Bytes.java:732)
> org.apache.hadoop.hbase.HTableDescriptor.setValue(org/apache/hadoop/hbase/HTableDescriptor.java:190){code}
> But it works on branch 1.2.0.
> so I view the source code, I find the issue is because the below code:
> {code:java}
> // admin.rb
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE])
> end
> {code}
> {code:java}
> // HTableDescriptor part
> public HTableDescriptor setValue(String key, String value) {
> getDelegateeForModification().setValue(Bytes.toBytes(key),
> Bytes.toBytes(value));
> return this;
> }
> {code}
> {code:java}
> // Bytes part
> public static byte[] toBytes(String s) {
> try {
> return s.getBytes(UTF8_CSN);
> } catch (UnsupportedEncodingException e) {
> // should never happen!
> throw new IllegalArgumentException("UTF8 decoding is not supported", e);
> }
> }
> {code}
> Call flow is:
> {code:java}
> admin.rb ---> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE]) --->
> Bytes.toBytes(key) && Bytes.toBytes(value) {code}
> from Bytes.toBytes, if s is null, the function will throw
> NullPointerException, but HTableDescriptor.setValue(String key, String value)
> does not check key and value.
> in admin.rb, it use arg.delete(SPLITS_FILE) to get the value, but this means,
> after using arg.delete(SPLITS_FILE), arg[SPLITS_FILE] will return nil. so
> HTableDescriptor.setValue(String key, String value) does not check key and
> value is root cause.
> why branch below 2.0.0 works fine, because old code is :
> {code:java}
> public HTableDescriptor setValue(String key, String value) {
> if (value == null) {
> remove(key);
> } else {
> setValue(Bytes.toBytes(key), Bytes.toBytes(value));
> }
> return this;
> }
> {code}
> it check the value.
> since branch 2.0.0, HBase add new function called 'TableDescriptorBuilder'
> it included:
> {code:java}
> public ModifyableTableDescriptor setValue(String key, String value) {
> return setValue(toBytesOrNull(key, Bytes::toBytes),
> toBytesOrNull(value, Bytes::toBytes));
> }
> {code}
> it checked key and value, but HTableDescriptor.setValue(String key, String
> value) does not call it, so
> just change HTableDescriptor.setValue(String key, String value) to call it,
> it will works
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21699) Create table failed when using SPLITS_FILE => 'splits.txt'
[
https://issues.apache.org/jira/browse/HBASE-21699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755651#comment-16755651
]
Guanghao Zhang commented on HBASE-21699:
Add a one line change to trigger hbase-server HADOOP QA ut.
> Create table failed when using SPLITS_FILE => 'splits.txt'
> ---
>
> Key: HBASE-21699
> URL: https://issues.apache.org/jira/browse/HBASE-21699
> Project: HBase
> Issue Type: Bug
> Components: Client, shell
>Affects Versions: 2.0.0, 2.0.1, 2.1.1, 2.0.2, 2.0.3, 2.1.2, 2.0.4
>Reporter: huan
>Priority: Blocker
> Attachments: HBASE-21699.master.006.patch, HBase-21699.v2.patch,
> HBase-21699.v3.patch, HBase-21699.v4.patch, HBase-21699.v5.patch,
> hbase-21699.001.patch
>
>
> Hi all:
> When I ran
> {code:java}
> create 't1', 'f1', SPLITS_FILE => 'splits.txt'
> {code}
> on HBase2.0.0, it failed, and no detailed error info, just like below:
> {code:java}
> ERROR:
> Creates a table. Pass a table name, and a set of column family
> specifications (at least one), and, optionally, table configuration.
> Column specification can be a simple string (name), or a dictionary
> (dictionaries are described below in main help output), necessarily
> including NAME attribute.
> Examples:
> {code}
> So I opened the debug:
> {code:java}
> hbase shell -d
> {code}
> and
> {code:java}
> ERROR:
> Backtrace:
> org.apache.hadoop.hbase.util.Bytes.toBytes(org/apache/hadoop/hbase/util/Bytes.java:732)
> org.apache.hadoop.hbase.HTableDescriptor.setValue(org/apache/hadoop/hbase/HTableDescriptor.java:190){code}
> But it works on branch 1.2.0.
> so I view the source code, I find the issue is because the below code:
> {code:java}
> // admin.rb
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE])
> end
> {code}
> {code:java}
> // HTableDescriptor part
> public HTableDescriptor setValue(String key, String value) {
> getDelegateeForModification().setValue(Bytes.toBytes(key),
> Bytes.toBytes(value));
> return this;
> }
> {code}
> {code:java}
> // Bytes part
> public static byte[] toBytes(String s) {
> try {
> return s.getBytes(UTF8_CSN);
> } catch (UnsupportedEncodingException e) {
> // should never happen!
> throw new IllegalArgumentException("UTF8 decoding is not supported", e);
> }
> }
> {code}
> Call flow is:
> {code:java}
> admin.rb ---> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE]) --->
> Bytes.toBytes(key) && Bytes.toBytes(value) {code}
> from Bytes.toBytes, if s is null, the function will throw
> NullPointerException, but HTableDescriptor.setValue(String key, String value)
> does not check key and value.
> in admin.rb, it use arg.delete(SPLITS_FILE) to get the value, but this means,
> after using arg.delete(SPLITS_FILE), arg[SPLITS_FILE] will return nil. so
> HTableDescriptor.setValue(String key, String value) does not check key and
> value is root cause.
> why branch below 2.0.0 works fine, because old code is :
> {code:java}
> public HTableDescriptor setValue(String key, String value) {
> if (value == null) {
> remove(key);
> } else {
> setValue(Bytes.toBytes(key), Bytes.toBytes(value));
> }
> return this;
> }
> {code}
> it check the value.
> since branch 2.0.0, HBase add new function called 'TableDescriptorBuilder'
> it included:
> {code:java}
> public ModifyableTableDescriptor setValue(String key, String value) {
> return setValue(toBytesOrNull(key, Bytes::toBytes),
> toBytesOrNull(value, Bytes::toBytes));
> }
> {code}
> it checked key and value, but HTableDescriptor.setValue(String key, String
> value) does not call it, so
> just change HTableDescriptor.setValue(String key, String value) to call it,
> it will works
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-21699) Create table failed when using SPLITS_FILE => 'splits.txt'
[
https://issues.apache.org/jira/browse/HBASE-21699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang reassigned HBASE-21699:
--
Assignee: huan
> Create table failed when using SPLITS_FILE => 'splits.txt'
> ---
>
> Key: HBASE-21699
> URL: https://issues.apache.org/jira/browse/HBASE-21699
> Project: HBase
> Issue Type: Bug
> Components: Client, shell
>Affects Versions: 2.0.0, 2.0.1, 2.1.1, 2.0.2, 2.0.3, 2.1.2, 2.0.4
>Reporter: huan
>Assignee: huan
>Priority: Blocker
> Attachments: HBASE-21699.master.006.patch, HBase-21699.v2.patch,
> HBase-21699.v3.patch, HBase-21699.v4.patch, HBase-21699.v5.patch,
> hbase-21699.001.patch
>
>
> Hi all:
> When I ran
> {code:java}
> create 't1', 'f1', SPLITS_FILE => 'splits.txt'
> {code}
> on HBase2.0.0, it failed, and no detailed error info, just like below:
> {code:java}
> ERROR:
> Creates a table. Pass a table name, and a set of column family
> specifications (at least one), and, optionally, table configuration.
> Column specification can be a simple string (name), or a dictionary
> (dictionaries are described below in main help output), necessarily
> including NAME attribute.
> Examples:
> {code}
> So I opened the debug:
> {code:java}
> hbase shell -d
> {code}
> and
> {code:java}
> ERROR:
> Backtrace:
> org.apache.hadoop.hbase.util.Bytes.toBytes(org/apache/hadoop/hbase/util/Bytes.java:732)
> org.apache.hadoop.hbase.HTableDescriptor.setValue(org/apache/hadoop/hbase/HTableDescriptor.java:190){code}
> But it works on branch 1.2.0.
> so I view the source code, I find the issue is because the below code:
> {code:java}
> // admin.rb
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE])
> end
> {code}
> {code:java}
> // HTableDescriptor part
> public HTableDescriptor setValue(String key, String value) {
> getDelegateeForModification().setValue(Bytes.toBytes(key),
> Bytes.toBytes(value));
> return this;
> }
> {code}
> {code:java}
> // Bytes part
> public static byte[] toBytes(String s) {
> try {
> return s.getBytes(UTF8_CSN);
> } catch (UnsupportedEncodingException e) {
> // should never happen!
> throw new IllegalArgumentException("UTF8 decoding is not supported", e);
> }
> }
> {code}
> Call flow is:
> {code:java}
> admin.rb ---> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE]) --->
> Bytes.toBytes(key) && Bytes.toBytes(value) {code}
> from Bytes.toBytes, if s is null, the function will throw
> NullPointerException, but HTableDescriptor.setValue(String key, String value)
> does not check key and value.
> in admin.rb, it use arg.delete(SPLITS_FILE) to get the value, but this means,
> after using arg.delete(SPLITS_FILE), arg[SPLITS_FILE] will return nil. so
> HTableDescriptor.setValue(String key, String value) does not check key and
> value is root cause.
> why branch below 2.0.0 works fine, because old code is :
> {code:java}
> public HTableDescriptor setValue(String key, String value) {
> if (value == null) {
> remove(key);
> } else {
> setValue(Bytes.toBytes(key), Bytes.toBytes(value));
> }
> return this;
> }
> {code}
> it check the value.
> since branch 2.0.0, HBase add new function called 'TableDescriptorBuilder'
> it included:
> {code:java}
> public ModifyableTableDescriptor setValue(String key, String value) {
> return setValue(toBytesOrNull(key, Bytes::toBytes),
> toBytesOrNull(value, Bytes::toBytes));
> }
> {code}
> it checked key and value, but HTableDescriptor.setValue(String key, String
> value) does not call it, so
> just change HTableDescriptor.setValue(String key, String value) to call it,
> it will works
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21739) Move grant/revoke from regionserver to master
[ https://issues.apache.org/jira/browse/HBASE-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755637#comment-16755637 ] Guanghao Zhang commented on HBASE-21739: [~Yi Mei] And add a patch for branch-2? There are a lot confilts when apply to branch-2... > Move grant/revoke from regionserver to master > - > > Key: HBASE-21739 > URL: https://issues.apache.org/jira/browse/HBASE-21739 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21739.master.001.patch, > HBASE-21739.master.002.patch, HBASE-21739.master.003.patch, > HBASE-21739.master.003.patch, HBASE-21739.master.004.patch, > HBASE-21739.master.005.patch > > > Create a sub-task to move grant/revoke from regionserver to master. Other > access control operations(getUserPermissions/ checkPermissions/ > hasPermission) will be moved in another sub-task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15320) HBase connector for Kafka Connect
[ https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15320: Resolution: Fixed Fix Version/s: (was: 3.0.0) connector-1.0.0 Status: Resolved (was: Patch Available) > HBase connector for Kafka Connect > - > > Key: HBASE-15320 > URL: https://issues.apache.org/jira/browse/HBASE-15320 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Andrew Purtell >Assignee: Mike Wingert >Priority: Major > Labels: beginner > Fix For: connector-1.0.0 > > Attachments: 15320.master.16.patch, 15320.master.16.patch, > HBASE-15320.master.1.patch, HBASE-15320.master.10.patch, > HBASE-15320.master.11.patch, HBASE-15320.master.12.patch, > HBASE-15320.master.14.patch, HBASE-15320.master.15.patch, > HBASE-15320.master.2.patch, HBASE-15320.master.3.patch, > HBASE-15320.master.4.patch, HBASE-15320.master.5.patch, > HBASE-15320.master.6.patch, HBASE-15320.master.7.patch, > HBASE-15320.master.8.patch, HBASE-15320.master.8.patch, > HBASE-15320.master.9.patch, HBASE-15320.pdf, HBASE-15320.pdf > > > Implement an HBase connector with source and sink tasks for the Connect > framework (http://docs.confluent.io/2.0.0/connect/index.html) available in > Kafka 0.9 and later. > See also: > http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines > An HBase source > (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task) > could be implemented as a replication endpoint or WALObserver, publishing > cluster wide change streams from the WAL to one or more topics, with > configurable mapping and partitioning of table changes to topics. > An HBase sink task > (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would > persist, with optional transformation (JSON? Avro?, map fields to native > schema?), Kafka SinkRecords into HBase tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-15320) HBase connector for Kafka Connect
[ https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755636#comment-16755636 ] Sean Busbey commented on HBASE-15320: - good catch [~MikeWingert] > HBase connector for Kafka Connect > - > > Key: HBASE-15320 > URL: https://issues.apache.org/jira/browse/HBASE-15320 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Andrew Purtell >Assignee: Mike Wingert >Priority: Major > Labels: beginner > Fix For: 3.0.0 > > Attachments: 15320.master.16.patch, 15320.master.16.patch, > HBASE-15320.master.1.patch, HBASE-15320.master.10.patch, > HBASE-15320.master.11.patch, HBASE-15320.master.12.patch, > HBASE-15320.master.14.patch, HBASE-15320.master.15.patch, > HBASE-15320.master.2.patch, HBASE-15320.master.3.patch, > HBASE-15320.master.4.patch, HBASE-15320.master.5.patch, > HBASE-15320.master.6.patch, HBASE-15320.master.7.patch, > HBASE-15320.master.8.patch, HBASE-15320.master.8.patch, > HBASE-15320.master.9.patch, HBASE-15320.pdf, HBASE-15320.pdf > > > Implement an HBase connector with source and sink tasks for the Connect > framework (http://docs.confluent.io/2.0.0/connect/index.html) available in > Kafka 0.9 and later. > See also: > http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines > An HBase source > (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task) > could be implemented as a replication endpoint or WALObserver, publishing > cluster wide change streams from the WAL to one or more topics, with > configurable mapping and partitioning of table changes to topics. > An HBase sink task > (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would > persist, with optional transformation (JSON? Avro?, map fields to native > schema?), Kafka SinkRecords into HBase tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21739) Move grant/revoke from regionserver to master
[ https://issues.apache.org/jira/browse/HBASE-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755631#comment-16755631 ] Guanghao Zhang commented on HBASE-21739: [~Yi Mei] Please add a release note. Thanks. > Move grant/revoke from regionserver to master > - > > Key: HBASE-21739 > URL: https://issues.apache.org/jira/browse/HBASE-21739 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21739.master.001.patch, > HBASE-21739.master.002.patch, HBASE-21739.master.003.patch, > HBASE-21739.master.003.patch, HBASE-21739.master.004.patch, > HBASE-21739.master.005.patch > > > Create a sub-task to move grant/revoke from regionserver to master. Other > access control operations(getUserPermissions/ checkPermissions/ > hasPermission) will be moved in another sub-task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21739) Move grant/revoke from regionserver to master
[ https://issues.apache.org/jira/browse/HBASE-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755628#comment-16755628 ] Guanghao Zhang commented on HBASE-21739: +1 > Move grant/revoke from regionserver to master > - > > Key: HBASE-21739 > URL: https://issues.apache.org/jira/browse/HBASE-21739 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21739.master.001.patch, > HBASE-21739.master.002.patch, HBASE-21739.master.003.patch, > HBASE-21739.master.003.patch, HBASE-21739.master.004.patch, > HBASE-21739.master.005.patch > > > Create a sub-task to move grant/revoke from regionserver to master. Other > access control operations(getUserPermissions/ checkPermissions/ > hasPermission) will be moved in another sub-task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21739) Move grant/revoke from regionserver to master
[ https://issues.apache.org/jira/browse/HBASE-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21739: --- Affects Version/s: 2.3.0 2.2.0 3.0.0 > Move grant/revoke from regionserver to master > - > > Key: HBASE-21739 > URL: https://issues.apache.org/jira/browse/HBASE-21739 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21739.master.001.patch, > HBASE-21739.master.002.patch, HBASE-21739.master.003.patch, > HBASE-21739.master.003.patch, HBASE-21739.master.004.patch, > HBASE-21739.master.005.patch > > > Create a sub-task to move grant/revoke from regionserver to master. Other > access control operations(getUserPermissions/ checkPermissions/ > hasPermission) will be moved in another sub-task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21791) Upgrade thrift dependency to 0.12.0
[
https://issues.apache.org/jira/browse/HBASE-21791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755626#comment-16755626
]
Hudson commented on HBASE-21791:
Results for branch master
[build #756 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/756/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/756//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/756//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/756//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Upgrade thrift dependency to 0.12.0
> ---
>
> Key: HBASE-21791
> URL: https://issues.apache.org/jira/browse/HBASE-21791
> Project: HBase
> Issue Type: Task
> Components: Thrift
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 1.2.10, 2.0.4
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0
>
> Attachments: HBASE-21791-branch-1.patch,
> HBASE-21791-branch-2.1.patch, HBASE-21791.patch
>
>
> As somebody have already known, that there is a CVE for thrift from 0.5.0 to
> 0.11.0.
> https://nvd.nist.gov/vuln/detail/CVE-2018-1320
> As the CVE is already public, let's upgrade our thrift dependency and release
> new versions ASAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21794) Update the Coprocessor observer example given in section 111.1 of the ref guide.
[
https://issues.apache.org/jira/browse/HBASE-21794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755619#comment-16755619
]
Hadoop QA commented on HBASE-21794:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
27s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
5s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 4m
59s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21794 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956825/hbase-21794.master.001.patch
|
| Optional Tests | dupname asflicense refguide |
| uname | Linux 2908e0a56ec8 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d82c1a6c2b |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/15778/artifact/patchprocess/branch-site/book.html
|
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/15778/artifact/patchprocess/whitespace-eol.txt
|
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/15778/artifact/patchprocess/patch-site/book.html
|
| Max. process+thread count | 77 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15778/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Update the Coprocessor observer example given in section 111.1 of the ref
> guide.
>
>
> Key: HBASE-21794
> URL: https://issues.apache.org/jira/browse/HBASE-21794
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, documentation
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Minor
> Attachments: hbase-21794.master.001.patch
>
>
> The given example should be changed after the CP changes (HBASE-17732)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21794) Update the Coprocessor observer example given in section 111.1 of the ref guide.
[ https://issues.apache.org/jira/browse/HBASE-21794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi updated HBASE-21794: --- Status: Patch Available (was: In Progress) > Update the Coprocessor observer example given in section 111.1 of the ref > guide. > > > Key: HBASE-21794 > URL: https://issues.apache.org/jira/browse/HBASE-21794 > Project: HBase > Issue Type: Bug > Components: Coprocessors, documentation >Reporter: Sakthi >Assignee: Sakthi >Priority: Minor > Attachments: hbase-21794.master.001.patch > > > The given example should be changed after the CP changes (HBASE-17732) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21794) Update the Coprocessor observer example given in section 111.1 of the ref guide.
[ https://issues.apache.org/jira/browse/HBASE-21794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi updated HBASE-21794: --- Attachment: hbase-21794.master.001.patch > Update the Coprocessor observer example given in section 111.1 of the ref > guide. > > > Key: HBASE-21794 > URL: https://issues.apache.org/jira/browse/HBASE-21794 > Project: HBase > Issue Type: Bug > Components: Coprocessors, documentation >Reporter: Sakthi >Assignee: Sakthi >Priority: Minor > Attachments: hbase-21794.master.001.patch > > > The given example should be changed after the CP changes (HBASE-17732) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21791) Upgrade thrift dependency to 0.12.0
[
https://issues.apache.org/jira/browse/HBASE-21791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755588#comment-16755588
]
Hudson commented on HBASE-21791:
Results for branch branch-2.1
[build #813 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/813/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/813//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/813//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/813//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Upgrade thrift dependency to 0.12.0
> ---
>
> Key: HBASE-21791
> URL: https://issues.apache.org/jira/browse/HBASE-21791
> Project: HBase
> Issue Type: Task
> Components: Thrift
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 1.2.10, 2.0.4
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0
>
> Attachments: HBASE-21791-branch-1.patch,
> HBASE-21791-branch-2.1.patch, HBASE-21791.patch
>
>
> As somebody have already known, that there is a CVE for thrift from 0.5.0 to
> 0.11.0.
> https://nvd.nist.gov/vuln/detail/CVE-2018-1320
> As the CVE is already public, let's upgrade our thrift dependency and release
> new versions ASAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21791) Upgrade thrift dependency to 0.12.0
[
https://issues.apache.org/jira/browse/HBASE-21791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755574#comment-16755574
]
Hudson commented on HBASE-21791:
Results for branch branch-1
[build #659 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/659/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/659//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/659//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/659//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Upgrade thrift dependency to 0.12.0
> ---
>
> Key: HBASE-21791
> URL: https://issues.apache.org/jira/browse/HBASE-21791
> Project: HBase
> Issue Type: Task
> Components: Thrift
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 1.2.10, 2.0.4
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0
>
> Attachments: HBASE-21791-branch-1.patch,
> HBASE-21791-branch-2.1.patch, HBASE-21791.patch
>
>
> As somebody have already known, that there is a CVE for thrift from 0.5.0 to
> 0.11.0.
> https://nvd.nist.gov/vuln/detail/CVE-2018-1320
> As the CVE is already public, let's upgrade our thrift dependency and release
> new versions ASAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21791) Upgrade thrift dependency to 0.12.0
[
https://issues.apache.org/jira/browse/HBASE-21791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755580#comment-16755580
]
Hudson commented on HBASE-21791:
Results for branch branch-2.0
[build #1297 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1297/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1297//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1297//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1297//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Upgrade thrift dependency to 0.12.0
> ---
>
> Key: HBASE-21791
> URL: https://issues.apache.org/jira/browse/HBASE-21791
> Project: HBase
> Issue Type: Task
> Components: Thrift
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 1.2.10, 2.0.4
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0
>
> Attachments: HBASE-21791-branch-1.patch,
> HBASE-21791-branch-2.1.patch, HBASE-21791.patch
>
>
> As somebody have already known, that there is a CVE for thrift from 0.5.0 to
> 0.11.0.
> https://nvd.nist.gov/vuln/detail/CVE-2018-1320
> As the CVE is already public, let's upgrade our thrift dependency and release
> new versions ASAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-15320) HBase connector for Kafka Connect
[ https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755578#comment-16755578 ] Mike Wingert commented on HBASE-15320: -- [~stack]: should we resolve this since it's part of hbase-connectors? > HBase connector for Kafka Connect > - > > Key: HBASE-15320 > URL: https://issues.apache.org/jira/browse/HBASE-15320 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Andrew Purtell >Assignee: Mike Wingert >Priority: Major > Labels: beginner > Fix For: 3.0.0 > > Attachments: 15320.master.16.patch, 15320.master.16.patch, > HBASE-15320.master.1.patch, HBASE-15320.master.10.patch, > HBASE-15320.master.11.patch, HBASE-15320.master.12.patch, > HBASE-15320.master.14.patch, HBASE-15320.master.15.patch, > HBASE-15320.master.2.patch, HBASE-15320.master.3.patch, > HBASE-15320.master.4.patch, HBASE-15320.master.5.patch, > HBASE-15320.master.6.patch, HBASE-15320.master.7.patch, > HBASE-15320.master.8.patch, HBASE-15320.master.8.patch, > HBASE-15320.master.9.patch, HBASE-15320.pdf, HBASE-15320.pdf > > > Implement an HBase connector with source and sink tasks for the Connect > framework (http://docs.confluent.io/2.0.0/connect/index.html) available in > Kafka 0.9 and later. > See also: > http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines > An HBase source > (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task) > could be implemented as a replication endpoint or WALObserver, publishing > cluster wide change streams from the WAL to one or more topics, with > configurable mapping and partitioning of table changes to topics. > An HBase sink task > (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would > persist, with optional transformation (JSON? Avro?, map fields to native > schema?), Kafka SinkRecords into HBase tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21807) Backport HBASE-21225 to branch-2.0 and branch-2.1
[
https://issues.apache.org/jira/browse/HBASE-21807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755573#comment-16755573
]
Hadoop QA commented on HBASE-21807:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
55s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
5s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
34s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
27s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
11s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
1s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 37s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
57s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}186m 44s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
52s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}235m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.util.TestFromClientSide3WoUnsafe |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f01af0 |
| JIRA Issue | HBASE-21807 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956795/hbase-21225.branch-2.0.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7ea73e88cebe 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.0 / 1f2241d714 |
| maven |
[jira] [Commented] (HBASE-21775) The BufferedMutator doesn't ever refresh region location cache
[
https://issues.apache.org/jira/browse/HBASE-21775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755571#comment-16755571
]
Duo Zhang commented on HBASE-21775:
---
On master branch if I revert the change here TestAsyncProcess can pass. And
this maybe a test issue, as I suppose you changed the retry behavior here, and
the failed UTs are all about how we do fail recovery, for example, testFail,
testErrorServers, etc.
> The BufferedMutator doesn't ever refresh region location cache
> --
>
> Key: HBASE-21775
> URL: https://issues.apache.org/jira/browse/HBASE-21775
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Tommy Li
>Assignee: Tommy Li
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.5, 1.3.4
>
> Attachments: HBASE-21775.master.001.patch,
> org.apache.hadoop.hbase.client.TestAsyncProcess-with-HBASE-21775.txt,
> org.apache.hadoop.hbase.client.TestAsyncProcess-without-HBASE-21775.txt
>
>
> {color:#22}I noticed in some of my writing jobs that the BufferedMutator
> would get stuck retrying writes against a dead server.{color}
> {code:java}
> 19/01/18 15:15:47 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:15:54 WARN [htable-pool3-t56] client.AsyncRequestFutureImpl:
> id=2, table=dummy_table, attempt=15/21, failureCount=1ops, last
> exception=org.apache.hadoop.hbase.DoNotRetryIOException: Operation rpcTimeout
> on ,17020,1547848193782, tracking started Fri Jan 18 14:55:37 PST
> 2019; NOT retrying, failed=1 -- final attempt!
> 19/01/18 15:15:54 ERROR [Executor task launch worker for task 0]
> IngestRawData.map(): [B@258bc2c7:
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: Operation rpcTimeout: 1 time, servers with issues:
> ,17020,1547848193782
> {code}
>
> After the single remaining action permanently failed, it would resume
> progress only to get stuck again retrying against the same dead server:
> {code:java}
> 19/01/18 15:21:18 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:21:18 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:21:20 INFO [htable-pool3-t55] client.AsyncRequestFutureImpl:
> id=2, table=dummy_table, attempt=6/21, failureCount=1ops, last
> exception=java.net.ConnectException: Call to failed on connection
> exception:
> org.apache.hbase.thirdparty.io.netty.channel.ConnectTimeoutException:
> connection timed out: on ,17020,1547848193782, tracking
> started null, retrying after=20089ms, operationsToReplay=1
> {code}
>
> Only restarting the client process to generate a new BufferedMutator instance
> would fix the issue, at least until the next regionserver crash
> The logs I've pasted show the issue happening with a
> ConnectionTimeoutException, but we've also seen it with
> NotServingRegionException and some others
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21806) add an option to roll WAL on very slow syncs
[ https://issues.apache.org/jira/browse/HBASE-21806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755568#comment-16755568 ] Sergey Shelukhin commented on HBASE-21806: -- The sync timeout is currently 5 minutes; I assume setting it too low might cause a lot of client failures on values that are high, but not ridiculously high... I was thinking we can just switch the WAL to avoid disruption on such intermediate values. > add an option to roll WAL on very slow syncs > > > Key: HBASE-21806 > URL: https://issues.apache.org/jira/browse/HBASE-21806 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HBASE-21806.patch > > > In large heterogeneous clusters sometimes a slow datanode can cause WAL syncs > to be very slow. In this case, before the bad datanode recovers, or is > discovered and repaired, it would be helpful to roll WAL on a very slow sync > to get a new pipeline. > Otherwise the slow WAL will impact write latency for a long time (slow writes > result in less writes result in the WAL not being rolled for longer) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21806) add an option to roll WAL on very slow syncs
[
https://issues.apache.org/jira/browse/HBASE-21806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755569#comment-16755569
]
Duo Zhang commented on HBASE-21806:
---
I think the default value is too high, even for OLAP clusters.
Skimmed the patch, overall LGTM. +1.
{code}
LOG.info("Trying to request a roll due to a very log sync ({} ms)", timeInNanos
/ 100);
{code}
This could be a typo? 'very log sync' => 'very long sync' or 'very long log
sync'?
> add an option to roll WAL on very slow syncs
>
>
> Key: HBASE-21806
> URL: https://issues.apache.org/jira/browse/HBASE-21806
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HBASE-21806.patch
>
>
> In large heterogeneous clusters sometimes a slow datanode can cause WAL syncs
> to be very slow. In this case, before the bad datanode recovers, or is
> discovered and repaired, it would be helpful to roll WAL on a very slow sync
> to get a new pipeline.
> Otherwise the slow WAL will impact write latency for a long time (slow writes
> result in less writes result in the WAL not being rolled for longer)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21806) add an option to roll WAL on very slow syncs
[ https://issues.apache.org/jira/browse/HBASE-21806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1679#comment-1679 ] Duo Zhang commented on HBASE-21806: --- One of my goal for introducing AsyncFSWAL is to set a smaller timeout when writing WAL, so if there are slow DNs we will fail to sync sonn and roll a new WAL... But anyway, the proposal here may still be useful, as for example, we can set the timeout to 5 seconds, and set a 2 seconds threshold, if we haven't got response after 5 seconds, the sync will fail, if the sync can come back but still spends more than 2 seconds, we roll a new WAL. > add an option to roll WAL on very slow syncs > > > Key: HBASE-21806 > URL: https://issues.apache.org/jira/browse/HBASE-21806 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HBASE-21806.patch > > > In large heterogeneous clusters sometimes a slow datanode can cause WAL syncs > to be very slow. In this case, before the bad datanode recovers, or is > discovered and repaired, it would be helpful to roll WAL on a very slow sync > to get a new pipeline. > Otherwise the slow WAL will impact write latency for a long time (slow writes > result in less writes result in the WAL not being rolled for longer) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17942) Disable region splits and merges per table
[
https://issues.apache.org/jira/browse/HBASE-17942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755521#comment-16755521
]
Hadoop QA commented on HBASE-17942:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
13s{color} | {color:red} hbase-server: The patch generated 2 new + 7 unchanged
- 0 fixed = 9 total (was 7) {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 20 new + 662 unchanged - 9 fixed =
682 total (was 671) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 19s{color} | {color:orange} The patch generated 39 new + 1228 unchanged - 5
fixed = 1267 total (was 1233) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 42s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
55s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}131m
8s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
13s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}193m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-17942 |
| JIRA Patch URL |
[jira] [Commented] (HBASE-21634) Print error message when user uses unacceptable values for LIMIT while setting quotas.
[
https://issues.apache.org/jira/browse/HBASE-21634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755493#comment-16755493
]
Hadoop QA commented on HBASE-21634:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
36s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
44s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
7s{color} | {color:red} The patch generated 1 new + 113 unchanged - 9 fixed =
114 total (was 122) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 6s{color} | {color:orange} The patch generated 82 new + 221 unchanged - 6
fixed = 303 total (was 227) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 30s{color}
| {color:red} hbase-shell in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAdminShell |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f01af0 |
| JIRA Issue | HBASE-21634 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956805/hbase-21634.branch-2.0.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit rubocop
ruby_lint |
| uname | Linux 8d0ba67ca5b8 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.0 / 1f2241d714 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| rubocop | v0.63.1 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/15775/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/15775/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/15775/artifact/patchprocess/patch-unit-hbase-shell.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15775/testReport/ |
| Max. process+thread count | 2518 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15775/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Print error message when user uses unacceptable values for LIMIT while
> setting quotas.
> --
>
> Key: HBASE-21634
> URL: https://issues.apache.org/jira/browse/HBASE-21634
> Project: HBase
> Issue Type: Improvement
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21634.branch-2.0.001.patch,
> hbase-21634.master.001.patch, hbase-21634.master.002.patch,
> hbase-21634.master.003.patch, hbase-21634.master.004.patch
>
>
> When unacceptable value(like 2.3G or 70H) to LIMIT are passed while setting
> quotas, we currently do not print any error message (informing the user about
> the erroneous
[jira] [Commented] (HBASE-21807) Backport HBASE-21225 to branch-2.0 and branch-2.1
[ https://issues.apache.org/jira/browse/HBASE-21807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755484#comment-16755484 ] Sakthi commented on HBASE-21807: Uploaded a patch for branch-2.0 > Backport HBASE-21225 to branch-2.0 and branch-2.1 > - > > Key: HBASE-21807 > URL: https://issues.apache.org/jira/browse/HBASE-21807 > Project: HBase > Issue Type: Task >Reporter: Sakthi >Assignee: Sakthi >Priority: Minor > Attachments: hbase-21225.branch-2.0.001.patch > > > Backport HBASE-21225 to branch-2.0 and branch-2.1. The specified Jira is > closed, hence filing this one to track the backport. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21806) add an option to roll WAL on very slow syncs
[ https://issues.apache.org/jira/browse/HBASE-21806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-21806: - Status: Patch Available (was: Open) > add an option to roll WAL on very slow syncs > > > Key: HBASE-21806 > URL: https://issues.apache.org/jira/browse/HBASE-21806 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HBASE-21806.patch > > > In large heterogeneous clusters sometimes a slow datanode can cause WAL syncs > to be very slow. In this case, before the bad datanode recovers, or is > discovered and repaired, it would be helpful to roll WAL on a very slow sync > to get a new pipeline. > Otherwise the slow WAL will impact write latency for a long time (slow writes > result in less writes result in the WAL not being rolled for longer) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21794) Update the Coprocessor observer example given in section 111.1 of the ref guide.
[ https://issues.apache.org/jira/browse/HBASE-21794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi updated HBASE-21794: --- Component/s: documentation Coprocessors > Update the Coprocessor observer example given in section 111.1 of the ref > guide. > > > Key: HBASE-21794 > URL: https://issues.apache.org/jira/browse/HBASE-21794 > Project: HBase > Issue Type: Bug > Components: Coprocessors, documentation >Reporter: Sakthi >Assignee: Sakthi >Priority: Minor > > The given example should be changed after the CP changes (HBASE-17732) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21800) RegionServer aborted due to NPE from MetaTableMetrics coprocessor
[
https://issues.apache.org/jira/browse/HBASE-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755483#comment-16755483
]
Sakthi commented on HBASE-21800:
Test failure is unrelated. Only log line was removed in the third patch. All
unit tests had passed in the second patch.
> RegionServer aborted due to NPE from MetaTableMetrics coprocessor
> -
>
> Key: HBASE-21800
> URL: https://issues.apache.org/jira/browse/HBASE-21800
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, meta, metrics, Operability
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Critical
> Labels: Meta
> Attachments: hbase-21800.master.001.patch,
> hbase-21800.master.002.patch, hbase-21800.master.003.patch
>
>
> I was just playing around the code, trying to capture "Top k" table metrics
> from MetaMetrics, when I bumped into this issue. Though currently we are not
> capturing "Top K" table metrics, but we can encounter this issue because of
> the "Top k Clients" that is implemented using the LossyAlgo.
>
> RegionServer gets aborted due to a NPE from MetaTableMetrics coprocessor. The
> log looks somewhat like this:
> {code:java}
> 2019-01-28 23:31:10,311 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> coprocessor.CoprocessorHost: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2019-01-28 23:31:10,314 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> regionserver.HRegionServer: * ABORTING region server
> 10.0.0.24,16020,1548747043814: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException *
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at
[jira] [Commented] (HBASE-21689) Make table/namespace specific current quota info available in shell(describe_namespace & describe)
[
https://issues.apache.org/jira/browse/HBASE-21689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755482#comment-16755482
]
Sakthi commented on HBASE-21689:
Updated with a Release Note.
> Make table/namespace specific current quota info available in
> shell(describe_namespace & describe)
> --
>
> Key: HBASE-21689
> URL: https://issues.apache.org/jira/browse/HBASE-21689
> Project: HBase
> Issue Type: Improvement
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21689.master.001.patch,
> hbase-21689.master.002.patch, hbase-21689.master.003.patch
>
>
> describe_namespace & describe command in shell should show current quota(if
> present) on the particular table/namespace.
> {noformat}
> // Namespace
> hbase(main):002:0> create_namespace 'ns1',
> {'hbase.namespace.quota.maxtables'=>'5'}
> Took 0.1703 seconds
> hbase(main):004:0> set_quota TYPE => SPACE, NAMESPACE => 'ns1', LIMIT =>
> '50T', POLICY => NO_WRITES
> Took 0.0644 seconds
> hbase(main):007:0> set_quota TYPE => THROTTLE, NAMESPACE => 'ns1', LIMIT =>
> '10m/sec'
> Took 0.0271 seconds
> hbase(main):005:0> describe_namespace 'ns1'
> DESCRIPTION
> {NAME => 'ns1', hbase.namespace.quota.maxtables => '5'}
> // Table
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't1', LIMIT => '1G',
> POLICY => NO_INSERTS
> Took 0.0155 seconds
> hbase(main):009:0> set_quota TYPE => THROTTLE, TABLE => 't1', LIMIT =>
> '10T/hour'
> Took 0.0309 seconds
> hbase(main):008:0> describe 't1'
> Table t1 is ENABLED
> t1
> COLUMN FAMILIES DESCRIPTION
> {NAME => 'cf', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS
> => 'FALSE', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE',
> TTL => 'FOREVER', MIN_VERSIONS => '0', REPL
> ICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false',
> IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRI
> TE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536'}
> 1 row(s)
> Took 0.0341 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21689) Make table/namespace specific current quota info available in shell(describe_namespace & describe)
[
https://issues.apache.org/jira/browse/HBASE-21689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi updated HBASE-21689:
---
Release Note: In shell commands "describe_namespace" and "describe", which
are used to see the descriptors of the namespaces and tables respectively,
quotas set on that particular namespace/table will also be printed along.
> Make table/namespace specific current quota info available in
> shell(describe_namespace & describe)
> --
>
> Key: HBASE-21689
> URL: https://issues.apache.org/jira/browse/HBASE-21689
> Project: HBase
> Issue Type: Improvement
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21689.master.001.patch,
> hbase-21689.master.002.patch, hbase-21689.master.003.patch
>
>
> describe_namespace & describe command in shell should show current quota(if
> present) on the particular table/namespace.
> {noformat}
> // Namespace
> hbase(main):002:0> create_namespace 'ns1',
> {'hbase.namespace.quota.maxtables'=>'5'}
> Took 0.1703 seconds
> hbase(main):004:0> set_quota TYPE => SPACE, NAMESPACE => 'ns1', LIMIT =>
> '50T', POLICY => NO_WRITES
> Took 0.0644 seconds
> hbase(main):007:0> set_quota TYPE => THROTTLE, NAMESPACE => 'ns1', LIMIT =>
> '10m/sec'
> Took 0.0271 seconds
> hbase(main):005:0> describe_namespace 'ns1'
> DESCRIPTION
> {NAME => 'ns1', hbase.namespace.quota.maxtables => '5'}
> // Table
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't1', LIMIT => '1G',
> POLICY => NO_INSERTS
> Took 0.0155 seconds
> hbase(main):009:0> set_quota TYPE => THROTTLE, TABLE => 't1', LIMIT =>
> '10T/hour'
> Took 0.0309 seconds
> hbase(main):008:0> describe 't1'
> Table t1 is ENABLED
> t1
> COLUMN FAMILIES DESCRIPTION
> {NAME => 'cf', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS
> => 'FALSE', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE',
> TTL => 'FOREVER', MIN_VERSIONS => '0', REPL
> ICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false',
> IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRI
> TE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536'}
> 1 row(s)
> Took 0.0341 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21791) Upgrade thrift dependency to 0.12.0
[
https://issues.apache.org/jira/browse/HBASE-21791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755479#comment-16755479
]
Hudson commented on HBASE-21791:
Results for branch branch-2.2
[build #2 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/2/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/2//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/2//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/2//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Upgrade thrift dependency to 0.12.0
> ---
>
> Key: HBASE-21791
> URL: https://issues.apache.org/jira/browse/HBASE-21791
> Project: HBase
> Issue Type: Task
> Components: Thrift
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 1.2.10, 2.0.4
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0
>
> Attachments: HBASE-21791-branch-1.patch,
> HBASE-21791-branch-2.1.patch, HBASE-21791.patch
>
>
> As somebody have already known, that there is a CVE for thrift from 0.5.0 to
> 0.11.0.
> https://nvd.nist.gov/vuln/detail/CVE-2018-1320
> As the CVE is already public, let's upgrade our thrift dependency and release
> new versions ASAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21634) Print error message when user uses unacceptable values for LIMIT while setting quotas.
[
https://issues.apache.org/jira/browse/HBASE-21634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755477#comment-16755477
]
Sakthi commented on HBASE-21634:
Have uploaded a patch for branch-2.0
> Print error message when user uses unacceptable values for LIMIT while
> setting quotas.
> --
>
> Key: HBASE-21634
> URL: https://issues.apache.org/jira/browse/HBASE-21634
> Project: HBase
> Issue Type: Improvement
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21634.branch-2.0.001.patch,
> hbase-21634.master.001.patch, hbase-21634.master.002.patch,
> hbase-21634.master.003.patch, hbase-21634.master.004.patch
>
>
> When unacceptable value(like 2.3G or 70H) to LIMIT are passed while setting
> quotas, we currently do not print any error message (informing the user about
> the erroneous input). Like below:
> {noformat}
> hbase(main):002:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '2.3G',
> POLICY => NO_WRITES
> Took 0.0792 seconds
> hbase(main):003:0> list_quotas
> OWNERQUOTAS
> TABLE => t2 TYPE => SPACE,
> TABLE => t2, LIMIT => 2B, VIOLATION_POLICY => NO_WRITES
> 1 row(s)
> Took 0.0512 seconds
> hbase(main):006:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '70H',
> POLICY => NO_WRITES
> Took 0.0088 seconds
> hbase(main):007:0> list_quotas
> OWNERQUOTAS
> TABLE => t2 TYPE => SPACE,
> TABLE => t2, LIMIT => 70B, VIOLATION_POLICY => NO_WRITES
> 1 row(s)
> Took 0.0448 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21634) Print error message when user uses unacceptable values for LIMIT while setting quotas.
[
https://issues.apache.org/jira/browse/HBASE-21634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi updated HBASE-21634:
---
Attachment: hbase-21634.branch-2.0.001.patch
> Print error message when user uses unacceptable values for LIMIT while
> setting quotas.
> --
>
> Key: HBASE-21634
> URL: https://issues.apache.org/jira/browse/HBASE-21634
> Project: HBase
> Issue Type: Improvement
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21634.branch-2.0.001.patch,
> hbase-21634.master.001.patch, hbase-21634.master.002.patch,
> hbase-21634.master.003.patch, hbase-21634.master.004.patch
>
>
> When unacceptable value(like 2.3G or 70H) to LIMIT are passed while setting
> quotas, we currently do not print any error message (informing the user about
> the erroneous input). Like below:
> {noformat}
> hbase(main):002:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '2.3G',
> POLICY => NO_WRITES
> Took 0.0792 seconds
> hbase(main):003:0> list_quotas
> OWNERQUOTAS
> TABLE => t2 TYPE => SPACE,
> TABLE => t2, LIMIT => 2B, VIOLATION_POLICY => NO_WRITES
> 1 row(s)
> Took 0.0512 seconds
> hbase(main):006:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '70H',
> POLICY => NO_WRITES
> Took 0.0088 seconds
> hbase(main):007:0> list_quotas
> OWNERQUOTAS
> TABLE => t2 TYPE => SPACE,
> TABLE => t2, LIMIT => 70B, VIOLATION_POLICY => NO_WRITES
> 1 row(s)
> Took 0.0448 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21806) add an option to roll WAL on very slow syncs
[ https://issues.apache.org/jira/browse/HBASE-21806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-21806: - Attachment: HBASE-21806.patch > add an option to roll WAL on very slow syncs > > > Key: HBASE-21806 > URL: https://issues.apache.org/jira/browse/HBASE-21806 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HBASE-21806.patch > > > In large heterogeneous clusters sometimes a slow datanode can cause WAL syncs > to be very slow. In this case, before the bad datanode recovers, or is > discovered and repaired, it would be helpful to roll WAL on a very slow sync > to get a new pipeline. > Otherwise the slow WAL will impact write latency for a long time (slow writes > result in less writes result in the WAL not being rolled for longer) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21791) Upgrade thrift dependency to 0.12.0
[
https://issues.apache.org/jira/browse/HBASE-21791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755468#comment-16755468
]
Hudson commented on HBASE-21791:
Results for branch branch-2
[build #1644 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1644/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1644//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1644//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1644//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Upgrade thrift dependency to 0.12.0
> ---
>
> Key: HBASE-21791
> URL: https://issues.apache.org/jira/browse/HBASE-21791
> Project: HBase
> Issue Type: Task
> Components: Thrift
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 1.2.10, 2.0.4
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0
>
> Attachments: HBASE-21791-branch-1.patch,
> HBASE-21791-branch-2.1.patch, HBASE-21791.patch
>
>
> As somebody have already known, that there is a CVE for thrift from 0.5.0 to
> 0.11.0.
> https://nvd.nist.gov/vuln/detail/CVE-2018-1320
> As the CVE is already public, let's upgrade our thrift dependency and release
> new versions ASAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21799) Update branch-2 version to 2.3.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/HBASE-21799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755469#comment-16755469
]
Hudson commented on HBASE-21799:
Results for branch branch-2
[build #1644 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1644/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1644//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1644//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1644//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Update branch-2 version to 2.3.0-SNAPSHOT
> -
>
> Key: HBASE-21799
> URL: https://issues.apache.org/jira/browse/HBASE-21799
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.3.0
>
> Attachments: HBASE-21799.branch-2.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21800) RegionServer aborted due to NPE from MetaTableMetrics coprocessor
[
https://issues.apache.org/jira/browse/HBASE-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755464#comment-16755464
]
Hadoop QA commented on HBASE-21800:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
11s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}260m 27s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}308m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAdmin1 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21800 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956760/hbase-21800.master.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 238261be1fe9 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d82c1a6c2b |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/15770/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15770/testReport/ |
| Max. process+thread count | 5167 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Updated] (HBASE-18999) Put in hbase shell cannot do multiple columns
[
https://issues.apache.org/jira/browse/HBASE-18999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-18999:
---
Release Note:
Adds support for multiple column op for following data manipulation commands:
- put
- delete
- deleteall
- append
- incr
- get_counter
> Put in hbase shell cannot do multiple columns
> -
>
> Key: HBASE-18999
> URL: https://issues.apache.org/jira/browse/HBASE-18999
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 1.0.0, 3.0.0, 2.0.0
>Reporter: Mike Drob
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-18999.master.001.patch,
> HBASE-18999.master.002.patch
>
>
> A {{Put}} can carry multiple cells, but doing so in the shell is very
> difficult to construct. We should make this easier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18999) Put in hbase shell cannot do multiple columns
[
https://issues.apache.org/jira/browse/HBASE-18999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755450#comment-16755450
]
Hadoop QA commented on HBASE-18999:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
36s{color} | {color:red} The patch generated 251 new + 415 unchanged - 14 fixed
= 666 total (was 429) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 23s{color} | {color:orange} The patch generated 92 new + 568 unchanged - 12
fixed = 660 total (was 580) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
34s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-18999 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956800/HBASE-18999.master.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit rubocop
ruby_lint |
| uname | Linux 8f3c6bd7c198 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d82c1a6c2b |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| rubocop | v0.62.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/15774/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/15774/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15774/testReport/ |
| Max. process+thread count | 2316 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15774/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Put in hbase shell cannot do multiple columns
> -
>
> Key: HBASE-18999
> URL: https://issues.apache.org/jira/browse/HBASE-18999
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 1.0.0, 3.0.0, 2.0.0
>Reporter: Mike Drob
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-18999.master.001.patch,
> HBASE-18999.master.002.patch
>
>
> A {{Put}} can carry multiple cells, but doing so in the shell is very
> difficult to construct. We should make this easier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18999) Put in hbase shell cannot do multiple columns
[
https://issues.apache.org/jira/browse/HBASE-18999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-18999:
---
Release Note:
Adds support for multiple column for following data manipulation commands:
- put
- delete
- deleteall
- append
- incr
- get_counter
was:
Adds support for multiple column op for following data manipulation commands:
- put
- delete
- deleteall
- append
- incr
- get_counter
> Put in hbase shell cannot do multiple columns
> -
>
> Key: HBASE-18999
> URL: https://issues.apache.org/jira/browse/HBASE-18999
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 1.0.0, 3.0.0, 2.0.0
>Reporter: Mike Drob
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-18999.master.001.patch,
> HBASE-18999.master.002.patch
>
>
> A {{Put}} can carry multiple cells, but doing so in the shell is very
> difficult to construct. We should make this easier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18999) Put in hbase shell cannot do multiple columns
[
https://issues.apache.org/jira/browse/HBASE-18999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-18999:
---
Release Note:
Adds shell support for multiple column for following data manipulation commands:
- put
- delete
- deleteall
- append
- incr
- get_counter
was:
Adds support for multiple column for following data manipulation commands:
- put
- delete
- deleteall
- append
- incr
- get_counter
> Put in hbase shell cannot do multiple columns
> -
>
> Key: HBASE-18999
> URL: https://issues.apache.org/jira/browse/HBASE-18999
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 1.0.0, 3.0.0, 2.0.0
>Reporter: Mike Drob
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-18999.master.001.patch,
> HBASE-18999.master.002.patch
>
>
> A {{Put}} can carry multiple cells, but doing so in the shell is very
> difficult to construct. We should make this easier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18999) Put in hbase shell cannot do multiple columns
[
https://issues.apache.org/jira/browse/HBASE-18999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-18999:
---
Attachment: HBASE-18999.master.002.patch
> Put in hbase shell cannot do multiple columns
> -
>
> Key: HBASE-18999
> URL: https://issues.apache.org/jira/browse/HBASE-18999
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 1.0.0, 3.0.0, 2.0.0
>Reporter: Mike Drob
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-18999.master.001.patch,
> HBASE-18999.master.002.patch
>
>
> A {{Put}} can carry multiple cells, but doing so in the shell is very
> difficult to construct. We should make this easier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21803) HBase website cleanup
[
https://issues.apache.org/jira/browse/HBASE-21803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755435#comment-16755435
]
Sakthi commented on HBASE-21803:
Ping [~psomogyi], [~stack], [~busbey] !
> HBase website cleanup
> -
>
> Key: HBASE-21803
> URL: https://issues.apache.org/jira/browse/HBASE-21803
> Project: HBase
> Issue Type: Umbrella
> Components: website
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
>
> This umbrella is to track the HBase website cleanup. The last Linkchecker
> job(which was run on 25th jan (takes approximately 20+ hours to finish)) has
> reported "1458 missing files" and "7143 missing named anchors". Have filed
> this Jira to track several sub-jiras that might follow.
> Also, any other issues, other than the LinkChecker job ones, can be tracked
> here. Please feel free to create sub-tasks here for the found website related
> issues.
> Here's a copy of the most recent Linkchecker job:
> {code:java}
> Index of Linklint results
> Sat, 26-Jan-2019 05:04:24 (local)
> Linklint version: 2.3.5_ingo_020
> summary.txt: summary of results
> log.txt: log of progress
> file.html: found 51187 files
>fileX.html: found 51187 files (cross referenced)
>fileF.html: found 51051 files with forward links
> remote.html: found 3431 other links
> remoteX.html: found 3431 other links (cross referenced)
> anchor.html: found 21525313 named anchors
> anchorX.html: found 21525313 named anchors (cross referenced)
> action.html: - 1 action skipped
> actionX.html: - 1 action skipped (cross referenced)
> skipped.html: - 2 files skipped
>skipX.html: - 2 files skipped (cross referenced)
> warn.html: warn 696 warnings
>warnX.html: warn 696 warnings (cross referenced)
>warnF.html: warn 253 files with warnings
>error.html: ERROR 1458 missing files
> errorX.html: ERROR 1458 missing files (cross referenced)
> errorF.html: ERROR 1417 files had broken links
> errorA.html: ERROR 7145 missing named anchors
> errorAX.html: ERROR 7145 missing named anchors (cross referenced)
> httpfail.html: - 1458 links: failed via http
> httpok.html: - 51187 links: ok via http
> mapped.html: - 4 links were mapped
> urlindex.html: results for remote urls
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HBASE-21803) HBase website cleanup
[
https://issues.apache.org/jira/browse/HBASE-21803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-21803 started by Sakthi.
--
> HBase website cleanup
> -
>
> Key: HBASE-21803
> URL: https://issues.apache.org/jira/browse/HBASE-21803
> Project: HBase
> Issue Type: Umbrella
> Components: website
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
>
> This umbrella is to track the HBase website cleanup. The last Linkchecker
> job(which was run on 25th jan (takes approximately 20+ hours to finish)) has
> reported "1458 missing files" and "7143 missing named anchors". Have filed
> this Jira to track several sub-jiras that might follow.
> Also, any other issues, other than the LinkChecker job ones, can be tracked
> here. Please feel free to create sub-tasks here for the found website related
> issues.
> Here's a copy of the most recent Linkchecker job:
> {code:java}
> Index of Linklint results
> Sat, 26-Jan-2019 05:04:24 (local)
> Linklint version: 2.3.5_ingo_020
> summary.txt: summary of results
> log.txt: log of progress
> file.html: found 51187 files
>fileX.html: found 51187 files (cross referenced)
>fileF.html: found 51051 files with forward links
> remote.html: found 3431 other links
> remoteX.html: found 3431 other links (cross referenced)
> anchor.html: found 21525313 named anchors
> anchorX.html: found 21525313 named anchors (cross referenced)
> action.html: - 1 action skipped
> actionX.html: - 1 action skipped (cross referenced)
> skipped.html: - 2 files skipped
>skipX.html: - 2 files skipped (cross referenced)
> warn.html: warn 696 warnings
>warnX.html: warn 696 warnings (cross referenced)
>warnF.html: warn 253 files with warnings
>error.html: ERROR 1458 missing files
> errorX.html: ERROR 1458 missing files (cross referenced)
> errorF.html: ERROR 1417 files had broken links
> errorA.html: ERROR 7145 missing named anchors
> errorAX.html: ERROR 7145 missing named anchors (cross referenced)
> httpfail.html: - 1458 links: failed via http
> httpok.html: - 51187 links: ok via http
> mapped.html: - 4 links were mapped
> urlindex.html: results for remote urls
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21800) RegionServer aborted due to NPE from MetaTableMetrics coprocessor
[
https://issues.apache.org/jira/browse/HBASE-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755432#comment-16755432
]
Sakthi commented on HBASE-21800:
Thanks [~xucang] and [~apurtell] for your reviews :)
> RegionServer aborted due to NPE from MetaTableMetrics coprocessor
> -
>
> Key: HBASE-21800
> URL: https://issues.apache.org/jira/browse/HBASE-21800
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, meta, metrics, Operability
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Critical
> Labels: Meta
> Attachments: hbase-21800.master.001.patch,
> hbase-21800.master.002.patch, hbase-21800.master.003.patch
>
>
> I was just playing around the code, trying to capture "Top k" table metrics
> from MetaMetrics, when I bumped into this issue. Though currently we are not
> capturing "Top K" table metrics, but we can encounter this issue because of
> the "Top k Clients" that is implemented using the LossyAlgo.
>
> RegionServer gets aborted due to a NPE from MetaTableMetrics coprocessor. The
> log looks somewhat like this:
> {code:java}
> 2019-01-28 23:31:10,311 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> coprocessor.CoprocessorHost: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2019-01-28 23:31:10,314 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> regionserver.HRegionServer: * ABORTING region server
> 10.0.0.24,16020,1548747043814: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException *
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at
[jira] [Commented] (HBASE-21800) RegionServer aborted due to NPE from MetaTableMetrics coprocessor
[
https://issues.apache.org/jira/browse/HBASE-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755429#comment-16755429
]
Hadoop QA commented on HBASE-21800:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 38s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}128m
33s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}166m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21800 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956778/hbase-21800.master.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 065bda5451ee 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d82c1a6c2b |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15771/testReport/ |
| Max. process+thread count | 5040 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15771/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> RegionServer aborted
[jira] [Updated] (HBASE-21807) Backport HBASE-21225 to branch-2.0 and branch-2.1
[ https://issues.apache.org/jira/browse/HBASE-21807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi updated HBASE-21807: --- Attachment: hbase-21225.branch-2.0.001.patch > Backport HBASE-21225 to branch-2.0 and branch-2.1 > - > > Key: HBASE-21807 > URL: https://issues.apache.org/jira/browse/HBASE-21807 > Project: HBase > Issue Type: Task >Reporter: Sakthi >Assignee: Sakthi >Priority: Minor > Attachments: hbase-21225.branch-2.0.001.patch > > > Backport HBASE-21225 to branch-2.0 and branch-2.1. The specified Jira is > closed, hence filing this one to track the backport. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21807) Backport HBASE-21225 to branch-2.0 and branch-2.1
[ https://issues.apache.org/jira/browse/HBASE-21807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi updated HBASE-21807: --- Status: Patch Available (was: In Progress) > Backport HBASE-21225 to branch-2.0 and branch-2.1 > - > > Key: HBASE-21807 > URL: https://issues.apache.org/jira/browse/HBASE-21807 > Project: HBase > Issue Type: Task >Reporter: Sakthi >Assignee: Sakthi >Priority: Minor > Attachments: hbase-21225.branch-2.0.001.patch > > > Backport HBASE-21225 to branch-2.0 and branch-2.1. The specified Jira is > closed, hence filing this one to track the backport. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-21807) Backport HBASE-21225 to branch-2.0 and branch-2.1
[ https://issues.apache.org/jira/browse/HBASE-21807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-21807 started by Sakthi. -- > Backport HBASE-21225 to branch-2.0 and branch-2.1 > - > > Key: HBASE-21807 > URL: https://issues.apache.org/jira/browse/HBASE-21807 > Project: HBase > Issue Type: Task >Reporter: Sakthi >Assignee: Sakthi >Priority: Minor > Attachments: hbase-21225.branch-2.0.001.patch > > > Backport HBASE-21225 to branch-2.0 and branch-2.1. The specified Jira is > closed, hence filing this one to track the backport. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21807) Backport HBASE-21225 to branch-2.0 and branch-2.1
Sakthi created HBASE-21807: -- Summary: Backport HBASE-21225 to branch-2.0 and branch-2.1 Key: HBASE-21807 URL: https://issues.apache.org/jira/browse/HBASE-21807 Project: HBase Issue Type: Task Reporter: Sakthi Assignee: Sakthi Backport HBASE-21225 to branch-2.0 and branch-2.1. The specified Jira is closed, hence filing this one to track the backport. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21806) add an option to roll WAL on very slow syncs
[ https://issues.apache.org/jira/browse/HBASE-21806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-21806: - Description: In large heterogeneous clusters sometimes a slow datanode can cause WAL syncs to be very slow. In this case, before the bad datanode recovers, or is discovered and repaired, it would be helpful to roll WAL on a very slow sync to get a new pipeline. Otherwise the slow WAL will impact write latency for a long time (slow writes result in less writes result in the WAL not being rolled for longer) was:In large heterogeneous clusters sometimes a slow datanode can cause WAL syncs to be very slow. In this case, before the bad datanode recovers, or is discovered and repaired, it would be helpful to roll WAL on a very slow sync to get a new pipeline. > add an option to roll WAL on very slow syncs > > > Key: HBASE-21806 > URL: https://issues.apache.org/jira/browse/HBASE-21806 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > > In large heterogeneous clusters sometimes a slow datanode can cause WAL syncs > to be very slow. In this case, before the bad datanode recovers, or is > discovered and repaired, it would be helpful to roll WAL on a very slow sync > to get a new pipeline. > Otherwise the slow WAL will impact write latency for a long time (slow writes > result in less writes result in the WAL not being rolled for longer) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21806) add an option to roll WAL on very slow syncs
Sergey Shelukhin created HBASE-21806: Summary: add an option to roll WAL on very slow syncs Key: HBASE-21806 URL: https://issues.apache.org/jira/browse/HBASE-21806 Project: HBase Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin In large heterogeneous clusters sometimes a slow datanode can cause WAL syncs to be very slow. In this case, before the bad datanode recovers, or is discovered and repaired, it would be helpful to roll WAL on a very slow sync to get a new pipeline. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21780) Avoid a wide line on the RegionServer webUI for many ZooKeeper servers
[ https://issues.apache.org/jira/browse/HBASE-21780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755403#comment-16755403 ] Sakthi commented on HBASE-21780: [~busbey], anything else we ought to do here? :) > Avoid a wide line on the RegionServer webUI for many ZooKeeper servers > -- > > Key: HBASE-21780 > URL: https://issues.apache.org/jira/browse/HBASE-21780 > Project: HBase > Issue Type: Improvement > Components: UI, Usability >Reporter: Sakthi >Assignee: Sakthi >Priority: Minor > Attachments: RS_ZKQuorum.png, RS_ZKQuorum_afterPatch.png, > hbase-21780.master.001.patch, hbase-21780.master.002.patch > > > HBASE-8812 made this change for MasterUI but not for RegionServer UI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21771) Cluster key in Master UI is incorrect
[
https://issues.apache.org/jira/browse/HBASE-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755401#comment-16755401
]
Sakthi commented on HBASE-21771:
[~apurtell] , [~busbey], how does the patch look to you guys?
> Cluster key in Master UI is incorrect
> -
>
> Key: HBASE-21771
> URL: https://issues.apache.org/jira/browse/HBASE-21771
> Project: HBase
> Issue Type: Bug
> Components: Replication, UI, Usability
>Affects Versions: 3.0.0, 2.1.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Major
> Fix For: 2.1.3
>
> Attachments: ClusterKey_afterPatch.png, hbase-21771.master.001.patch,
> hbase-21771.master.002.patch
>
>
> The master UI is supposed to give us a cluster key we can copy/paste to add a
> replication peer in the hbase shell. the shell explains that it should look
> like this:
> {quote}
> {code}
> For a HBase cluster peer, a cluster key must be provided and is composed like
> this:
> hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent
> This gives a full path for HBase to connect to another HBase cluster.
> ...
> hbase> add_peer '1', CLUSTER_KEY => "server1.cie.com:2181:/hbase"
> hbase> add_peer '1', CLUSTER_KEY => "server1.cie.com:2181:/hbase", STATE =>
> "ENABLED"
> hbase> add_peer '1', CLUSTER_KEY => "server1.cie.com:2181:/hbase", STATE =>
> "DISABLED"
> hbase> add_peer '2', CLUSTER_KEY => "zk1,zk2,zk3:2182:/hbase-prod",
> TABLE_CFS => { "table1" => [], "table2" => ["cf1"], "table3" => ["cf1",
> "cf2"] }
> hbase> add_peer '2', CLUSTER_KEY => "zk1,zk2,zk3:2182:/hbase-prod",
> NAMESPACES => ["ns1", "ns2", "ns3"]
> {code}
> {quote}
> However, on my example cluster with ZK quorum with 3 servers, the master ui
> gives this:
> {quote}
> {code}
> Cluster Key busbey-training-1.gce.cloudera.com:2181
> busbey-training-2.gce.cloudera.com:2181
> busbey-training-3.gce.cloudera.com:2181:/hbaseKey to add this cluster
> as a peer for replication. Use 'help "add_peer"' in the shell for details.
> {code}
> {quote}
> Note that the quorum members are newline separated instead of comma and that
> the port appears on each member instead of after the set of host names.
> Workaround is to manually construct the cluster key from the details in the
> field. :(
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-17942) Disable region splits and merges per table
[ https://issues.apache.org/jira/browse/HBASE-17942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-17942: --- Status: Patch Available (was: In Progress) > Disable region splits and merges per table > -- > > Key: HBASE-17942 > URL: https://issues.apache.org/jira/browse/HBASE-17942 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Nihal Jain >Priority: Major > Attachments: HBASE-17942.master.001.patch, > HBASE-17942.master.002.patch > > > Snapshot of a large tables usually fails when region split/merge happens > during snapshot. HBASE-15128 has introduced enable/disable switch for split > /merges, but only cluster - wide which is not always good in a large > deployments. The new feature will improve snapshot stability ands performance > for a single table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21775) The BufferedMutator doesn't ever refresh region location cache
[
https://issues.apache.org/jira/browse/HBASE-21775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755347#comment-16755347
]
Tommy Li commented on HBASE-21775:
--
Thanks for the link, [~stack]. I took a look at the report from before my
change went in and indeed TestAsyncProcess [is not listed
there|[https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.1/168/artifact/dashboard.html].]
Could this be a build caching issue?
> The BufferedMutator doesn't ever refresh region location cache
> --
>
> Key: HBASE-21775
> URL: https://issues.apache.org/jira/browse/HBASE-21775
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Tommy Li
>Assignee: Tommy Li
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.5, 1.3.4
>
> Attachments: HBASE-21775.master.001.patch,
> org.apache.hadoop.hbase.client.TestAsyncProcess-with-HBASE-21775.txt,
> org.apache.hadoop.hbase.client.TestAsyncProcess-without-HBASE-21775.txt
>
>
> {color:#22}I noticed in some of my writing jobs that the BufferedMutator
> would get stuck retrying writes against a dead server.{color}
> {code:java}
> 19/01/18 15:15:47 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:15:54 WARN [htable-pool3-t56] client.AsyncRequestFutureImpl:
> id=2, table=dummy_table, attempt=15/21, failureCount=1ops, last
> exception=org.apache.hadoop.hbase.DoNotRetryIOException: Operation rpcTimeout
> on ,17020,1547848193782, tracking started Fri Jan 18 14:55:37 PST
> 2019; NOT retrying, failed=1 -- final attempt!
> 19/01/18 15:15:54 ERROR [Executor task launch worker for task 0]
> IngestRawData.map(): [B@258bc2c7:
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: Operation rpcTimeout: 1 time, servers with issues:
> ,17020,1547848193782
> {code}
>
> After the single remaining action permanently failed, it would resume
> progress only to get stuck again retrying against the same dead server:
> {code:java}
> 19/01/18 15:21:18 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:21:18 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:21:20 INFO [htable-pool3-t55] client.AsyncRequestFutureImpl:
> id=2, table=dummy_table, attempt=6/21, failureCount=1ops, last
> exception=java.net.ConnectException: Call to failed on connection
> exception:
> org.apache.hbase.thirdparty.io.netty.channel.ConnectTimeoutException:
> connection timed out: on ,17020,1547848193782, tracking
> started null, retrying after=20089ms, operationsToReplay=1
> {code}
>
> Only restarting the client process to generate a new BufferedMutator instance
> would fix the issue, at least until the next regionserver crash
> The logs I've pasted show the issue happening with a
> ConnectionTimeoutException, but we've also seen it with
> NotServingRegionException and some others
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21800) RegionServer aborted due to NPE from MetaTableMetrics coprocessor
[
https://issues.apache.org/jira/browse/HBASE-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755351#comment-16755351
]
Xu Cang commented on HBASE-21800:
-
Nice bug fix and improvements. Looks godo to me [~jatsakthi] Thanks!
> RegionServer aborted due to NPE from MetaTableMetrics coprocessor
> -
>
> Key: HBASE-21800
> URL: https://issues.apache.org/jira/browse/HBASE-21800
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, meta, metrics, Operability
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Critical
> Labels: Meta
> Attachments: hbase-21800.master.001.patch,
> hbase-21800.master.002.patch, hbase-21800.master.003.patch
>
>
> I was just playing around the code, trying to capture "Top k" table metrics
> from MetaMetrics, when I bumped into this issue. Though currently we are not
> capturing "Top K" table metrics, but we can encounter this issue because of
> the "Top k Clients" that is implemented using the LossyAlgo.
>
> RegionServer gets aborted due to a NPE from MetaTableMetrics coprocessor. The
> log looks somewhat like this:
> {code:java}
> 2019-01-28 23:31:10,311 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> coprocessor.CoprocessorHost: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2019-01-28 23:31:10,314 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> regionserver.HRegionServer: * ABORTING region server
> 10.0.0.24,16020,1548747043814: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException *
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
>
[jira] [Comment Edited] (HBASE-21800) RegionServer aborted due to NPE from MetaTableMetrics coprocessor
[
https://issues.apache.org/jira/browse/HBASE-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755351#comment-16755351
]
Xu Cang edited comment on HBASE-21800 at 1/29/19 8:13 PM:
--
Nice bug fix and improvements. Looks good to me [~jatsakthi] Thanks!
was (Author: xucang):
Nice bug fix and improvements. Looks godo to me [~jatsakthi] Thanks!
> RegionServer aborted due to NPE from MetaTableMetrics coprocessor
> -
>
> Key: HBASE-21800
> URL: https://issues.apache.org/jira/browse/HBASE-21800
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, meta, metrics, Operability
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Critical
> Labels: Meta
> Attachments: hbase-21800.master.001.patch,
> hbase-21800.master.002.patch, hbase-21800.master.003.patch
>
>
> I was just playing around the code, trying to capture "Top k" table metrics
> from MetaMetrics, when I bumped into this issue. Though currently we are not
> capturing "Top K" table metrics, but we can encounter this issue because of
> the "Top k Clients" that is implemented using the LossyAlgo.
>
> RegionServer gets aborted due to a NPE from MetaTableMetrics coprocessor. The
> log looks somewhat like this:
> {code:java}
> 2019-01-28 23:31:10,311 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> coprocessor.CoprocessorHost: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2019-01-28 23:31:10,314 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> regionserver.HRegionServer: * ABORTING region server
> 10.0.0.24,16020,1548747043814: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException *
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
>
[jira] [Comment Edited] (HBASE-21775) The BufferedMutator doesn't ever refresh region location cache
[
https://issues.apache.org/jira/browse/HBASE-21775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755347#comment-16755347
]
Tommy Li edited comment on HBASE-21775 at 1/29/19 8:13 PM:
---
Thanks for the link, [~stack]. I took a look at the report from before my
change went in and indeed TestAsyncProcess [is not listed
there|https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.1/168/artifact/dashboard.html]
.Could this be a build caching issue?
was (Author: tommyzli):
Thanks for the link, [~stack]. I took a look at the report from before my
change went in and indeed TestAsyncProcess [is not listed
there|[https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.1/168/artifact/dashboard.html].]
Could this be a build caching issue?
> The BufferedMutator doesn't ever refresh region location cache
> --
>
> Key: HBASE-21775
> URL: https://issues.apache.org/jira/browse/HBASE-21775
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Tommy Li
>Assignee: Tommy Li
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.5, 1.3.4
>
> Attachments: HBASE-21775.master.001.patch,
> org.apache.hadoop.hbase.client.TestAsyncProcess-with-HBASE-21775.txt,
> org.apache.hadoop.hbase.client.TestAsyncProcess-without-HBASE-21775.txt
>
>
> {color:#22}I noticed in some of my writing jobs that the BufferedMutator
> would get stuck retrying writes against a dead server.{color}
> {code:java}
> 19/01/18 15:15:47 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:15:54 WARN [htable-pool3-t56] client.AsyncRequestFutureImpl:
> id=2, table=dummy_table, attempt=15/21, failureCount=1ops, last
> exception=org.apache.hadoop.hbase.DoNotRetryIOException: Operation rpcTimeout
> on ,17020,1547848193782, tracking started Fri Jan 18 14:55:37 PST
> 2019; NOT retrying, failed=1 -- final attempt!
> 19/01/18 15:15:54 ERROR [Executor task launch worker for task 0]
> IngestRawData.map(): [B@258bc2c7:
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: Operation rpcTimeout: 1 time, servers with issues:
> ,17020,1547848193782
> {code}
>
> After the single remaining action permanently failed, it would resume
> progress only to get stuck again retrying against the same dead server:
> {code:java}
> 19/01/18 15:21:18 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:21:18 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:21:20 INFO [htable-pool3-t55] client.AsyncRequestFutureImpl:
> id=2, table=dummy_table, attempt=6/21, failureCount=1ops, last
> exception=java.net.ConnectException: Call to failed on connection
> exception:
> org.apache.hbase.thirdparty.io.netty.channel.ConnectTimeoutException:
> connection timed out: on ,17020,1547848193782, tracking
> started null, retrying after=20089ms, operationsToReplay=1
> {code}
>
> Only restarting the client process to generate a new BufferedMutator instance
> would fix the issue, at least until the next regionserver crash
> The logs I've pasted show the issue happening with a
> ConnectionTimeoutException, but we've also seen it with
> NotServingRegionException and some others
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21775) The BufferedMutator doesn't ever refresh region location cache
[
https://issues.apache.org/jira/browse/HBASE-21775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755342#comment-16755342
]
Tommy Li commented on HBASE-21775:
--
So I pulled branch-2.1 and ran `mvn test
-Dtest=org.apache.hadoop.hbase.client.TestAsyncProcess
-Dskip.license.check=true` locally both with my change and without, and I see
the same 5 test failures in both runs. I've attached the surefire output of
both runs.
[^org.apache.hadoop.hbase.client.TestAsyncProcess-without-HBASE-21775.txt] has
one extra failure which is the test that I added.
Unless I'm looking at the wrong tests, I don't think the failures are
introduced by my change
> The BufferedMutator doesn't ever refresh region location cache
> --
>
> Key: HBASE-21775
> URL: https://issues.apache.org/jira/browse/HBASE-21775
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Tommy Li
>Assignee: Tommy Li
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.5, 1.3.4
>
> Attachments: HBASE-21775.master.001.patch,
> org.apache.hadoop.hbase.client.TestAsyncProcess-with-HBASE-21775.txt,
> org.apache.hadoop.hbase.client.TestAsyncProcess-without-HBASE-21775.txt
>
>
> {color:#22}I noticed in some of my writing jobs that the BufferedMutator
> would get stuck retrying writes against a dead server.{color}
> {code:java}
> 19/01/18 15:15:47 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:15:54 WARN [htable-pool3-t56] client.AsyncRequestFutureImpl:
> id=2, table=dummy_table, attempt=15/21, failureCount=1ops, last
> exception=org.apache.hadoop.hbase.DoNotRetryIOException: Operation rpcTimeout
> on ,17020,1547848193782, tracking started Fri Jan 18 14:55:37 PST
> 2019; NOT retrying, failed=1 -- final attempt!
> 19/01/18 15:15:54 ERROR [Executor task launch worker for task 0]
> IngestRawData.map(): [B@258bc2c7:
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: Operation rpcTimeout: 1 time, servers with issues:
> ,17020,1547848193782
> {code}
>
> After the single remaining action permanently failed, it would resume
> progress only to get stuck again retrying against the same dead server:
> {code:java}
> 19/01/18 15:21:18 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:21:18 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:21:20 INFO [htable-pool3-t55] client.AsyncRequestFutureImpl:
> id=2, table=dummy_table, attempt=6/21, failureCount=1ops, last
> exception=java.net.ConnectException: Call to failed on connection
> exception:
> org.apache.hbase.thirdparty.io.netty.channel.ConnectTimeoutException:
> connection timed out: on ,17020,1547848193782, tracking
> started null, retrying after=20089ms, operationsToReplay=1
> {code}
>
> Only restarting the client process to generate a new BufferedMutator instance
> would fix the issue, at least until the next regionserver crash
> The logs I've pasted show the issue happening with a
> ConnectionTimeoutException, but we've also seen it with
> NotServingRegionException and some others
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21775) The BufferedMutator doesn't ever refresh region location cache
[
https://issues.apache.org/jira/browse/HBASE-21775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tommy Li updated HBASE-21775:
-
Attachment:
org.apache.hadoop.hbase.client.TestAsyncProcess-without-HBASE-21775.txt
org.apache.hadoop.hbase.client.TestAsyncProcess-with-HBASE-21775.txt
> The BufferedMutator doesn't ever refresh region location cache
> --
>
> Key: HBASE-21775
> URL: https://issues.apache.org/jira/browse/HBASE-21775
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Tommy Li
>Assignee: Tommy Li
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.5, 1.3.4
>
> Attachments: HBASE-21775.master.001.patch,
> org.apache.hadoop.hbase.client.TestAsyncProcess-with-HBASE-21775.txt,
> org.apache.hadoop.hbase.client.TestAsyncProcess-without-HBASE-21775.txt
>
>
> {color:#22}I noticed in some of my writing jobs that the BufferedMutator
> would get stuck retrying writes against a dead server.{color}
> {code:java}
> 19/01/18 15:15:47 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:15:54 WARN [htable-pool3-t56] client.AsyncRequestFutureImpl:
> id=2, table=dummy_table, attempt=15/21, failureCount=1ops, last
> exception=org.apache.hadoop.hbase.DoNotRetryIOException: Operation rpcTimeout
> on ,17020,1547848193782, tracking started Fri Jan 18 14:55:37 PST
> 2019; NOT retrying, failed=1 -- final attempt!
> 19/01/18 15:15:54 ERROR [Executor task launch worker for task 0]
> IngestRawData.map(): [B@258bc2c7:
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: Operation rpcTimeout: 1 time, servers with issues:
> ,17020,1547848193782
> {code}
>
> After the single remaining action permanently failed, it would resume
> progress only to get stuck again retrying against the same dead server:
> {code:java}
> 19/01/18 15:21:18 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:21:18 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:21:20 INFO [htable-pool3-t55] client.AsyncRequestFutureImpl:
> id=2, table=dummy_table, attempt=6/21, failureCount=1ops, last
> exception=java.net.ConnectException: Call to failed on connection
> exception:
> org.apache.hbase.thirdparty.io.netty.channel.ConnectTimeoutException:
> connection timed out: on ,17020,1547848193782, tracking
> started null, retrying after=20089ms, operationsToReplay=1
> {code}
>
> Only restarting the client process to generate a new BufferedMutator instance
> would fix the issue, at least until the next regionserver crash
> The logs I've pasted show the issue happening with a
> ConnectionTimeoutException, but we've also seen it with
> NotServingRegionException and some others
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21775) The BufferedMutator doesn't ever refresh region location cache
[
https://issues.apache.org/jira/browse/HBASE-21775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755331#comment-16755331
]
stack commented on HBASE-21775:
---
Does it pass locally for you [~tommyzli]?
I believe the mighty [~Apache9] is referring to this
https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.1/lastSuccessfulBuild/artifact/dashboard.html
It looks like the test fails 100% of the time in the flakey test finder. Click
on the 'show' to look at particular failure types.
I took a look and the failures seem to be other tests, not TestAsyncProcess?
Yeah, does it pass locally for you?
> The BufferedMutator doesn't ever refresh region location cache
> --
>
> Key: HBASE-21775
> URL: https://issues.apache.org/jira/browse/HBASE-21775
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Tommy Li
>Assignee: Tommy Li
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.5, 1.3.4
>
> Attachments: HBASE-21775.master.001.patch
>
>
> {color:#22}I noticed in some of my writing jobs that the BufferedMutator
> would get stuck retrying writes against a dead server.{color}
> {code:java}
> 19/01/18 15:15:47 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:15:54 WARN [htable-pool3-t56] client.AsyncRequestFutureImpl:
> id=2, table=dummy_table, attempt=15/21, failureCount=1ops, last
> exception=org.apache.hadoop.hbase.DoNotRetryIOException: Operation rpcTimeout
> on ,17020,1547848193782, tracking started Fri Jan 18 14:55:37 PST
> 2019; NOT retrying, failed=1 -- final attempt!
> 19/01/18 15:15:54 ERROR [Executor task launch worker for task 0]
> IngestRawData.map(): [B@258bc2c7:
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: Operation rpcTimeout: 1 time, servers with issues:
> ,17020,1547848193782
> {code}
>
> After the single remaining action permanently failed, it would resume
> progress only to get stuck again retrying against the same dead server:
> {code:java}
> 19/01/18 15:21:18 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:21:18 INFO [Executor task launch worker for task 0]
> client.AsyncRequestFutureImpl: #2, waiting for 1 actions to finish on table:
> dummy_table
> 19/01/18 15:21:20 INFO [htable-pool3-t55] client.AsyncRequestFutureImpl:
> id=2, table=dummy_table, attempt=6/21, failureCount=1ops, last
> exception=java.net.ConnectException: Call to failed on connection
> exception:
> org.apache.hbase.thirdparty.io.netty.channel.ConnectTimeoutException:
> connection timed out: on ,17020,1547848193782, tracking
> started null, retrying after=20089ms, operationsToReplay=1
> {code}
>
> Only restarting the client process to generate a new BufferedMutator instance
> would fix the issue, at least until the next regionserver crash
> The logs I've pasted show the issue happening with a
> ConnectionTimeoutException, but we've also seen it with
> NotServingRegionException and some others
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21800) RegionServer aborted due to NPE from MetaTableMetrics coprocessor
[
https://issues.apache.org/jira/browse/HBASE-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi updated HBASE-21800:
---
Attachment: hbase-21800.master.003.patch
> RegionServer aborted due to NPE from MetaTableMetrics coprocessor
> -
>
> Key: HBASE-21800
> URL: https://issues.apache.org/jira/browse/HBASE-21800
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, meta, metrics, Operability
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Critical
> Labels: Meta
> Attachments: hbase-21800.master.001.patch,
> hbase-21800.master.002.patch, hbase-21800.master.003.patch
>
>
> I was just playing around the code, trying to capture "Top k" table metrics
> from MetaMetrics, when I bumped into this issue. Though currently we are not
> capturing "Top K" table metrics, but we can encounter this issue because of
> the "Top k Clients" that is implemented using the LossyAlgo.
>
> RegionServer gets aborted due to a NPE from MetaTableMetrics coprocessor. The
> log looks somewhat like this:
> {code:java}
> 2019-01-28 23:31:10,311 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> coprocessor.CoprocessorHost: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2019-01-28 23:31:10,314 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> regionserver.HRegionServer: * ABORTING region server
> 10.0.0.24,16020,1548747043814: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException *
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
>
[jira] [Commented] (HBASE-21800) RegionServer aborted due to NPE from MetaTableMetrics coprocessor
[
https://issues.apache.org/jira/browse/HBASE-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755313#comment-16755313
]
Sakthi commented on HBASE-21800:
Sure [~apurtell], will do. Thanks for taking a look :)
> RegionServer aborted due to NPE from MetaTableMetrics coprocessor
> -
>
> Key: HBASE-21800
> URL: https://issues.apache.org/jira/browse/HBASE-21800
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, meta, metrics, Operability
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Critical
> Labels: Meta
> Attachments: hbase-21800.master.001.patch,
> hbase-21800.master.002.patch
>
>
> I was just playing around the code, trying to capture "Top k" table metrics
> from MetaMetrics, when I bumped into this issue. Though currently we are not
> capturing "Top K" table metrics, but we can encounter this issue because of
> the "Top k Clients" that is implemented using the LossyAlgo.
>
> RegionServer gets aborted due to a NPE from MetaTableMetrics coprocessor. The
> log looks somewhat like this:
> {code:java}
> 2019-01-28 23:31:10,311 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> coprocessor.CoprocessorHost: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2019-01-28 23:31:10,314 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> regionserver.HRegionServer: * ABORTING region server
> 10.0.0.24,16020,1548747043814: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException *
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at
[jira] [Updated] (HBASE-21800) RegionServer aborted due to NPE from MetaTableMetrics coprocessor
[
https://issues.apache.org/jira/browse/HBASE-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi updated HBASE-21800:
---
Labels: Meta (was: )
> RegionServer aborted due to NPE from MetaTableMetrics coprocessor
> -
>
> Key: HBASE-21800
> URL: https://issues.apache.org/jira/browse/HBASE-21800
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, meta, metrics, Operability
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Critical
> Labels: Meta
> Attachments: hbase-21800.master.001.patch,
> hbase-21800.master.002.patch
>
>
> I was just playing around the code, trying to capture "Top k" table metrics
> from MetaMetrics, when I bumped into this issue. Though currently we are not
> capturing "Top K" table metrics, but we can encounter this issue because of
> the "Top k Clients" that is implemented using the LossyAlgo.
>
> RegionServer gets aborted due to a NPE from MetaTableMetrics coprocessor. The
> log looks somewhat like this:
> {code:java}
> 2019-01-28 23:31:10,311 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> coprocessor.CoprocessorHost: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2019-01-28 23:31:10,314 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> regionserver.HRegionServer: * ABORTING region server
> 10.0.0.24,16020,1548747043814: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException *
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
>
[jira] [Commented] (HBASE-21800) RegionServer aborted due to NPE from MetaTableMetrics coprocessor
[
https://issues.apache.org/jira/browse/HBASE-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755304#comment-16755304
]
Andrew Purtell commented on HBASE-21800:
I think this is a nice improvement. What do you think [~xucang]?
However, please remove debug logs from LossyCounting while we are at it, that
is going to be too chatty in the logs. Otherwise, looks good
> RegionServer aborted due to NPE from MetaTableMetrics coprocessor
> -
>
> Key: HBASE-21800
> URL: https://issues.apache.org/jira/browse/HBASE-21800
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, meta, metrics, Operability
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Critical
> Labels: Meta
> Attachments: hbase-21800.master.001.patch,
> hbase-21800.master.002.patch
>
>
> I was just playing around the code, trying to capture "Top k" table metrics
> from MetaMetrics, when I bumped into this issue. Though currently we are not
> capturing "Top K" table metrics, but we can encounter this issue because of
> the "Top k Clients" that is implemented using the LossyAlgo.
>
> RegionServer gets aborted due to a NPE from MetaTableMetrics coprocessor. The
> log looks somewhat like this:
> {code:java}
> 2019-01-28 23:31:10,311 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> coprocessor.CoprocessorHost: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2019-01-28 23:31:10,314 ERROR
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> regionserver.HRegionServer: * ABORTING region server
> 10.0.0.24,16020,1548747043814: The coprocessor
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw
> java.lang.NullPointerException *
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
> at
>
[jira] [Updated] (HBASE-21791) Upgrade thrift dependency to 0.12.0
[ https://issues.apache.org/jira/browse/HBASE-21791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21791: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: (was: 1.4.10) Status: Resolved (was: Patch Available) Pushed the branch-1 patch. I think that's everything, so resolving this. > Upgrade thrift dependency to 0.12.0 > --- > > Key: HBASE-21791 > URL: https://issues.apache.org/jira/browse/HBASE-21791 > Project: HBase > Issue Type: Task > Components: Thrift >Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 1.2.10, 2.0.4 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0 > > Attachments: HBASE-21791-branch-1.patch, > HBASE-21791-branch-2.1.patch, HBASE-21791.patch > > > As somebody have already known, that there is a CVE for thrift from 0.5.0 to > 0.11.0. > https://nvd.nist.gov/vuln/detail/CVE-2018-1320 > As the CVE is already public, let's upgrade our thrift dependency and release > new versions ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
