[jira] [Updated] (HBASE-21276) hbase scan operation cannot scan some rowkey

2018-10-07 Thread xiaolerzheng (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xiaolerzheng updated HBASE-21276:
-
Description: 
the table ZEUS.LOAN_CONSUMER_CONTACT has some row,

we can get the row from "get",

but we cannot scan it from scan mr and neither can get the row from "scan"  
with timestamp, nor can get the row from 'scan' with timestamp and startrow

hbase(main):001:0> get 'ZEUS.LOAN_CONSUMER_CONTACT', '72520206#KB 2 
83#13971083626'
 COLUMN CELL
 0:BINLOG_TIME timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 0:CREATE_TIME timestamp=1537254107291, value=2018-09-18 15:01:46
 0:LAST_MOD_TIME timestamp=1537254107291, value=2018-09-18 15:01:46
 0:PHONE_NO timestamp=1537254107291, value=13971083626
 0:SOURCE timestamp=1537254107291, value=\x80\x00\x00\x01
 0:UID timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 0:USER_NAME timestamp=1537254107291, 
value=KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\xA5\xE9\xB3\x8583
 0:_0 timestamp=1537254107291, value=x
 8 row(s) in 0.2280 seconds

 

 

hbase(main):002:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT',

{ TIMERANGE => [1537254107291, 1537254107293]}

ROW COLUMN+CELL
 0 row(s) in 1410.9010 seconds

hbase(main):003:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT',

{ TIMERANGE => [1537254107280, 1537254107294]}

ROW COLUMN+CELL
 0 row(s) in 1410.5480 seconds

 

hbase(main):004:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT',

{ STARTROW => '72520206#KB 2 83#13971083626', TIMERANGE => [1537254107280, 
1537254107294]}

ROW COLUMN+CELL
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:BINLOG_TIME, 
timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:CREATE_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:LAST_MOD_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:PHONE_NO, 
timestamp=1537254107291, value=13971083626
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:SOURCE, 
timestamp=1537254107291, value=\x80\x00\x00\x01
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:UID, 
timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:USER_NAME, 
timestamp=1537254107291, 
value=KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\xA5\xE9\xB3\x8583
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:_0, 
timestamp=1537254107291, value=x
 xA5\xE9\xB3\x8583#13971083626
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:BINLOG_TIME, 
timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:CREATE_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:LAST_MOD_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:PHONE_NO, 
timestamp=1537254107291, value=18694056338
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:SOURCE, 
timestamp=1537254107291, value=\x80\x00\x00\x01
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:UID, 
timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:USER_NAME, 
timestamp=1537254107291, value=K\xE6\x96\x8C\xE5\x93\xA5
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:_0, 
timestamp=1537254107291, value=x

  was:
the table ZEUS.LOAN_CONSUMER_CONTACT has some row, we can get the row from 
"get",

but we cannot get the row from "scan"  with timestamp,

but we can get the row from 'scan' with timestamp and startrow

hbase(main):001:0> get 'ZEUS.LOAN_CONSUMER_CONTACT', '72520206#KB 2 
83#13971083626'
 COLUMN CELL
 0:BINLOG_TIME timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 0:CREATE_TIME timestamp=1537254107291, value=2018-09-18 15:01:46
 0:LAST_MOD_TIME timestamp=1537254107291, value=2018-09-18 15:01:46
 0:PHONE_NO timestamp=1537254107291, value=13971083626
 0:SOURCE timestamp=1537254107291, value=\x80\x00\x00\x01
 0:UID timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 0:USER_NAME timestamp=1537254107291, 
value=KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\xA5\xE9\xB3\x8583
 0:_0 timestamp=1537254107291, value=x
 8 row(s) in 0.2280 seconds

 

 

hbase(main):002:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', { TIMERANGE => 
[1537254107291, 1537254107293]}
 ROW COLUMN+CELL
 0 row(s) in 1410.9010 seconds

hbase(main):003:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', { TIMERANGE => 
[1537254107280, 1537254107294]}
 ROW COLUMN+CELL
 0 row(s) in 1410.5480 seconds

 

hbase(main):004:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', { STARTROW => 
'72520206#KB 2 83#139710

[jira] [Updated] (HBASE-21254) Need to find a way to limit the number of proc wal files

2018-10-07 Thread Duo Zhang (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-21254:
--
Attachment: HBASE-21254.patch

> Need to find a way to limit the number of proc wal files
> 
>
> Key: HBASE-21254
> URL: https://issues.apache.org/jira/browse/HBASE-21254
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21254.patch
>
>
> For regionserver, we have a max wal file limitation, if we reach the 
> limitation, we will trigger a flush on specific regions so that we can delete 
> old wal files. But for proc wals, we do not have this mechanism, and it will 
> be worse after HBASE-21233, as if there is an old procedure which can not 
> make progress and do not persist its state, we need to keep the old proc wal 
> file for ever...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-21254) Need to find a way to limit the number of proc wal files

2018-10-07 Thread Duo Zhang (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-21254:
--
Status: Patch Available  (was: Open)

No UT yet. Will add soon.

> Need to find a way to limit the number of proc wal files
> 
>
> Key: HBASE-21254
> URL: https://issues.apache.org/jira/browse/HBASE-21254
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21254.patch
>
>
> For regionserver, we have a max wal file limitation, if we reach the 
> limitation, we will trigger a flush on specific regions so that we can delete 
> old wal files. But for proc wals, we do not have this mechanism, and it will 
> be worse after HBASE-21233, as if there is an old procedure which can not 
> make progress and do not persist its state, we need to keep the old proc wal 
> file for ever...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-21276) hbase scan operation cannot scan some rowkey

2018-10-07 Thread xiaolerzheng (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xiaolerzheng updated HBASE-21276:
-
Description: 
the table ZEUS.LOAN_CONSUMER_CONTACT has some row, we can get the row from 
"get",

but we cannot get the row from "scan"  with timestamp,

but we can get the row from 'scan' with timestamp and startrow

hbase(main):001:0> get 'ZEUS.LOAN_CONSUMER_CONTACT', '72520206#KB 2 
83#13971083626'
 COLUMN CELL
 0:BINLOG_TIME timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 0:CREATE_TIME timestamp=1537254107291, value=2018-09-18 15:01:46
 0:LAST_MOD_TIME timestamp=1537254107291, value=2018-09-18 15:01:46
 0:PHONE_NO timestamp=1537254107291, value=13971083626
 0:SOURCE timestamp=1537254107291, value=\x80\x00\x00\x01
 0:UID timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 0:USER_NAME timestamp=1537254107291, 
value=KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\xA5\xE9\xB3\x8583
 0:_0 timestamp=1537254107291, value=x
 8 row(s) in 0.2280 seconds

 

 

hbase(main):002:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', { TIMERANGE => 
[1537254107291, 1537254107293]}
 ROW COLUMN+CELL
 0 row(s) in 1410.9010 seconds

hbase(main):003:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', { TIMERANGE => 
[1537254107280, 1537254107294]}
 ROW COLUMN+CELL
 0 row(s) in 1410.5480 seconds

 

hbase(main):004:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', { STARTROW => 
'72520206#KB 2 83#13971083626', TIMERANGE => [1537254107280, 1537254107294]}
 ROW COLUMN+CELL
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:BINLOG_TIME, 
timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:CREATE_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:LAST_MOD_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:PHONE_NO, 
timestamp=1537254107291, value=13971083626
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:SOURCE, 
timestamp=1537254107291, value=\x80\x00\x00\x01
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:UID, 
timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:USER_NAME, 
timestamp=1537254107291, 
value=KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\xA5\xE9\xB3\x8583
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:_0, 
timestamp=1537254107291, value=x
 xA5\xE9\xB3\x8583#13971083626
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:BINLOG_TIME, 
timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:CREATE_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:LAST_MOD_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:PHONE_NO, 
timestamp=1537254107291, value=18694056338
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:SOURCE, 
timestamp=1537254107291, value=\x80\x00\x00\x01
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:UID, 
timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:USER_NAME, 
timestamp=1537254107291, value=K\xE6\x96\x8C\xE5\x93\xA5
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:_0, 
timestamp=1537254107291, value=x

  was:
the table ZEUS.LOAN_CONSUMER_CONTACT has some row, we can get the row from 
"get",

but we cannot get the row from "scan"  

hbase(main):001:0> get 'ZEUS.LOAN_CONSUMER_CONTACT', '72520206#KB 2 
83#13971083626'
COLUMN CELL
 0:BINLOG_TIME timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 0:CREATE_TIME timestamp=1537254107291, value=2018-09-18 15:01:46
 0:LAST_MOD_TIME timestamp=1537254107291, value=2018-09-18 15:01:46
 0:PHONE_NO timestamp=1537254107291, value=13971083626
 0:SOURCE timestamp=1537254107291, value=\x80\x00\x00\x01
 0:UID timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 0:USER_NAME timestamp=1537254107291, 
value=KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\xA5\xE9\xB3\x8583
 0:_0 timestamp=1537254107291, value=x
8 row(s) in 0.2280 seconds

 

 

hbase(main):002:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', \{ TIMERANGE => 
[1537254107291, 1537254107293]}
ROW COLUMN+CELL
0 row(s) in 1410.9010 seconds

hbase(main):003:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', \{ TIMERANGE => 
[1537254107280, 1537254107294]}
ROW COLUMN+CELL
0 row(s) in 1410.5480 seconds

 

hbase(main):004:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', \{ STARTROW => 
'72520206#KB 2 83#13971083626', TIMERANGE => [1537254107280, 1537254107294]}
ROW COLUMN+CELL
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ c

[jira] [Created] (HBASE-21276) hbase scan operation cannot scan some rowkey

2018-10-07 Thread xiaolerzheng (JIRA) 
xiaolerzheng created HBASE-21276:


 Summary: hbase scan operation  cannot scan some rowkey
 Key: HBASE-21276
 URL: https://issues.apache.org/jira/browse/HBASE-21276
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.1.3
Reporter: xiaolerzheng


the table ZEUS.LOAN_CONSUMER_CONTACT has some row, we can get the row from 
"get",

but we cannot get the row from "scan"  

hbase(main):001:0> get 'ZEUS.LOAN_CONSUMER_CONTACT', '72520206#KB 2 
83#13971083626'
COLUMN CELL
 0:BINLOG_TIME timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 0:CREATE_TIME timestamp=1537254107291, value=2018-09-18 15:01:46
 0:LAST_MOD_TIME timestamp=1537254107291, value=2018-09-18 15:01:46
 0:PHONE_NO timestamp=1537254107291, value=13971083626
 0:SOURCE timestamp=1537254107291, value=\x80\x00\x00\x01
 0:UID timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 0:USER_NAME timestamp=1537254107291, 
value=KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\xA5\xE9\xB3\x8583
 0:_0 timestamp=1537254107291, value=x
8 row(s) in 0.2280 seconds

 

 

hbase(main):002:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', \{ TIMERANGE => 
[1537254107291, 1537254107293]}
ROW COLUMN+CELL
0 row(s) in 1410.9010 seconds

hbase(main):003:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', \{ TIMERANGE => 
[1537254107280, 1537254107294]}
ROW COLUMN+CELL
0 row(s) in 1410.5480 seconds

 

hbase(main):004:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', \{ STARTROW => 
'72520206#KB 2 83#13971083626', TIMERANGE => [1537254107280, 1537254107294]}
ROW COLUMN+CELL
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:BINLOG_TIME, 
timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:CREATE_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:LAST_MOD_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:PHONE_NO, 
timestamp=1537254107291, value=13971083626
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:SOURCE, 
timestamp=1537254107291, value=\x80\x00\x00\x01
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:UID, 
timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:USER_NAME, 
timestamp=1537254107291, 
value=KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\xA5\xE9\xB3\x8583
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:_0, 
timestamp=1537254107291, value=x
 xA5\xE9\xB3\x8583#13971083626
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:BINLOG_TIME, 
timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:CREATE_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:LAST_MOD_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:PHONE_NO, 
timestamp=1537254107291, value=18694056338
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:SOURCE, 
timestamp=1537254107291, value=\x80\x00\x00\x01
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:UID, 
timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:USER_NAME, 
timestamp=1537254107291, value=K\xE6\x96\x8C\xE5\x93\xA5
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:_0, 
timestamp=1537254107291, value=x



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21230) BackupUtils#checkTargetDir doesn't compose error message correctly

2018-10-07 Thread liubangchen (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641320#comment-16641320
 ] 

liubangchen commented on HBASE-21230:
-

I think so. Let me fix it.

> BackupUtils#checkTargetDir doesn't compose error message correctly
> --
>
> Key: HBASE-21230
> URL: https://issues.apache.org/jira/browse/HBASE-21230
> Project: HBase
>  Issue Type: Bug
>  Components: backup&restore
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Minor
> Attachments: HBASE-21230-1.patch
>
>
> Here is related code from BackupUtils#checkTargetDir:
> {code}
>   String expMsg = e.getMessage();
>   String newMsg = null;
>   if (expMsg.contains("No FileSystem for scheme")) {
> newMsg =
> "Unsupported filesystem scheme found in the backup target url. 
> Error Message: "
> + newMsg;
> {code}
> I think the intention was to concatenate expMsg at the end of newMsg.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-21230) BackupUtils#checkTargetDir doesn't compose error message correctly

2018-10-07 Thread liubangchen (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liubangchen updated HBASE-21230:

Attachment: HBASE-21230-1.patch
Status: Patch Available  (was: Open)

> BackupUtils#checkTargetDir doesn't compose error message correctly
> --
>
> Key: HBASE-21230
> URL: https://issues.apache.org/jira/browse/HBASE-21230
> Project: HBase
>  Issue Type: Bug
>  Components: backup&restore
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Minor
> Attachments: HBASE-21230-1.patch
>
>
> Here is related code from BackupUtils#checkTargetDir:
> {code}
>   String expMsg = e.getMessage();
>   String newMsg = null;
>   if (expMsg.contains("No FileSystem for scheme")) {
> newMsg =
> "Unsupported filesystem scheme found in the backup target url. 
> Error Message: "
> + newMsg;
> {code}
> I think the intention was to concatenate expMsg at the end of newMsg.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HBASE-21269) Forward-port to branch-2 " HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign"

2018-10-07 Thread Jingyun Tian (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingyun Tian reassigned HBASE-21269:


Assignee: Jingyun Tian

> Forward-port to branch-2 " HBASE-21213 [hbck2] bypass leaves behind state 
> in RegionStates when assign/unassign"
> ---
>
> Key: HBASE-21269
> URL: https://issues.apache.org/jira/browse/HBASE-21269
> Project: HBase
>  Issue Type: Sub-task
>  Components: amv2
>Reporter: stack
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21269.branch-2.001.patch
>
>
> A bunch of this patch does not apply to branch-2 and master now we don't have 
> AP or UP anymore. Need to figure if we need override in branch-2 and master. 
> Let me upload the forward-port done so far. Can finish this when move to 
> branch-2.2 exercise. FYI [~Apache9]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21269) Forward-port to branch-2 " HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign"

2018-10-07 Thread stack (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641298#comment-16641298
 ] 

stack commented on HBASE-21269:
---

@jingyoun tian that would be great. 

> Forward-port to branch-2 " HBASE-21213 [hbck2] bypass leaves behind state 
> in RegionStates when assign/unassign"
> ---
>
> Key: HBASE-21269
> URL: https://issues.apache.org/jira/browse/HBASE-21269
> Project: HBase
>  Issue Type: Sub-task
>  Components: amv2
>Reporter: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21269.branch-2.001.patch
>
>
> A bunch of this patch does not apply to branch-2 and master now we don't have 
> AP or UP anymore. Need to figure if we need override in branch-2 and master. 
> Let me upload the forward-port done so far. Can finish this when move to 
> branch-2.2 exercise. FYI [~Apache9]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21269) Forward-port to branch-2 " HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign"

2018-10-07 Thread Jingyun Tian (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641296#comment-16641296
 ] 

Jingyun Tian commented on HBASE-21269:
--

[~stack] Can I take this one? I am currently working on how to make the HBCK 
works for TSRP.

> Forward-port to branch-2 " HBASE-21213 [hbck2] bypass leaves behind state 
> in RegionStates when assign/unassign"
> ---
>
> Key: HBASE-21269
> URL: https://issues.apache.org/jira/browse/HBASE-21269
> Project: HBase
>  Issue Type: Sub-task
>  Components: amv2
>Reporter: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21269.branch-2.001.patch
>
>
> A bunch of this patch does not apply to branch-2 and master now we don't have 
> AP or UP anymore. Need to figure if we need override in branch-2 and master. 
> Let me upload the forward-port done so far. Can finish this when move to 
> branch-2.2 exercise. FYI [~Apache9]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-21216) TestSnapshotFromMaster#testSnapshotHFileArchiving is flaky

2018-10-07 Thread Ted Yu (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-21216:
---
Description: 
>From 
>https://builds.apache.org/job/HBase-Flaky-Tests/job/branch-2/794/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testSnapshotHFileArchiving/
> :
{code}
java.lang.AssertionError: Archived hfiles [] and table hfiles 
[9ca09392705f425f9c916beedc10d63c] is missing snapshot 
file:6739a09747e54189a4112a6d8f37e894
at 
org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:370)
{code}
The file appeared in archive dir before hfile cleaners were run:
{code}
2018-09-20 10:38:53,187 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-archive/
2018-09-20 10:38:53,188 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|data/
2018-09-20 10:38:53,189 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|---default/
2018-09-20 10:38:53,190 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|--test/
2018-09-20 10:38:53,191 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-1237d57b63a7bdf067a930441a02514a/
2018-09-20 10:38:53,192 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|recovered.edits/
2018-09-20 10:38:53,193 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---4.seqid
2018-09-20 10:38:53,193 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-29e1700e09b51223ad2f5811105a4d51/
2018-09-20 10:38:53,194 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|fam/
2018-09-20 10:38:53,195 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---2c66a18f6c1a4074b84ffbb3245268c4
2018-09-20 10:38:53,196 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---45bb396c6a5e49629e45a4d56f1e9b14
2018-09-20 10:38:53,196 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---6739a09747e54189a4112a6d8f37e894
{code}
However, the archive dir became empty after hfile cleaners were run:
{code}
2018-09-20 10:38:53,312 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-archive/
2018-09-20 10:38:53,313 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-corrupt/
{code}
Leading to the assertion failure.

This test is one of the top flaky tests.

  was:
>From 
>https://builds.apache.org/job/HBase-Flaky-Tests/job/branch-2/794/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testSnapshotHFileArchiving/
> :
{code}
java.lang.AssertionError: Archived hfiles [] and table hfiles 
[9ca09392705f425f9c916beedc10d63c] is missing snapshot 
file:6739a09747e54189a4112a6d8f37e894
at 
org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:370)
{code}
The file appeared in archive dir before hfile cleaners were run:
{code}
2018-09-20 10:38:53,187 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-archive/
2018-09-20 10:38:53,188 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|data/
2018-09-20 10:38:53,189 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|---default/
2018-09-20 10:38:53,190 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|--test/
2018-09-20 10:38:53,191 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-1237d57b63a7bdf067a930441a02514a/
2018-09-20 10:38:53,192 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|recovered.edits/
2018-09-20 10:38:53,193 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---4.seqid
2018-09-20 10:38:53,193 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-29e1700e09b51223ad2f5811105a4d51/
2018-09-20 10:38:53,194 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|fam/
2018-09-20 10:38:53,195 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---2c66a18f6c1a4074b84ffbb3245268c4
2018-09-20 10:38:53,196 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---45bb396c6a5e49629e45a4d56f1e9b14
2018-09-20 10:38:53,196 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---6739a09747e54189a4112a6d8f37e894
{code}
However, the archive dir became empty after hfile cleaners were run:
{code}
2018-09-20 10:38:53,312 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-archive/
2018-09-20 10:38:53,313 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-corrupt/
{code}
Leading to the assertion failure.


> TestSnapshotFromMaster#testSnapshotHFileArchiving is flaky
> --
>
> Key: HBASE-21216
> URL: https://issues.apache.org/jira/browse/HBASE-21216
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21216.v1.t

[jira] [Updated] (HBASE-21230) BackupUtils#checkTargetDir doesn't compose error message correctly

2018-10-07 Thread Ted Yu (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-21230:
---
Description: 
Here is related code from BackupUtils#checkTargetDir:
{code}
  String expMsg = e.getMessage();
  String newMsg = null;
  if (expMsg.contains("No FileSystem for scheme")) {
newMsg =
"Unsupported filesystem scheme found in the backup target url. 
Error Message: "
+ newMsg;
{code}
I think the intention was to concatenate expMsg at the end of newMsg.

  was:
Here is related code:
{code}
  String expMsg = e.getMessage();
  String newMsg = null;
  if (expMsg.contains("No FileSystem for scheme")) {
newMsg =
"Unsupported filesystem scheme found in the backup target url. 
Error Message: "
+ newMsg;
{code}
I think the intention was to concatenate expMsg at the end of newMsg.


> BackupUtils#checkTargetDir doesn't compose error message correctly
> --
>
> Key: HBASE-21230
> URL: https://issues.apache.org/jira/browse/HBASE-21230
> Project: HBase
>  Issue Type: Bug
>  Components: backup&restore
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Minor
>
> Here is related code from BackupUtils#checkTargetDir:
> {code}
>   String expMsg = e.getMessage();
>   String newMsg = null;
>   if (expMsg.contains("No FileSystem for scheme")) {
> newMsg =
> "Unsupported filesystem scheme found in the backup target url. 
> Error Message: "
> + newMsg;
> {code}
> I think the intention was to concatenate expMsg at the end of newMsg.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-21238) MapReduceHFileSplitterJob#run shouldn't call System.exit

2018-10-07 Thread Ted Yu (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-21238:
---
Labels: mapreduce  (was: )

> MapReduceHFileSplitterJob#run shouldn't call System.exit
> 
>
> Key: HBASE-21238
> URL: https://issues.apache.org/jira/browse/HBASE-21238
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>  Labels: mapreduce
>
> {code}
> if (args.length < 2) {
>   usage("Wrong number of arguments: " + args.length);
>   System.exit(-1);
> {code}
> Correct way of handling error condition is through return value of run method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21250) Refactor WALProcedureStore and add more comments for better understanding the implementation

2018-10-07 Thread Hudson (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641144#comment-16641144
 ] 

Hudson commented on HBASE-21250:


Results for branch branch-2.1
[build #430 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/430/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/430//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console 
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/430//console].


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/430//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Refactor WALProcedureStore and add more comments for better understanding the 
> implementation
> 
>
> Key: HBASE-21250
> URL: https://issues.apache.org/jira/browse/HBASE-21250
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21250-v1.patch, HBASE-21250-v2.patch, 
> HBASE-21250-v3.patch, HBASE-21250.patch
>
>
> The implementation is complicated and lack of comments to say how it works.
> {code}
> /**
>  * WAL implementation of the ProcedureStore.
>  * @see ProcedureWALPrettyPrinter for printing content of a single WAL.
>  * @see #main(String[]) to parse a directory of MasterWALProcs.
>  */
> {code}
> I think at least we can move sub classes to separated files to make the class 
> smaller, and add more comments to describe what is going on here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21250) Refactor WALProcedureStore and add more comments for better understanding the implementation

2018-10-07 Thread Hudson (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641118#comment-16641118
 ] 

Hudson commented on HBASE-21250:


Results for branch branch-2
[build #1354 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1354/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- Something went wrong running this stage, please [check relevant console 
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1354//console].




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1354//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1354//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Refactor WALProcedureStore and add more comments for better understanding the 
> implementation
> 
>
> Key: HBASE-21250
> URL: https://issues.apache.org/jira/browse/HBASE-21250
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21250-v1.patch, HBASE-21250-v2.patch, 
> HBASE-21250-v3.patch, HBASE-21250.patch
>
>
> The implementation is complicated and lack of comments to say how it works.
> {code}
> /**
>  * WAL implementation of the ProcedureStore.
>  * @see ProcedureWALPrettyPrinter for printing content of a single WAL.
>  * @see #main(String[]) to parse a directory of MasterWALProcs.
>  */
> {code}
> I think at least we can move sub classes to separated files to make the class 
> smaller, and add more comments to describe what is going on here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21250) Refactor WALProcedureStore and add more comments for better understanding the implementation

2018-10-07 Thread Hudson (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641112#comment-16641112
 ] 

Hudson commented on HBASE-21250:


Results for branch branch-2.0
[build #917 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/917/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/917//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console 
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/917//console].


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/917//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Refactor WALProcedureStore and add more comments for better understanding the 
> implementation
> 
>
> Key: HBASE-21250
> URL: https://issues.apache.org/jira/browse/HBASE-21250
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21250-v1.patch, HBASE-21250-v2.patch, 
> HBASE-21250-v3.patch, HBASE-21250.patch
>
>
> The implementation is complicated and lack of comments to say how it works.
> {code}
> /**
>  * WAL implementation of the ProcedureStore.
>  * @see ProcedureWALPrettyPrinter for printing content of a single WAL.
>  * @see #main(String[]) to parse a directory of MasterWALProcs.
>  */
> {code}
> I think at least we can move sub classes to separated files to make the class 
> smaller, and add more comments to describe what is going on here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21275) Thrift Server (branch 1 fix) -> Disable TRACE HTTP method for thrift http server (branch 1 only)

2018-10-07 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641109#comment-16641109
 ] 

Hadoop QA commented on HBASE-21275:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} branch-1.2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m 
25s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
43s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
55s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
33s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
2s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
7s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
49s{color} | {color:red} hbase-thrift: The patch generated 2 new + 24 unchanged 
- 0 fixed = 26 total (was 24) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
22s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
11m 54s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.1 2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
54s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
7s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  6m  
2s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
24s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 44s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:34a9b27 |
| JIRA Issue | HBASE-21275 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12942692/HBASE-21275-branch-1.2.001.patch
 |
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  
shadedjars  hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 03501fc574ae 3.13.0-143-generic #192-Ubunt

[jira] [Commented] (HBASE-20623) Introduce the helper method "getCellBuilder()" to Mutation

2018-10-07 Thread Chia-Ping Tsai (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-20623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641078#comment-16641078
 ] 

Chia-Ping Tsai commented on HBASE-20623:


bq. Sorry for my absence for a long time.I have polished up this patch 
again.the QA report is green.can we move on?
Thanks for the patch. Some comments are already in review board.

> Introduce the helper method "getCellBuilder()" to Mutation
> --
>
> Key: HBASE-20623
> URL: https://issues.apache.org/jira/browse/HBASE-20623
> Project: HBase
>  Issue Type: Task
>  Components: API
>Reporter: Chia-Ping Tsai
>Assignee: maoling
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20623.master.001.patch, 
> HBASE-20623.master.002.patch, HBASE-20623.master.003.patch, 
> HBASE-20623.master.004.patch, HBASE-20623.master.005.patch
>
>
> see 
> [https://lists.apache.org/thread.html/d05bfaa0134502a47f6e1aca56cb0b096d4dd32ddefbbdf28db4952a@%3Cdev.hbase.apache.org%3E]
>  for more details.
> {code:java}
> How about a "getCellBuilder" or "getCellBuilderFactory" method for
> Mutation implementations that gives you a CellBuilder instance that
> already has relevant parts set? Like for a Put instance it should be
> able to already have the Type and Row set.{code}
> mentioned a day or so ago by [~busbey]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20623) Introduce the helper method "getCellBuilder()" to Mutation

2018-10-07 Thread maoling (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-20623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641074#comment-16641074
 ] 

maoling commented on HBASE-20623:
-

[~chia7712] 
Sorry for my absence for a long time.I have polished up this patch again.the QA 
report is green.can we move on?

> Introduce the helper method "getCellBuilder()" to Mutation
> --
>
> Key: HBASE-20623
> URL: https://issues.apache.org/jira/browse/HBASE-20623
> Project: HBase
>  Issue Type: Task
>  Components: API
>Reporter: Chia-Ping Tsai
>Assignee: maoling
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20623.master.001.patch, 
> HBASE-20623.master.002.patch, HBASE-20623.master.003.patch, 
> HBASE-20623.master.004.patch, HBASE-20623.master.005.patch
>
>
> see 
> [https://lists.apache.org/thread.html/d05bfaa0134502a47f6e1aca56cb0b096d4dd32ddefbbdf28db4952a@%3Cdev.hbase.apache.org%3E]
>  for more details.
> {code:java}
> How about a "getCellBuilder" or "getCellBuilderFactory" method for
> Mutation implementations that gives you a CellBuilder instance that
> already has relevant parts set? Like for a Put instance it should be
> able to already have the Type and Row set.{code}
> mentioned a day or so ago by [~busbey]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21219) Hbase incremental backup fails with null pointer exception

2018-10-07 Thread Hudson (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641050#comment-16641050
 ] 

Hudson commented on HBASE-21219:


Results for branch master
[build #530 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/530/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- Something went wrong running this stage, please [check relevant console 
output|https://builds.apache.org/job/HBase%20Nightly/job/master/530//console].




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/530//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/530//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Hbase incremental backup fails with null pointer exception
> --
>
> Key: HBASE-21219
> URL: https://issues.apache.org/jira/browse/HBASE-21219
> Project: HBase
>  Issue Type: Bug
>  Components: backup&restore
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21219-v1.patch
>
>
> hbase backup create incremental hdfs:///bkpHbase_Test/bkpHbase_Test2 -t 
> bkpHbase_Test2 
> 2018-09-21 15:35:31,421 INFO [main] impl.TableBackupClient: Backup 
> backup_1537524313995 started at 1537524331419. 2018-09-21 15:35:31,454 INFO 
> [main] impl.IncrementalBackupManager: Execute roll log procedure for 
> incremental backup ... 2018-09-21 15:35:32,985 ERROR [main] 
> impl.TableBackupClient: Unexpected Exception : java.lang.NullPointerException 
> java.lang.NullPointerException at 
> org.apache.hadoop.hbase.backup.impl.IncrementalBackupManager.getLogFilesForNewBackup(IncrementalBackupManager.java:309)
>  at 
> org.apache.hadoop.hbase.backup.impl.IncrementalBackupManager.getIncrBackupLogFileMap(IncrementalBackupManager.java:103)
>  at 
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:276)
>  at 
> org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:601)
>  at 
> org.apache.hadoop.hbase.backup.impl.BackupCommands$CreateCommand.execute(BackupCommands.java:347)
>  at 
> org.apache.hadoop.hbase.backup.BackupDriver.parseAndRun(BackupDriver.java:138)
>  at org.apache.hadoop.hbase.backup.BackupDriver.doWork(BackupDriver.java:171) 
> at org.apache.hadoop.hbase.backup.BackupDriver.run(BackupDriver.java:204) at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at 
> org.apache.hadoop.hbase.backup.BackupDriver.main(BackupDriver.java:179) 
> 2018-09-21 15:35:32,989 ERROR [main] impl.TableBackupClient: 
> BackupId=backup_1537524313995,startts=1537524331419,failedts=1537524332989,failedphase=PREPARE_INCREMENTAL,failedmessage=null
>  2018-09-21 15:35:57,167 ERROR [main] impl.TableBackupClient: Backup 
> backup_1537524313995 failed. 
> Backup session finished. Status: FAILURE 2018-09-21 15:35:57,175 ERROR [main] 
> backup.BackupDriver: Error running 
> command-line tool java.io.IOException: java.lang.NullPointerException at 
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:281)
>  at 
> org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:601)
>  at 
> org.apache.hadoop.hbase.backup.impl.BackupCommands$CreateCommand.execute(BackupCommands.java:347)
>  at 
> org.apache.hadoop.hbase.backup.BackupDriver.parseAndRun(BackupDriver.java:138)
>  at org.apache.hadoop.hbase.backup.BackupDriver.doWork(BackupDriver.java:171) 
> at org.apache.hadoop.hbase.backup.BackupDriver.run(BackupDriver.java:204) at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at 
> org.apache.hadoop.hbase.backup.BackupDriver.main(BackupDriver.java:179) 
> Caused by: java.lang.NullPointerException at 
> org.apache.hadoop.hbase.backup.impl.IncrementalBackupManager.getLogFilesForNewBackup(IncrementalBackupManager.java:309)
>  at 
> org.apache.hadoop.hbase.backup.impl.IncrementalBackupManager.getIncrBackupLogFileMap(IncrementalBackupManager.java:103)
>  at 
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:276)
>  ... 7 more



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-21250) Refactor WALProcedureStore and add more comments for better understanding the implementation

2018-10-07 Thread Duo Zhang (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-21250:
--
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to branch-2.0+.

Thanks [~stack] and [~allan163] for reviewing.

> Refactor WALProcedureStore and add more comments for better understanding the 
> implementation
> 
>
> Key: HBASE-21250
> URL: https://issues.apache.org/jira/browse/HBASE-21250
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21250-v1.patch, HBASE-21250-v2.patch, 
> HBASE-21250-v3.patch, HBASE-21250.patch
>
>
> The implementation is complicated and lack of comments to say how it works.
> {code}
> /**
>  * WAL implementation of the ProcedureStore.
>  * @see ProcedureWALPrettyPrinter for printing content of a single WAL.
>  * @see #main(String[]) to parse a directory of MasterWALProcs.
>  */
> {code}
> I think at least we can move sub classes to separated files to make the class 
> smaller, and add more comments to describe what is going on here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HBASE-21254) Need to find a way to limit the number of proc wal files

2018-10-07 Thread Duo Zhang (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang reassigned HBASE-21254:
-

Assignee: Duo Zhang

> Need to find a way to limit the number of proc wal files
> 
>
> Key: HBASE-21254
> URL: https://issues.apache.org/jira/browse/HBASE-21254
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
>
> For regionserver, we have a max wal file limitation, if we reach the 
> limitation, we will trigger a flush on specific regions so that we can delete 
> old wal files. But for proc wals, we do not have this mechanism, and it will 
> be worse after HBASE-21233, as if there is an old procedure which can not 
> make progress and do not persist its state, we need to keep the old proc wal 
> file for ever...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21250) Refactor WALProcedureStore and add more comments for better understanding the implementation

2018-10-07 Thread Duo Zhang (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641004#comment-16641004
 ] 

Duo Zhang commented on HBASE-21250:
---

Let me commit.

> Refactor WALProcedureStore and add more comments for better understanding the 
> implementation
> 
>
> Key: HBASE-21250
> URL: https://issues.apache.org/jira/browse/HBASE-21250
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21250-v1.patch, HBASE-21250-v2.patch, 
> HBASE-21250-v3.patch, HBASE-21250.patch
>
>
> The implementation is complicated and lack of comments to say how it works.
> {code}
> /**
>  * WAL implementation of the ProcedureStore.
>  * @see ProcedureWALPrettyPrinter for printing content of a single WAL.
>  * @see #main(String[]) to parse a directory of MasterWALProcs.
>  */
> {code}
> I think at least we can move sub classes to separated files to make the class 
> smaller, and add more comments to describe what is going on here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-21250) Refactor WALProcedureStore and add more comments for better understanding the implementation

2018-10-07 Thread Duo Zhang (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-21250:
--
Fix Version/s: 2.0.3
   2.1.1

> Refactor WALProcedureStore and add more comments for better understanding the 
> implementation
> 
>
> Key: HBASE-21250
> URL: https://issues.apache.org/jira/browse/HBASE-21250
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21250-v1.patch, HBASE-21250-v2.patch, 
> HBASE-21250-v3.patch, HBASE-21250.patch
>
>
> The implementation is complicated and lack of comments to say how it works.
> {code}
> /**
>  * WAL implementation of the ProcedureStore.
>  * @see ProcedureWALPrettyPrinter for printing content of a single WAL.
>  * @see #main(String[]) to parse a directory of MasterWALProcs.
>  */
> {code}
> I think at least we can move sub classes to separated files to make the class 
> smaller, and add more comments to describe what is going on here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)