[jira] [Commented] (HBASE-20817) Infinite loop when executing ReopenTableRegionsProcedure
[ https://issues.apache.org/jira/browse/HBASE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529451#comment-16529451 ] Duo Zhang commented on HBASE-20817: --- Good. I think the fix here is OK. For the region replica feature, AFAICT, AMv2 does not work well with it. So I tend to add a note in our ref guide about this. Let me try the patch here and also verify the UT. Thanks a lot. > Infinite loop when executing ReopenTableRegionsProcedure > - > > Key: HBASE-20817 > URL: https://issues.apache.org/jira/browse/HBASE-20817 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.1.0, 2.0.2, 2.2.0 > > Attachments: HBASE-20817.patch > > > As discussed in HBASE-20792, it seems that a region's openSeqNum could remain > the same after a sucessful reopen, which causes the RTRP loop infinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20805) Document for CreateSnapshot
[ https://issues.apache.org/jira/browse/HBASE-20805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingdao Yang reassigned HBASE-20805: Assignee: Mingdao Yang > Document for CreateSnapshot > --- > > Key: HBASE-20805 > URL: https://issues.apache.org/jira/browse/HBASE-20805 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Assignee: Mingdao Yang >Priority: Major > > Add documentation for CreateSnapshot. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529364#comment-16529364 ] Kuan-Po Tseng commented on HBASE-18201: --- No problem, I will add comments for every bug fixes, thanks for your patience. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[
https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529362#comment-16529362
]
Reid Chan commented on HBASE-18201:
---
Dug deeper, and got your concern.
Would you mind adding some comments {{// ...}} on the if branch why this branch
is needed.
> add UT and docs for DataBlockEncodingTool
> -
>
> Key: HBASE-18201
> URL: https://issues.apache.org/jira/browse/HBASE-18201
> Project: HBase
> Issue Type: Sub-task
> Components: tooling
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-18201.master.001.patch,
> HBASE-18201.master.002.patch, HBASE-18201.master.002.patch,
> HBASE-18201.master.003.patch
>
>
> There is no example, documents, or tests for DataBlockEncodingTool. We should
> have it friendly if any use case exists. Otherwise, we should just get rid of
> it because DataBlockEncodingTool presumes that the implementation of cell
> returned from DataBlockEncoder is KeyValue. The presume may obstruct the
> cleanup of KeyValue references in the code base of read/write path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[
https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529358#comment-16529358
]
Reid Chan commented on HBASE-18201:
---
Got your point.
But still we can avoid this if branch: {{return baos.toByteArray()}}, or add a
finally block {{baosBytes = baos.toByteArray()}}.
> add UT and docs for DataBlockEncodingTool
> -
>
> Key: HBASE-18201
> URL: https://issues.apache.org/jira/browse/HBASE-18201
> Project: HBase
> Issue Type: Sub-task
> Components: tooling
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-18201.master.001.patch,
> HBASE-18201.master.002.patch, HBASE-18201.master.002.patch,
> HBASE-18201.master.003.patch
>
>
> There is no example, documents, or tests for DataBlockEncodingTool. We should
> have it friendly if any use case exists. Otherwise, we should just get rid of
> it because DataBlockEncodingTool presumes that the implementation of cell
> returned from DataBlockEncoder is KeyValue. The presume may obstruct the
> cleanup of KeyValue references in the code base of read/write path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20808) Wrong shutdown order between Chores and ChoreService
[
https://issues.apache.org/jira/browse/HBASE-20808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529353#comment-16529353
]
Reid Chan commented on HBASE-20808:
---
Does code base has any parallel clean up or shutdown? {{(one-by-one)}} this
comment is unnecessary.
> Wrong shutdown order between Chores and ChoreService
>
>
> Key: HBASE-20808
> URL: https://issues.apache.org/jira/browse/HBASE-20808
> Project: HBase
> Issue Type: Bug
>Reporter: Reid Chan
>Assignee: Nihal Jain
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-20808.master.001.patch,
> HBASE-20808.master.002.patch
>
>
> When stopping master, {{ChoreService}}, which serves all the chores, is
> stopped before canceling all running chores.
> It should cancel all running chores, then shutdown {{ChoreService}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[
https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529342#comment-16529342
]
Kuan-Po Tseng commented on HBASE-18201:
---
Sorry, I gave the wrong explain. The probolm is in
{code:java}
baosBytes = baos.toByteArray();
{code}
Encoder ROW_INDEX_V1 write integer in outputStream while the others write
integer in byte array(baosBytes), we should change code like this
{code:java}
baos.flush();
baosBytes = baos.toByteArray();
this.dataBlockEncoder.endBlockEncoding(encodingCtx, out, baosBytes);
if(this.encodingCtx.getDataBlockEncoding().equals(DataBlockEncoding.ROW_INDEX_V1))
{
baosBytes = baos.toByteArray();
}{code}
Add baos.toByteArray() after endBlockEncoding again to make sure the integer
writes in outputstream with Encoder ROW_INDEX_V1 dump to byte array (baosBytes).
The if branch is neccessary because Encoders excepts ROW_INDEX_V1 write integer
in baosBytes directly, without if branch and do toByteArray() again, baosBytes
won't contains the integer wrotten in endBlockEncoding.
> add UT and docs for DataBlockEncodingTool
> -
>
> Key: HBASE-18201
> URL: https://issues.apache.org/jira/browse/HBASE-18201
> Project: HBase
> Issue Type: Sub-task
> Components: tooling
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-18201.master.001.patch,
> HBASE-18201.master.002.patch, HBASE-18201.master.002.patch,
> HBASE-18201.master.003.patch
>
>
> There is no example, documents, or tests for DataBlockEncodingTool. We should
> have it friendly if any use case exists. Otherwise, we should just get rid of
> it because DataBlockEncodingTool presumes that the implementation of cell
> returned from DataBlockEncoder is KeyValue. The presume may obstruct the
> cleanup of KeyValue references in the code base of read/write path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20746) Release 2.1.0
[ https://issues.apache.org/jira/browse/HBASE-20746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529333#comment-16529333 ] Duo Zhang commented on HBASE-20746: --- The link to track unfinished issues https://issues.apache.org/jira/browse/HBASE-20826?jql=project%20%3D%20HBASE%20AND%20fixVersion%20%3D%202.1.0%20AND%20resolution%20%3D%20Unresolved > Release 2.1.0 > - > > Key: HBASE-20746 > URL: https://issues.apache.org/jira/browse/HBASE-20746 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > After HBASE-20708 I do no think we will have unresolvable problems for 2.1.0 > release any more. So let's create a issue to track the release processing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20691) Storage policy should allow deferring to HDFS
[ https://issues.apache.org/jira/browse/HBASE-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529331#comment-16529331 ] Yu Li commented on HBASE-20691: --- Ping, mind take a look here and give your +1s if no more comments? [~busbey] [~mdrob] Thanks. To be honest, this one has taken way more time than expected and I'd like to close it ASAP (smile). > Storage policy should allow deferring to HDFS > - > > Key: HBASE-20691 > URL: https://issues.apache.org/jira/browse/HBASE-20691 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Affects Versions: 1.5.0, 2.0.0 >Reporter: Sean Busbey >Assignee: Yu Li >Priority: Minor > Fix For: 2.1.0, 1.5.0 > > Attachments: HBASE-20691.patch, HBASE-20691.v2.patch, > HBASE-20691.v3.patch, HBASE-20691.v4.patch, HBASE-20691.v5.patch, > HBASE-20691.v6.patch > > > In HBase 1.1 - 1.4 we can defer storage policy decisions to HDFS by using > "NONE" as the storage policy in hbase configs. > As described on this [dev@hbase thread "WAL storage policies and interactions > with Hadoop admin > tools."|https://lists.apache.org/thread.html/d220726fab4bb4c9e117ecc8f44246402dd97bfc986a57eb2237@%3Cdev.hbase.apache.org%3E] > we no longer have that option in 2.0.0 and 1.5.0 (as the branch is now). > Additionally, we can't set the policy to HOT in the event that HDFS has > changed the policy for a parent directory of our WALs. > We should put back that ability. Presuming this is done by re-adopting the > "NONE" placeholder variable, we need to ensure that value doesn't get passed > to HDFS APIs. Since it isn't a valid storage policy attempting to use it will > cause a bunch of logging churn (which will be a regression of the problem > HBASE-18118 sought to fix). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20789) TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky
[ https://issues.apache.org/jira/browse/HBASE-20789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529330#comment-16529330 ] Duo Zhang commented on HBASE-20789: --- Left some comments on RB. > TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky > --- > > Key: HBASE-20789 > URL: https://issues.apache.org/jira/browse/HBASE-20789 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.6, 2.0.2 > > Attachments: > 0001-HBASE-20789-TestBucketCache-testCacheBlockNextBlockM.patch, > HBASE-20789.v1.patch, HBASE-20789.v2.patch, HBASE-20789.v3.patch, > bucket-33718.out > > > The UT failed frequently in our internal branch-2... Will dig into the UT. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20816) Run ITBLL for branch-2.1
[ https://issues.apache.org/jira/browse/HBASE-20816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529325#comment-16529325 ] Duo Zhang commented on HBASE-20816: --- This time we finished 3 iterations, each with 1B rows. And the fourth iteration failed at very stage because of call timeout exception and RegionOpeningException, and I checked the cluster status this morning, it is fine, all regions are onlined. So I think it is just overloaded, no critical bugs. > Run ITBLL for branch-2.1 > > > Key: HBASE-20816 > URL: https://issues.apache.org/jira/browse/HBASE-20816 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20789) TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky
[ https://issues.apache.org/jira/browse/HBASE-20789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529314#comment-16529314 ] Zheng Hu commented on HBASE-20789: -- [~zyork], [~Apache9] any other concern ? Thanks. > TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky > --- > > Key: HBASE-20789 > URL: https://issues.apache.org/jira/browse/HBASE-20789 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.6, 2.0.2 > > Attachments: > 0001-HBASE-20789-TestBucketCache-testCacheBlockNextBlockM.patch, > HBASE-20789.v1.patch, HBASE-20789.v2.patch, HBASE-20789.v3.patch, > bucket-33718.out > > > The UT failed frequently in our internal branch-2... Will dig into the UT. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20817) Infinite loop when executing ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529306#comment-16529306
]
Ankit Singhal commented on HBASE-20817:
---
bq. Hey Josh Elser, is this your case? Read only table?
[~Apache9], Josh is on holiday so not sure if he will access his mailbox for
next couple of days. but from his last comments, it seems we are talking about
the same tables(table_x) created by Atlas/Titan.
{code}
2018-06-27 17:16:25,270 INFO [PEWorker-16] procedure2.ProcedureExecutor:
Finished subprocedure(s) of pid=15024, ppid=15023,
state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
ReopenTableRegionsProcedure table=table_izljd; resume parent processing.
2018-06-27 17:16:25,270 INFO [PEWorker-16] procedure2.ProcedureExecutor:
Finished pid=320739, ppid=15024, state=SUCCESS; MoveRegionProcedure
hri=523007f77f96474d01d74ed3d048e173,
source=ctr-e138-1518143905142-381863-01-02.hwx.site,16020,1530113920688,
destination=ctr-e138-1518143905142-381863-01-02.hwx.site,16020,1530113920688
in 497msec
2018-06-27 17:16:25,279 INFO [PEWorker-2] procedure2.ProcedureExecutor:
Initialized subprocedures=[{pid=320742, ppid=15024,
state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
hri=523007f77f96474d01d74ed3d048e173,
source=ctr-e138-1518143905142-381863-01-02.hwx.site,16020,1530113920688,
destination=ctr-e138-1518143905142-381863-01-02.hwx.site,16020,1530113920688}]
2018-06-27 17:16:25,291 INFO [PEWorker-2] procedure.MasterProcedureScheduler:
pid=320742, ppid=15024, state=RUNNABLE:MOVE_REGION_UNASSIGN;
MoveRegionProcedure hri=523007f77f96474d01d74ed3d048e173,
source=ctr-e138-1518143905142-381863-01-02.hwx.site,16020,1530113920688,
destination=ctr-e138-1518143905142-381863-01-02.hwx.site,16020,1530113920688
checking lock on 523007f77f96474d01d74ed3d048e173
2018-06-27 17:16:25,291 INFO [PEWorker-2] procedure2.ProcedureExecutor:
Initialized subprocedures=[{pid=320743, ppid=320742,
state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure table=table_izljd,
region=523007f77f96474d01d74ed3d048e173,
server=ctr-e138-1518143905142-381863-01-02.hwx.site,16020,1530113920688}]
2018-06-27 17:16:25,292 INFO [PEWorker-2] procedure.MasterProcedureScheduler:
pid=320743, ppid=320742, state=RUNNABLE:REGION_TRANSITION_DISPATCH;
UnassignProcedure table=table_izljd, region=523007f77f96474d01d74ed3d048e173,
server=ctr-e138-1518143905142-381863-01-02.hwx.site,16020,1530113920688
checking lock on 523007f77f96474d01d74ed3d048e173
2018-06-27 17:16:25,292 INFO [PEWorker-2] assignment.RegionStateStore:
pid=320743 updating hbase:meta row=523007f77f96474d01d74ed3d048e173,
regionState=CLOSING
{code}
> Infinite loop when executing ReopenTableRegionsProcedure
> -
>
> Key: HBASE-20817
> URL: https://issues.apache.org/jira/browse/HBASE-20817
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.2, 2.2.0
>
> Attachments: HBASE-20817.patch
>
>
> As discussed in HBASE-20792, it seems that a region's openSeqNum could remain
> the same after a sucessful reopen, which causes the RTRP loop infinitely.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20817) Infinite loop when executing ReopenTableRegionsProcedure
[ https://issues.apache.org/jira/browse/HBASE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529300#comment-16529300 ] Duo Zhang commented on HBASE-20817: --- Hey [~elserj], is this your case? Read only table? > Infinite loop when executing ReopenTableRegionsProcedure > - > > Key: HBASE-20817 > URL: https://issues.apache.org/jira/browse/HBASE-20817 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.1.0, 2.0.2, 2.2.0 > > Attachments: HBASE-20817.patch > > > As discussed in HBASE-20792, it seems that a region's openSeqNum could remain > the same after a sucessful reopen, which causes the RTRP loop infinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20817) Infinite loop when executing ReopenTableRegionsProcedure
[ https://issues.apache.org/jira/browse/HBASE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529298#comment-16529298 ] Duo Zhang commented on HBASE-20817: --- Checked the code, I think for read only table it is OK to write open region marker, and also increase the openSeqNum, as there is no actual data written. But there are also region replicas... Let me check the code carefully. > Infinite loop when executing ReopenTableRegionsProcedure > - > > Key: HBASE-20817 > URL: https://issues.apache.org/jira/browse/HBASE-20817 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.1.0, 2.0.2, 2.2.0 > > Attachments: HBASE-20817.patch > > > As discussed in HBASE-20792, it seems that a region's openSeqNum could remain > the same after a sucessful reopen, which causes the RTRP loop infinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20828) Finish-up AMv2 Design/List of Tenets/Specification of operation
[ https://issues.apache.org/jira/browse/HBASE-20828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529295#comment-16529295 ] Duo Zhang commented on HBASE-20828: --- Talked with [~stack] offline with the high level design of AMv2, this is what we think is useful to make the implementation cleaner 1. Region will be created with CLOSED state. 2. For a region with a state other than OPEN state, there will be a RIT associated with it, and it will finally be transited to OPEN state(unless we are disabling the table) 3. Regions for a disabled table are in CLOSED state. 4. The typical transiting path: OPEN -> CLOSING -> CLOSED -> OPENING -> OPEN. 5. If a server is crashed, all regions on it will be transited to CLOSED directly without CLOSING state. 6. A RIT procedure will never fail, unless the target server is crashed, i.e, only SCP can break the execution of RIT procedure. #6 is the most difficult part here. What in my mind is that, let's get rid of the AssignProcedure, UnassignProcedure, MoveRegionProcedure, and only introduce a single RegionTransition procedure. It can cover all the transiting life cycle of a region, and can also start/stop at a particular state, i.e, for creating a table or SCP, we could start from CLOSED state directly, and if we are disabling a table, we could let it stop at CLOSED state. The advantage here will be that, we can make sure that there is only one RIT procedure for a region. In the old time, MRP is not a RIT so we may miss it... And also, the logic for breaking the execution of the RIT in SCP will be easier, if there is already a RIT for it, just tell it what to do, if not, schedule one. We do not need to fear that if there is a MRP and after we fail the UnassignProcedure it will schedule a AssignProcedure soon and cause something wrong... > Finish-up AMv2 Design/List of Tenets/Specification of operation > --- > > Key: HBASE-20828 > URL: https://issues.apache.org/jira/browse/HBASE-20828 > Project: HBase > Issue Type: Umbrella > Components: amv2 >Reporter: stack >Priority: Major > > AMv2 is missing specification. There are too many grey-areas still. Also > missing are a concise listing of the tenets of AMv2 operation. Here are some > examples: > * HBASE-19529 "Handle null states in AM": Asks how we should treat null > state in hbase:meta. What does it 'mean'. We seem to treat it differently > dependent on context. Needs clarification. [~Apache9] recently asked similar > about the meaning of OFFLINE. > * Logging needs to have a particular form to help trace Procedure progress; > needs a write-up. > Lets fill in items to address in this umbrella issue. Can address in > subissues and produce specification doc too. We have the below but these are > mostly (incomplete) description for devs on pv2 and amv2; the specification > is missing: > http://hbase.apache.org/book.html#pv2 > http://hbase.apache.org/book.html#amv2 > (Other areas include addressing what is up w/ rollback -- when, how much, and > when it is not appropriate -- as well as recommendation on Procedures > coarseness, locking -- is it ok to lock table in alter table procedure for > the life of the procedure? -- and so on). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20542) Better heap utilization for IMC with MSLABs
[ https://issues.apache.org/jira/browse/HBASE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529293#comment-16529293 ] Duo Zhang commented on HBASE-20542: --- [~stack] has been doing the performance testing for 2.0.x, there is no much difference between 2.1.x and 2.0.x I believe, most performance related patches are applied to both branches. Turn on IMC by default maybe too late for 2.1.0, let's do it in 2.2? We have also planned to fix the AMv2 on branch-2, see HBASE-20828, so 2.1 will not be the last minor release line of 2.x. Take it easy. > Better heap utilization for IMC with MSLABs > --- > > Key: HBASE-20542 > URL: https://issues.apache.org/jira/browse/HBASE-20542 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-20542.branch-2.001.patch, > HBASE-20542.branch-2.003.patch, HBASE-20542.branch-2.004.patch, > HBASE-20542.branch-2.005.patch, HBASE-20542.master.003.patch, run.sh, > workloada, workloadc, workloadx, workloady > > > Following HBASE-20188 we realized in-memory compaction combined with MSLABs > may suffer from heap under-utilization due to internal fragmentation. This > jira presents a solution to circumvent this problem. The main idea is to have > each update operation check if it will cause overflow in the active segment > *before* it is writing the new value (instead of checking the size after the > write is completed), and if it is then the active segment is atomically > swapped with a new empty segment, and is pushed (full-yet-not-overflowed) to > the compaction pipeline. Later on the IMC deamon will run its compaction > operation (flatten index/merge indices/data compaction) in the background. > Some subtle concurrency issues should be handled with care. We next elaborate > on them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20817) Infinite loop when executing ReopenTableRegionsProcedure
[ https://issues.apache.org/jira/browse/HBASE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529289#comment-16529289 ] Duo Zhang commented on HBASE-20817: --- Oh, read only table is a problem... Let me check the code first, not very familiar with it. Thanks for pointing this out. > Infinite loop when executing ReopenTableRegionsProcedure > - > > Key: HBASE-20817 > URL: https://issues.apache.org/jira/browse/HBASE-20817 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.1.0, 2.0.2, 2.2.0 > > Attachments: HBASE-20817.patch > > > As discussed in HBASE-20792, it seems that a region's openSeqNum could remain > the same after a sucessful reopen, which causes the RTRP loop infinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20817) Infinite loop when executing ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529280#comment-16529280
]
Hadoop QA commented on HBASE-20817:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-20817 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20817 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12929889/HBASE-20817.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13464/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Infinite loop when executing ReopenTableRegionsProcedure
> -
>
> Key: HBASE-20817
> URL: https://issues.apache.org/jira/browse/HBASE-20817
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.2, 2.2.0
>
> Attachments: HBASE-20817.patch
>
>
> As discussed in HBASE-20792, it seems that a region's openSeqNum could remain
> the same after a sucessful reopen, which causes the RTRP loop infinitely.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20817) Infinite loop when executing ReopenTableRegionsProcedure
[ https://issues.apache.org/jira/browse/HBASE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal updated HBASE-20817: -- Affects Version/s: 2.0.0 Status: Patch Available (was: Open) > Infinite loop when executing ReopenTableRegionsProcedure > - > > Key: HBASE-20817 > URL: https://issues.apache.org/jira/browse/HBASE-20817 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.1.0, 2.0.2, 2.2.0 > > Attachments: HBASE-20817.patch > > > As discussed in HBASE-20792, it seems that a region's openSeqNum could remain > the same after a sucessful reopen, which causes the RTRP loop infinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20817) Infinite loop when executing ReopenTableRegionsProcedure
[ https://issues.apache.org/jira/browse/HBASE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529275#comment-16529275 ] Ankit Singhal commented on HBASE-20817: --- [~Apache9], [~stack], have attached a test case which is failing if we are not updating max sequence id. Is there any impact of updating the sequence id file for READ ONLY table? > Infinite loop when executing ReopenTableRegionsProcedure > - > > Key: HBASE-20817 > URL: https://issues.apache.org/jira/browse/HBASE-20817 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.1.0, 2.0.2, 2.2.0 > > Attachments: HBASE-20817.patch > > > As discussed in HBASE-20792, it seems that a region's openSeqNum could remain > the same after a sucessful reopen, which causes the RTRP loop infinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20817) Infinite loop when executing ReopenTableRegionsProcedure
[ https://issues.apache.org/jira/browse/HBASE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal updated HBASE-20817: -- Attachment: HBASE-20817.patch > Infinite loop when executing ReopenTableRegionsProcedure > - > > Key: HBASE-20817 > URL: https://issues.apache.org/jira/browse/HBASE-20817 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.1.0, 2.0.2, 2.2.0 > > Attachments: HBASE-20817.patch > > > As discussed in HBASE-20792, it seems that a region's openSeqNum could remain > the same after a sucessful reopen, which causes the RTRP loop infinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20542) Better heap utilization for IMC with MSLABs
[
https://issues.apache.org/jira/browse/HBASE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529252#comment-16529252
]
Hudson commented on HBASE-20542:
Results for branch branch-2
[build #932 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/932/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/932//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/932//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/932//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Better heap utilization for IMC with MSLABs
> ---
>
> Key: HBASE-20542
> URL: https://issues.apache.org/jira/browse/HBASE-20542
> Project: HBase
> Issue Type: Sub-task
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Attachments: HBASE-20542.branch-2.001.patch,
> HBASE-20542.branch-2.003.patch, HBASE-20542.branch-2.004.patch,
> HBASE-20542.branch-2.005.patch, HBASE-20542.master.003.patch, run.sh,
> workloada, workloadc, workloadx, workloady
>
>
> Following HBASE-20188 we realized in-memory compaction combined with MSLABs
> may suffer from heap under-utilization due to internal fragmentation. This
> jira presents a solution to circumvent this problem. The main idea is to have
> each update operation check if it will cause overflow in the active segment
> *before* it is writing the new value (instead of checking the size after the
> write is completed), and if it is then the active segment is atomically
> swapped with a new empty segment, and is pushed (full-yet-not-overflowed) to
> the compaction pipeline. Later on the IMC deamon will run its compaction
> operation (flatten index/merge indices/data compaction) in the background.
> Some subtle concurrency issues should be handled with care. We next elaborate
> on them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20817) Infinite loop when executing ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529247#comment-16529247
]
Ankit Singhal commented on HBASE-20817:
---
For Atlas test, it is going in an infinite loop because we are not updating the
seqId in the WAL for READ ONLY tables and our Reopen table procedure keep on
spawning MRP.
HRegion#openHRegion
{code}
if (wal != null && getRegionServerServices() != null && !writestate.readOnly) {
// Only write the region open event marker to WAL if we are not read-only.
writeRegionOpenMarker(wal, openSeqNum);
}
{code}
> Infinite loop when executing ReopenTableRegionsProcedure
> -
>
> Key: HBASE-20817
> URL: https://issues.apache.org/jira/browse/HBASE-20817
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Reporter: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.2, 2.2.0
>
>
> As discussed in HBASE-20792, it seems that a region's openSeqNum could remain
> the same after a sucessful reopen, which causes the RTRP loop infinitely.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20542) Better heap utilization for IMC with MSLABs
[ https://issues.apache.org/jira/browse/HBASE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529187#comment-16529187 ] Eshcar Hillel commented on HBASE-20542: --- Thanks [~Apache9], pushed the patch to master and branch-2. So what are the plans for testing performance of 2.1.0 and ensuing releases? Is it all in the context of HBASE-20188? Running load (a) and then workloada and workloadc of YCSB? Would you also consider workloadx? The reason I ask is that we would like to suggest setting IMC as default again. > Better heap utilization for IMC with MSLABs > --- > > Key: HBASE-20542 > URL: https://issues.apache.org/jira/browse/HBASE-20542 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-20542.branch-2.001.patch, > HBASE-20542.branch-2.003.patch, HBASE-20542.branch-2.004.patch, > HBASE-20542.branch-2.005.patch, HBASE-20542.master.003.patch, run.sh, > workloada, workloadc, workloadx, workloady > > > Following HBASE-20188 we realized in-memory compaction combined with MSLABs > may suffer from heap under-utilization due to internal fragmentation. This > jira presents a solution to circumvent this problem. The main idea is to have > each update operation check if it will cause overflow in the active segment > *before* it is writing the new value (instead of checking the size after the > write is completed), and if it is then the active segment is atomically > swapped with a new empty segment, and is pushed (full-yet-not-overflowed) to > the compaction pipeline. Later on the IMC deamon will run its compaction > operation (flatten index/merge indices/data compaction) in the background. > Some subtle concurrency issues should be handled with care. We next elaborate > on them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20804) Document and add tests for HBaseConfTool
[
https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529183#comment-16529183
]
Hadoop QA commented on HBASE-20804:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
28s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
51s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
4s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
44s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 6m
41s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 45s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}280m
24s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
58s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}370m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20804 |
| JIRA Patch URL |
https://issue
[jira] [Commented] (HBASE-18477) Umbrella JIRA for HBase Read Replica clusters
[
https://issues.apache.org/jira/browse/HBASE-18477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529141#comment-16529141
]
Hudson commented on HBASE-18477:
Results for branch HBASE-18477
[build #251 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/251/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/251//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/251//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/251//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/251//console].
> Umbrella JIRA for HBase Read Replica clusters
> -
>
> Key: HBASE-18477
> URL: https://issues.apache.org/jira/browse/HBASE-18477
> Project: HBase
> Issue Type: New Feature
>Reporter: Zach York
>Assignee: Zach York
>Priority: Major
> Attachments: HBase Read-Replica Clusters Scope doc.docx, HBase
> Read-Replica Clusters Scope doc.pdf, HBase Read-Replica Clusters Scope
> doc_v2.docx, HBase Read-Replica Clusters Scope doc_v2.pdf
>
>
> Recently, changes (such as HBASE-17437) have unblocked HBase to run with a
> root directory external to the cluster (such as in Amazon S3). This means
> that the data is stored outside of the cluster and can be accessible after
> the cluster has been terminated. One use case that is often asked about is
> pointing multiple clusters to one root directory (sharing the data) to have
> read resiliency in the case of a cluster failure.
>
> This JIRA is an umbrella JIRA to contain all the tasks necessary to create a
> read-replica HBase cluster that is pointed at the same root directory.
>
> This requires making the Read-Replica cluster Read-Only (no metadata
> operation or data operations).
> Separating the hbase:meta table for each cluster (Otherwise HBase gets
> confused with multiple clusters trying to update the meta table with their ip
> addresses)
> Adding refresh functionality for the meta table to ensure new metadata is
> picked up on the read replica cluster.
> Adding refresh functionality for HFiles for a given table to ensure new data
> is picked up on the read replica cluster.
>
> This can be used with any existing cluster that is backed by an external
> filesystem.
>
> Please note that this feature is still quite manual (with the potential for
> automation later).
>
> More information on this particular feature can be found here:
> https://aws.amazon.com/blogs/big-data/setting-up-read-replica-clusters-with-hbase-on-amazon-s3/
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529101#comment-16529101
]
Hadoop QA commented on HBASE-20662:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
56s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} hbase-client: The patch generated 0 new + 5
unchanged - 1 fixed = 5 total (was 6) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} hbase-server: The patch generated 0 new + 164
unchanged - 4 fixed = 164 total (was 168) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
58s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 55s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
0s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}109m 1s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}153m 40s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.replication.TestReplicationDroppedTables |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20662 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12929851/HBASE-20662.master.003.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fe6fa52f3b4f 4.4.0-43-generic #63-Ubuntu SMP
[jira] [Commented] (HBASE-20805) Document for CreateSnapshot
[ https://issues.apache.org/jira/browse/HBASE-20805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529070#comment-16529070 ] Andy Lin commented on HBASE-20805: -- NO, you can take it. > Document for CreateSnapshot > --- > > Key: HBASE-20805 > URL: https://issues.apache.org/jira/browse/HBASE-20805 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Priority: Major > > Add documentation for CreateSnapshot. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-13428) [DOC] Migration to hbase-2.0.0
[ https://issues.apache.org/jira/browse/HBASE-13428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-13428: -- Fix Version/s: (was: 2.1.0) 2.2.0 > [DOC] Migration to hbase-2.0.0 > -- > > Key: HBASE-13428 > URL: https://issues.apache.org/jira/browse/HBASE-13428 > Project: HBase > Issue Type: Umbrella > Components: migration >Reporter: stack >Priority: Blocker > Fix For: 3.0.0, 2.2.0 > > > Opening a 2.0 umbrella migration issue. Lets hang off this one any tools and > expectations migrating from 1.0 (or earlier) to 2.0. So far there are none > that I know of though there is an expectation in HBASE-13373 that hfiles are > at least major version 2 and minor version 3. Lets list all such > expectations, etc., here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20792) info:servername and info:sn inconsistent for OPEN region
[
https://issues.apache.org/jira/browse/HBASE-20792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529068#comment-16529068
]
Duo Zhang commented on HBASE-20792:
---
Do we want to commit the patch to branch-2.0? [~stack] [~elserj]
> info:servername and info:sn inconsistent for OPEN region
>
>
> Key: HBASE-20792
> URL: https://issues.apache.org/jira/browse/HBASE-20792
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.2, 2.2.0
>
> Attachments: HBASE-20792.patch, TestRegionMoveAndAbandon.java,
> hbase-hbase-master-ctr-e138-1518143905142-380753-01-04.hwx.site.log
>
>
> Next problem we've run into after HBASE-20752 and HBASE-20708
> After a rolling restart of a cluster, we'll see situations where a collection
> of regions will simply not be assigned out to the RS. I was able to reproduce
> this my mimic the restart patterns our tests do internally (ignore whether
> this is the best way to restart nodes for now :)). The general pattern is
> this:
> {code:java}
> for rs in regionservers:
> stop(server, rs, RS)
> for master in masters:
> stop(server, master, MASTER)
> sleep(15)
> for master in masters:
> start(server, master, MASTER)
> for rs in regionservers:
> start(server, rs, RS){code}
> Looking at meta, we can see why the Master is ignoring some regions:
> {noformat}
> test
> column=table:state, timestamp=1529871718998, value=\x08\x00
> test,,1529871718122.0297f680df6dc0166a44f9536346268e.
> column=info:regioninfo, timestamp=1529967103390, value={ENCODED =>
> 0297f680df6dc0166a44f9536346268e, NAME =>
> 'test,,1529871718122.0297f680df6dc0166a44f9536346268e.', STARTKEY
> => '', ENDKEY =>
> ''}
> test,,1529871718122.0297f680df6dc0166a44f9536346268e.
> column=info:seqnumDuringOpen, timestamp=1529967103390,
> value=\x00\x00\x00\x00\x00\x00\x00*
> test,,1529871718122.0297f680df6dc0166a44f9536346268e.
> column=info:server, timestamp=1529967103390,
> value=ctr-e138-1518143905142-378097-02-12.hwx.site:16020
> test,,1529871718122.0297f680df6dc0166a44f9536346268e.
> column=info:serverstartcode, timestamp=1529967103390, value=1529966776248
> test,,1529871718122.0297f680df6dc0166a44f9536346268e. column=info:sn,
> timestamp=1529967096482,
> value=ctr-e138-1518143905142-378097-02-06.hwx.site,16020,1529966755170
> test,,1529871718122.0297f680df6dc0166a44f9536346268e.
> column=info:state, timestamp=1529967103390, value=OPEN{noformat}
> The region is marked as {{OPEN}}. The master doesn't know any better.
> However, the interesting bit is that {{info:server}} and {{info:sn}} are
> inconsistent (which, according to the javadoc should not be possible for an
> {{OPEN}} region).{{}}
> This doesn't happen every time, but I caught it yesterday on the 2nd or 3rd
> attempt, so I'm hopeful it's not a bear to repro.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20805) Document for CreateSnapshot
[ https://issues.apache.org/jira/browse/HBASE-20805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529069#comment-16529069 ] Mingdao Yang commented on HBASE-20805: -- [~Andyming] Are you working on this? If not, I'd like to take up this task. > Document for CreateSnapshot > --- > > Key: HBASE-20805 > URL: https://issues.apache.org/jira/browse/HBASE-20805 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Priority: Major > > Add documentation for CreateSnapshot. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20796) STUCK RIT though region successfully assigned
[
https://issues.apache.org/jira/browse/HBASE-20796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20796:
--
Fix Version/s: (was: 2.1.0)
2.1.1
> STUCK RIT though region successfully assigned
> -
>
> Key: HBASE-20796
> URL: https://issues.apache.org/jira/browse/HBASE-20796
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.1.1
>
> Attachments: HBASE-20796.branch-2.0.001.patch
>
>
> This is a good one. We keep logging messages like this:
> {code}
> 2018-06-26 12:32:24,859 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: STUCK
> Region-In-Transition rit=OPENING,
> location=vd0410.X.Y.com,22101,1529611445046,
> table=IntegrationTestBigLinkedList_20180525080406,
> region=e10b35d49528e2453a04c7038e3393d7
> {code}
> ...though the region is successfully assigned.
> Story:
> * Dispatch an assign 2018-06-26 12:31:27,390 INFO
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure: Dispatch
> pid=370829, ppid=370391, state=RUNNABLE:REGION_TRANSITION_DISPATCH;
> AssignProcedure table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2; rit=OPENING,
> location=vd0410.X.Y.Z,22101,1529611445046
> * It gets stuck 2018-06-26 12:32:29,860 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: STUCK
> Region-In-Transition rit=OPENING, location=vd0410.X.Y.Z,22101,1529611445046,
> table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2 (Because the server was killed)
> * We stay STUCK for a while.
> * The Master notices the server as crashed and starts a SCP.
> * SCP kills ongoing assign: 2018-06-26 12:32:54,809 INFO
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure: pid=371105
> found RIT pid=370829, ppid=370391, state=RUNNABLE:REGION_TRANSITION_DISPATCH;
> AssignProcedure table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2; rit=OPENING,
> location=vd0410.X.Y.Z,22101,1529611445046
> * The kill brings on a retry ... 2018-06-26 12:32:54,810 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure: Remote
> call failed pid=370829, ppid=370391,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH; AssignProcedure
> table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2; rit=OPENING,
> location=vd0410.X.Y.Z,22101,1529611445046; exception=ServerCrashProcedure
> pid=371105, server=vd0410.X.Y.Z,22101,1529611445046
> * Which eventually succeeds. Successfully deployed to new server
> 2018-06-26 12:32:55,429 INFO
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Finished pid=370829,
> ppid=370391, state=SUCCESS; AssignProcedure
> table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2 in 1mins, 35.379sec
> * But then, it looks like the RPC was ongoing and it broke in following way
> 2018-06-26 12:33:06,378 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure: Remote
> call failed pid=370829, ppid=370391, state=SUCCESS; AssignProcedure
> table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2; rit=OPEN,
> location=vc0614.halxg.cloudera.com,22101,1529611443424; exception=Call to
> vd0410.X.Y.Z/10.10.10.10:22101 failed on local exception:
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> syscall:read(..) failed: Connection reset by peer (Notice how state for
> region is OPEN and 'SUCCESS').
> * Then says 2018-06-26 12:33:06,380 INFO
> org.apache.hadoop.hbase.master.assignment.AssignProcedure: Retry=1 of max=10;
> pid=370829, ppid=370391, state=SUCCESS; AssignProcedure
> table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2; rit=OPEN,
> location=vc0614.X.Y.Z,22101,1529611443424
> * And finally... 2018-06-26 12:34:10,727 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: STUCK
> Region-In-Transition rit=OFFLINE, location=null,
> table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2
> Restart of Master got rid of the STUCK complaints.
> This is interesting because the stuck rpc and the successful reassign are all
> riding on the same pid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20331) clean up shaded packaging for 2.1
[ https://issues.apache.org/jira/browse/HBASE-20331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529067#comment-16529067 ] Duo Zhang commented on HBASE-20331: --- Any updates here [~busbey]? > clean up shaded packaging for 2.1 > - > > Key: HBASE-20331 > URL: https://issues.apache.org/jira/browse/HBASE-20331 > Project: HBase > Issue Type: Umbrella > Components: Client, mapreduce, shading >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 2.1.0 > > > polishing pass on shaded modules for 2.0 based on trying to use them in more > contexts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20542) Better heap utilization for IMC with MSLABs
[ https://issues.apache.org/jira/browse/HBASE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529064#comment-16529064 ] Duo Zhang commented on HBASE-20542: --- It is fine to commit to branch-2, but not branch-2.1 for now. We will release 2.1.0 soon. Thanks. > Better heap utilization for IMC with MSLABs > --- > > Key: HBASE-20542 > URL: https://issues.apache.org/jira/browse/HBASE-20542 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-20542.branch-2.001.patch, > HBASE-20542.branch-2.003.patch, HBASE-20542.branch-2.004.patch, > HBASE-20542.branch-2.005.patch, HBASE-20542.master.003.patch, run.sh, > workloada, workloadc, workloadx, workloady > > > Following HBASE-20188 we realized in-memory compaction combined with MSLABs > may suffer from heap under-utilization due to internal fragmentation. This > jira presents a solution to circumvent this problem. The main idea is to have > each update operation check if it will cause overflow in the active segment > *before* it is writing the new value (instead of checking the size after the > write is completed), and if it is then the active segment is atomically > swapped with a new empty segment, and is pushed (full-yet-not-overflowed) to > the compaction pipeline. Later on the IMC deamon will run its compaction > operation (flatten index/merge indices/data compaction) in the background. > Some subtle concurrency issues should be handled with care. We next elaborate > on them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20712) HBase eclipse formatter should not format the ASF license header
[
https://issues.apache.org/jira/browse/HBASE-20712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20712:
---
Component/s: documentation
> HBase eclipse formatter should not format the ASF license header
>
>
> Key: HBASE-20712
> URL: https://issues.apache.org/jira/browse/HBASE-20712
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-20712.master.001.patch
>
>
> Whenever we add a new class along with the ASF license header, we cannot
> press {{ctrl + A}} to format the whole file as it also formats the header
> text.
> IMO we should disable formatting of headers in our code formatter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20822) TestAsyncNonMetaRegionLocator is flakey
[ https://issues.apache.org/jira/browse/HBASE-20822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529042#comment-16529042 ] Duo Zhang commented on HBASE-20822: --- It has fallen out from the flakey dashboard... Will keep an eye on it. > TestAsyncNonMetaRegionLocator is flakey > --- > > Key: HBASE-20822 > URL: https://issues.apache.org/jira/browse/HBASE-20822 > Project: HBase > Issue Type: Bug > Components: asyncclient >Reporter: Duo Zhang >Priority: Major > Attachments: > org.apache.hadoop.hbase.client.TestAsyncNonMetaRegionLocator-output.txt > > > This is a critical one, the error is result mismatch. Need to find out why. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20829) TestSyncReplicationStandbyKillRS is flakey
[
https://issues.apache.org/jira/browse/HBASE-20829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang reassigned HBASE-20829:
-
Assignee: Duo Zhang
> TestSyncReplicationStandbyKillRS is flakey
> --
>
> Key: HBASE-20829
> URL: https://issues.apache.org/jira/browse/HBASE-20829
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20829-debug.patch
>
>
> Timed out.
> {noformat}
> 2018-06-30 01:32:33,823 ERROR [Time-limited test]
> replication.TestSyncReplicationStandbyKillRS(93): Failed to transit standby
> cluster to DOWNGRADE_ACTIVE
> {noformat}
> We failed to transit the state to DA and then wait for it to become DA so
> hang there.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20829) TestSyncReplicationStandbyKillRS is flakey
[
https://issues.apache.org/jira/browse/HBASE-20829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529040#comment-16529040
]
Duo Zhang commented on HBASE-20829:
---
What I pushed to master branch. Add the exception stacktrace to log so we can
know what is going on.
> TestSyncReplicationStandbyKillRS is flakey
> --
>
> Key: HBASE-20829
> URL: https://issues.apache.org/jira/browse/HBASE-20829
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20829-debug.patch
>
>
> Timed out.
> {noformat}
> 2018-06-30 01:32:33,823 ERROR [Time-limited test]
> replication.TestSyncReplicationStandbyKillRS(93): Failed to transit standby
> cluster to DOWNGRADE_ACTIVE
> {noformat}
> We failed to transit the state to DA and then wait for it to become DA so
> hang there.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20829) TestSyncReplicationStandbyKillRS is flakey
[
https://issues.apache.org/jira/browse/HBASE-20829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20829:
--
Attachment: HBASE-20829-debug.patch
> TestSyncReplicationStandbyKillRS is flakey
> --
>
> Key: HBASE-20829
> URL: https://issues.apache.org/jira/browse/HBASE-20829
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20829-debug.patch
>
>
> Timed out.
> {noformat}
> 2018-06-30 01:32:33,823 ERROR [Time-limited test]
> replication.TestSyncReplicationStandbyKillRS(93): Failed to transit standby
> cluster to DOWNGRADE_ACTIVE
> {noformat}
> We failed to transit the state to DA and then wait for it to become DA so
> hang there.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20804) Document and add tests for HBaseConfTool
[ https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20804: --- Attachment: (was: HBASE-20804.master.001.patch) > Document and add tests for HBaseConfTool > > > Key: HBASE-20804 > URL: https://issues.apache.org/jira/browse/HBASE-20804 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20804.master.001.patch > > > Add documentation and tests for HBaseConfTool. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20804) Document and add tests for HBaseConfTool
[ https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20804: --- Attachment: HBASE-20804.master.001.patch > Document and add tests for HBaseConfTool > > > Key: HBASE-20804 > URL: https://issues.apache.org/jira/browse/HBASE-20804 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20804.master.001.patch > > > Add documentation and tests for HBaseConfTool. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20804) Document and add tests for HBaseConfTool
[ https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20804: --- Fix Version/s: 3.0.0 Status: Patch Available (was: In Progress) > Document and add tests for HBaseConfTool > > > Key: HBASE-20804 > URL: https://issues.apache.org/jira/browse/HBASE-20804 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20804.master.001.patch > > > Add documentation and tests for HBaseConfTool. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20804) Document and add tests for HBaseConfTool
[ https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20804: --- Attachment: HBASE-20804.master.001.patch > Document and add tests for HBaseConfTool > > > Key: HBASE-20804 > URL: https://issues.apache.org/jira/browse/HBASE-20804 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Assignee: Nihal Jain >Priority: Major > Attachments: HBASE-20804.master.001.patch > > > Add documentation and tests for HBaseConfTool. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-20804) Document and add tests for HBaseConfTool
[ https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-20804 started by Nihal Jain. -- > Document and add tests for HBaseConfTool > > > Key: HBASE-20804 > URL: https://issues.apache.org/jira/browse/HBASE-20804 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Assignee: Nihal Jain >Priority: Major > > Add documentation and tests for HBaseConfTool. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20804) Document and add tests for HBaseConfTool
[ https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20804: --- Description: Add documentation and tests for HBaseConfTool. (was: Add documentation for HBaseConfTool.) > Document and add tests for HBaseConfTool > > > Key: HBASE-20804 > URL: https://issues.apache.org/jira/browse/HBASE-20804 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Assignee: Nihal Jain >Priority: Major > > Add documentation and tests for HBaseConfTool. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20804) Document and add tests for HBaseConfTool
[ https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20804: --- Summary: Document and add tests for HBaseConfTool (was: Document for HBaseConfTool) > Document and add tests for HBaseConfTool > > > Key: HBASE-20804 > URL: https://issues.apache.org/jira/browse/HBASE-20804 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Assignee: Nihal Jain >Priority: Major > > Add documentation for HBaseConfTool. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20542) Better heap utilization for IMC with MSLABs
[ https://issues.apache.org/jira/browse/HBASE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529035#comment-16529035 ] Eshcar Hillel commented on HBASE-20542: --- Thanks [~mdrob]. So except for these warnings, QA passed. I am pushing this patch to master. [~Apache9] [~stack] should I commit this to branch-2/branch2.0 or would you like to take a closer look at it? > Better heap utilization for IMC with MSLABs > --- > > Key: HBASE-20542 > URL: https://issues.apache.org/jira/browse/HBASE-20542 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-20542.branch-2.001.patch, > HBASE-20542.branch-2.003.patch, HBASE-20542.branch-2.004.patch, > HBASE-20542.branch-2.005.patch, HBASE-20542.master.003.patch, run.sh, > workloada, workloadc, workloadx, workloady > > > Following HBASE-20188 we realized in-memory compaction combined with MSLABs > may suffer from heap under-utilization due to internal fragmentation. This > jira presents a solution to circumvent this problem. The main idea is to have > each update operation check if it will cause overflow in the active segment > *before* it is writing the new value (instead of checking the size after the > write is completed), and if it is then the active segment is atomically > swapped with a new empty segment, and is pushed (full-yet-not-overflowed) to > the compaction pipeline. Later on the IMC deamon will run its compaction > operation (flatten index/merge indices/data compaction) in the background. > Some subtle concurrency issues should be handled with care. We next elaborate > on them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20804) Document for HBaseConfTool
[ https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529032#comment-16529032 ] Nihal Jain edited comment on HBASE-20804 at 7/1/18 9:52 AM: I am assigning it to me. [~chia7712], [~Andyming] was (Author: nihaljain.cs): I am assigning it to me. [~chia7712] > Document for HBaseConfTool > -- > > Key: HBASE-20804 > URL: https://issues.apache.org/jira/browse/HBASE-20804 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Assignee: Nihal Jain >Priority: Major > > Add documentation for HBaseConfTool. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20829) TestSyncReplicationStandbyKillRS is flakey
Duo Zhang created HBASE-20829:
-
Summary: TestSyncReplicationStandbyKillRS is flakey
Key: HBASE-20829
URL: https://issues.apache.org/jira/browse/HBASE-20829
Project: HBase
Issue Type: Bug
Components: Replication
Reporter: Duo Zhang
Fix For: 3.0.0
Timed out.
{noformat}
2018-06-30 01:32:33,823 ERROR [Time-limited test]
replication.TestSyncReplicationStandbyKillRS(93): Failed to transit standby
cluster to DOWNGRADE_ACTIVE
{noformat}
We failed to transit the state to DA and then wait for it to become DA so hang
there.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-20804) Document for HBaseConfTool
[ https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain reassigned HBASE-20804: -- Assignee: Nihal Jain > Document for HBaseConfTool > -- > > Key: HBASE-20804 > URL: https://issues.apache.org/jira/browse/HBASE-20804 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Assignee: Nihal Jain >Priority: Major > > Add documentation for HBaseConfTool. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20804) Document for HBaseConfTool
[ https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529032#comment-16529032 ] Nihal Jain commented on HBASE-20804: I am assigning it to me. [~chia7712] > Document for HBaseConfTool > -- > > Key: HBASE-20804 > URL: https://issues.apache.org/jira/browse/HBASE-20804 > Project: HBase > Issue Type: Sub-task > Components: documentation, tooling >Reporter: Andy Lin >Priority: Major > > Add documentation for HBaseConfTool. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529021#comment-16529021
]
Nihal Jain edited comment on HBASE-20662 at 7/1/18 9:18 AM:
Attached a new patch [^HBASE-20662.master.003.patch] , removing all the
redundant refactoring/code changes.
This patch does the following:
* Increasing space quota on a violated table will remove
SpaceViolationPolicy.DISABLE enforcement
* The table disable/enable logic (in case of SpaceViolationPolicy.DISABLE) has
been shifted at master side, saving us on multiple disable RPCs from RSs
The new namespace test cases (which were added in
[^HBASE-20662.master.002.patch]) have been removed in this patch; They will be
added as part of HBASE-20820.
was (Author: nihaljain.cs):
Attached a new patch [^HBASE-20662.master.003.patch] , removing all the
redundant refactoring/code changes.
This patch does the following:
* Increasing space quota on a violated table will remove
SpaceViolationPolicy.DISABLE enforcement
* The table disable/enable logic (in case of SpaceViolationPolicy.DISABLE) has
been shifted at master side, saving us on multiple disable RPCs from RSs
The new namespace test cases (which were added in Patch-2) have been removed in
this patch; They will be added as part of HBASE-20820.
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$Maste
[jira] [Updated] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20662:
---
Attachment: HBASE-20662.master.003.patch
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:100)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:90)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:360)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.
[jira] [Updated] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20662:
---
Attachment: (was: HBASE-20662.master.003.patch)
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:100)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:90)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:360)
> at
> org.apache.hadoop.hbase.shaded.protobuf.Pr
[jira] [Comment Edited] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529021#comment-16529021
]
Nihal Jain edited comment on HBASE-20662 at 7/1/18 9:16 AM:
Attached a new patch [^HBASE-20662.master.003.patch] , removing all the
redundant refactoring/code changes.
This patch does the following:
* Increasing space quota on a violated table will remove
SpaceViolationPolicy.DISABLE enforcement
* The table disable/enable logic (in case of SpaceViolationPolicy.DISABLE) has
been shifted at master side, saving us on multiple disable RPCs from RSs
The new namespace test cases (which were added in Patch-2) have been removed in
this patch; They will be added as part of HBASE-20820.
was (Author: nihaljain.cs):
Attached a new patch, removing all the redundant refactoring/code changes.
This patch does the following:
* Increasing space quota on a violated table will remove
SpaceViolationPolicy.DISABLE enforcement
* The table disable/enable logic (in case of
[^HBASE-20662.master.003.patch]SpaceViolationPolicy.DISABLE) has been shifted
at master side, saving us on multiple disable RPCs from RSs
The new namespace test cases (which were added in Patch-2) have been removed in
this patch; They will be added as part of HBASE-20820.
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMeth
[jira] [Updated] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20662:
---
Attachment: (was: HBASE-20662.master.003.patch)
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:100)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:90)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:360)
> at
> org.apache.hadoop.hbase.shaded.protobuf.Pr
[jira] [Updated] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20662:
---
Attachment: HBASE-20662.master.003.patch
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:100)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:90)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:360)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.
[jira] [Updated] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20662:
---
Attachment: HBASE-20662.master.003.patch
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:100)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:90)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:360)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.
[jira] [Commented] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529021#comment-16529021
]
Nihal Jain commented on HBASE-20662:
Attached a new patch, removing all the redundant refactoring/code changes.
This patch does the following:
* Increasing space quota on a violated table will remove
SpaceViolationPolicy.DISABLE enforcement
* The table disable/enable logic (in case of
[^HBASE-20662.master.003.patch]SpaceViolationPolicy.DISABLE) has been shifted
at master side, saving us on multiple disable RPCs from RSs
The new namespace test cases (which were added in Patch-2) have been removed in
this patch; They will be added as part of HBASE-20820.
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.ja
[jira] [Commented] (HBASE-20808) Wrong shutdown order between Chores and ChoreService
[
https://issues.apache.org/jira/browse/HBASE-20808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529003#comment-16529003
]
Nihal Jain commented on HBASE-20808:
{quote}For a logic clear code base, having a clear start
getChoreService().scheduleChore(ScheduledChore);, it is better to have a clear
end getChoreService().cancelChore(ScheduledChore);.
{quote}
Isn't it sometimes better to have a cleaner solution than an ideal solution?
Anyways, I have attached a new patch with the suggested change:
[^HBASE-20808.master.002.patch]
> Wrong shutdown order between Chores and ChoreService
>
>
> Key: HBASE-20808
> URL: https://issues.apache.org/jira/browse/HBASE-20808
> Project: HBase
> Issue Type: Bug
>Reporter: Reid Chan
>Assignee: Nihal Jain
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-20808.master.001.patch,
> HBASE-20808.master.002.patch
>
>
> When stopping master, {{ChoreService}}, which serves all the chores, is
> stopped before canceling all running chores.
> It should cancel all running chores, then shutdown {{ChoreService}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20808) Wrong shutdown order between Chores and ChoreService
[
https://issues.apache.org/jira/browse/HBASE-20808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20808:
---
Attachment: HBASE-20808.master.002.patch
> Wrong shutdown order between Chores and ChoreService
>
>
> Key: HBASE-20808
> URL: https://issues.apache.org/jira/browse/HBASE-20808
> Project: HBase
> Issue Type: Bug
>Reporter: Reid Chan
>Assignee: Nihal Jain
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-20808.master.001.patch,
> HBASE-20808.master.002.patch
>
>
> When stopping master, {{ChoreService}}, which serves all the chores, is
> stopped before canceling all running chores.
> It should cancel all running chores, then shutdown {{ChoreService}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
