[jira] [Updated] (HBASE-19863) java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
[
https://issues.apache.org/jira/browse/HBASE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Soldatov updated HBASE-19863:
Attachment: HBASE-19863.v3-branch-2.patch
> java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter
> is used
> -
>
> Key: HBASE-19863
> URL: https://issues.apache.org/jira/browse/HBASE-19863
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.4.1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Attachments: HBASE-19863-branch-2.patch, HBASE-19863-branch1.patch,
> HBASE-19863-test.patch, HBASE-19863.v2-branch-2.patch,
> HBASE-19863.v3-branch-2.patch
>
>
> Under some circumstances scan with SingleColumnValueFilter may fail with an
> exception
> {noformat}
> java.lang.IllegalStateException: isDelete failed: deleteBuffer=C3,
> qualifier=C2, timestamp=1516433595543, comparison result: 1
> at
> org.apache.hadoop.hbase.regionserver.ScanDeleteTracker.isDeleted(ScanDeleteTracker.java:149)
> at
> org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:386)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:545)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5876)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6027)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2552)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32385)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> {noformat}
> Conditions:
> table T with a single column family 0 that uses ROWCOL bloom filter
> (important) and column qualifiers C1,C2,C3,C4,C5.
> When we fill the table for every row we put deleted cell for C3.
> The table has a single region with two HStore:
> A: start row: 0, stop row: 99
> B: start row: 10 stop row: 99
> B has newer versions of rows 10-99. Store files have several blocks each
> (important).
> Store A is the result of major compaction, so it doesn't have any deleted
> cells (important).
> So, we are running a scan like:
> {noformat}
> scan 'T', { COLUMNS => ['0:C3','0:C5'], FILTER => "SingleColumnValueFilter
> ('0','C5',=,'binary:whatever')"}
> {noformat}
> How the scan performs:
> First, we iterate A for rows 0 and 1 without any problems.
> Next, we start to iterate A for row 10, so read the first cell and set hfs
> scanner to A :
> 10:0/C1/0/Put/x but found that we have a newer version of the cell in B :
> 10:0/C1/1/Put/x,
> so we make B as our current store scanner. Since we are looking for
> particular columns
> C3 and C5, we perform the optimization StoreScanner.seekOrSkipToNextColumn
> which
> would run reseek for all store scanners.
> For store A the following magic would happen in requestSeek:
> 1. bloom filter check passesGeneralBloomFilter would set haveToSeek to
> false because row 10 doesn't have C3 qualifier in store A.
> 2. Since we don't have to seek we just create a fake row
> 10:0/C3/OLDEST_TIMESTAMP/Maximum, an optimization that is quite important for
> us and it commented with :
> {noformat}
> // Multi-column Bloom filter optimization.
> // Create a fake key/value, so that this scanner only bubbles up to the
> top
> // of the KeyValueHeap in StoreScanner after we scanned this row/column in
> // all other store files. The query matcher will then just skip this fake
> // key/value and the store scanner will progress to the next column. This
> // is obviously not a "real real" seek, but unlike the fake KV earlier in
> // this method, we want this to be propagated to ScanQueryMatcher.
> {noformat}
>
> For store B we would set it to fake 10:0/C3/createFirstOnRowColTS()/Maximum
> to skip C3 entirely.
> After that we start searching for qualifier C5 using seekOrSkipToNextColumn
> which run first trySkipToNextColumn:
> {noformat}
> protected boolean trySkipToNextColumn(Cell cell) throws IOException {
> Cell nextCell = null;
> do {
> Cell nextIndexedKey = getNextIndexedKey();
> if (nextIndexedKey !=
[jira] [Commented] (HBASE-19400) Add missing security hooks for MasterService RPCs
[
https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366679#comment-16366679
]
Appy commented on HBASE-19400:
--
Updated the patch with tests. Ptal [~stack].
TestAccessController failed with following locally, will take a look tomorrow:
{noformat}
[ERROR] Tests run: 70, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 96.359
s <<< FAILURE! - in org.apache.hadoop.hbase.security.access.TestAccessController
[ERROR] org.apache.hadoop.hbase.security.access.TestAccessController Time
elapsed: 2.844 s <<< FAILURE!
java.lang.AssertionError: Unexpected reference count: 3
at
org.apache.hadoop.hbase.security.access.TestAccessController.tearDownAfterClass(TestAccessController.java:266)
{noformat}
> Add missing security hooks for MasterService RPCs
> -
>
> Key: HBASE-19400
> URL: https://issues.apache.org/jira/browse/HBASE-19400
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0-beta-1
>Reporter: Balazs Meszaros
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-19400.master.001.patch,
> HBASE-19400.master.002.patch, HBASE-19400.master.003.patch,
> HBASE-19400.master.004.patch
>
>
> The following RPC methods do not call the observers, therefore they are not
> guarded by AccessController:
> - normalize
> - setNormalizerRunning
> - runCatalogScan
> - enableCatalogJanitor
> - runCleanerChore
> - setCleanerChoreRunning
> - execMasterService
> - execProcedure
> - execProcedureWithRet
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-19400) Add missing security hooks for MasterService RPCs
[ https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy reassigned HBASE-19400: Assignee: Appy (was: Balazs Meszaros) > Add missing security hooks for MasterService RPCs > - > > Key: HBASE-19400 > URL: https://issues.apache.org/jira/browse/HBASE-19400 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > Attachments: HBASE-19400.master.001.patch, > HBASE-19400.master.002.patch, HBASE-19400.master.003.patch, > HBASE-19400.master.004.patch > > > The following RPC methods do not call the observers, therefore they are not > guarded by AccessController: > - normalize > - setNormalizerRunning > - runCatalogScan > - enableCatalogJanitor > - runCleanerChore > - setCleanerChoreRunning > - execMasterService > - execProcedure > - execProcedureWithRet -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19400) Add missing security hooks for MasterService RPCs
[ https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19400: - Attachment: HBASE-19400.master.004.patch > Add missing security hooks for MasterService RPCs > - > > Key: HBASE-19400 > URL: https://issues.apache.org/jira/browse/HBASE-19400 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Attachments: HBASE-19400.master.001.patch, > HBASE-19400.master.002.patch, HBASE-19400.master.003.patch, > HBASE-19400.master.004.patch > > > The following RPC methods do not call the observers, therefore they are not > guarded by AccessController: > - normalize > - setNormalizerRunning > - runCatalogScan > - enableCatalogJanitor > - runCleanerChore > - setCleanerChoreRunning > - execMasterService > - execProcedure > - execProcedureWithRet -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20000) Remove the quantum logic in FairQueue, always put high priority queue in front
[
https://issues.apache.org/jira/browse/HBASE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366671#comment-16366671
]
Appy commented on HBASE-2:
--
bq. Unfortunately this is a bug fix...
If this is fixing bugs, that's fortunate! :)
I missed to see that it was a sub-task, so without context i thought it was
general refactor. Sorry.
bq. When implementing HBASE-19397 I found that this is not trivial... Usually
the parent procedure will release the lock after switching out so anyone can
acquire the lock again... We have a holdLock method for Procedure which can
control whether to always hold the lock even if you are not running at the
time...
Yups, the whole framework is!
Parent procedure giving away the lock is covered by that invariant no? it's
child procedure *may* get blocked. ("will" if some other proc acquire it, 'will
not' if the child proc itself acquire it. (But we can always reword it better,
it was just first cut suggestion).
To me, holdlock seemed to be just optimization, give away the lock if you
(proc) are yielding and don't need exclusion - just good samaritan way? Not
sure though if any proc is actually doing that or all just keep the lock held.
Or maybe it's not worth the complexity in framework. Yeah let's see after 2.0.
Given that quantum logic is gone, it does't look "fair" anymore, should be
renamed to "PriorityQueue" in future?
> Remove the quantum logic in FairQueue, always put high priority queue in front
> --
>
> Key: HBASE-2
> URL: https://issues.apache.org/jira/browse/HBASE-2
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-2-branch-2.patch, HBASE-2-v1.patch,
> HBASE-2.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19851) ProcV2 framework's rollback() needs to maintain stack like execute()
[ https://issues.apache.org/jira/browse/HBASE-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366648#comment-16366648 ] stack commented on HBASE-19851: --- Maybe I should file the above as MergeRegionProcedure is incomplete... > ProcV2 framework's rollback() needs to maintain stack like execute() > > > Key: HBASE-19851 > URL: https://issues.apache.org/jira/browse/HBASE-19851 > Project: HBase > Issue Type: Improvement > Components: proc-v2 >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Priority: Critical > Fix For: 2.0.0 > > > Refer to HBASE-19839 and detailed description there. Bug showed up as a flaky > test TestMergeTableRegionsProcedure.testRollbackAndDoubleExecution(). In > short, when rollback() is called after step 5 > (MERGE_TABLE_REGIONS_CLOSE_REGIONS), rollback of the procedure tries to > reassign two regions that are to be merged. It spawns 2 AssignProcedures and > don't wait on them to finish. Rollback should have ability to have > sub-procedures and separate proc stack. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19166) Add translation for handling hbase.regionserver.wal.WALEdit
[
https://issues.apache.org/jira/browse/HBASE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366617#comment-16366617
]
stack commented on HBASE-19166:
---
That might work. Needs doc in code on why this is being done. Put up a cluster
with hbase2 and hbase1 servers and see if WAL split works? Or at least that a
WALPlayer from hbase1 can read an hbase2 written file. Thanks.
> Add translation for handling hbase.regionserver.wal.WALEdit
> ---
>
> Key: HBASE-19166
> URL: https://issues.apache.org/jira/browse/HBASE-19166
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Ted Yu
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: 19166-async-log-writer.v1.txt
>
>
> For hlog generated by 1.x, using WALPlayer from hbase2 would result in:
> {code}
> 2017-11-02 21:22:40,907 INFO [main] mapreduce.Job: Task Id :
> attempt_1509641483571_0003_m_00_0, Status : FAILED
> Error: java.lang.ClassCastException:
> org.apache.hadoop.hbase.regionserver.wal.WALEdit cannot be cast to
> org.apache.hadoop.hbase.wal.WALEdit
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALCellMapper.map(WALPlayer.java:143)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
> {code}
> HBASE-16479 relocated WALEdit.
> Chatting with Enis, he mentioned adding translation for handling
> hbase.regionserver.wal.WALEdit
> This way, WAL from 1.x can be recognized by hbase-2
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20006) TestRestoreSnapshotFromClientWithRegionReplicas is flakey
[
https://issues.apache.org/jira/browse/HBASE-20006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366610#comment-16366610
]
stack edited comment on HBASE-20006 at 2/16/18 4:47 AM:
I pushed patch to master and branch-2. Lets see how it does. Also disabled
TestHTrace while in here. We don't support htrace in 2.0.0, not yet anyways,
and the test is flakey.
Leaving open to see how it does post commit.
was (Author: stack):
I pushed patch to master and branch-2. Lets see how it does. Also disabled
TestHTrace while in here. We don't support htrace in 2.0.0, not yet anyways,
and the test is flakey.
> TestRestoreSnapshotFromClientWithRegionReplicas is flakey
> -
>
> Key: HBASE-20006
> URL: https://issues.apache.org/jira/browse/HBASE-20006
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20006.branch-2.001.patch
>
>
> Failing 10% of the time. Interestingly, it is below that causes fail. We go
> to split but it is already split. We will then fail the split with an
> internal assert which messes up procedures; at a minimum we should just not
> split (this is in the prepare stage).
> {code}
> 2018-02-15 23:21:42,162 INFO [PEWorker-12]
> procedure.MasterProcedureScheduler(571): pid=105,
> state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> parent=3f850cea7d71a7ebd019f2f009efca4d,
> daughterA=06b5e6366efbef155d70e56cfdf58dc9,
> daughterB=8c175de1b33765a5683ac1e502edb0bd,
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.
> 2018-02-15 23:21:42,162 INFO [PEWorker-12]
> assignment.SplitTableRegionProcedure(440): Split of {ENCODED =>
> 3f850cea7d71a7ebd019f2f009efca4d, NAME =>
> 'testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.',
> STARTKEY => '', ENDKEY => '1'} skipped; state is already SPLIT
> 2018-02-15 23:21:42,163 ERROR [PEWorker-12]
> procedure2.ProcedureExecutor(1480): CODE-BUG: Uncaught runtime exception:
> pid=105, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> parent=3f850cea7d71a7ebd019f2f009efca4d,
> daughterA=06b5e6366efbef155d70e56cfdf58dc9,
> daughterB=8c175de1b33765a5683ac1e502edb0bd
> java.lang.AssertionError: split region should have an exception here
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:228)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:89)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:180)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1455)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1224)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20006) TestRestoreSnapshotFromClientWithRegionReplicas is flakey
[
https://issues.apache.org/jira/browse/HBASE-20006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366610#comment-16366610
]
stack commented on HBASE-20006:
---
I pushed patch to master and branch-2. Lets see how it does. Also disabled
TestHTrace while in here. We don't support htrace in 2.0.0, not yet anyways,
and the test is flakey.
> TestRestoreSnapshotFromClientWithRegionReplicas is flakey
> -
>
> Key: HBASE-20006
> URL: https://issues.apache.org/jira/browse/HBASE-20006
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20006.branch-2.001.patch
>
>
> Failing 10% of the time. Interestingly, it is below that causes fail. We go
> to split but it is already split. We will then fail the split with an
> internal assert which messes up procedures; at a minimum we should just not
> split (this is in the prepare stage).
> {code}
> 2018-02-15 23:21:42,162 INFO [PEWorker-12]
> procedure.MasterProcedureScheduler(571): pid=105,
> state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> parent=3f850cea7d71a7ebd019f2f009efca4d,
> daughterA=06b5e6366efbef155d70e56cfdf58dc9,
> daughterB=8c175de1b33765a5683ac1e502edb0bd,
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.
> 2018-02-15 23:21:42,162 INFO [PEWorker-12]
> assignment.SplitTableRegionProcedure(440): Split of {ENCODED =>
> 3f850cea7d71a7ebd019f2f009efca4d, NAME =>
> 'testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.',
> STARTKEY => '', ENDKEY => '1'} skipped; state is already SPLIT
> 2018-02-15 23:21:42,163 ERROR [PEWorker-12]
> procedure2.ProcedureExecutor(1480): CODE-BUG: Uncaught runtime exception:
> pid=105, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> parent=3f850cea7d71a7ebd019f2f009efca4d,
> daughterA=06b5e6366efbef155d70e56cfdf58dc9,
> daughterB=8c175de1b33765a5683ac1e502edb0bd
> java.lang.AssertionError: split region should have an exception here
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:228)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:89)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:180)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1455)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1224)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20001) cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
[
https://issues.apache.org/jira/browse/HBASE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366609#comment-16366609
]
Chia-Ping Tsai commented on HBASE-20001:
How you find out this issue? Does the incorrect encoded name break something in
your cluster?
> cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
> -
>
> Key: HBASE-20001
> URL: https://issues.apache.org/jira/browse/HBASE-20001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.1.7
>Reporter: Francis Liu
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.3.2, 1.5.0, 1.4.3
>
> Attachments: HBASE-20001.branch-1.4.001.patch
>
>
> In RegionStates.cleanIfNoMetaEntry()
> {{if (MetaTableAccessor.getRegion(server.getConnection(),
> hri.getEncodedNameAsBytes()) == null) {}}
> {{regionOffline(hri);}}
> {{FSUtils.deleteRegionDir(server.getConfiguration(), hri);}}
> }
> But api expects regionname
> {{public static Pair getRegion(Connection
> connection, byte [] regionName)}}
> So we might end up cleaning good regions.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20006) TestRestoreSnapshotFromClientWithRegionReplicas is flakey
[
https://issues.apache.org/jira/browse/HBASE-20006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20006:
--
Fix Version/s: 2.0.0-beta-2
> TestRestoreSnapshotFromClientWithRegionReplicas is flakey
> -
>
> Key: HBASE-20006
> URL: https://issues.apache.org/jira/browse/HBASE-20006
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20006.branch-2.001.patch
>
>
> Failing 10% of the time. Interestingly, it is below that causes fail. We go
> to split but it is already split. We will then fail the split with an
> internal assert which messes up procedures; at a minimum we should just not
> split (this is in the prepare stage).
> {code}
> 2018-02-15 23:21:42,162 INFO [PEWorker-12]
> procedure.MasterProcedureScheduler(571): pid=105,
> state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> parent=3f850cea7d71a7ebd019f2f009efca4d,
> daughterA=06b5e6366efbef155d70e56cfdf58dc9,
> daughterB=8c175de1b33765a5683ac1e502edb0bd,
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.
> 2018-02-15 23:21:42,162 INFO [PEWorker-12]
> assignment.SplitTableRegionProcedure(440): Split of {ENCODED =>
> 3f850cea7d71a7ebd019f2f009efca4d, NAME =>
> 'testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.',
> STARTKEY => '', ENDKEY => '1'} skipped; state is already SPLIT
> 2018-02-15 23:21:42,163 ERROR [PEWorker-12]
> procedure2.ProcedureExecutor(1480): CODE-BUG: Uncaught runtime exception:
> pid=105, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> parent=3f850cea7d71a7ebd019f2f009efca4d,
> daughterA=06b5e6366efbef155d70e56cfdf58dc9,
> daughterB=8c175de1b33765a5683ac1e502edb0bd
> java.lang.AssertionError: split region should have an exception here
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:228)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:89)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:180)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1455)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1224)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20006) TestRestoreSnapshotFromClientWithRegionReplicas is flakey
[
https://issues.apache.org/jira/browse/HBASE-20006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20006:
--
Attachment: HBASE-20006.branch-2.001.patch
> TestRestoreSnapshotFromClientWithRegionReplicas is flakey
> -
>
> Key: HBASE-20006
> URL: https://issues.apache.org/jira/browse/HBASE-20006
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20006.branch-2.001.patch
>
>
> Failing 10% of the time. Interestingly, it is below that causes fail. We go
> to split but it is already split. We will then fail the split with an
> internal assert which messes up procedures; at a minimum we should just not
> split (this is in the prepare stage).
> {code}
> 2018-02-15 23:21:42,162 INFO [PEWorker-12]
> procedure.MasterProcedureScheduler(571): pid=105,
> state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> parent=3f850cea7d71a7ebd019f2f009efca4d,
> daughterA=06b5e6366efbef155d70e56cfdf58dc9,
> daughterB=8c175de1b33765a5683ac1e502edb0bd,
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.
> 2018-02-15 23:21:42,162 INFO [PEWorker-12]
> assignment.SplitTableRegionProcedure(440): Split of {ENCODED =>
> 3f850cea7d71a7ebd019f2f009efca4d, NAME =>
> 'testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.',
> STARTKEY => '', ENDKEY => '1'} skipped; state is already SPLIT
> 2018-02-15 23:21:42,163 ERROR [PEWorker-12]
> procedure2.ProcedureExecutor(1480): CODE-BUG: Uncaught runtime exception:
> pid=105, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> parent=3f850cea7d71a7ebd019f2f009efca4d,
> daughterA=06b5e6366efbef155d70e56cfdf58dc9,
> daughterB=8c175de1b33765a5683ac1e502edb0bd
> java.lang.AssertionError: split region should have an exception here
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:228)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:89)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:180)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1455)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1224)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20006) TestRestoreSnapshotFromClientWithRegionReplicas is flakey
[
https://issues.apache.org/jira/browse/HBASE-20006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366606#comment-16366606
]
stack commented on HBASE-20006:
---
So, what we have here is a split table followed by a split region. The split
region encounters a parent that has been successfully split but that has not
yet been GC'd. This is legit case. The assert is not right. Removing.
> TestRestoreSnapshotFromClientWithRegionReplicas is flakey
> -
>
> Key: HBASE-20006
> URL: https://issues.apache.org/jira/browse/HBASE-20006
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Priority: Major
>
> Failing 10% of the time. Interestingly, it is below that causes fail. We go
> to split but it is already split. We will then fail the split with an
> internal assert which messes up procedures; at a minimum we should just not
> split (this is in the prepare stage).
> {code}
> 2018-02-15 23:21:42,162 INFO [PEWorker-12]
> procedure.MasterProcedureScheduler(571): pid=105,
> state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> parent=3f850cea7d71a7ebd019f2f009efca4d,
> daughterA=06b5e6366efbef155d70e56cfdf58dc9,
> daughterB=8c175de1b33765a5683ac1e502edb0bd,
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.
> 2018-02-15 23:21:42,162 INFO [PEWorker-12]
> assignment.SplitTableRegionProcedure(440): Split of {ENCODED =>
> 3f850cea7d71a7ebd019f2f009efca4d, NAME =>
> 'testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.',
> STARTKEY => '', ENDKEY => '1'} skipped; state is already SPLIT
> 2018-02-15 23:21:42,163 ERROR [PEWorker-12]
> procedure2.ProcedureExecutor(1480): CODE-BUG: Uncaught runtime exception:
> pid=105, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure
> table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
> parent=3f850cea7d71a7ebd019f2f009efca4d,
> daughterA=06b5e6366efbef155d70e56cfdf58dc9,
> daughterB=8c175de1b33765a5683ac1e502edb0bd
> java.lang.AssertionError: split region should have an exception here
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:228)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:89)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:180)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1455)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1224)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19166) Add translation for handling hbase.regionserver.wal.WALEdit
[
https://issues.apache.org/jira/browse/HBASE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366604#comment-16366604
]
Ted Yu commented on HBASE-19166:
19166-async-log-writer.v1.txt shows what I mean.
Ran thru hbase-server tests without seeing failed test other than flaky tests.
> Add translation for handling hbase.regionserver.wal.WALEdit
> ---
>
> Key: HBASE-19166
> URL: https://issues.apache.org/jira/browse/HBASE-19166
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Ted Yu
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: 19166-async-log-writer.v1.txt
>
>
> For hlog generated by 1.x, using WALPlayer from hbase2 would result in:
> {code}
> 2017-11-02 21:22:40,907 INFO [main] mapreduce.Job: Task Id :
> attempt_1509641483571_0003_m_00_0, Status : FAILED
> Error: java.lang.ClassCastException:
> org.apache.hadoop.hbase.regionserver.wal.WALEdit cannot be cast to
> org.apache.hadoop.hbase.wal.WALEdit
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALCellMapper.map(WALPlayer.java:143)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
> {code}
> HBASE-16479 relocated WALEdit.
> Chatting with Enis, he mentioned adding translation for handling
> hbase.regionserver.wal.WALEdit
> This way, WAL from 1.x can be recognized by hbase-2
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19166) Add translation for handling hbase.regionserver.wal.WALEdit
[
https://issues.apache.org/jira/browse/HBASE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19166:
---
Attachment: 19166-async-log-writer.v1.txt
> Add translation for handling hbase.regionserver.wal.WALEdit
> ---
>
> Key: HBASE-19166
> URL: https://issues.apache.org/jira/browse/HBASE-19166
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Ted Yu
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: 19166-async-log-writer.v1.txt
>
>
> For hlog generated by 1.x, using WALPlayer from hbase2 would result in:
> {code}
> 2017-11-02 21:22:40,907 INFO [main] mapreduce.Job: Task Id :
> attempt_1509641483571_0003_m_00_0, Status : FAILED
> Error: java.lang.ClassCastException:
> org.apache.hadoop.hbase.regionserver.wal.WALEdit cannot be cast to
> org.apache.hadoop.hbase.wal.WALEdit
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALCellMapper.map(WALPlayer.java:143)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
> {code}
> HBASE-16479 relocated WALEdit.
> Chatting with Enis, he mentioned adding translation for handling
> hbase.regionserver.wal.WALEdit
> This way, WAL from 1.x can be recognized by hbase-2
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20006) TestRestoreSnapshotFromClientWithRegionReplicas is flakey
stack created HBASE-20006:
-
Summary: TestRestoreSnapshotFromClientWithRegionReplicas is flakey
Key: HBASE-20006
URL: https://issues.apache.org/jira/browse/HBASE-20006
Project: HBase
Issue Type: Sub-task
Reporter: stack
Failing 10% of the time. Interestingly, it is below that causes fail. We go to
split but it is already split. We will then fail the split with an internal
assert which messes up procedures; at a minimum we should just not split (this
is in the prepare stage).
{code}
2018-02-15 23:21:42,162 INFO [PEWorker-12]
procedure.MasterProcedureScheduler(571): pid=105,
state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure
table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
parent=3f850cea7d71a7ebd019f2f009efca4d,
daughterA=06b5e6366efbef155d70e56cfdf58dc9,
daughterB=8c175de1b33765a5683ac1e502edb0bd,
table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.
2018-02-15 23:21:42,162 INFO [PEWorker-12]
assignment.SplitTableRegionProcedure(440): Split of {ENCODED =>
3f850cea7d71a7ebd019f2f009efca4d, NAME =>
'testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.',
STARTKEY => '', ENDKEY => '1'} skipped; state is already SPLIT
2018-02-15 23:21:42,163 ERROR [PEWorker-12] procedure2.ProcedureExecutor(1480):
CODE-BUG: Uncaught runtime exception: pid=105,
state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure
table=testOnlineSnapshotAfterSplittingRegions-1518736887838,
parent=3f850cea7d71a7ebd019f2f009efca4d,
daughterA=06b5e6366efbef155d70e56cfdf58dc9,
daughterB=8c175de1b33765a5683ac1e502edb0bd
java.lang.AssertionError: split region should have an exception here
at
org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:228)
at
org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:89)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:180)
at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1455)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1224)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366540#comment-16366540
]
Francis Liu commented on HBASE-19920:
-
Test looks good. +1 post QA issues. Thanks [~mdrob].
> TokenUtil.obtainToken unnecessarily creates a local directory
> -
>
> Key: HBASE-19920
> URL: https://issues.apache.org/jira/browse/HBASE-19920
> Project: HBase
> Issue Type: Bug
>Reporter: Rohini Palaniswamy
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0
>
> Attachments: HBASE-19920.patch, HBASE-19920.v2.patch,
> HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch,
> HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch,
> HBASE-19920.v9.patch
>
>
> On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil
> which in its static block initializes DynamicClassLoader and that creates the
> directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class
> to access hbase.dynamic.jars.dir.
> https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127
> Since this is region server specific code, not expecting this to happen when
> one accesses hbase as a client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20001) cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
[
https://issues.apache.org/jira/browse/HBASE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366534#comment-16366534
]
Thiruvel Thirumoolan commented on HBASE-20001:
--
[~chia7712] - I have added a comment in the reviewboard, can we pls continue
the conversation there? thanks for your feedback.
Some tests in TestSplitTransactionOnCluster also failed, guessing its mostly
test based, will check it out.
> cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
> -
>
> Key: HBASE-20001
> URL: https://issues.apache.org/jira/browse/HBASE-20001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.1.7
>Reporter: Francis Liu
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.3.2, 1.5.0, 1.4.3
>
> Attachments: HBASE-20001.branch-1.4.001.patch
>
>
> In RegionStates.cleanIfNoMetaEntry()
> {{if (MetaTableAccessor.getRegion(server.getConnection(),
> hri.getEncodedNameAsBytes()) == null) {}}
> {{regionOffline(hri);}}
> {{FSUtils.deleteRegionDir(server.getConfiguration(), hri);}}
> }
> But api expects regionname
> {{public static Pair getRegion(Connection
> connection, byte [] regionName)}}
> So we might end up cleaning good regions.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19978) The keepalive logic is incomplete in ProcedureExecutor
[ https://issues.apache.org/jira/browse/HBASE-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19978: -- Issue Type: Sub-task (was: Bug) Parent: HBASE-19976 > The keepalive logic is incomplete in ProcedureExecutor > -- > > Key: HBASE-19978 > URL: https://issues.apache.org/jira/browse/HBASE-19978 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > > The worker thread will just exit after keep alive time, and we never add it > back. The only way to add it back is through the stuck checker, this is not > correct. Here we should start new worker thread if it is under the core pool > size and there are pending procedures. > For now the default keep alive time is Long.MAX_VALUE which means no timeout > so no problem, but we do allow users to set it so we need to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20002) Reserve worker thread in ProcedureExecutor for high priority procedures
[ https://issues.apache.org/jira/browse/HBASE-20002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20002: -- Resolution: Later Status: Resolved (was: Patch Available) > Reserve worker thread in ProcedureExecutor for high priority procedures > --- > > Key: HBASE-20002 > URL: https://issues.apache.org/jira/browse/HBASE-20002 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-20002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20002) Reserve worker thread in ProcedureExecutor for high priority procedures
[ https://issues.apache.org/jira/browse/HBASE-20002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366528#comment-16366528 ] Duo Zhang commented on HBASE-20002: --- I think with HBASE-2, the problem is partially solved. We have a stuck checker which will start new worker thread if half of the workers are stuck, and then with HBASE-2, first it will select ServerCrashProcedure if any, and it will not block, and then finally it will select RecoverMetaProcedure since it is always placed at beginning, and then we can make progress. Then let me address the keep alive problem first, I've already filed an issue I think. The patch would be the last choice if all other approach are failed. > Reserve worker thread in ProcedureExecutor for high priority procedures > --- > > Key: HBASE-20002 > URL: https://issues.apache.org/jira/browse/HBASE-20002 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-20002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20001) cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
[
https://issues.apache.org/jira/browse/HBASE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20001:
---
Fix Version/s: (was: 1.4.2)
1.4.3
> cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
> -
>
> Key: HBASE-20001
> URL: https://issues.apache.org/jira/browse/HBASE-20001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.1.7
>Reporter: Francis Liu
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.3.2, 1.5.0, 1.4.3
>
> Attachments: HBASE-20001.branch-1.4.001.patch
>
>
> In RegionStates.cleanIfNoMetaEntry()
> {{if (MetaTableAccessor.getRegion(server.getConnection(),
> hri.getEncodedNameAsBytes()) == null) {}}
> {{regionOffline(hri);}}
> {{FSUtils.deleteRegionDir(server.getConfiguration(), hri);}}
> }
> But api expects regionname
> {{public static Pair getRegion(Connection
> connection, byte [] regionName)}}
> So we might end up cleaning good regions.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19166) Add translation for handling hbase.regionserver.wal.WALEdit
[
https://issues.apache.org/jira/browse/HBASE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366510#comment-16366510
]
Ted Yu commented on HBASE-19166:
w.r.t. "Got unknown writer class: AsyncProtobufLogWriter", maybe
AsyncProtobufLogWriter can override the following method:
{code}
protected String getWriterClassName() {
{code}
by returning "ProtobufLogWriter" which would be recognized by branch-1 codebase.
> Add translation for handling hbase.regionserver.wal.WALEdit
> ---
>
> Key: HBASE-19166
> URL: https://issues.apache.org/jira/browse/HBASE-19166
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Ted Yu
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
>
> For hlog generated by 1.x, using WALPlayer from hbase2 would result in:
> {code}
> 2017-11-02 21:22:40,907 INFO [main] mapreduce.Job: Task Id :
> attempt_1509641483571_0003_m_00_0, Status : FAILED
> Error: java.lang.ClassCastException:
> org.apache.hadoop.hbase.regionserver.wal.WALEdit cannot be cast to
> org.apache.hadoop.hbase.wal.WALEdit
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALCellMapper.map(WALPlayer.java:143)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
> {code}
> HBASE-16479 relocated WALEdit.
> Chatting with Enis, he mentioned adding translation for handling
> hbase.regionserver.wal.WALEdit
> This way, WAL from 1.x can be recognized by hbase-2
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20002) Reserve worker thread in ProcedureExecutor for high priority procedures
[ https://issues.apache.org/jira/browse/HBASE-20002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366509#comment-16366509 ] Duo Zhang commented on HBASE-20002: --- Let me check the failed UTs. And seems the TestDLSXXX have already been fixed by HBASE-19998? Then we need new UTs to reproduce the dead lock problem... > Reserve worker thread in ProcedureExecutor for high priority procedures > --- > > Key: HBASE-20002 > URL: https://issues.apache.org/jira/browse/HBASE-20002 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-20002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19953) Avoid calling post* hook when procedure fails
[
https://issues.apache.org/jira/browse/HBASE-19953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366499#comment-16366499
]
Hadoop QA commented on HBASE-19953:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
56s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
31s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
58s{color} | {color:red} hbase-server: The patch generated 21 new + 227
unchanged - 2 fixed = 248 total (was 229) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 2m
32s{color} | {color:red} patch has 10 errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 0s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}101m 51s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
21s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}130m 28s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestMasterObserverPostCalls |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19953 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12910825/HBASE-19952.001.branch-2.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7243ffd9ddff 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 30dd595d3b |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/11547/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11547/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11547/testReport/ |
| asflicense |
[jira] [Commented] (HBASE-19980) NullPointerException when restoring a snapshot after splitting a region
[
https://issues.apache.org/jira/browse/HBASE-19980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366494#comment-16366494
]
Toshihiro Suzuki commented on HBASE-19980:
--
[~yuzhih...@gmail.com] Please wait. I will retry to reproduce this issue in
branch-1 again.
> NullPointerException when restoring a snapshot after splitting a region
> ---
>
> Key: HBASE-19980
> URL: https://issues.apache.org/jira/browse/HBASE-19980
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19980-UT.patch, HBASE-19980.branch-1.001.patch,
> HBASE-19980.master.001.patch, HBASE-19980.master.002.patch
>
>
> Investigating HBASE-19893, I'm facing the following NullPointerException when
> restoring a snapshot after splitting a region:
> {code}
> 2018-02-13 01:59:56,033 ERROR [PEWorker-1]
> procedure.RestoreSnapshotProcedure(403): restore snapshot={
> ss=snaptb1-1518454780480
> table=testRestoreSnapshotAfterSplittingRegions-1518454780480 type=FLUSH }
> failed in on-disk restore. Try re-running the restore command.
> java.io.IOException
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.editRegions(ModifyRegionUtils.java:222)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:424)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:227)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:177)
> at
> org.apache.hadoop.hbase.master.procedure.RestoreSnapshotProcedure.restoreSnapshot(RestoreSnapshotProcedure.java:395)
> at
> org.apache.hadoop.hbase.master.procedure.RestoreSnapshotProcedure.executeFromState(RestoreSnapshotProcedure.java:143)
> at
> org.apache.hadoop.hbase.master.procedure.RestoreSnapshotProcedure.executeFromState(RestoreSnapshotProcedure.java:58)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:180)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1455)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1224)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:751)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:675)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreRegion(RestoreSnapshotHelper.java:521)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreRegion(RestoreSnapshotHelper.java:464)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$300(RestoreSnapshotHelper.java:116)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.editRegion(RestoreSnapshotHelper.java:427)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$2.call(ModifyRegionUtils.java:209)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$2.call(ModifyRegionUtils.java:206)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19998) Flakey TestVisibilityLabelsWithDefaultVisLabelService
[
https://issues.apache.org/jira/browse/HBASE-19998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366386#comment-16366386
]
Hudson commented on HBASE-19998:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4591 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4591/])
HBASE-19998 Flakey TestVisibilityLabelsWithDefaultVisLabelService (stack: rev
50c705dad9825d3095352b997be35a2568bd6190)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionServerTracker.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
> Flakey TestVisibilityLabelsWithDefaultVisLabelService
> -
>
> Key: HBASE-19998
> URL: https://issues.apache.org/jira/browse/HBASE-19998
> Project: HBase
> Issue Type: Bug
> Components: flakey, test
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19998.master.001.patch,
> hbase-19988.master.001.patch
>
>
> This is a good one. Its a timeout and though it has lots of test methods, the
> problem is one of them gets stuck. The test method kills a RegionServers then
> starts a new one. Usually all works out fine but the odd time there is this
> unexplained MOVE that gets interjected just as ServerCrashProcedure starts
> up. hbase:meta gets stuck (perhaps this is what is being referred to here:
> https://issues.apache.org/jira/browse/HBASE-19929?focusedCommentId=16356906=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16356906).
> It is trying to run the MOVE by first unassigning from the server that has
> just crashed. It never succeeds. Need to fix this. Need to figure where these
> Move operations are coming from too. Let me add some debug.
> See here how we are well into ServerCrashProcedure... and then two MOVEs
> cut-in... for hbase:meta and for namespace:
> {code}
>
> 2018-02-14 02:35:19,806 DEBUG [PEWorker-6]
> procedure.ServerCrashProcedure(192): pid=10,
> state=RUNNABLE:SERVER_CRASH_PROCESS_META; ServerCrashProcedure
> server=asf903.gq1.ygridcore.net,59608,1518575711969, splitWal=true,
> meta=true; Processing hbase:meta that was on
> asf903.gq1.ygridcore.net,59608,1518575711969
> 2018-02-14 02:35:19,807 INFO [PEWorker-6]
> procedure2.ProcedureExecutor(1498): Initialized subprocedures=[{pid=12,
> ppid=10, state=RUNNABLE:RECOVER_META_SPLIT_LOGS; RecoverMetaProcedure
> failedMetaServer=asf903.gq1.ygridcore.net,59608,1518575711969, splitWal=true}]
> 2018-02-14 02:35:19,811 DEBUG [Thread-214] procedure2.ProcedureExecutor(868):
> Stored pid=11, state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=hbase:meta,,1.1588230740,
> source=asf903.gq1.ygridcore.net,59608,1518575711969, destination=
> 2018-02-14 02:35:19,813 INFO [PEWorker-8]
> procedure.MasterProcedureScheduler(813): pid=11,
> state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=hbase:meta,,1.1588230740,
> source=asf903.gq1.ygridcore.net,59608,1518575711969, destination= hbase:meta
> hbase:meta,,1.1588230740
> 2018-02-14 02:35:19,814 INFO [PEWorker-8]
> procedure2.ProcedureExecutor(1498): Initialized subprocedures=[{pid=14,
> ppid=11, state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
> table=hbase:meta, region=1588230740,
> server=asf903.gq1.ygridcore.net,59608,1518575711969}]
> 2018-02-14 02:35:19,831 DEBUG [Thread-214] procedure2.ProcedureExecutor(868):
> Stored pid=13, state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=hbase:namespace,,1518575716296.e52a160b3f3a57ab50d710eba62d9b15.,
> source=asf903.gq1.ygridcore.net,59608,1518575711969, destination=
> 2018-02-14 02:35:19,833 INFO [PEWorker-10]
> procedure.MasterProcedureScheduler(813): pid=13,
> state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=hbase:namespace,,1518575716296.e52a160b3f3a57ab50d710eba62d9b15.,
> source=asf903.gq1.ygridcore.net,59608,1518575711969, destination=
> hbase:namespace
> hbase:namespace,,1518575716296.e52a160b3f3a57ab50d710eba62d9b15.
> 2018-02-14 02:35:19,837 INFO [PEWorker-10]
> procedure2.ProcedureExecutor(1498): Initialized subprocedures=[{pid=15,
> ppid=13, state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
> table=hbase:namespace, region=e52a160b3f3a57ab50d710eba62d9b15,
> server=asf903.gq1.ygridcore.net,59608,1518575711969}]
>
> {code}
> Here is the failure of the unassign:
> {code}
> 2018-02-14 02:35:19,944 WARN [PEWorker-8]
> assignment.RegionTransitionProcedure(187): Remote call failed pid=14,
> ppid=11, state=RUNNABLE:REGION_TRANSITION_DISPATCH;
[jira] [Updated] (HBASE-19953) Avoid calling post* hook when procedure fails
[
https://issues.apache.org/jira/browse/HBASE-19953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-19953:
---
Status: Patch Available (was: Open)
Here's an .001 which fixes deleteNamespace and modifyNamespace with some tests
to boot.
It looks like nearly every DDL operation in HMaster is affected by this
because, for 2.0 clients, we don't take a real latch (we get the NoopLatch
inside ProcedurePrepareLatch). Thankfully, I have a test case which lets us see
this very easily. Definitely have more methods to fix.
Comments welcome on the approach. Not too elegant at this point (lots of
copy-paste).
> Avoid calling post* hook when procedure fails
> -
>
> Key: HBASE-19953
> URL: https://issues.apache.org/jira/browse/HBASE-19953
> Project: HBase
> Issue Type: Bug
> Components: master, proc-v2
>Reporter: Ramesh Mani
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19952.001.branch-2.patch
>
>
> Ramesh pointed out a case where I think we're mishandling some post\*
> MasterObserver hooks. Specifically, I'm looking at the deleteNamespace.
> We synchronously execute the DeleteNamespace procedure. When the user
> provides a namespace that isn't empty, the procedure does a rollback (which
> is just a no-op), but this doesn't propagate an exception up to the
> NonceProcedureRunnable in {{HMaster#deleteNamespace}}. It took Ramesh
> pointing it out a bit better to me that the code executes a bit differently
> than we actually expect.
> I think we need to double-check our post hooks and make sure we aren't
> invoking them when the procedure actually failed. cc/ [~Apache9], [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19903) Split TestShell so it will not time out
[ https://issues.apache.org/jira/browse/HBASE-19903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366387#comment-16366387 ] Hudson commented on HBASE-19903: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4591 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4591/]) HBASE-19903 Split TestShell so it will not time out (zhangduo: rev 01a2869363d45c7692f3984f31ab8c7a50d117fc) * (edit) hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestShell.java * (add) hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestQuotasShell.java * (add) hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestTableShell.java * (add) hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestAdminShell.java * (edit) hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestShellNoCluster.java > Split TestShell so it will not time out > --- > > Key: HBASE-19903 > URL: https://issues.apache.org/jira/browse/HBASE-19903 > Project: HBase > Issue Type: Sub-task > Components: shell, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19903.patch > > > It runs more than 10 minutes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19953) Avoid calling post* hook when procedure fails
[
https://issues.apache.org/jira/browse/HBASE-19953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-19953:
---
Attachment: HBASE-19952.001.branch-2.patch
> Avoid calling post* hook when procedure fails
> -
>
> Key: HBASE-19953
> URL: https://issues.apache.org/jira/browse/HBASE-19953
> Project: HBase
> Issue Type: Bug
> Components: master, proc-v2
>Reporter: Ramesh Mani
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19952.001.branch-2.patch
>
>
> Ramesh pointed out a case where I think we're mishandling some post\*
> MasterObserver hooks. Specifically, I'm looking at the deleteNamespace.
> We synchronously execute the DeleteNamespace procedure. When the user
> provides a namespace that isn't empty, the procedure does a rollback (which
> is just a no-op), but this doesn't propagate an exception up to the
> NonceProcedureRunnable in {{HMaster#deleteNamespace}}. It took Ramesh
> pointing it out a bit better to me that the code executes a bit differently
> than we actually expect.
> I think we need to double-check our post hooks and make sure we aren't
> invoking them when the procedure actually failed. cc/ [~Apache9], [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20005) TestFromClientSideWithCoprocessor#testCheckAndDeleteWithCompareOp occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-20005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20005:
---
Attachment: testCheckAndDeleteWithCompareOp.out
> TestFromClientSideWithCoprocessor#testCheckAndDeleteWithCompareOp
> occasionally fails
>
>
> Key: HBASE-20005
> URL: https://issues.apache.org/jira/browse/HBASE-20005
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Major
> Attachments: testCheckAndDeleteWithCompareOp.out
>
>
> By adding message for the assertions, the test failed at:
> {code}
> ok = table.checkAndMutate(ROW, FAMILY).qualifier(QUALIFIER)
> .ifMatches(CompareOperator.GREATER, value4).thenDelete(delete);
> -assertTrue(ok);
> +assertTrue("gr", ok);
> {code}
> I ran the test with MemoryCompactionPolicy.NONE but it still failed at the
> same place.
> Both branch-2 and master have this intermittent test failure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20005) TestFromClientSideWithCoprocessor#testCheckAndDeleteWithCompareOp occasionally fails
Ted Yu created HBASE-20005:
--
Summary:
TestFromClientSideWithCoprocessor#testCheckAndDeleteWithCompareOp occasionally
fails
Key: HBASE-20005
URL: https://issues.apache.org/jira/browse/HBASE-20005
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
By adding message for the assertions, the test failed at:
{code}
ok = table.checkAndMutate(ROW, FAMILY).qualifier(QUALIFIER)
.ifMatches(CompareOperator.GREATER, value4).thenDelete(delete);
-assertTrue(ok);
+assertTrue("gr", ok);
{code}
I ran the test with MemoryCompactionPolicy.NONE but it still failed at the same
place.
Both branch-2 and master have this intermittent test failure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HBASE-18276) Release 1.2.7
[ https://issues.apache.org/jira/browse/HBASE-18276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18276 started by Sean Busbey. --- > Release 1.2.7 > - > > Key: HBASE-18276 > URL: https://issues.apache.org/jira/browse/HBASE-18276 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.7 > > > about time to get rolling on 1.2.7 for ~monthly -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19988) HRegion#lockRowsAndBuildMiniBatch() is too chatty when interrupted while waiting for a row lock
[ https://issues.apache.org/jira/browse/HBASE-19988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366298#comment-16366298 ] Umesh Agashe commented on HBASE-19988: -- hmm... failures doesn't seem to be related to the changes. 4 runs, 4 different tests failed each time: hadoop.hbase.replication.TestReplicationDroppedTables hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures hadoop.hbase.quotas.TestQuotaThrottle hadoop.hbase.TestJMXListener > HRegion#lockRowsAndBuildMiniBatch() is too chatty when interrupted while > waiting for a row lock > --- > > Key: HBASE-19988 > URL: https://issues.apache.org/jira/browse/HBASE-19988 > Project: HBase > Issue Type: Improvement > Components: amv2 >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: hbase-19988.master.001.patch, > hbase-19988.master.001.patch, hbase-19988.master.002.patch, > hbase-19988.master.002.patch > > > See HBASE-19970, TestHRegionWithInMemoryFlush created 4.2g log file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19988) HRegion#lockRowsAndBuildMiniBatch() is too chatty when interrupted while waiting for a row lock
[
https://issues.apache.org/jira/browse/HBASE-19988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366293#comment-16366293
]
Hadoop QA commented on HBASE-19988:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
33s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
27s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
36s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
52s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}104m 20s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}133m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestJMXListener |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19988 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12910789/hbase-19988.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fda3c6d43cd3 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 01a2869363 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
[jira] [Commented] (HBASE-15911) NPE in AssignmentManager.onRegionTransition after Master restart
[ https://issues.apache.org/jira/browse/HBASE-15911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366288#comment-16366288 ] Ben Lau commented on HBASE-15911: - Thanks [~pankaj2461], I will take a look at master to see if this is broken there as well or just in branch-1 and/or branch-2 and then submit forward ports of our patch later (we are on 1.3). Master code looks somewhat different. > NPE in AssignmentManager.onRegionTransition after Master restart > > > Key: HBASE-15911 > URL: https://issues.apache.org/jira/browse/HBASE-15911 > Project: HBase > Issue Type: Bug > Components: master, Region Assignment >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Major > > 16/05/27 17:49:18 ERROR ipc.RpcServer: Unexpected throwable object > java.lang.NullPointerException > at > org.apache.hadoop.hbase.master.AssignmentManager.onRegionTransition(AssignmentManager.java:4364) > at > org.apache.hadoop.hbase.master.MasterRpcServices.reportRegionStateTransition(MasterRpcServices.java:1421) > at > org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:8623) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2239) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:116) > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:137) > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:112) > at java.lang.Thread.run(Thread.java:745) > I'm pretty sure I've seen it before and more than once, but never got to dig > in. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-19710) hbase:namespace table was stuck in transition
[
https://issues.apache.org/jira/browse/HBASE-19710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-19710.
Resolution: Duplicate
Dup of HBASE-19998
> hbase:namespace table was stuck in transition
> -
>
> Key: HBASE-19710
> URL: https://issues.apache.org/jira/browse/HBASE-19710
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Critical
> Attachments: master-005-log.tar.gz, master-006.tar.gz,
> rs-009.log.tar.gz
>
>
> ITBLL with chaos monkey failed due to namespace table getting stuck in
> transition.
> From hbase-hbase-master-ctr-e137-1514896590304-3629-01-06.hwx.site.log ,
> we can see that master closed namespace table on 09:
> {code}
> 2018-01-04 17:24:35,067 DEBUG [main-EventThread] zookeeper.ZKWatcher:
> master:2-0x160c222710c0028,
> quorum=ctr-e137-1514896590304-3629-01-11.hwx.site:2181,ctr-e137-
> 1514896590304-3629-01-14.hwx.site:2181,ctr-e137-1514896590304-3629-01-09.hwx.site:2181,ctr-e137-1514896590304-3629-01-06.hwx.site:2181,ctr-e137-1514896590304-3629-
>
> 01-03.hwx.site:2181,ctr-e137-1514896590304-3629-01-07.hwx.site:2181,ctr-e137-1514896590304-3629-01-13.hwx.site:2181,ctr-e137-1514896590304-3629-01-02.hwx.site:
>
> 2181,ctr-e137-1514896590304-3629-01-12.hwx.site:2181,ctr-e137-1514896590304-3629-01-08.hwx.site:2181,ctr-e137-1514896590304-3629-01-10.hwx.site:2181,
> baseZNode=/ hbase-unsecure Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected, path=/hbase-unsecure/rs
> 2018-01-04 17:24:35,067 INFO [ProcExecWrkr-5] assignment.RegionStateStore:
> pid=643 updating hbase:meta
> row=hbase:namespace,,1515085217343.a95ed2d7434a43390fbec73abeeb9fd9.,
> regionState=CLOSING,
> regionLocation=ctr-e137-1514896590304-3629-01-09.hwx.site,16020,1515086643872
> ...
> 2018-01-04 17:24:35,246 INFO [ProcExecWrkr-12]
> procedure.MasterProcedureScheduler: pid=647, ppid=642,
> state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure table=hbase:
> namespace, region=a95ed2d7434a43390fbec73abeeb9fd9 hbase:namespace
> hbase:namespace,,1515085217343.a95ed2d7434a43390fbec73abeeb9fd9.
> 2018-01-04 17:25:17,041 DEBUG
> [ctr-e137-1514896590304-3629-01-06:2.masterManager]
> procedure2.ProcedureExecutor: Loading pid=641,
> state=WAITING:MOVE_REGION_ASSIGN; MoveRegionProcedure
> hri=hbase:namespace,,1515085217343.a95ed2d7434a43390fbec73abeeb9fd9.,
> source=ctr-e137-1514896590304-3629-01-09.hwx.site,16020,1515086643872,
> destination=
> {code}
> For the move operation, from ctr-e137-1514896590304-3629-01-09.hwx.site
> log:
> {code}
> 2018-01-04 17:24:34,855 DEBUG
> [RS_CLOSE_REGION-ctr-e137-1514896590304-3629-01-09:16020-0]
> coprocessor.CoprocessorHost: Stop coprocessor
> org.apache.hadoop.hbase.security. access.SecureBulkLoadEndpoint
> 2018-01-04 17:24:34,855 INFO
> [RS_CLOSE_REGION-ctr-e137-1514896590304-3629-01-09:16020-0]
> regionserver.HRegion: Closed hbase:namespace,,1515085217343.
> a95ed2d7434a43390fbec73abeeb9fd9.
> 2018-01-04 17:24:34,856 DEBUG
> [RS_CLOSE_REGION-ctr-e137-1514896590304-3629-01-09:16020-0]
> handler.CloseRegionHandler: Closed hbase:namespace,,1515085217343.
> a95ed2d7434a43390fbec73abeeb9fd9.
> ...
> 2018-01-04 17:25:47,607 DEBUG
> [RpcServer.priority.FPBQ.Fifo.handler=18,queue=0,port=16020] ipc.RpcServer:
> callId: 16 service: ClientService methodName: Get size: 103
> connection: 172.27.13.80:36738 deadline: 1515086837568
> org.apache.hadoop.hbase.NotServingRegionException:
> hbase:namespace,,1515085217343.a95ed2d7434a43390fbec73abeeb9fd9. is not
> online on ctr-e137-1514896590304-3629-01-09.hwx. site,16020,1515086719163
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3312)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3289)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1354)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2360)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41544)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:403)
> {code}
> We can see that the region server was not serving the region.
> After that, the masters kept thinking namespace table was on 0009, leading to
> prolonged downtime.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19851) ProcV2 framework's rollback() needs to maintain stack like execute()
[
https://issues.apache.org/jira/browse/HBASE-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366272#comment-16366272
]
stack commented on HBASE-19851:
---
I saw another version of this issue in nightly. We have reached one of the
states below:
{code}
case MERGE_TABLE_REGIONS_POST_OPERATION:
case MERGE_TABLE_REGIONS_OPEN_MERGED_REGION:
case MERGE_TABLE_REGIONS_POST_MERGE_COMMIT_OPERATION:
case MERGE_TABLE_REGIONS_UPDATE_META:
String msg = this + " We are in the " + state + " state."
+ " It is complicated to rollback the merge operation that region
server is working on."
+ " Rollback is not supported and we should let the merge operation
to complete";
LOG.warn(msg);
// PONR
throw new UnsupportedOperationException(this + " unhandled state=" +
state);
{code}
We throw the UnsupportedOperationException which then tickles a CODE-BUG
warning in the framework which is its way of saying not-supposed-to-happen and
it is right; the merge procedure is not complete.
{code}
2018-02-15 16:18:58,492 WARN [PEWorker-1]
assignment.MergeTableRegionsProcedure(311): Failed rollback attempt step
MERGE_TABLE_REGIONS_UPDATE_META for merging the regions
[dcd17e940fbb453cf6df1cee87b0cec7, 2b03bbfd418c9b3533bc2bb3cf6ce748] in table
testRollbackAndDoubleExecution
java.lang.UnsupportedOperationException: pid=46,
state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META,
exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via
MergeTableRegionsProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException:
abort requested; MergeTableRegionsProcedure
table=testRollbackAndDoubleExecution,
regions=[dcd17e940fbb453cf6df1cee87b0cec7, 2b03bbfd418c9b3533bc2bb3cf6ce748],
forcibly=false unhandled state=MERGE_TABLE_REGIONS_UPDATE_META
at
org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.rollbackState(MergeTableRegionsProcedure.java:291)
at
org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.rollbackState(MergeTableRegionsProcedure.java:78)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:199)
at org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:859)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1356)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1312)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1181)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
2018-02-15 16:18:58,497 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
CODE-BUG: Uncaught runtime exception for pid=46,
state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META,
exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via
MergeTableRegionsProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException:
abort requested; MergeTableRegionsProcedure
table=testRollbackAndDoubleExecution,
regions=[dcd17e940fbb453cf6df1cee87b0cec7, 2b03bbfd418c9b3533bc2bb3cf6ce748],
forcibly=false
java.lang.UnsupportedOperationException: pid=46,
state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META,
exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via
MergeTableRegionsProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException:
abort requested; MergeTableRegionsProcedure
table=testRollbackAndDoubleExecution,
regions=[dcd17e940fbb453cf6df1cee87b0cec7, 2b03bbfd418c9b3533bc2bb3cf6ce748],
forcibly=false unhandled state=MERGE_TABLE_REGIONS_UPDATE_META
at
org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.rollbackState(MergeTableRegionsProcedure.java:291)
at
org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.rollbackState(MergeTableRegionsProcedure.java:78)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:199)
at org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:859)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1356)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1312)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1181)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
{code}
> ProcV2 framework's rollback() needs to maintain stack like execute()
>
>
> Key:
[jira] [Updated] (HBASE-19851) ProcV2 framework's rollback() needs to maintain stack like execute()
[ https://issues.apache.org/jira/browse/HBASE-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19851: -- Priority: Critical (was: Major) > ProcV2 framework's rollback() needs to maintain stack like execute() > > > Key: HBASE-19851 > URL: https://issues.apache.org/jira/browse/HBASE-19851 > Project: HBase > Issue Type: Improvement > Components: proc-v2 >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Priority: Critical > Fix For: 2.0.0 > > > Refer to HBASE-19839 and detailed description there. Bug showed up as a flaky > test TestMergeTableRegionsProcedure.testRollbackAndDoubleExecution(). In > short, when rollback() is called after step 5 > (MERGE_TABLE_REGIONS_CLOSE_REGIONS), rollback of the procedure tries to > reassign two regions that are to be merged. It spawns 2 AssignProcedures and > don't wait on them to finish. Rollback should have ability to have > sub-procedures and separate proc stack. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19988) HRegion#lockRowsAndBuildMiniBatch() is too chatty when interrupted while waiting for a row lock
[ https://issues.apache.org/jira/browse/HBASE-19988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366229#comment-16366229 ] Appy commented on HBASE-19988: -- Yes, sir, am good with the patch. > HRegion#lockRowsAndBuildMiniBatch() is too chatty when interrupted while > waiting for a row lock > --- > > Key: HBASE-19988 > URL: https://issues.apache.org/jira/browse/HBASE-19988 > Project: HBase > Issue Type: Improvement > Components: amv2 >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: hbase-19988.master.001.patch, > hbase-19988.master.001.patch, hbase-19988.master.002.patch, > hbase-19988.master.002.patch > > > See HBASE-19970, TestHRegionWithInMemoryFlush created 4.2g log file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19770) Add '--return-values' option to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-19770:
---
Fix Version/s: 1.4.2
1.5.0
> Add '--return-values' option to Shell to print return values of commands in
> interactive mode
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19770.000.branch-2.addendum.patch,
> HBASE-19770.001.branch-2.patch, HBASE-19770.002.branch-2.patch,
> HBASE-19770.003.branch-2.patch, HBASE-19770.004.branch-2.patch,
> HBASE-19770.ADDENDUM.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366210#comment-16366210
]
Hadoop QA commented on HBASE-19920:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
46s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
31s{color} | {color:red} hbase-client: The patch generated 3 new + 297
unchanged - 0 fixed = 300 total (was 297) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
59s{color} | {color:red} hbase-server: The patch generated 1 new + 5 unchanged
- 0 fixed = 6 total (was 5) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
19s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
27s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
40s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
58s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
3s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}102m
18s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}137m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19920 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12910777/HBASE-19920.v9.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux
[jira] [Commented] (HBASE-19953) Avoid calling post* hook when procedure fails
[
https://issues.apache.org/jira/browse/HBASE-19953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366207#comment-16366207
]
Josh Elser commented on HBASE-19953:
So, copy-paste'ing the latch code already there elsewhere didn't work because
the default latch implementation is a no-op which doesn't wait (because we have
a client which is async-procedure compatible)
I can force a latch which will wait as a temporary workaround, which will have
the (undesired) side-effect of holding the RPC open. Not the end of the world
for now.
> Avoid calling post* hook when procedure fails
> -
>
> Key: HBASE-19953
> URL: https://issues.apache.org/jira/browse/HBASE-19953
> Project: HBase
> Issue Type: Bug
> Components: master, proc-v2
>Reporter: Ramesh Mani
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> Ramesh pointed out a case where I think we're mishandling some post\*
> MasterObserver hooks. Specifically, I'm looking at the deleteNamespace.
> We synchronously execute the DeleteNamespace procedure. When the user
> provides a namespace that isn't empty, the procedure does a rollback (which
> is just a no-op), but this doesn't propagate an exception up to the
> NonceProcedureRunnable in {{HMaster#deleteNamespace}}. It took Ramesh
> pointing it out a bit better to me that the code executes a bit differently
> than we actually expect.
> I think we need to double-check our post hooks and make sure we aren't
> invoking them when the procedure actually failed. cc/ [~Apache9], [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19352) Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags
[ https://issues.apache.org/jira/browse/HBASE-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-19352: Status: Open (was: Patch Available) Moving out of patch available since it doesn't apply to master anymore. Want some help in closing this out [~esteban]? > Port HADOOP-10379: Protect authentication cookies with the HttpOnly and > Secure flags > > > Key: HBASE-19352 > URL: https://issues.apache.org/jira/browse/HBASE-19352 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez >Priority: Major > Attachments: HBASE-19352.master.v0.patch > > > This came via a security scanner, since we have a fork of HttpServer2 in > HBase we should include it too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19352) Port HADOOP-10379: Protect authentication cookies with the HttpOnly and Secure flags
[
https://issues.apache.org/jira/browse/HBASE-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366180#comment-16366180
]
Hadoop QA commented on HBASE-19352:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-19352 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19352 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12899872/HBASE-19352.master.v0.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11546/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Port HADOOP-10379: Protect authentication cookies with the HttpOnly and
> Secure flags
>
>
> Key: HBASE-19352
> URL: https://issues.apache.org/jira/browse/HBASE-19352
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
>Priority: Major
> Attachments: HBASE-19352.master.v0.patch
>
>
> This came via a security scanner, since we have a fork of HttpServer2 in
> HBase we should include it too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19770) Add '--return-values' option to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366176#comment-16366176
]
Josh Elser commented on HBASE-19770:
Thanks, Andrew! Will do so (lopping the addendum in to make one commit on
those).
> Add '--return-values' option to Shell to print return values of commands in
> interactive mode
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.000.branch-2.addendum.patch,
> HBASE-19770.001.branch-2.patch, HBASE-19770.002.branch-2.patch,
> HBASE-19770.003.branch-2.patch, HBASE-19770.004.branch-2.patch,
> HBASE-19770.ADDENDUM.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19770) Add '--return-values' option to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366169#comment-16366169
]
Andrew Purtell commented on HBASE-19770:
Sorry I missed the at-mention. I see this neither committed to branch-1.4 nor
branch-1. No issues at all with commit to either. Would certainly be fine to
put it in branch-1 for 1.5.
> Add '--return-values' option to Shell to print return values of commands in
> interactive mode
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.000.branch-2.addendum.patch,
> HBASE-19770.001.branch-2.patch, HBASE-19770.002.branch-2.patch,
> HBASE-19770.003.branch-2.patch, HBASE-19770.004.branch-2.patch,
> HBASE-19770.ADDENDUM.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366167#comment-16366167
]
Hadoop QA commented on HBASE-20004:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 15m
48s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
20s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} branch-1 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} branch-1 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
14s{color} | {color:red} hbase-server in branch-1 has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} branch-1 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} branch-1 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
32s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
29s{color} | {color:red} The patch causes 44 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
21s{color} | {color:red} The patch causes 44 errors with Hadoop v2.5.2. {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}118m 58s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}163m 3s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.TestMasterBalanceThrottling |
| | hadoop.hbase.util.TestHBaseFsck |
[jira] [Commented] (HBASE-20001) cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
[
https://issues.apache.org/jira/browse/HBASE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366158#comment-16366158
]
Thiruvel Thirumoolan commented on HBASE-20001:
--
[~chia7712] When we were investigating this problem internally, it was hard to
trace through regions that were cleaned up, we had to lookup namenode audit
logs.
> cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
> -
>
> Key: HBASE-20001
> URL: https://issues.apache.org/jira/browse/HBASE-20001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.1.7
>Reporter: Francis Liu
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.3.2, 1.5.0, 1.4.2
>
> Attachments: HBASE-20001.branch-1.4.001.patch
>
>
> In RegionStates.cleanIfNoMetaEntry()
> {{if (MetaTableAccessor.getRegion(server.getConnection(),
> hri.getEncodedNameAsBytes()) == null) {}}
> {{regionOffline(hri);}}
> {{FSUtils.deleteRegionDir(server.getConfiguration(), hri);}}
> }
> But api expects regionname
> {{public static Pair getRegion(Connection
> connection, byte [] regionName)}}
> So we might end up cleaning good regions.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19988) HRegion#lockRowsAndBuildMiniBatch() is too chatty when interrupted while waiting for a row lock
[ https://issues.apache.org/jira/browse/HBASE-19988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366156#comment-16366156 ] stack commented on HBASE-19988: --- Retry. The Quota failure is not in https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html but it has failed for me on local builds. Lets see if issue. Hadoop checks are the thrift schema download failure. [ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) on project hbase-thrift: Failed to install metadata org.apache.hbase:hbase-thrift:3.0.0-SNAPSHOT/maven-metadata.xml: Could not parse metadata /home/jenkins/.m2/repository/org/apache/hbase/hbase-thrift/3.0.0-SNAPSHOT/maven-metadata-local.xml: in epilog non whitespace content is not allowed but got / (position: END_TAG seen ...\n/... @25:2) -> [Help 1] [~appy] you good w/ this? > HRegion#lockRowsAndBuildMiniBatch() is too chatty when interrupted while > waiting for a row lock > --- > > Key: HBASE-19988 > URL: https://issues.apache.org/jira/browse/HBASE-19988 > Project: HBase > Issue Type: Improvement > Components: amv2 >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: hbase-19988.master.001.patch, > hbase-19988.master.001.patch, hbase-19988.master.002.patch, > hbase-19988.master.002.patch > > > See HBASE-19970, TestHRegionWithInMemoryFlush created 4.2g log file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19988) HRegion#lockRowsAndBuildMiniBatch() is too chatty when interrupted while waiting for a row lock
[ https://issues.apache.org/jira/browse/HBASE-19988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19988: -- Attachment: hbase-19988.master.002.patch > HRegion#lockRowsAndBuildMiniBatch() is too chatty when interrupted while > waiting for a row lock > --- > > Key: HBASE-19988 > URL: https://issues.apache.org/jira/browse/HBASE-19988 > Project: HBase > Issue Type: Improvement > Components: amv2 >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: hbase-19988.master.001.patch, > hbase-19988.master.001.patch, hbase-19988.master.002.patch, > hbase-19988.master.002.patch > > > See HBASE-19970, TestHRegionWithInMemoryFlush created 4.2g log file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19991) lots of hbase-rest test failures against hadoop 3
[ https://issues.apache.org/jira/browse/HBASE-19991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366150#comment-16366150 ] stack commented on HBASE-19991: --- +1 That list of exclusions doesn't look like it was figuring. > lots of hbase-rest test failures against hadoop 3 > - > > Key: HBASE-19991 > URL: https://issues.apache.org/jira/browse/HBASE-19991 > Project: HBase > Issue Type: Bug > Components: REST, test >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19991.WIP.patch, HBASE-19991.v1.patch, > HBASE-19991.v2.patch > > > mvn clean test -pl hbase-rest -Dhadoop.profile=3.0 > [ERROR] Tests run: 106, Failures: 95, Errors: 8, Skipped: 1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19996) Some nonce procs might not be cleaned up (follow up HBASE-19756)
[ https://issues.apache.org/jira/browse/HBASE-19996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366147#comment-16366147 ] Hudson commented on HBASE-19996: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4590 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4590/]) HBASE-19996 Some nonce procs might not be cleaned up (follow up (tedyu: rev 816d8602288f094eed5eaee27fa39730e0b11a98) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestFailedProcCleanup.java > Some nonce procs might not be cleaned up (follow up HBASE-19756) > > > Key: HBASE-19996 > URL: https://issues.apache.org/jira/browse/HBASE-19996 > Project: HBase > Issue Type: Bug >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 1.3.2, 1.5.0, 2.0.0-beta-2, 1.4.2 > > Attachments: HBASE-19996.branch-1.4.001.patch, > HBASE-19996.branch-1.4.001.patch, HBASE-19996.master.001.patch > > > Follow up to HBASE-19756 which dealt with NPEs during proc cleanup. > Unfortunately, the patch for branch-1 might not remove some valid procs too. > The branch-2 patch doesn't have this problem. This fixes the branch-1 bug and > also adds another test to branch-2. Thanks to [~toffer] for flagging this > internally. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19930) fix ImmutableMemStoreLAB#forceCopyOfBigCellInto
[ https://issues.apache.org/jira/browse/HBASE-19930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366146#comment-16366146 ] Hudson commented on HBASE-19930: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4590 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4590/]) HBASE-19930: Fixing the bug, in the rare case when there is a merge into (anastas: rev 07e93458fe69b7701ea2045926b4afe30d166cb5) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ImmutableMemStoreLAB.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java > fix ImmutableMemStoreLAB#forceCopyOfBigCellInto > --- > > Key: HBASE-19930 > URL: https://issues.apache.org/jira/browse/HBASE-19930 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Gali Sheffi >Assignee: Gali Sheffi >Priority: Major > Attachments: HBASE-19930-V01.patch, HBASE-19930-V02.patch, > HBASE-19930-V03.patch, HBASE-19930-V04.patch, HBASE-19930-V05.patch, > HBASE-19930-V06.patch > > > This issue is about fixing ImmutableMemStoreLAB#forceCopyOfBigCellInto. > Following a comment in HBASE-19133 regarding a bug in > ImmutableMemStoreLAB#forceCopyOfBigCellInto (assuming this method is never > called for an ImmutableMemStoreLAB, and just throwing an > IllegalStateException whenever called), the forceCopyOfBigCellInto method now > performs the copy of big cells on the first MSLABImpl in its mslabs > linked-list. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20002) Reserve worker thread in ProcedureExecutor for high priority procedures
[
https://issues.apache.org/jira/browse/HBASE-20002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366078#comment-16366078
]
Hadoop QA commented on HBASE-20002:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
22s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} hbase-procedure: The patch generated 0 new + 33
unchanged - 6 fixed = 33 total (was 39) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} The patch hbase-server passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
6s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
1s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
8s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
24s{color} | {color:red} hbase-server generated 1 new + 2 unchanged - 0 fixed =
3 total (was 2) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 3s{color}
| {color:red} hbase-procedure in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}109m 46s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}141m 16s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.procedure2.TestYieldProcedures |
| | hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures |
| | hadoop.hbase.master.procedure.TestCreateNamespaceProcedure |
| | hadoop.hbase.master.procedure.TestModifyTableProcedure |
| | hadoop.hbase.master.procedure.TestModifyNamespaceProcedure |
| |
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366069#comment-16366069
]
Hadoop QA commented on HBASE-18294:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 18 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 7m
22s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} The patch hbase-common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} The patch hbase-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} hbase-server: The patch generated 0 new + 805
unchanged - 9 fixed = 805 total (was 814) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
54s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 1s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
33s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
26s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}118m
45s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m
23s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}184m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366068#comment-16366068 ] Sean Busbey commented on HBASE-20004: - HBASE-10473 came in as a result of handling a security incident on private@hbase that was the result of a user report for a scan from some tool named 'QualysGuard'. I'll dig at the cited support documentation to see if OPTIONS is really a problem if we're disallowing TRACE. > Client is not able to execute REST queries through browser in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Critical > Attachments: HBASE-20004.branch-1.patch, HBASE-20004.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19680) BufferedMutatorImpl#mutate should wait the result from AP in order to throw the failed mutations
[ https://issues.apache.org/jira/browse/HBASE-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366067#comment-16366067 ] Chia-Ping Tsai commented on HBASE-19680: bq. perf-wise? Maybe I could help sir. yep. Will finish the test asap. > BufferedMutatorImpl#mutate should wait the result from AP in order to throw > the failed mutations > > > Key: HBASE-19680 > URL: https://issues.apache.org/jira/browse/HBASE-19680 > Project: HBase > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19680.v0.patch, HBASE-19680.v1.patch > > > Currently, BMI#mutate doesn't wait the result from AP so the errors are > stored in AP. The only way which can return the errors to user is, calling > the flush to catch the exception. That is non-intuitive. > I feel BMI#mutate should wait the result. That is to say, user can parse the > exception thrown by BM#mutate to get the failed mutations. Also, we can > remove the global error from AP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19980) NullPointerException when restoring a snapshot after splitting a region
[
https://issues.apache.org/jira/browse/HBASE-19980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366063#comment-16366063
]
Ted Yu commented on HBASE-19980:
bq. the NullPointerException was not reproduced in the unit test in branch-1
That means branch-1 doesn't have this issue.
Shall we resolve this JIRA ?
> NullPointerException when restoring a snapshot after splitting a region
> ---
>
> Key: HBASE-19980
> URL: https://issues.apache.org/jira/browse/HBASE-19980
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19980-UT.patch, HBASE-19980.branch-1.001.patch,
> HBASE-19980.master.001.patch, HBASE-19980.master.002.patch
>
>
> Investigating HBASE-19893, I'm facing the following NullPointerException when
> restoring a snapshot after splitting a region:
> {code}
> 2018-02-13 01:59:56,033 ERROR [PEWorker-1]
> procedure.RestoreSnapshotProcedure(403): restore snapshot={
> ss=snaptb1-1518454780480
> table=testRestoreSnapshotAfterSplittingRegions-1518454780480 type=FLUSH }
> failed in on-disk restore. Try re-running the restore command.
> java.io.IOException
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.editRegions(ModifyRegionUtils.java:222)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:424)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:227)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:177)
> at
> org.apache.hadoop.hbase.master.procedure.RestoreSnapshotProcedure.restoreSnapshot(RestoreSnapshotProcedure.java:395)
> at
> org.apache.hadoop.hbase.master.procedure.RestoreSnapshotProcedure.executeFromState(RestoreSnapshotProcedure.java:143)
> at
> org.apache.hadoop.hbase.master.procedure.RestoreSnapshotProcedure.executeFromState(RestoreSnapshotProcedure.java:58)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:180)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1455)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1224)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:751)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:675)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreRegion(RestoreSnapshotHelper.java:521)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreRegion(RestoreSnapshotHelper.java:464)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$300(RestoreSnapshotHelper.java:116)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.editRegion(RestoreSnapshotHelper.java:427)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$2.call(ModifyRegionUtils.java:209)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$2.call(ModifyRegionUtils.java:206)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366058#comment-16366058 ] Sean Busbey commented on HBASE-20004: - looks like we've been stoping TRACE and OPTIONS since HBASE-10473 > Client is not able to execute REST queries through browser in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Critical > Attachments: HBASE-20004.branch-1.patch, HBASE-20004.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366051#comment-16366051 ] Sean Busbey commented on HBASE-20004: - I'm -1 as-is without some analysis on why we're restricting OPTIONS on HTTP requests in the first place and what the impact is of no longer restricting it. > Client is not able to execute REST queries through browser in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Critical > Attachments: HBASE-20004.branch-1.patch, HBASE-20004.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19506) Support variable sized chunks from ChunkCreator
[
https://issues.apache.org/jira/browse/HBASE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366045#comment-16366045
]
Hadoop QA commented on HBASE-19506:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
44s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} hbase-server: The patch generated 0 new + 29
unchanged - 3 fixed = 29 total (was 32) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 16s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}135m 51s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}174m 30s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.cleaner.TestSnapshotFromMaster |
| | hadoop.hbase.client.TestSnapshotFromClientWithRegionReplicas |
| | hadoop.hbase.client.TestSnapshotMetadata |
| | hadoop.hbase.regionserver.TestCompactSplitThread |
| | hadoop.hbase.client.TestSizeFailures |
| | hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd |
| | hadoop.hbase.client.TestSnapshotCloneIndependence |
| | hadoop.hbase.snapshot.TestRegionSnapshotTask |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19506 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12910755/HBASE-19506-V07.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 561f084617d1 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 50c705dad9 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366044#comment-16366044
]
Sean Busbey commented on HBASE-19920:
-
+1 pending QA feedback.
{code}
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/security/token/TokenUtil.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/security/token/TokenUtil.java
...
+import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
+
{code}
nit: no longer used. fine to fix this on commit.
> TokenUtil.obtainToken unnecessarily creates a local directory
> -
>
> Key: HBASE-19920
> URL: https://issues.apache.org/jira/browse/HBASE-19920
> Project: HBase
> Issue Type: Bug
>Reporter: Rohini Palaniswamy
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0
>
> Attachments: HBASE-19920.patch, HBASE-19920.v2.patch,
> HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch,
> HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch,
> HBASE-19920.v9.patch
>
>
> On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil
> which in its static block initializes DynamicClassLoader and that creates the
> directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class
> to access hbase.dynamic.jars.dir.
> https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127
> Since this is region server specific code, not expecting this to happen when
> one accesses hbase as a client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366019#comment-16366019
]
Mike Drob commented on HBASE-19920:
---
v9 to address Sean's comments
> TokenUtil.obtainToken unnecessarily creates a local directory
> -
>
> Key: HBASE-19920
> URL: https://issues.apache.org/jira/browse/HBASE-19920
> Project: HBase
> Issue Type: Bug
>Reporter: Rohini Palaniswamy
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0
>
> Attachments: HBASE-19920.patch, HBASE-19920.v2.patch,
> HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch,
> HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch,
> HBASE-19920.v9.patch
>
>
> On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil
> which in its static block initializes DynamicClassLoader and that creates the
> directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class
> to access hbase.dynamic.jars.dir.
> https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127
> Since this is region server specific code, not expecting this to happen when
> one accesses hbase as a client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob updated HBASE-19920:
--
Attachment: HBASE-19920.v9.patch
> TokenUtil.obtainToken unnecessarily creates a local directory
> -
>
> Key: HBASE-19920
> URL: https://issues.apache.org/jira/browse/HBASE-19920
> Project: HBase
> Issue Type: Bug
>Reporter: Rohini Palaniswamy
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0
>
> Attachments: HBASE-19920.patch, HBASE-19920.v2.patch,
> HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch,
> HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch,
> HBASE-19920.v9.patch
>
>
> On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil
> which in its static block initializes DynamicClassLoader and that creates the
> directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class
> to access hbase.dynamic.jars.dir.
> https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127
> Since this is region server specific code, not expecting this to happen when
> one accesses hbase as a client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19680) BufferedMutatorImpl#mutate should wait the result from AP in order to throw the failed mutations
[ https://issues.apache.org/jira/browse/HBASE-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365992#comment-16365992 ] stack commented on HBASE-19680: --- What were you thinking [~chia7712] perf-wise? Maybe I could help sir. > BufferedMutatorImpl#mutate should wait the result from AP in order to throw > the failed mutations > > > Key: HBASE-19680 > URL: https://issues.apache.org/jira/browse/HBASE-19680 > Project: HBase > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19680.v0.patch, HBASE-19680.v1.patch > > > Currently, BMI#mutate doesn't wait the result from AP so the errors are > stored in AP. The only way which can return the errors to user is, calling > the flush to catch the exception. That is non-intuitive. > I feel BMI#mutate should wait the result. That is to say, user can parse the > exception thrown by BM#mutate to get the failed mutations. Also, we can > remove the global error from AP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365980#comment-16365980
]
Mike Drob commented on HBASE-19920:
---
bq. my intuition is telling me that the two ProtobufUtils should be sharing a
common place for it though
Probably. There's a TODO at the top of one to generate it from the other, I'd
rather leave it until then.
bq. Is there a reason we can't just hide the holder behind something like
DynamicClassloader.getInstance()?
Not all DCLs are created the same. There's one in RemoteWithExtrasException too
that I will update to use this same lazy pattern though.
{quote}
So long as you're changing this up to do lazy init, should we also clean up how
we init?
HBC.create can throw (like if the xml config version is off from library). do
we want that to propagate or should we do a try/catch to set CLASS_LOADER to
null? AFAICT the use of CLASS_LOADER would then default to the bootstrap
classloader, which is probably correct.
Also HBC.create expressly sets the classloader for the Configuration to be
whatever classloader was used to make the HBaseConfiguration instance. Should
it be created with parent?
{quote}
Probably better to do in a follow on?
bq. This feels wrong. I think this variable needs to be volatile or
AtomicBoolean? It doesn't have the same guarantees about visibility since it's
not within the holder, right?
It works currently because it's only accessed by a single thread, but can make
it volatile for future proofing, sure.
bq. Given that, why not also make the fault injection private and/or final and
then use reflection in the test to allow changing it in the test?
private, sure, we can call setAccessible via the reflection. final, i'd rather
not have to deal with manually removing flags though and I don't see much
benefit.
bq. Should specifically check that it's the injected ServiceException. I guess
by text of the message?
can make this explicit setting exception instead of boolean
> TokenUtil.obtainToken unnecessarily creates a local directory
> -
>
> Key: HBASE-19920
> URL: https://issues.apache.org/jira/browse/HBASE-19920
> Project: HBase
> Issue Type: Bug
>Reporter: Rohini Palaniswamy
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0
>
> Attachments: HBASE-19920.patch, HBASE-19920.v2.patch,
> HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch,
> HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch
>
>
> On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil
> which in its static block initializes DynamicClassLoader and that creates the
> directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class
> to access hbase.dynamic.jars.dir.
> https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127
> Since this is region server specific code, not expecting this to happen when
> one accesses hbase as a client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365978#comment-16365978 ] Ted Yu commented on HBASE-20004: +1 Build error was from hbase-thrift module which is unrelated. > Client is not able to execute REST queries through browser in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Critical > Attachments: HBASE-20004.branch-1.patch, HBASE-20004.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365937#comment-16365937
]
Hadoop QA commented on HBASE-20004:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
3s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
2s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
5s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
47s{color} | {color:green} hbase-http in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20004 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12910766/HBASE-20004.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fd34a68bdc28 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 01a2869363 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11542/testReport/ |
| Max. process+thread count | 292 (vs. ulimit of 1) |
| modules
[jira] [Commented] (HBASE-19529) Handle null states in AM
[ https://issues.apache.org/jira/browse/HBASE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365913#comment-16365913 ] Appy commented on HBASE-19529: -- makes sense. > Handle null states in AM > > > Key: HBASE-19529 > URL: https://issues.apache.org/jira/browse/HBASE-19529 > Project: HBase > Issue Type: Bug >Reporter: Appy >Priority: Critical > Fix For: 2.0.0 > > > From debugging in HBASE-19457, found some questions that need concrete > answers: > 1) What does a region state of null in meta means? Currently AM treats it as > OFFLINE > 2) What does a table state of null in meta means? Currently TSM treats it as > ENABLED. > More importantly, we need to fix holes in AM so that our state machine is > well defined and doesn't end up in these uncertainties. > Figuring out answers to above questions will help in that direction. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365888#comment-16365888 ] Ashish Singhi commented on HBASE-20004: --- I hope these two patches should apply to all branches. I will be on vacation from tomorrow for next 5 days. > Client is not able to execute REST queries through browser in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Critical > Attachments: HBASE-20004.branch-1.patch, HBASE-20004.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19903) Split TestShell so it will not time out
[ https://issues.apache.org/jira/browse/HBASE-19903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365886#comment-16365886 ] stack commented on HBASE-19903: --- Happy New Year [~Apache9] > Split TestShell so it will not time out > --- > > Key: HBASE-19903 > URL: https://issues.apache.org/jira/browse/HBASE-19903 > Project: HBase > Issue Type: Sub-task > Components: shell, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19903.patch > > > It runs more than 10 minutes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-20004: -- Attachment: HBASE-20004.branch-1.patch > Client is not able to execute REST queries through browser in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Critical > Attachments: HBASE-20004.branch-1.patch, HBASE-20004.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365863#comment-16365863 ] Ashish Singhi commented on HBASE-20004: --- The patch here fixes the problem for us. Please review. > Client is not able to execute REST queries through browser in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Critical > Attachments: HBASE-20004.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-20004: -- Status: Patch Available (was: Open) > Client is not able to execute REST queries through browser in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Critical > Attachments: HBASE-20004.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-20004: -- Attachment: HBASE-20004.patch > Client is not able to execute REST queries through browser in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Critical > Attachments: HBASE-20004.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365845#comment-16365845 ] Ashish Singhi commented on HBASE-20004: --- I don't have exact details at the moment with me. But the problem here was the browser client negotiate with server by first sending a HTTP request with method type as OPTIONS and this method we have added as a constraint in the security handler. > Client is not able to execute REST queries through browser in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Critical > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19903) Split TestShell so it will not time out
[ https://issues.apache.org/jira/browse/HBASE-19903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19903: -- Component/s: test shell > Split TestShell so it will not time out > --- > > Key: HBASE-19903 > URL: https://issues.apache.org/jira/browse/HBASE-19903 > Project: HBase > Issue Type: Sub-task > Components: shell, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19903.patch > > > It runs more than 10 minutes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19903) Split TestShell so it will not time out
[ https://issues.apache.org/jira/browse/HBASE-19903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19903: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to master and branch-2. Thanks [~stack] for reviewing. Going to sleep now, it is already the new year in China :) > Split TestShell so it will not time out > --- > > Key: HBASE-19903 > URL: https://issues.apache.org/jira/browse/HBASE-19903 > Project: HBase > Issue Type: Sub-task > Components: shell, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19903.patch > > > It runs more than 10 minutes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365830#comment-16365830
]
Sean Busbey commented on HBASE-19920:
-
I like the move to the singleton holder idiom. I don't have a specific reason,
but my intuition is telling me that the two ProtobufUtils should be sharing a
common place for it though. Is there a reason we can't just hide the holder
behind something like *DynamicClassloader.getInstance()*?
{code}
/**
- * Dynamic class loader to load filter/comparators
- */
- private final static ClassLoader CLASS_LOADER;
+ * Dynamic class loader to load filter/comparators
+ */
+ private final static class ClassLoaderHolder {
+ private final static ClassLoader CLASS_LOADER;
- static {
- ClassLoader parent = ProtobufUtil.class.getClassLoader();
- Configuration conf = HBaseConfiguration.create();
- CLASS_LOADER = new DynamicClassLoader(conf, parent);
+ static {
+ ClassLoader parent = ProtobufUtil.class.getClassLoader();
+ Configuration conf = HBaseConfiguration.create();
+ CLASS_LOADER = new DynamicClassLoader(conf, parent);
+ }
}
{code}
So long as you're changing this up to do lazy init, should we also clean up how
we init?
HBC.create can throw (like if the xml config version is off from library). do
we want that to propagate or should we do a try/catch to set CLASS_LOADER to
null? AFAICT the use of CLASS_LOADER would then default to the bootstrap
classloader, which is probably correct.
Also HBC.create expressly sets the classloader for the Configuration to be
whatever classloader was used to make the HBaseConfiguration instance. Should
it be created with parent?
{code}
+ private static boolean classLoaderLoaded = false;
+
{code}
This feels wrong. I think this variable needs to be volatile or AtomicBoolean?
It doesn't have the same guarantees about visibility since it's not within the
holder, right?
{code}
+ClassLoader cl = new URLClassLoader(new URL[] { urlPU, urlTU },
getClass().getClassLoader());
+
+Class tokenUtil = cl.loadClass(TokenUtil.class.getCanonicalName());
+tokenUtil.getDeclaredField("shouldInjectFault").setBoolean(null, true);
{code}
We have to go through all this reflection so that the TokenUtil class
specifically won't be in the classloader of the test in case some other test
needed to do something that caused the dynamic classloader to get loaded, right?
Given that, why not also make the fault injection private and/or final and then
use reflection in the test to allow changing it in the test?
{code}
+} catch (InvocationTargetException e) {
+ Throwable t = e;
+ boolean serviceExceptionFound = false;
+ while ((t = t.getCause()) != null) {
+if (t instanceof com.google.protobuf.ServiceException) {
+ serviceExceptionFound = true;
+ break;
+}
+ }
{code}
Should specifically check that it's the injected ServiceException. I guess by
text of the message?
> TokenUtil.obtainToken unnecessarily creates a local directory
> -
>
> Key: HBASE-19920
> URL: https://issues.apache.org/jira/browse/HBASE-19920
> Project: HBase
> Issue Type: Bug
>Reporter: Rohini Palaniswamy
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0
>
> Attachments: HBASE-19920.patch, HBASE-19920.v2.patch,
> HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch,
> HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch
>
>
> On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil
> which in its static block initializes DynamicClassLoader and that creates the
> directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class
> to access hbase.dynamic.jars.dir.
> https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127
> Since this is region server specific code, not expecting this to happen when
> one accesses hbase as a client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20002) Reserve worker thread in ProcedureExecutor for high priority procedures
[ https://issues.apache.org/jira/browse/HBASE-20002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365816#comment-16365816 ] Duo Zhang commented on HBASE-20002: --- Review board link: https://reviews.apache.org/r/65675/ > Reserve worker thread in ProcedureExecutor for high priority procedures > --- > > Key: HBASE-20002 > URL: https://issues.apache.org/jira/browse/HBASE-20002 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-20002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20002) Reserve worker thread in ProcedureExecutor for high priority procedures
[ https://issues.apache.org/jira/browse/HBASE-20002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20002: -- Attachment: HBASE-20002.patch > Reserve worker thread in ProcedureExecutor for high priority procedures > --- > > Key: HBASE-20002 > URL: https://issues.apache.org/jira/browse/HBASE-20002 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-20002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20002) Reserve worker thread in ProcedureExecutor for high priority procedures
[ https://issues.apache.org/jira/browse/HBASE-20002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20002: -- Status: Patch Available (was: Open) > Reserve worker thread in ProcedureExecutor for high priority procedures > --- > > Key: HBASE-20002 > URL: https://issues.apache.org/jira/browse/HBASE-20002 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-20002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365766#comment-16365766
]
Sean Busbey commented on HBASE-19920:
-
oh it grabbed the first patch instead of v8. let me figure out if I need to
upgrade or manually specify the version or what.
> TokenUtil.obtainToken unnecessarily creates a local directory
> -
>
> Key: HBASE-19920
> URL: https://issues.apache.org/jira/browse/HBASE-19920
> Project: HBase
> Issue Type: Bug
>Reporter: Rohini Palaniswamy
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0
>
> Attachments: HBASE-19920.patch, HBASE-19920.v2.patch,
> HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch,
> HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch
>
>
> On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil
> which in its static block initializes DynamicClassLoader and that creates the
> directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class
> to access hbase.dynamic.jars.dir.
> https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127
> Since this is region server specific code, not expecting this to happen when
> one accesses hbase as a client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.01.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.02.patch, HBASE-18294.03.patch, > HBASE-18294.04.patch, HBASE-18294.05.patch, HBASE-18294.06.patch, > HBASE-18294.07.patch, HBASE-18294.07.patch, HBASE-18294.08.patch, > HBASE-18294.09.patch, HBASE-18294.10.patch, HBASE-18294.11.patch, > HBASE-18294.11.patch, HBASE-18294.12.patch, HBASE-18294.13.patch, > HBASE-18294.15.patch, HBASE-18294.16.patch, HBASE-18294.master.01.patch, > HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365720#comment-16365720
]
Sean Busbey commented on HBASE-19920:
-
hurm. lemme see what smart-apply-patch did.
> TokenUtil.obtainToken unnecessarily creates a local directory
> -
>
> Key: HBASE-19920
> URL: https://issues.apache.org/jira/browse/HBASE-19920
> Project: HBase
> Issue Type: Bug
>Reporter: Rohini Palaniswamy
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0
>
> Attachments: HBASE-19920.patch, HBASE-19920.v2.patch,
> HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch,
> HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch
>
>
> On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil
> which in its static block initializes DynamicClassLoader and that creates the
> directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class
> to access hbase.dynamic.jars.dir.
> https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127
> Since this is region server specific code, not expecting this to happen when
> one accesses hbase as a client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20003) WALLess HBase on Persistent Memory
[ https://issues.apache.org/jira/browse/HBASE-20003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365708#comment-16365708 ] Anoop Sam John commented on HBASE-20003: bq.Possibility of bigger memstores There were some concerns of having this.. But with off heap memstore and the Compacting Memstore we can overcome those. > WALLess HBase on Persistent Memory > -- > > Key: HBASE-20003 > URL: https://issues.apache.org/jira/browse/HBASE-20003 > Project: HBase > Issue Type: New Feature >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Major > > This JIRA aims to make use of persistent memory (pmem) technologies in HBase. > One such usage is to make the Memstore to reside on pmem. Making a persistent > memstore would remove the need for WAL and paves way for a WALLess HBase. > The existing region replica feature could be used here and ensure the data > written to memstores are synchronously replicated to the replicas and ensure > strong consistency of the data. (pipeline model) > Advantages : > -Data Availability : Since the data across replicas are consistent > (synchronously written) our data is always 100 % available. > -Lower MTTR : It becomes easier/faster to switch over to the replicas on a > primary region failure as there is no WAL replay involved. Building the > memstore map data also is much faster than reading the WAL and replaying the > WAL. > -Possibility of bigger memstores : These pmems are designed to have more > memory than DRAMs so it would also enable us to have bigger sized memstores > which leads to lesser flushes/compaction IO. > -Removes the dependency of HDFS on the write path > Initial PoC has been designed and developed. Testing is underway and we would > publish the PoC results along with the design doc sooner. The PoC doc will > talk about the design decisions, the libraries considered to work with these > pmem devices, pros and cons of those libraries and the performance results. > Note : Next gen memory technologies using 3DXPoint gives persistent memory > feature. Such memory DIMMs are soon to appear in the market. The PoC is done > around Intel's ApachePass (AEP) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19506) Support variable sized chunks from ChunkCreator
[ https://issues.apache.org/jira/browse/HBASE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gali Sheffi updated HBASE-19506: Attachment: HBASE-19506-V07.patch > Support variable sized chunks from ChunkCreator > --- > > Key: HBASE-19506 > URL: https://issues.apache.org/jira/browse/HBASE-19506 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Priority: Major > Attachments: HBASE-19506-V01.patch, HBASE-19506-V02.patch, > HBASE-19506-V03.patch, HBASE-19506-V04.patch, HBASE-19506-V05.patch, > HBASE-19506-V06.patch, HBASE-19506-V07.patch > > > When CellChunkMap is created it allocates a special index chunk (or chunks) > where array of cell-representations is stored. When the number of > cell-representations is small, it is preferable to allocate a chunk smaller > than a default value which is 2MB. > On the other hand, those "non-standard size" chunks can not be used in pool. > On-demand allocations in off-heap are costly. So this JIRA is about to > investigate the trade of between memory usage and the final performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20003) WALLess HBase on Persistent Memory
[ https://issues.apache.org/jira/browse/HBASE-20003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365705#comment-16365705 ] Anoop Sam John commented on HBASE-20003: Sure. The doc will be coming in soon. The one we have needs some cleanup.. Stuck with some tests these days. > WALLess HBase on Persistent Memory > -- > > Key: HBASE-20003 > URL: https://issues.apache.org/jira/browse/HBASE-20003 > Project: HBase > Issue Type: New Feature >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Major > > This JIRA aims to make use of persistent memory (pmem) technologies in HBase. > One such usage is to make the Memstore to reside on pmem. Making a persistent > memstore would remove the need for WAL and paves way for a WALLess HBase. > The existing region replica feature could be used here and ensure the data > written to memstores are synchronously replicated to the replicas and ensure > strong consistency of the data. (pipeline model) > Advantages : > -Data Availability : Since the data across replicas are consistent > (synchronously written) our data is always 100 % available. > -Lower MTTR : It becomes easier/faster to switch over to the replicas on a > primary region failure as there is no WAL replay involved. Building the > memstore map data also is much faster than reading the WAL and replaying the > WAL. > -Possibility of bigger memstores : These pmems are designed to have more > memory than DRAMs so it would also enable us to have bigger sized memstores > which leads to lesser flushes/compaction IO. > -Removes the dependency of HDFS on the write path > Initial PoC has been designed and developed. Testing is underway and we would > publish the PoC results along with the design doc sooner. The PoC doc will > talk about the design decisions, the libraries considered to work with these > pmem devices, pros and cons of those libraries and the performance results. > Note : Next gen memory technologies using 3DXPoint gives persistent memory > feature. Such memory DIMMs are soon to appear in the market. The PoC is done > around Intel's ApachePass (AEP) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365692#comment-16365692
]
Mike Drob commented on HBASE-19920:
---
You were looking at the wrong patch, boss. {{ProtobufExceptionUtil}} has been
dropped.
> TokenUtil.obtainToken unnecessarily creates a local directory
> -
>
> Key: HBASE-19920
> URL: https://issues.apache.org/jira/browse/HBASE-19920
> Project: HBase
> Issue Type: Bug
>Reporter: Rohini Palaniswamy
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0
>
> Attachments: HBASE-19920.patch, HBASE-19920.v2.patch,
> HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch,
> HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch
>
>
> On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil
> which in its static block initializes DynamicClassLoader and that creates the
> directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class
> to access hbase.dynamic.jars.dir.
> https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127
> Since this is region server specific code, not expecting this to happen when
> one accesses hbase as a client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19903) Split TestShell so it will not time out
[ https://issues.apache.org/jira/browse/HBASE-19903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365673#comment-16365673 ] stack commented on HBASE-19903: --- +1 > Split TestShell so it will not time out > --- > > Key: HBASE-19903 > URL: https://issues.apache.org/jira/browse/HBASE-19903 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19903.patch > > > It runs more than 10 minutes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18882) [TEST] Run MR branch-1 jobs against hbase2 cluster
[ https://issues.apache.org/jira/browse/HBASE-18882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18882: -- Summary: [TEST] Run MR branch-1 jobs against hbase2 cluster (was: Run MR branch-1 jobs against hbase2 cluster) > [TEST] Run MR branch-1 jobs against hbase2 cluster > -- > > Key: HBASE-18882 > URL: https://issues.apache.org/jira/browse/HBASE-18882 > Project: HBase > Issue Type: Task > Components: mapreduce, test >Reporter: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > > Ensure this works still. Run all our bundled MR tools at least. Find some > custom if time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18758) [TEST][compat 1-2] Test delegation tokens continue to work when hbase1 going against hbase2 cluster
[ https://issues.apache.org/jira/browse/HBASE-18758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18758: -- Summary: [TEST][compat 1-2] Test delegation tokens continue to work when hbase1 going against hbase2 cluster (was: [compat 1-2] Test delegation tokens continue to work when hbase1 going against hbase2 cluster) > [TEST][compat 1-2] Test delegation tokens continue to work when hbase1 going > against hbase2 cluster > --- > > Key: HBASE-18758 > URL: https://issues.apache.org/jira/browse/HBASE-18758 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Priority: Critical > Fix For: 2.0.0-beta-2 > > > Suggested by [~toffer] Need to test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18596) [TEST] A hbase1 cluster should be able to replicate to a hbase2 cluster; verify
[ https://issues.apache.org/jira/browse/HBASE-18596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18596: -- Summary: [TEST] A hbase1 cluster should be able to replicate to a hbase2 cluster; verify (was: A hbase1 cluster should be able to replicate to a hbase2 cluster; verify) > [TEST] A hbase1 cluster should be able to replicate to a hbase2 cluster; > verify > --- > > Key: HBASE-18596 > URL: https://issues.apache.org/jira/browse/HBASE-18596 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: Esteban Gutierrez >Priority: Blocker > Fix For: 2.0.0-beta-2 > > > From the mailing list thread "[DISCUSS] hbase-2.0.0 compatibility > expectations", [~esteban] asks: > bq. Should we add additional details around replication as well? for > instance, shall we consider a hbase-1.x cluster as a client for a hbase-2.x > cluster? > The latter should be a blocker. Verify it works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17730) [DOC] Migration to 2.0 for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17730: -- Summary: [DOC] Migration to 2.0 for coprocessors (was: Migration to 2.0 for coprocessors ) > [DOC] Migration to 2.0 for coprocessors > > > Key: HBASE-17730 > URL: https://issues.apache.org/jira/browse/HBASE-17730 > Project: HBase > Issue Type: Sub-task > Components: documentation, migration >Reporter: Appy >Priority: Blocker > Fix For: 2.0.0-beta-2 > > > Jiras breaking coprocessor compatibility should be marked with component ' > Coprocessor', and label 'incompatible'. > Close to releasing 2.0, we should go through all such jiras and write down > steps for migrating coprocessor easily. > The idea is, it might be very hard to fix coprocessor breakages by reverse > engineering errors, but will be easier we suggest easiest way to fix > breakages resulting from each individual incompatible change. > For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors > because BaseXXXObserver classes are gone and will probably result in a lot of > confusion, but if we explicitly mention the fix which is just one line change > - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - > it makes it very easy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-13428) [DOC] Migration to hbase-2.0.0
[ https://issues.apache.org/jira/browse/HBASE-13428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13428: -- Summary: [DOC] Migration to hbase-2.0.0 (was: Migration to hbase-2.0.0) > [DOC] Migration to hbase-2.0.0 > -- > > Key: HBASE-13428 > URL: https://issues.apache.org/jira/browse/HBASE-13428 > Project: HBase > Issue Type: Umbrella > Components: migration >Reporter: stack >Priority: Blocker > Fix For: 2.0.0-beta-2 > > > Opening a 2.0 umbrella migration issue. Lets hang off this one any tools and > expectations migrating from 1.0 (or earlier) to 2.0. So far there are none > that I know of though there is an expectation in HBASE-13373 that hfiles are > at least major version 2 and minor version 3. Lets list all such > expectations, etc., here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19529) Handle null states in AM
[ https://issues.apache.org/jira/browse/HBASE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365641#comment-16365641 ] stack commented on HBASE-19529: --- Moved it out of beta-2. I think this is an ongoing project. Still Critical but not something to block beta-2. Change it if you think otherwise [~appy] > Handle null states in AM > > > Key: HBASE-19529 > URL: https://issues.apache.org/jira/browse/HBASE-19529 > Project: HBase > Issue Type: Bug >Reporter: Appy >Priority: Critical > Fix For: 2.0.0 > > > From debugging in HBASE-19457, found some questions that need concrete > answers: > 1) What does a region state of null in meta means? Currently AM treats it as > OFFLINE > 2) What does a table state of null in meta means? Currently TSM treats it as > ENABLED. > More importantly, we need to fix holes in AM so that our state machine is > well defined and doesn't end up in these uncertainties. > Figuring out answers to above questions will help in that direction. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19529) Handle null states in AM
[ https://issues.apache.org/jira/browse/HBASE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19529: -- Fix Version/s: (was: 2.0.0-beta-2) 2.0.0 > Handle null states in AM > > > Key: HBASE-19529 > URL: https://issues.apache.org/jira/browse/HBASE-19529 > Project: HBase > Issue Type: Bug >Reporter: Appy >Priority: Critical > Fix For: 2.0.0 > > > From debugging in HBASE-19457, found some questions that need concrete > answers: > 1) What does a region state of null in meta means? Currently AM treats it as > OFFLINE > 2) What does a table state of null in meta means? Currently TSM treats it as > ENABLED. > More importantly, we need to fix holes in AM so that our state machine is > well defined and doesn't end up in these uncertainties. > Figuring out answers to above questions will help in that direction. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16550) Procedure v2 - Add AM compatibility for 2.x Master and 1.x RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
[ https://issues.apache.org/jira/browse/HBASE-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16550: -- Fix Version/s: (was: 2.0.0-beta-2) > Procedure v2 - Add AM compatibility for 2.x Master and 1.x RSs; i.e. support > Rolling Upgrade from hbase-1 to -2. > > > Key: HBASE-16550 > URL: https://issues.apache.org/jira/browse/HBASE-16550 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Priority: Blocker > > Core AM HBASE-14614 relies on the RS to be using zkless assignment. Add > support for the old a plain non zkless AM -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-16550) Procedure v2 - Add AM compatibility for 2.x Master and 1.x RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
[ https://issues.apache.org/jira/browse/HBASE-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365628#comment-16365628 ] stack commented on HBASE-16550: --- Big job mirroring Region State Changes to zk (See Release Note on HBASE-14614). A flag to enable/disable (same flag could turn off zk mirroring of table state HBASE-16060). > Procedure v2 - Add AM compatibility for 2.x Master and 1.x RSs; i.e. support > Rolling Upgrade from hbase-1 to -2. > > > Key: HBASE-16550 > URL: https://issues.apache.org/jira/browse/HBASE-16550 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Priority: Blocker > > Core AM HBASE-14614 relies on the RS to be using zkless assignment. Add > support for the old a plain non zkless AM -- This message was sent by Atlassian JIRA (v7.6.3#76005)
