[jira] [Created] (HBASE-20589) Don't need to assign meta to a new RS when standby master become active
Guanghao Zhang created HBASE-20589: -- Summary: Don't need to assign meta to a new RS when standby master become active Key: HBASE-20589 URL: https://issues.apache.org/jira/browse/HBASE-20589 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang I found this problem when I write ut for HBASE-20569. Now the master finishActiveMasterInitialization introduce a new RecoverMetaProcedure(HBASE-18261) and it has a sub procedure AssignProcedure. AssignProcedure will skip assign a region when regions state is OPEN and server is online. But for the new regiog state node is created with state OFFLINE. So it will assign the meta to a new RS. And kill the old RS when old RS report to master. This will make the master initialization cost a long time. I will attatch a ut to show this. FYI [~stack] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20584) TestRestoreSnapshotFromClient failed
[
https://issues.apache.org/jira/browse/HBASE-20584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingyun Tian resolved HBASE-20584.
--
Resolution: Duplicate
> TestRestoreSnapshotFromClient failed
>
>
> Key: HBASE-20584
> URL: https://issues.apache.org/jira/browse/HBASE-20584
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
>
> org.apache.hadoop.hbase.snapshot.HBaseSnapshotException:
> org.apache.hadoop.hbase.snapshot.HBaseSnapshotException: Snapshot {
> ss=snaptb1-1526376636687
> table=testRestoreSnapshotAfterSplittingRegions-1526376636687 type=FLUSH } had
> an error. Procedure snaptb1-1526376636687 { waiting=[] done=[] }
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:380)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.isSnapshotDone(MasterRpcServices.java:1128)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException via
> Failed taking snapshot { ss=snaptb1-1526376636687
> table=testRestoreSnapshotAfterSplittingRegions-1526376636687 type=FLUSH } due
> to exception:Regions moved during the snapshot '{ ss=snaptb1-1526376636687
> table=testRestoreSnapshotAfterSplittingRegions-1526376636687 type=FLUSH }'.
> expected=6
> snapshotted=7.:org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException:
> Regions moved during the snapshot '{ ss=snaptb1-1526376636687
> table=testRestoreSnapshotAfterSplittingRegions-1526376636687 type=FLUSH }'.
> expected=6 snapshotted=7.
> at
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:82)
> at
> org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.rethrowExceptionIfFailed(TakeSnapshotHandler.java:311)
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:369)
> ... 6 more
> Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException:
> Regions moved during the snapshot '{ ss=snaptb1-1526376636687
> table=testRestoreSnapshotAfterSplittingRegions-1526376636687 type=FLUSH }'.
> expected=6 snapshotted=7.
> at
> org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifyRegions(MasterSnapshotVerifier.java:217)
> at
> org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifySnapshot(MasterSnapshotVerifier.java:121)
> at
> org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.process(TakeSnapshotHandler.java:207)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:100)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:90)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:360)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:348)
> at
> org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3061)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3053)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.snapshot(HBaseAdmin.java:2532)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.snapshot(HBaseAdmin.java:2499)
> at
> org.apache.hadoop.hb
Re: Lots of broken tests on flaky dashboard
Seems 2.0 is also effected? Mike Drob 于2018年5月16日 周三03:03写道： > I took a look at the flaky dashboard[1] this morning and noticed an > alarming number of always failed tests. > > Is the dashboard still accurate? I know we've moved our unit testing > infrastructure around a little bit and wasn't sure if the dashboard needed > updates as well. > > The tests are failing on both branch-2 and master. Are folks already > looking at these, and is there a JIRA to follow along for discussion? I did > a quick search but didn't see anything obvious. > > Thanks, > Mike > > > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html >
[jira] [Reopened] (HBASE-20564) Tighter ByteBufferKeyValue Cell Comparator
[ https://issues.apache.org/jira/browse/HBASE-20564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-20564: --- Reopening to fix. Broke stuff (found by [~yuzhih...@gmail.com]) > Tighter ByteBufferKeyValue Cell Comparator > -- > > Key: HBASE-20564 > URL: https://issues.apache.org/jira/browse/HBASE-20564 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: 1.4.pe.write.0510.96203.cpu.svg, > 2.p3.write2.0514.104236.cpu.svg, 2.pe.write.135142.cpu.svg, 20564.addendum, > HBASE-20564.branch-2.0.001.patch, HBASE-20564.branch-2.0.002.patch, > HBASE-20564.branch-2.patch, hits.png > > > Comparing Cells in hbase2 takes almost 3x the CPU. > In hbase1, its a keyValue backed by a byte array caching a few important > values.. In hbase2, its a NoTagByteBufferChunkKeyValue(?) deserializing the > row/family/qualifier lengths repeatedly. > I tried making a purposed comparator -- one that was not generic -- and it > seemed to have a nicer profile coming close to hbase1 in percentage used > (I'll post graphs) when I ran it in my perpetual memstore filler (See scripts > attached to HBASE-20483). It doesn't work when I try to run it on cluster. > Let me run unit tests to see if it can figure what I have wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20588) Space quota change after quota violation doesn't seem to take in effect
Biju Nair created HBASE-20588:
-
Summary: Space quota change after quota violation doesn't seem to
take in effect
Key: HBASE-20588
URL: https://issues.apache.org/jira/browse/HBASE-20588
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0
Reporter: Biju Nair
Steps followed
- Through h{{base shell}}
{noformat}
set_quota TYPE => SPACE, TABLE => 'TestTable', LIMIT => '2M', POLICY =>
NO_INSERTS{noformat}
- Run {{PE}} until the quota is reached
{noformat}
hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --rows=2000
sequentialWrite 1{noformat}
- Through {{HBase}} shell
{noformat}
set_quota TYPE => SPACE, TABLE => 'TestTable', LIMIT => NONE{noformat}
- Through {{HBase}} shell verify the effective Quotas
{noformat}
> list_quotas
OWNER QUOTAS
0 row(s)
Took 0.0365 seconds{noformat}
- Wait for some time (at least 5 mins) and try to add data to the table
{noformat}
> put 'TestTable','r1','info:0','v1'
ERROR: org.apache.hadoop.hbase.quotas.SpaceLimitingException: NO_INSERTS Puts
are disallowed due to a space quota.
at
org.apache.hadoop.hbase.quotas.policies.NoInsertsViolationPolicyEnforcement.check(NoInsertsViolationPolicyEnforcement.java:47){noformat}
To resolve the issue, {{RSes}} need to be restarted which points to in memory
data not getting reset.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Lots of broken tests on flaky dashboard
I took a look at the flaky dashboard[1] this morning and noticed an alarming number of always failed tests. Is the dashboard still accurate? I know we've moved our unit testing infrastructure around a little bit and wasn't sure if the dashboard needed updates as well. The tests are failing on both branch-2 and master. Are folks already looking at these, and is there a JIRA to follow along for discussion? I did a quick search but didn't see anything obvious. Thanks, Mike https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html
[jira] [Created] (HBASE-20587) Remove client-side Jackson dependency
Josh Elser created HBASE-20587: -- Summary: Remove client-side Jackson dependency Key: HBASE-20587 URL: https://issues.apache.org/jira/browse/HBASE-20587 Project: HBase Issue Type: Bug Components: dependencies Reporter: Josh Elser Assignee: Josh Elser HBASE-20582 got me looking at how we use Jackson. It appears that we moved some JSON code from hbase-server into hbase-common via HBASE-19053. But, there seems to be no good reason why this code should live there and not in hbase-http instead. Keeping Jackson off the user's classpath is a nice goal. FYI [~appy], [~mdrob] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: WAL storage policies and interactions with Hadoop admin tools.
Thanks Sean for the insight & detailed analysis! I think it makes sense to revert HBASE-18118. It's not as trivial to maintain backward compatibility. Kepping "NONE" as the default hsm doesn't harm. Having additional documentation is helpful to avoid confusion (since "NONE" is not a supported HDFS HSM option) On Mon, May 14, 2018 at 9:38 PM Yu Li wrote: > Thanks for pointing this out Sean. IMHO, after re-checking the codes, > HBASE-18118 needs an addendum (at least). The proposal was to set the > storage policy of WAL directory to HOT by default, but the current > implementation could not achieve this: it follows the old "NONE" logic to > escape calling the API if policy matches default, but for "HOT" we need an > explicit call to HDFS. > > Further more, I think the old logic to leave default to "NONE" is even > better: if admin set hbase.root.dir to some policy like ALL_SSD the WAL > will simply follow, and if not the policy is HOT by default > So maybe reverting HBASE-18118 is a better choice although I could see my > own +1 on HBASE-18118 there?... @Andrew what's your opinion here? > > And btw, I have opened HBASE-20479 for documenting the whole HSM solution > in hbase including HFile/WAL/Bulkload etc. (but still haven't got enough > time to complete it) JFYI. > > > Best Regards, > Yu > > On 15 May 2018 at 05:14, Sean Busbey wrote: > >> Hi folks! >> >> I'm trying to reason through our "set a storage policy for WALs" >> feature and having some difficulty. I want to get some feedback before >> I fix our docs or submit a patch to change behavior. >> >> Here's the history of the feature as I understand it: >> >> 1) Starting in HBase 1.1 you can change the setting >> "hbase.wal.storage.policy" and if the underlying Hadoop installation >> supports storage policies[1] then we'll call the needed APIs to set >> policies as we create WALs. >> >> The main use case is to tell HDFS that you want the HBase WAL on SSDs >> in a mixed hardware deployment. >> >> 2) In HBase 1.1 - 1.4, the above setting defaulted to the value >> "NONE". Our utility code for setting storage policies expressly checks >> any config value against the default and when it matches opts to log a >> message rather than call the actual Hadoop API[2]. This is important >> since "NONE" isn't actually a valid storage policy, so if we pass it >> to the Hadoop API we'll get a bunch of log noise. >> >> 3) In HBase 2 and 1.5+, the setting defaults to "HOT" as of >> HBASE-18118. Now if we were to pass the value to the Hadoop API we >> won't get log noise. The utility code does the same check against our >> default. The Hadoop default storage policy is "HOT" so presumably we >> save an RPC call by not setting it again. >> >> >> >> If the above is correct, how do I specify that I want WALs to have a >> storage policy of HOT in the event that HDFS already has some other >> policy in place for a parent directory? >> >> e.g. In HBase 1.1 - 1.4, I can set the storage policy (via Hadoop >> admin tools) for "/hbase" to be COLD and I can change >> "hbase.wal.storage.policy" to HOT. In HBase 2 and 1.5+, AFAICT my WALs >> will still have the COLD policy. >> >> Related, but different problem: I can use Hadoop admin tools to set >> the storage policy for "/hbase" to be "ALL_SSD" and if I leave HBase >> configs on defaults then I end up with WALs having "ALL_SSD" as their >> policy in all versions. But in HBase 2 and 1.5+ the HBase configs >> claim the policy is HOT. >> >> Should we always set the policy if the api is available? To avoid >> having to double-configure in something like the second case, do we >> still need a way to say "please do not expressly set a storage >> policy"? (as an alternative we could just call out "be sure to update >> your WAL config" in docs) >> >> >> >> [1]: "Storage Policy" gets called several things in Hadoop, like >> Archival Storage, Heterogenous Storage, HSM, and "Hierarchical >> Storage". In all cases I'm talking about the feature documented here: >> >> >> http://hadoop.apache.org/docs/r2.7.5/hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html >> >> http://hadoop.apache.org/docs/r3.0.2/hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html >> >> I think it's available in Hadoop 2.6.0+, 3.0.0+. >> >> [2]: >> >> In rel/1.2.0 you can see the default check by tracing starting at FSHLog: >> >> https://s.apache.org/BqAk >> >> The constants referred to in that code are in HConstants: >> >> https://s.apache.org/OJyR >> >> And in FSUtils we exit the function early when the default matches >> what we pull out of configs: >> >> https://s.apache.org/A4GA >> >> In rel/2.0.0 the code works essentially the same but has moved around. >> The starting point is now AbstractFSWAL: >> >> https://s.apache.org/pp6T >> >> The constants now use HOT instead of NONE as a default: >> >> https://s.apache.org/7K2J >> >> and in CommonFSUtils we do the same early return: >> >> https://s.apache.org/fYKr >> > > -- A very happy Clouderan
[jira] [Created] (HBASE-20586) SyncTable tool: Add support for cross-realm remote clusters
Wellington Chevreuil created HBASE-20586:
Summary: SyncTable tool: Add support for cross-realm remote
clusters
Key: HBASE-20586
URL: https://issues.apache.org/jira/browse/HBASE-20586
Project: HBase
Issue Type: Improvement
Components: mapreduce
Reporter: Wellington Chevreuil
Assignee: Wellington Chevreuil
One possible scenario for HashTable/SyncTable is for synchronize different
clusters, for instance, when replication has been enabled but data existed
already, or due replication issues that may had caused long lags in the
replication.
For secured clusters under different kerberos realms (with cross-realm properly
set), though, current SyncTable version would fail to authenticate with the
remote cluster when trying to read HashTable outputs (when *sourcehashdir* is
remote) and also when trying to read table data on the remote cluster (when
*sourcezkcluster* is remote).
The hdfs error would look like this:
{noformat}
INFO mapreduce.Job: Task Id : attempt_1524358175778_105392_m_00_0, Status :
FAILED
Error: java.io.IOException: Failed on local exception: java.io.IOException:
org.apache.hadoop.security.AccessControlException: Client cannot authenticate
via:[TOKEN, KERBEROS]; Host Details : local host is: "local-host/1.1.1.1";
destination host is: "remote-nn":8020;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
at org.apache.hadoop.ipc.Client.call(Client.java:1506)
at org.apache.hadoop.ipc.Client.call(Client.java:1439)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
at com.sun.proxy.$Proxy13.getBlockLocations(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getBlockLocations(ClientNamenodeProtocolTranslatorPB.java:256)
...
at
org.apache.hadoop.hbase.mapreduce.HashTable$TableHash.readPropertiesFile(HashTable.java:144)
at
org.apache.hadoop.hbase.mapreduce.HashTable$TableHash.read(HashTable.java:105)
at
org.apache.hadoop.hbase.mapreduce.SyncTable$SyncMapper.setup(SyncTable.java:188)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
...
Caused by: java.io.IOException:
org.apache.hadoop.security.AccessControlException: Client cannot authenticate
via:[TOKEN, KERBEROS]{noformat}
The above can be sorted if the SyncTable job acquires a DT for the remote NN.
Once hdfs related authentication is done, it's also necessary to authenticate
against remote HBase, as the below error would arise:
{noformat}
INFO mapreduce.Job: Task Id : attempt_1524358175778_172414_m_00_0, Status :
FAILED
Error: org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the
location
at
org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:326)
...
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:867)
at
org.apache.hadoop.hbase.mapreduce.SyncTable$SyncMapper.syncRange(SyncTable.java:331)
...
Caused by: java.io.IOException: Could not set up IO Streams to
remote-rs-host/1.1.1.2:60020
at
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:786)
...
Caused by: java.lang.RuntimeException: SASL authentication failed. The most
likely cause is missing or invalid credentials. Consider 'kinit'.
...
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed
to find any Kerberos tgt)
...{noformat}
The above would need additional authentication logic against the remote hbase
cluster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Accidentally pushed HBASE-19064 as master
I also thought that we have disabled force push on master and other release branches but I tried and succeeded... There are 20+ commits and it is only a few seconds after the accidental push so I just used a force push to revert everything back... 2018-05-15 21:32 GMT+08:00 Sean Busbey : > In the future please don't do force pushes. I thought we had disabled > them for the master branch. I'll check up with Infra on this. > > Next time just push a revert. > > On Tue, May 15, 2018 at 8:11 AM, 张铎(Duo Zhang) > wrote: > > Thanks. > > > > I pushed HBASE-20457 to master and then the Github repo is back to > normal... > > > > 2018-05-15 19:10 GMT+08:00 Tim Robertson : > > > >> Perhaps someone can verify this first: > >> > >> // assuming you have ASF upstream of GH (git remote add upstream > > >> git fetch upstream > >> git checkout master > >> git reset --hard upstream/master > >> git push origin master --force > >> > >> > >> This should force (i.e. destructive operation) the GH master branch to > be > >> the same as the ASF git master. > >> > >> HTH, > >> Tim > >> > >> On Tue, May 15, 2018 at 12:52 PM, 张铎(Duo Zhang) > >> wrote: > >> > >> > When committing HBASE-20576... > >> > > >> > I've done a force push to master immediately and the asf git is back > to > >> > normal, but the github repo has already been messed up. > >> > > >> > Does anyone know how to get the github repo back? > >> > > >> > Really sorry about this... > >> > > >> >
Re: Accidentally pushed HBASE-19064 as master
FYI for those interested I filed INFRA-16532 On Tue, May 15, 2018 at 8:32 AM, Sean Busbey wrote: > In the future please don't do force pushes. I thought we had disabled > them for the master branch. I'll check up with Infra on this. > > Next time just push a revert. > > On Tue, May 15, 2018 at 8:11 AM, 张铎(Duo Zhang) wrote: >> Thanks. >> >> I pushed HBASE-20457 to master and then the Github repo is back to normal... >> >> 2018-05-15 19:10 GMT+08:00 Tim Robertson : >> >>> Perhaps someone can verify this first: >>> >>> // assuming you have ASF upstream of GH (git remote add upstream >>> git fetch upstream >>> git checkout master >>> git reset --hard upstream/master >>> git push origin master --force >>> >>> >>> This should force (i.e. destructive operation) the GH master branch to be >>> the same as the ASF git master. >>> >>> HTH, >>> Tim >>> >>> On Tue, May 15, 2018 at 12:52 PM, 张铎(Duo Zhang) >>> wrote: >>> >>> > When committing HBASE-20576... >>> > >>> > I've done a force push to master immediately and the asf git is back to >>> > normal, but the github repo has already been messed up. >>> > >>> > Does anyone know how to get the github repo back? >>> > >>> > Really sorry about this... >>> > >>>
Re: Accidentally pushed HBASE-19064 as master
In the future please don't do force pushes. I thought we had disabled them for the master branch. I'll check up with Infra on this. Next time just push a revert. On Tue, May 15, 2018 at 8:11 AM, 张铎(Duo Zhang) wrote: > Thanks. > > I pushed HBASE-20457 to master and then the Github repo is back to normal... > > 2018-05-15 19:10 GMT+08:00 Tim Robertson : > >> Perhaps someone can verify this first: >> >> // assuming you have ASF upstream of GH (git remote add upstream >> git fetch upstream >> git checkout master >> git reset --hard upstream/master >> git push origin master --force >> >> >> This should force (i.e. destructive operation) the GH master branch to be >> the same as the ASF git master. >> >> HTH, >> Tim >> >> On Tue, May 15, 2018 at 12:52 PM, 张铎(Duo Zhang) >> wrote: >> >> > When committing HBASE-20576... >> > >> > I've done a force push to master immediately and the asf git is back to >> > normal, but the github repo has already been messed up. >> > >> > Does anyone know how to get the github repo back? >> > >> > Really sorry about this... >> > >>
[jira] [Created] (HBASE-20585) Need to clear peer map when clearing MasterProcedureScheduler
Duo Zhang created HBASE-20585: - Summary: Need to clear peer map when clearing MasterProcedureScheduler Key: HBASE-20585 URL: https://issues.apache.org/jira/browse/HBASE-20585 Project: HBase Issue Type: Bug Components: proc-v2 Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 3.0.0, 2.1.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Accidentally pushed HBASE-19064 as master
Thanks. I pushed HBASE-20457 to master and then the Github repo is back to normal... 2018-05-15 19:10 GMT+08:00 Tim Robertson : > Perhaps someone can verify this first: > > // assuming you have ASF upstream of GH (git remote add upstream > git fetch upstream > git checkout master > git reset --hard upstream/master > git push origin master --force > > > This should force (i.e. destructive operation) the GH master branch to be > the same as the ASF git master. > > HTH, > Tim > > On Tue, May 15, 2018 at 12:52 PM, 张铎(Duo Zhang) > wrote: > > > When committing HBASE-20576... > > > > I've done a force push to master immediately and the asf git is back to > > normal, but the github repo has already been messed up. > > > > Does anyone know how to get the github repo back? > > > > Really sorry about this... > > >
Re: Accidentally pushed HBASE-19064 as master
Perhaps someone can verify this first: // assuming you have ASF upstream of GH (git remote add upstream git fetch upstream git checkout master git reset --hard upstream/master git push origin master --force This should force (i.e. destructive operation) the GH master branch to be the same as the ASF git master. HTH, Tim On Tue, May 15, 2018 at 12:52 PM, 张铎(Duo Zhang) wrote: > When committing HBASE-20576... > > I've done a force push to master immediately and the asf git is back to > normal, but the github repo has already been messed up. > > Does anyone know how to get the github repo back? > > Really sorry about this... >
Accidentally pushed HBASE-19064 as master
When committing HBASE-20576... I've done a force push to master immediately and the asf git is back to normal, but the github repo has already been messed up. Does anyone know how to get the github repo back? Really sorry about this...
[jira] [Created] (HBASE-20584) TestRestoreSnapshotFromClient failed
Jingyun Tian created HBASE-20584:
Summary: TestRestoreSnapshotFromClient failed
Key: HBASE-20584
URL: https://issues.apache.org/jira/browse/HBASE-20584
Project: HBase
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Jingyun Tian
Assignee: Jingyun Tian
org.apache.hadoop.hbase.snapshot.HBaseSnapshotException:
org.apache.hadoop.hbase.snapshot.HBaseSnapshotException: Snapshot {
ss=snaptb1-1526376636687
table=testRestoreSnapshotAfterSplittingRegions-1526376636687 type=FLUSH } had
an error. Procedure snaptb1-1526376636687 { waiting=[] done=[] }
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:380)
at
org.apache.hadoop.hbase.master.MasterRpcServices.isSnapshotDone(MasterRpcServices.java:1128)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException via
Failed taking snapshot { ss=snaptb1-1526376636687
table=testRestoreSnapshotAfterSplittingRegions-1526376636687 type=FLUSH } due
to exception:Regions moved during the snapshot '{ ss=snaptb1-1526376636687
table=testRestoreSnapshotAfterSplittingRegions-1526376636687 type=FLUSH }'.
expected=6
snapshotted=7.:org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException:
Regions moved during the snapshot '{ ss=snaptb1-1526376636687
table=testRestoreSnapshotAfterSplittingRegions-1526376636687 type=FLUSH }'.
expected=6 snapshotted=7.
at
org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:82)
at
org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.rethrowExceptionIfFailed(TakeSnapshotHandler.java:311)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:369)
... 6 more
Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Regions
moved during the snapshot '{ ss=snaptb1-1526376636687
table=testRestoreSnapshotAfterSplittingRegions-1526376636687 type=FLUSH }'.
expected=6 snapshotted=7.
at
org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifyRegions(MasterSnapshotVerifier.java:217)
at
org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifySnapshot(MasterSnapshotVerifier.java:121)
at
org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.process(TakeSnapshotHandler.java:207)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:100)
at
org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:90)
at
org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:360)
at
org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:348)
at
org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101)
at
org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3061)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3053)
at
org.apache.hadoop.hbase.client.HBaseAdmin.snapshot(HBaseAdmin.java:2532)
at
org.apache.hadoop.hbase.client.HBaseAdmin.snapshot(HBaseAdmin.java:2499)
at
org.apache.hadoop.hbase.client.HBaseAdmin.snapshot(HBaseAdmin.java:2492)
at
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient.testRestoreSnapshotAfterSplittingRegions(TestRestoreSnapshotFromClient.java:311)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.refl
