[jira] [Resolved] (HBASE-27510) Should use 'org.apache.hbase.thirdparty.io.netty.tryReflectionSetAccessible'
[ https://issues.apache.org/jira/browse/HBASE-27510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-27510. --- Fix Version/s: 2.6.0 3.0.0-alpha-4 2.5.3 Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2.5+. Thanks [~weichiu] for reviewing! > Should use 'org.apache.hbase.thirdparty.io.netty.tryReflectionSetAccessible' > > > Key: HBASE-27510 > URL: https://issues.apache.org/jira/browse/HBASE-27510 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.3 > > > We current use '-Dio.netty.tryReflectionSetAccessible=true' in pom, but after > shading, it should be > '-Dorg.apache.hbase.thirdparty.io.netty.tryReflectionSetAccessible=true' now. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27484) FNFE on StoreFileScanner after a flush followed by a compaction
[
https://issues.apache.org/jira/browse/HBASE-27484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil resolved HBASE-27484.
--
Resolution: Fixed
Merged to master, branch-2, branch-2.5 and branch-2.4. Thanks for reviewing it,
[~psomogyi] !
> FNFE on StoreFileScanner after a flush followed by a compaction
> ---
>
> Key: HBASE-27484
> URL: https://issues.apache.org/jira/browse/HBASE-27484
> Project: HBase
> Issue Type: Bug
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> One of our customers was running SyncTable from a 1.2 based cluster, where
> SyncTable map tasks were open scanners on a 2.4 based cluster for comparing
> the two clusters. Few of the map tasks failed with a DoNotRetryException
> caused by a FileNotFoundException blowing all the way up to the client:
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.io.FileNotFoundException:
> open
> s3a://x--xxx/hbase/data/default/xx/4c53da8c2ab9b7d7a0d6046ef3bb701c/0/e2c58350e4e54c21b0f713a4c271b8c0
> at 7225 on
> s3a://x--xxx/hbase/data/default/xx/4c53da8c2ab9b7d7a0d6046ef3bb701c/0/e2c58350e4e54c21b0f713a4c271b8c0:
> com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does
> not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey;
> Request ID: KBRNC67WZGCS4SCF; S3 Extended Request ID:
> wWEJm8tlFSj/8g+xFpmD1vgWzT/n7HzBcOFAZ8ayIqKMsDZGN/d2kEhdusPLhMd540h+QAPP1xw=;
> Proxy: null), S3 Extended Request ID:
> wWEJm8tlFSj/8g+xFpmD1vgWzT/n7HzBcOFAZ8ayIqKMsDZGN/d2kEhdusPLhMd540h+QAPP1xw=:NoSuchKey
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3712)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45819)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:392)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:359)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:339)
> Caused by: java.io.FileNotFoundException: open
> s3a://x--xxx/hbase/data/default/xx/4c53da8c2ab9b7d7a0d6046ef3bb701c/0/e2c58350e4e54c21b0f713a4c271b8c0
> at 7225 on
> s3a://x--xxx/hbase/data/default/xx/4c53da8c2ab9b7d7a0d6046ef3bb701c/0/e2c58350e4e54c21b0f713a4c271b8c0:
> com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does
> not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey;
> Request ID: KBRNC67WZGCS4SCF; S3 Extended Request ID:
> wWEJm8tlFSj/8g+xFpmD1vgWzT/n7HzBcOFAZ8ayIqKMsDZGN/d2kEhdusPLhMd540h+QAPP1xw=;
> Proxy: null), S3 Extended Request ID:
> wWEJm8tlFSj/8g+xFpmD1vgWzT/n7HzBcOFAZ8ayIqKMsDZGN/d2kEhdusPLhMd540h+QAPP1xw=:NoSuchKey
> ...
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.seekTo(HFileReaderImpl.java:632)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:315)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:216)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:417)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.reopenAfterFlush(StoreScanner.java:1018)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:552)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:155)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:7399)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:7567)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:7331)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3373)
> {noformat}
> We can see on the RS logs that the above file got recently create as an
> outcome of a memstore flush, then compaction is triggered shortly:
> {noformat}
> 2022-11-11 22:16:50,322 INFO
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher: Flushed memstore
> data size=208.15 KB at sequenceid=4949703 (bloomFilter=false),
> to=s3a://x--xxx/hbase/data/default/xx/4c53da8c2ab9b7d7a0d6046ef3bb701c/0/e2c58350e4e54c21b0f713a4c271b8c0
> 2022-11-11 22:16:50,513 INFO org.apache.hadoop.hbase.regionserver.HStore:
> Added
> s3a://x--xxx/hbase/data/default/x
Re: [DISCUSS] How to deal with the disabling of public sign ups for jira.a.o(enable github issues?)
Bump and also send this to user@hbase. We need to find a way to deal with the current situation where contributors can not create a Jira account on their own... At least, we need to change the readme on github page, web site and also the ref guide to tell users how to acquire a jira account... Thanks. 张铎(Duo Zhang) 于2022年11月27日周日 22:06写道： > > For me, I think most developers already have a github account, so > enabling it could help us get more feedback. For lots of younger > Chinese developers, they rarely use email in their daily life... > No doubt later we need to modify our readme on github. If we just let > users go to github issues on the readme, they will soon open an issue > there. But if we ask users to first send an email to a mailing list, > for acquiring a jira account, and then wait for a PMC member to submit > the request, and receive the email response, set up their account, and > then they can finally open an issue on jira. I'm afraid lots of users > will just give up, it is not very friendly... > > And I do not mean separate issue systems for users and devs. Users can > still open jira issues or ask in the mailing list if they want, github > issues is just another channel. If a user asks something in the > mailing list and we think it is a bug, we will ask the user to file an > issue or we will file an issue for it. It is just the same with github > issues. > > Thanks. > > Nick Dimiduk 于2022年11月24日周四 15:44写道： > > > > This new situation around JIRA seems very similar to the existing situation > > around Slack. A new community member currently must acquire a Slack invite > > somehow, usually by emailing one of the lists. Mailing lists themselves > > involve a signup process, though it may be possible to email user/-zh/dev > > without first subscribing to the list. > > > > I have a -0 opinion on using GitHub Issues to manage JIRA subscription > > access. It seems like a comical cascade of complexity. I’d prefer to keep > > GitHub Issues available to us as a future alternative to JIRA for project > > issue tracking. I agree with you that migrating away from JIRA will be > > painful. > > > > I’m not a big fan of having separate issue systems for users vs. devs. It > > emphasizes the idea that users and devs are different groups of people with > > unequal voice in the project direction. I suppose it could be done well, > > but I think it is more likely to be done poorly. > > > > I follow the Infra list, but only casually. It seems there’s a plan to > > eventually adopt some Atlassian Cloud service, which has better anti-spam > > controls. If that is on the roadmap, I’m content to let JIRA access follow > > Slack access: using some existing outreach to request access. Introducing a > > dedicated list would be fine with me as well. > > > > -n > > > > On Thu, Nov 24, 2022 at 03:19 Duo Zhang wrote: > > > > > I've forwarded an announcement email from the INFRA team recently > > > about disabling the public sign ups for jira.a.o because of spamming. > > > And now the rule is finally applied, when you open jira.a.o, you can > > > see there is a gray bar on the top to tell you 'Public signup for this > > > instance is disabled. Our Jira Guidelines page explains how to get an > > > account.'. > > > > > > For me, I do not think it is easy for us to completely drop jira since > > > it is the issue tracker we have been using for years and all our > > > release processes are bound to it. So at least we need to find a way > > > to let our contributors know how to acquire a jira account. > > > > > > The hive project decided to use a dedicated mailing list for acquiring > > > a jira account. > > > > > > > > > https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-JIRA > > > > > > For me, I think maybe we could enable github issues for our users and > > > contributors. They can ask questions and report issues there and if we > > > think it is a bug that needs to be fixed, we could open a jira issue > > > for it. And we could also create a special issue template for > > > acquiring jira accounts. > > > > > > Thoughts? > > > > > > Thanks. > > >
[jira] [Created] (HBASE-27511) Lock contention when doing multiple parallel preads due to StoreFileReader reuse
Wellington Chevreuil created HBASE-27511:
Summary: Lock contention when doing multiple parallel preads due
to StoreFileReader reuse
Key: HBASE-27511
URL: https://issues.apache.org/jira/browse/HBASE-27511
Project: HBase
Issue Type: Bug
Reporter: Wellington Chevreuil
Assignee: Wellington Chevreuil
Attachments: rs-stack-lock-contention
In HStoreFile, we reuse the StoreFileReader created during HStoreFile
initialization when creating a StoreFileScanner for preads:
[https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStoreFile.java#L545]
When using S3 as hbase storage, we noticed this caused lock contention when
multiple clients were doing preads in parallel:
{noformat}
...
"RpcServer.default.FPBQ.Fifo.handler=38,queue=8,port=16020" #125 daemon prio=5
os_prio=0 tid=0x7fc11d83c000 nid=0x73f2 waiting for monitor entry
[0x7fc1154e6000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:73)
- waiting to lock <0x7fc4c2769660> (a
org.apache.hadoop.fs.s3a.S3AInputStream)
...
"RpcServer.default.FPBQ.Fifo.handler=37,queue=7,port=16020" #124 daemon prio=5
os_prio=0 tid=0x7fc11d83a000 nid=0x73f1 waiting for monitor entry
[0x7fc1155e7000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:73)
- waiting to lock <0x7fc4c2769660> (a
org.apache.hadoop.fs.s3a.S3AInputStream)
...
"RpcServer.default.FPBQ.Fifo.handler=36,queue=6,port=16020" #123 daemon prio=5
os_prio=0 tid=0x7fc11d838000 nid=0x73f0 runnable [0x7fc1156e8000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
...
at org.apache.hadoop.fs.s3a.S3AInputStream.reopen(S3AInputStream.java:216)
- locked <0x7fc4c2769660> (a
org.apache.hadoop.fs.s3a.S3AInputStream)
... {noformat}
We should create a new instance of StoreFileReader for each StoreFileScanner
when doing preads, instead,
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [DISCUSS] Allow namespace admins to clone snapshots created by them
This should not break any existing use case so I see no reason to not add this to branch-2.5 and branch-2.4. On Thu, Nov 24, 2022 at 3:03 AM 张铎(Duo Zhang) wrote: > I'm OK with this change. > > But maybe we still need to determine which branches we can apply this > change to? Is it OK to include this change for branch-2.5 and > branch-2.4? > > Tak Lon (Stephen) Wu 于2022年11月22日周二 06:31写道： > > > > FYI the PR is https://github.com/apache/hbase/pull/4885 and > > https://issues.apache.org/jira/browse/HBASE-27493. > > > > the proposal seems to be, should we allow cloning snapshot to any > > namespace if they're not the global admin. > > > > logically, it should be fine because they're the admin for the > > namespace, and should be able to do whatever within that namespace. > > > > Thanks, > > Stephen > > > > > > On Mon, Nov 21, 2022 at 11:38 AM Szabolcs Bukros > > wrote: > > > > > > Hi Everyone, > > > > > > Creating a snapshot requires table admin permissions. But cloning it > > > requires global admin permissions unless the user owns the snapshot and > > > wants to recreate the original table the snapshot was based on using > the > > > same table name. This puts unnecessary load on the few users having > global > > > admin permissions on the cluster. I would like to relax this rule a > bit and > > > allow the owner of the snapshot to clone it into any namespace where > they > > > have admin permissions regardless of the table name used. > > > > > > Please let me know what you think about this proposal. And if you find > it > > > acceptable which branch do you think this could land on. > > > > > > Thanks, > > > Szabolcs Bukros >
