[
https://issues.apache.org/jira/browse/HDDS-15335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-15335:
------------------------------
Description:
NSSummaryTask is a ReconOmTask that the dispatcher fans out on every batch of
OM RocksDB updates Recon ingests.
Inside its process() method, three sub-tasks (FSO / Legacy / OBS) ran
sequentially even though they operate on disjoint slices of the event stream
(filtered by table and bucket layout) and write to disjoint NSSummary entries.
The Legacy and OBS sub-tasks were also each individually slower than necessary
because every event triggered a fresh RocksDB point read of the corresponding
OmBucketInfo from Recon's local OM snapshot DB (via
{{getBucketTable().getSkipCache(...)}}), even though bucket layout and objectID
never change once a bucket exists.
was:NSSummaryTask is a ReconOmTask that the dispatcher fans out on every
batch of OM RocksDB updates Recon ingests. Inside its process() method, three
sub-tasks (FSO / Legacy / OBS) ran sequentially even though they operate on
disjoint slices of the event stream (filtered by table and bucket layout) and
write to disjoint NSSummary entries. The Legacy and OBS sub-tasks were also
each individually slower than necessary because every event triggered a fresh
RocksDB point read of the corresponding OmBucketInfo from Recon's local OM
snapshot DB (via {{getBucketTable().getSkipCache(...)}}), even though bucket
layout and objectID never change once a bucket exists.
> Recon: parallelize NSSummaryTask sub-tasks and cache OmBucketInfo lookups
> -------------------------------------------------------------------------
>
> Key: HDDS-15335
> URL: https://issues.apache.org/jira/browse/HDDS-15335
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Recon
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
>
> NSSummaryTask is a ReconOmTask that the dispatcher fans out on every batch of
> OM RocksDB updates Recon ingests.
> Inside its process() method, three sub-tasks (FSO / Legacy / OBS) ran
> sequentially even though they operate on disjoint slices of the event stream
> (filtered by table and bucket layout) and write to disjoint NSSummary
> entries. The Legacy and OBS sub-tasks were also each individually slower than
> necessary because every event triggered a fresh RocksDB point read of the
> corresponding OmBucketInfo from Recon's local OM snapshot DB (via
> {{getBucketTable().getSkipCache(...)}}), even though bucket layout and
> objectID never change once a bucket exists.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
