[
https://issues.apache.org/jira/browse/HDDS-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajesh reassigned HDDS-16998:
-----------------------------
Assignee: rajesh
> Skip set the lastScanned fields for Lifecycle task during cache scan
> --------------------------------------------------------------------
>
> Key: HDDS-16998
> URL: https://issues.apache.org/jira/browse/HDDS-16998
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sammi Chen
> Assignee: rajesh
> Priority: Major
>
> In evaluateKeyAndDirTable (FSO), each directory first iterates
> keyTable.cacheIterator() and then the RocksDB iterator, and both set
> lastScannedKey. The file table uses PartialTableCache, which is a
> ConcurrentHashMap, so the cache pass runs in hash order and lastScannedKey
> can end up at an arbitrary key. If scan state is saved at that point (either
> the expired list fills inside the cache loop and handleAndClearFullList sends
> the state, or the task aborts on shouldRun() at the start of the DB loop), we
> persist that cache key. On resume we seek(lastScannedKey) in the DB iterator,
> which skips every key in that directory sorting before it. Those keys aren't
> evaluated until the next full scan cycle.
> Example: directory has DB keys a, b, c, x, and key m is only in cache. The
> cache pass sets lastScannedKey = m, the leader changes, state is saved with
> m, and on resume we seek to m and skip a, b, c.
> Since the cache pass doesn't seek and is fully re-evaluated on every resume
> anyway, the simple fix is to not update lastScannedKey in the cache loop, so
> the saved position only comes from the sorted DB iteration. OBS
> (evaluateBucket) only iterates the DB, so it isn't affected
> It's found by Chandolu Rajesh Kumar.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
