[
https://issues.apache.org/jira/browse/HDFS-5431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846101#comment-13846101
]
Andrew Wang commented on HDFS-5431:
-----------------------------------
Here's a new version. I made the suggested changes as follows:
- "force" RPC parameter, which forces a directive to be added or modified
regardless of pool limits. Propagated this out to CacheAdmin and redid all the
usages. When {{force=false}}, we synchronously wait for a CRM rescan before
checking limits for more accuracy.
- {{CRM#kickAndWaitForRescan}} tries to be fancy with volatile longs and a
condition variable, checking this would be appreciated
- Do incremental stats updates on add/remove/modify of a directive, with
caveats about consistency and usage of {{force}}
- Made it so CRM is only kicked for {{force=false}} operations and
modify/remove pool. I feel like the normal usage pattern will be to do a bunch
of caching commands at once, so it'd be bad to do rescans during this. Instead,
I lowered the default refresh time for CRM to 1 min, and maybe we should lower
it even more once we benchmark it.
- Rejiggered overlimit bytes to simply be Math.max(bytesNeeded-limit, 0) for a
pool, simpler.
> support cachepool-based limit management in path-based caching
> --------------------------------------------------------------
>
> Key: HDFS-5431
> URL: https://issues.apache.org/jira/browse/HDFS-5431
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Affects Versions: 3.0.0
> Reporter: Colin Patrick McCabe
> Assignee: Andrew Wang
> Attachments: hdfs-5431-1.patch, hdfs-5431-2.patch, hdfs-5431-3.patch
>
>
> We should support cachepool-based quota management in path-based caching.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
