[
https://issues.apache.org/jira/browse/HDFS-5431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843560#comment-13843560
]
Andrew Wang commented on HDFS-5431:
-----------------------------------
Thanks Colin, new patch. Rundown of changes:
* I re-did the FSImager serialization to use a flag int. I also adjusted
CacheDirective to use an int instead of byte, and moved all that code to
FSImageSerialization and cleaned it up a bit. I think it's better not to have
these methods exposed in a public class.
* Removed reservation and weight, le sigh
* Renamed quota to limit, hope that's reasonable
* Added pool stat for "bytesOverlimit". I guess this could be in the cache
directive too, but logically it's more of a pool setting.
* Used parseShort
I still like having a hard fail when you're at quota. This is how we force
someone to go look at their pool stats and fix things. Accepting further
directives in this situation feels like a silent failure, since the user might
go off their merry way and only much later realize their data wasn't cached. I
imagine monitoring tools will still be watching the statistics and throwing up
their own errors too. It might also be worth noting that DN liveness won't
affect limits since file sizes are kept on the NN, but it will be affected by
adding more files to a directory or writing more data to a file.
I agree that we could still use more debugging-type information here. If a user
has to go to the logs in an expected scenario, we've failed.
> support cachepool-based quota management in path-based caching
> --------------------------------------------------------------
>
> Key: HDFS-5431
> URL: https://issues.apache.org/jira/browse/HDFS-5431
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Affects Versions: 3.0.0
> Reporter: Colin Patrick McCabe
> Assignee: Andrew Wang
> Attachments: hdfs-5431-1.patch, hdfs-5431-2.patch
>
>
> We should support cachepool-based quota management in path-based caching.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
