[ https://issues.apache.org/jira/browse/HDFS-5431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843560#comment-13843560 ]
Andrew Wang commented on HDFS-5431: ----------------------------------- Thanks Colin, new patch. Rundown of changes: * I re-did the FSImager serialization to use a flag int. I also adjusted CacheDirective to use an int instead of byte, and moved all that code to FSImageSerialization and cleaned it up a bit. I think it's better not to have these methods exposed in a public class. * Removed reservation and weight, le sigh * Renamed quota to limit, hope that's reasonable * Added pool stat for "bytesOverlimit". I guess this could be in the cache directive too, but logically it's more of a pool setting. * Used parseShort I still like having a hard fail when you're at quota. This is how we force someone to go look at their pool stats and fix things. Accepting further directives in this situation feels like a silent failure, since the user might go off their merry way and only much later realize their data wasn't cached. I imagine monitoring tools will still be watching the statistics and throwing up their own errors too. It might also be worth noting that DN liveness won't affect limits since file sizes are kept on the NN, but it will be affected by adding more files to a directory or writing more data to a file. I agree that we could still use more debugging-type information here. If a user has to go to the logs in an expected scenario, we've failed. > support cachepool-based quota management in path-based caching > -------------------------------------------------------------- > > Key: HDFS-5431 > URL: https://issues.apache.org/jira/browse/HDFS-5431 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode, namenode > Affects Versions: 3.0.0 > Reporter: Colin Patrick McCabe > Assignee: Andrew Wang > Attachments: hdfs-5431-1.patch, hdfs-5431-2.patch > > > We should support cachepool-based quota management in path-based caching. -- This message was sent by Atlassian JIRA (v6.1.4#6159)