[ https://issues.apache.org/jira/browse/YARN-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14487631#comment-14487631 ]
Craig Welch commented on YARN-3318: ----------------------------------- bq. ...Do we really see non-comparator based ordering-policy. We are unnecessarily adding two abstractions - adding policies and comparators... In the context of the code so far, the "comparator based" approach is specific to compounding comparators to achieve functionality (priority + fifo, fair + fifo, etc). This was the initial motivation for the two level api & configuration, the broader surface of the policy which would allow for different collection types, sorting on demand, etc, (the original policy) and the narrower one within that (comparator) for the cases where comparator logic was sufficient, e.g. sharing a collection (for composition) and a collection type (a tree, for efficient resorting of individual elements when required) was possible. The two level api & configuration was not well received. Offline Wangda has indicated that he thinks there are policies coming up which will need the wider, initial api, with control over the collection, sorting, etc. Supporting policy composition for those cases would be very awkward & is not really worth pursuing. The various competing tradeoffs, the aversion to a multilevel api, the need for the higher level api, and the ability to compose policies creates something of a tension, I don't think it's realistic to try and accomplish it all together, the result will be Frankensteinian at best. Something has to go. Originally, I chose the multilevel api to resolve the dilemma, I like that choice, it seems unpopular with the crowd. Given that, the other "optional" dynamic is the ability to compose policies (there's no requirement for either of these as far as I can tell, it is a "bonus feature"). While I like the composition approach, it can't be maintained as such with the broader api and without the multilevel config/api. As one of these has to go and it appears it can't be the broader api or the multilevel api I suppose it will have to be composition. Internally there can be some composition of course, but it won't be transparent/exposed/configurable as it was initially. I'll put out a patch with that in a bit. > Create Initial OrderingPolicy Framework and FifoOrderingPolicy > -------------------------------------------------------------- > > Key: YARN-3318 > URL: https://issues.apache.org/jira/browse/YARN-3318 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler > Reporter: Craig Welch > Assignee: Craig Welch > Attachments: YARN-3318.13.patch, YARN-3318.14.patch, > YARN-3318.17.patch, YARN-3318.34.patch, YARN-3318.35.patch, > YARN-3318.36.patch, YARN-3318.39.patch, YARN-3318.45.patch, > YARN-3318.47.patch, YARN-3318.48.patch > > > Create the initial framework required for using OrderingPolicies and an > initial FifoOrderingPolicy -- This message was sent by Atlassian JIRA (v6.3.4#6332)