[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16196544#comment-16196544 ]
Eric Yang commented on YARN-7202: --------------------------------- [~jianhe] ServiceClient code is partially completed in YARN-7202. "updateService" code is not quite testable because service operation and config change sequence are inverse of what they are supposed to be. Prior to this patch, the PUT method is: If Service object should be in STARTED state, then return. If Service object should be in STOPPED state, then return. If Service object should increase or decrease containers, perform the operation. The problem arises, if someone retrieve Service status object via GET /ws/v1/services/{service_name}, and change number of container for the service. It will not perform the container changes. The code got to this point because there is no separation between configuration, and status. If status object is reused to perform configuration changes, then the code doesn't perform what it was designed to do. For clarity, flex number of container operation should be considered as configuration changes because even existing containers may need to have information of newly spawning containers. Hence, it is best that we treat flexing container numbers as configuration change, and restart all containers with in the service. Giving the above reasons, the proper design should be: If Service config changed, save configuration. If Service should increase/decrease containers, perform flex operation. If Service should be in STARTED state, then start containers. If Service should be in STOPPED state, then stop containers. At this point, ServiceClient is not written to support the proper sequence of operation to perform. I only made changes to ApiServer updateService method to contain the scope of required changes. Therefore, I use mock object here to allow TestApiService to pass without making a failure test case to block other development. ServiceClient will not pass the test code at this point, but ApiServer code can pass the test with improved logic. This is the reason that mock object is used here. In YARN-7215, after spec and status are separated two calls. I added additional logic in ServiceClient code to have ability to make service config changes. In YARN-7215, you can see that TestApiService is switched to the real ServiceClient, and both positive and negative test cases passed. The dependency are required to import MiniCluster. For some reason, I think the code refactoring has some issues. Hadoop-yarn-service-api is not a sibling project to hadoop-yarn-services, or hadoop-yarn-services-core. Therefore, it doesn't inherit MiniCluster imports. This is the reason that test jar files are imported again. I think YARN-7202, YARN-7216, and YARN-7215 should be one transaction. Giving degree of complexity in the design bug, it is good that we discuss the reasons for these changes. The issues are raised out of order due to late discovery of the bugs in the inherited code. Hence, the course correction is only made in later JIRAs. Hope this clarifies that the end goal of this JIRA is met, but the proper code will only happen after YARN-7216 and YARN-7215 are committed. > End-to-end UT for api-server > ---------------------------- > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Jian He > Assignee: Eric Yang > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch, > YARN-7202.yarn-native-services.003.patch, > YARN-7202.yarn-native-services.004.patch, > YARN-7202.yarn-native-services.005.patch, > YARN-7202.yarn-native-services.006.patch, > YARN-7202.yarn-native-services.007.patch, > YARN-7202.yarn-native-services.008.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org