[jira] [Updated] (YARN-6613) Update json validation for new native services providers
[ https://issues.apache.org/jira/browse/YARN-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-6613: - Attachment: YARN-6613-yarn-native-services.006.patch > Update json validation for new native services providers > > > Key: YARN-6613 > URL: https://issues.apache.org/jira/browse/YARN-6613 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Fix For: yarn-native-services > > Attachments: YARN-6613-yarn-native-services.001.patch, > YARN-6613-yarn-native-services.002.patch, > YARN-6613-yarn-native-services.003.patch, > YARN-6613-yarn-native-services.004.patch, > YARN-6613-yarn-native-services.005.patch, > YARN-6613-yarn-native-services.006.patch > > > YARN-6160 started some work enabling different validation for each native > services provider. The validation done in > ServiceApiUtil#validateApplicationPayload needs to updated accordingly. This > validation should also be updated to handle the APPLICATION artifact type, > which does not have an associated provider. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6613) Update json validation for new native services providers
[ https://issues.apache.org/jira/browse/YARN-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-6613: - Attachment: YARN-6613-yarn-native-services.005.patch Thanks very much for the review, [~jianhe]! bq. remove the unused method AbstractClientProvider#processClientOperation This will be used in SliderClient doClientInstall, but it's commented out right now. bq. Application configurations are merged into components before persisting, this will increase app json file size. For hdfs, it won't be a problem though. for zk that's relatively sensitive to file size, may be an issue. Any reason need to resolve it before persisting? The problem is for external applications. The external components are read and added to the current application, and these are supposed to use the external application's global configuration. Since we don't have multiple global configurations per app, the external application configurations need to have their external global configurations merged right away. So the easiest way to handle it is to do all the merging before persisting, instead of having some resolution performed before persisting and some resolution performed after. It's also easier to test that the resulting Application will be correct if all the resolution is performed at once. Previously Slider also allowed configuration properties to be overridden for external components, which was pretty useful. For example, maybe you want to use the external application hbase, but change the number of region servers, or tweak the configuration properties. In my new patch, I will allow this as well. bq. In actionStart, why is it required to write back to hdfs? It's because the applicationId is part of the Application now. If the Application is not written back to HDFS, the id will be wrong. > Update json validation for new native services providers > > > Key: YARN-6613 > URL: https://issues.apache.org/jira/browse/YARN-6613 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Fix For: yarn-native-services > > Attachments: YARN-6613-yarn-native-services.001.patch, > YARN-6613-yarn-native-services.002.patch, > YARN-6613-yarn-native-services.003.patch, > YARN-6613-yarn-native-services.004.patch, > YARN-6613-yarn-native-services.005.patch > > > YARN-6160 started some work enabling different validation for each native > services provider. The validation done in > ServiceApiUtil#validateApplicationPayload needs to updated accordingly. This > validation should also be updated to handle the APPLICATION artifact type, > which does not have an associated provider. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6613) Update json validation for new native services providers
[ https://issues.apache.org/jira/browse/YARN-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-6613: - Attachment: YARN-6613-yarn-native-services.004.patch Fixing findbugs and checkstyle errors. > Update json validation for new native services providers > > > Key: YARN-6613 > URL: https://issues.apache.org/jira/browse/YARN-6613 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Fix For: yarn-native-services > > Attachments: YARN-6613-yarn-native-services.001.patch, > YARN-6613-yarn-native-services.002.patch, > YARN-6613-yarn-native-services.003.patch, > YARN-6613-yarn-native-services.004.patch > > > YARN-6160 started some work enabling different validation for each native > services provider. The validation done in > ServiceApiUtil#validateApplicationPayload needs to updated accordingly. This > validation should also be updated to handle the APPLICATION artifact type, > which does not have an associated provider. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6613) Update json validation for new native services providers
[ https://issues.apache.org/jira/browse/YARN-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-6613: - Attachment: YARN-6613-yarn-native-services.003.patch > Update json validation for new native services providers > > > Key: YARN-6613 > URL: https://issues.apache.org/jira/browse/YARN-6613 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Fix For: yarn-native-services > > Attachments: YARN-6613-yarn-native-services.001.patch, > YARN-6613-yarn-native-services.002.patch, > YARN-6613-yarn-native-services.003.patch > > > YARN-6160 started some work enabling different validation for each native > services provider. The validation done in > ServiceApiUtil#validateApplicationPayload needs to updated accordingly. This > validation should also be updated to handle the APPLICATION artifact type, > which does not have an associated provider. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6613) Update json validation for new native services providers
[ https://issues.apache.org/jira/browse/YARN-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-6613: - Attachment: YARN-6613-yarn-native-services.002.patch > Update json validation for new native services providers > > > Key: YARN-6613 > URL: https://issues.apache.org/jira/browse/YARN-6613 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Fix For: yarn-native-services > > Attachments: YARN-6613-yarn-native-services.001.patch, > YARN-6613-yarn-native-services.002.patch > > > YARN-6160 started some work enabling different validation for each native > services provider. The validation done in > ServiceApiUtil#validateApplicationPayload needs to updated accordingly. This > validation should also be updated to handle the APPLICATION artifact type, > which does not have an associated provider. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6613) Update json validation for new native services providers
[ https://issues.apache.org/jira/browse/YARN-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-6613: - Attachment: YARN-6613-yarn-native-services.001.patch This patch applies on top of YARN-6160. > Update json validation for new native services providers > > > Key: YARN-6613 > URL: https://issues.apache.org/jira/browse/YARN-6613 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Fix For: yarn-native-services > > Attachments: YARN-6613-yarn-native-services.001.patch > > > YARN-6160 started some work enabling different validation for each native > services provider. The validation done in > ServiceApiUtil#validateApplicationPayload needs to updated accordingly. This > validation should also be updated to handle the APPLICATION artifact type, > which does not have an associated provider. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org