[jira] [Updated] (YARN-6355) Preprocessor framework for AM and Client interactions with the RM
[ https://issues.apache.org/jira/browse/YARN-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6355: -- Attachment: YARN-6355.002.patch Updating patch with some code cleanup, javadocs and testcases > Preprocessor framework for AM and Client interactions with the RM > - > > Key: YARN-6355 > URL: https://issues.apache.org/jira/browse/YARN-6355 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: Arun Suresh > Labels: amrmproxy, resourcemanager > Attachments: YARN-6355.001.patch, YARN-6355.002.patch > > > Currently on the NM, we have the {{AMRMProxy}} framework to intercept the AM > <-> RM communication and enforce policies. This is used both by YARN > federation (YARN-2915) as well as Distributed Scheduling (YARN-2877). > This JIRA proposes to introduce a similar framework on the the RM side, so > that pluggable policies can be enforced on ApplicationMasterService centrally > as well. > This would be similar in spirit to a Java Servlet Filter Chain. Where the > order of the interceptors can declared externally. > Once possible usecase would be: > the {{OpportunisticContainerAllocatorAMService}} is implemented as a wrapper > over the {{ApplicationMasterService}}. It would probably be better to > implement it as an Interceptor. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6355) Preprocessor framework for AM and Client interactions with the RM
[ https://issues.apache.org/jira/browse/YARN-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6355: -- Attachment: YARN-6355-one-pager.pdf Attaching a one pager design doc. > Preprocessor framework for AM and Client interactions with the RM > - > > Key: YARN-6355 > URL: https://issues.apache.org/jira/browse/YARN-6355 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: Arun Suresh > Labels: amrmproxy, resourcemanager > Attachments: YARN-6355.001.patch, YARN-6355.002.patch, > YARN-6355-one-pager.pdf > > > Currently on the NM, we have the {{AMRMProxy}} framework to intercept the AM > <-> RM communication and enforce policies. This is used both by YARN > federation (YARN-2915) as well as Distributed Scheduling (YARN-2877). > This JIRA proposes to introduce a similar framework on the the RM side, so > that pluggable policies can be enforced on ApplicationMasterService centrally > as well. > This would be similar in spirit to a Java Servlet Filter Chain. Where the > order of the interceptors can declared externally. > Once possible usecase would be: > the {{OpportunisticContainerAllocatorAMService}} is implemented as a wrapper > over the {{ApplicationMasterService}}. It would probably be better to > implement it as an Interceptor. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6355) Preprocessor framework for AM and Client interactions with the RM
[ https://issues.apache.org/jira/browse/YARN-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6355: -- Attachment: YARN-6355.003.patch Updated patch based on suggestions from [~subru] > Preprocessor framework for AM and Client interactions with the RM > - > > Key: YARN-6355 > URL: https://issues.apache.org/jira/browse/YARN-6355 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: Arun Suresh > Labels: amrmproxy, resourcemanager > Attachments: YARN-6355.001.patch, YARN-6355.002.patch, > YARN-6355.003.patch, YARN-6355-one-pager.pdf > > > Currently on the NM, we have the {{AMRMProxy}} framework to intercept the AM > <-> RM communication and enforce policies. This is used both by YARN > federation (YARN-2915) as well as Distributed Scheduling (YARN-2877). > This JIRA proposes to introduce a similar framework on the the RM side, so > that pluggable policies can be enforced on ApplicationMasterService centrally > as well. > This would be similar in spirit to a Java Servlet Filter Chain. Where the > order of the interceptors can declared externally. > Once possible usecase would be: > the {{OpportunisticContainerAllocatorAMService}} is implemented as a wrapper > over the {{ApplicationMasterService}}. It would probably be better to > implement it as an Interceptor. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6355) Preprocessor framework for AM and Client interactions with the RM
[ https://issues.apache.org/jira/browse/YARN-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6355: -- Attachment: YARN-6355.004.patch Further cleanup and test case fixes. > Preprocessor framework for AM and Client interactions with the RM > - > > Key: YARN-6355 > URL: https://issues.apache.org/jira/browse/YARN-6355 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: Arun Suresh > Labels: amrmproxy, resourcemanager > Attachments: YARN-6355.001.patch, YARN-6355.002.patch, > YARN-6355.003.patch, YARN-6355.004.patch, YARN-6355-one-pager.pdf > > > Currently on the NM, we have the {{AMRMProxy}} framework to intercept the AM > <-> RM communication and enforce policies. This is used both by YARN > federation (YARN-2915) as well as Distributed Scheduling (YARN-2877). > This JIRA proposes to introduce a similar framework on the the RM side, so > that pluggable policies can be enforced on ApplicationMasterService centrally > as well. > This would be similar in spirit to a Java Servlet Filter Chain. Where the > order of the interceptors can declared externally. > Once possible usecase would be: > the {{OpportunisticContainerAllocatorAMService}} is implemented as a wrapper > over the {{ApplicationMasterService}}. It would probably be better to > implement it as an Interceptor. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6355) Preprocessor framework for AM and Client interactions with the RM
[ https://issues.apache.org/jira/browse/YARN-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6355: -- Attachment: YARN-6355.005.patch Attaching a combined patch for YARN-6776 and YARN-6777 to run it through Jenkins > Preprocessor framework for AM and Client interactions with the RM > - > > Key: YARN-6355 > URL: https://issues.apache.org/jira/browse/YARN-6355 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: Arun Suresh > Labels: amrmproxy, resourcemanager > Attachments: YARN-6355.001.patch, YARN-6355.002.patch, > YARN-6355.003.patch, YARN-6355.004.patch, YARN-6355.005.patch, > YARN-6355-one-pager.pdf > > > Currently on the NM, we have the {{AMRMProxy}} framework to intercept the AM > <-> RM communication and enforce policies. This is used both by YARN > federation (YARN-2915) as well as Distributed Scheduling (YARN-2877). > This JIRA proposes to introduce a similar framework on the the RM side, so > that pluggable policies can be enforced on ApplicationMasterService centrally > as well. > This would be similar in spirit to a Java Servlet Filter Chain. Where the > order of the interceptors can declared externally. > Once possible usecase would be: > the {{OpportunisticContainerAllocatorAMService}} is implemented as a wrapper > over the {{ApplicationMasterService}}. It would probably be better to > implement it as an Interceptor. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6355) Preprocessor framework for AM and Client interactions with the RM
[ https://issues.apache.org/jira/browse/YARN-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6355: -- Attachment: YARN-6355.006.patch > Preprocessor framework for AM and Client interactions with the RM > - > > Key: YARN-6355 > URL: https://issues.apache.org/jira/browse/YARN-6355 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: Arun Suresh > Labels: amrmproxy, resourcemanager > Attachments: YARN-6355.001.patch, YARN-6355.002.patch, > YARN-6355.003.patch, YARN-6355.004.patch, YARN-6355.005.patch, > YARN-6355.006.patch, YARN-6355-one-pager.pdf > > > Currently on the NM, we have the {{AMRMProxy}} framework to intercept the AM > <-> RM communication and enforce policies. This is used both by YARN > federation (YARN-2915) as well as Distributed Scheduling (YARN-2877). > This JIRA proposes to introduce a similar framework on the the RM side, so > that pluggable policies can be enforced on ApplicationMasterService centrally > as well. > This would be similar in spirit to a Java Servlet Filter Chain. Where the > order of the interceptors can declared externally. > Once possible usecase would be: > the {{OpportunisticContainerAllocatorAMService}} is implemented as a wrapper > over the {{ApplicationMasterService}}. It would probably be better to > implement it as an Interceptor. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6355) Preprocessor framework for AM and Client interactions with the RM
[ https://issues.apache.org/jira/browse/YARN-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6355: -- Attachment: YARN-6355.007.patch > Preprocessor framework for AM and Client interactions with the RM > - > > Key: YARN-6355 > URL: https://issues.apache.org/jira/browse/YARN-6355 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: Arun Suresh > Labels: amrmproxy, resourcemanager > Attachments: YARN-6355.001.patch, YARN-6355.002.patch, > YARN-6355.003.patch, YARN-6355.004.patch, YARN-6355.005.patch, > YARN-6355.006.patch, YARN-6355.007.patch, YARN-6355-one-pager.pdf > > > Currently on the NM, we have the {{AMRMProxy}} framework to intercept the AM > <-> RM communication and enforce policies. This is used both by YARN > federation (YARN-2915) as well as Distributed Scheduling (YARN-2877). > This JIRA proposes to introduce a similar framework on the the RM side, so > that pluggable policies can be enforced on ApplicationMasterService centrally > as well. > This would be similar in spirit to a Java Servlet Filter Chain. Where the > order of the interceptors can declared externally. > Once possible usecase would be: > the {{OpportunisticContainerAllocatorAMService}} is implemented as a wrapper > over the {{ApplicationMasterService}}. It would probably be better to > implement it as an Interceptor. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6355) Preprocessor framework for AM and Client interactions with the RM
[ https://issues.apache.org/jira/browse/YARN-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6355: -- Summary: Preprocessor framework for AM and Client interactions with the RM (was: Interceptor framework for the YARN ApplicationMasterService) > Preprocessor framework for AM and Client interactions with the RM > - > > Key: YARN-6355 > URL: https://issues.apache.org/jira/browse/YARN-6355 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: Arun Suresh > Labels: amrmproxy, resourcemanager > > Currently on the NM, we have the {{AMRMProxy}} framework to intercept the AM > <-> RM communication and enforce policies. This is used both by YARN > federation (YARN-2915) as well as Distributed Scheduling (YARN-2877). > This JIRA proposes to introduce a similar framework on the the RM side, so > that pluggable policies can be enforced on ApplicationMasterService centrally > as well. > This would be similar in spirit to a Java Servlet Filter Chain. Where the > order of the interceptors can declared externally. > Once possible usecase would be: > the {{OpportunisticContainerAllocatorAMService}} is implemented as a wrapper > over the {{ApplicationMasterService}}. It would probably be better to > implement it as an Interceptor. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6355) Preprocessor framework for AM and Client interactions with the RM
[ https://issues.apache.org/jira/browse/YARN-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6355: -- Attachment: YARN-6355.001.patch Attaching initial version of the patch. * I think we should stick to the "Interceptor" name, since we need something that does both preprocessing of the request and maybe handle the response as well - A preprocessor implies that we just pre process the request. * I have modified the OpportunisticContainerAllocatorAMService to use the framework. > Preprocessor framework for AM and Client interactions with the RM > - > > Key: YARN-6355 > URL: https://issues.apache.org/jira/browse/YARN-6355 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Arun Suresh >Assignee: Arun Suresh > Labels: amrmproxy, resourcemanager > Attachments: YARN-6355.001.patch > > > Currently on the NM, we have the {{AMRMProxy}} framework to intercept the AM > <-> RM communication and enforce policies. This is used both by YARN > federation (YARN-2915) as well as Distributed Scheduling (YARN-2877). > This JIRA proposes to introduce a similar framework on the the RM side, so > that pluggable policies can be enforced on ApplicationMasterService centrally > as well. > This would be similar in spirit to a Java Servlet Filter Chain. Where the > order of the interceptors can declared externally. > Once possible usecase would be: > the {{OpportunisticContainerAllocatorAMService}} is implemented as a wrapper > over the {{ApplicationMasterService}}. It would probably be better to > implement it as an Interceptor. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org