[
https://issues.apache.org/jira/browse/SLIDER-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shanyu zhao updated SLIDER-1119:
--------------------------------
Attachment: (was: SLIDER-1119.patch)
> Strip off Zookeeper dependency if Yarn Registry is not enabled
> --------------------------------------------------------------
>
> Key: SLIDER-1119
> URL: https://issues.apache.org/jira/browse/SLIDER-1119
> Project: Slider
> Issue Type: Bug
> Affects Versions: Slider 0.90.2
> Reporter: shanyu zhao
>
> In some Hadoop cluster where Yarn registry is not enabled, we should not use
> it in Slider. To check if Yarn registry is enabled or not, we can check an
> existing hadoop configuration (hadoop.registry.rm.enabled).
> Today the main use of Yarn registry is to securely publish Slider AM
> operation URL, so that Slider agent can connect to Slider AM. Without Yarn
> registry, we can directly pass in the AM operation URL as a command line
> parameter to containers when launching the container.
> Now admittedly without Yarn registry we lose Slider AM HA. This is because
> when AM restarts, there is no way to tell containers where to find the new
> AM. In this case AM would restart those containers. This may not be desirable
> for some usage but is fine with some other usage scenarios.
> Detailed changes needed:
> 1) In SliderAppMaster, disable service/component registration code if Yarn
> registry is not enabled.
> In SliderClient, disable zookeeper node creation/deletion if Yarn registry is
> not enabled.
> This is done in AbstractSliderLaunchedService.
> 2) In AbstractProviderService and AgentProviderService, do not use Yarn
> registry if not enabled.
> 3) In AgentProviderService.buildContainerLaunchContext(), add "--am-ops-url"
> parameter to the command line, pass in AM Ops URL from cluster info:
> String amOpsUrl =
> getClusterInfoPropertyValue(StatusKeys.INFO_AM_AGENT_OPS_URL);
> 4) In agent python code, read "--am-ops-url" parameter and use it instead of
> reading the value from Zookeeper
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
