[
https://issues.apache.org/jira/browse/YARN-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15757531#comment-15757531
]
Billie Rinaldi commented on YARN-5993:
--------------------------------------
bq.
org.apache.slider.providers.docker.DockerProviderService.publishExportGroups(String,
String, String, String, List) makes inefficient use of keySet iterator instead
of entrySet iterator
My patch does not introduce this issue and it is being fixed in YARN-5967.
bq. Why is this method synchronized now?
I think I was just being careful and don't recall a stronger justification. I
will remove it.
bq. Since we are discarding all but the first IP, I am thinking that we should
probably introduce a IPS_KEY_FORMAT of ${%s_IPS} and replace it with a comma
separated list of IPs when there are more than one IP for a single container.
This gives application owners a placeholder to fetch all the IPs as well (if
they want to).
I experimented with HOSTS and IPS formats in addition to HOST and IP.
Ultimately I found it to be more confusing to configure than just having HOST
and IP substitutions and having a separate export for each HOST and IP.
Currently in this patch each container can only export one export entry for a
given export name. We could allow a container to export multiple entries, one
for each IP, in the case where multiple IPs are returned in the container
status. However, it should be noted that we would still have only one
ServiceRecord registered in ZooKeeper. I am not aware of Registry DNS
supporting multiple IPs per ServiceRecord. If we want to support multiple IPs
in the ServiceRecord, we'll need to open a separate ticket for that.
bq. Just to confirm, if roleNameKey is null then there is no roleGroupKey,
right?
Let's say we have an external app external-app-name and that external app has a
unique component COMP with instances COMP1 and COMP2. When used in an assembly
app, the role names will be external-app-name-COMP1 and
external-app-name-COMP2. The role group for both will be
external-app-name-COMP. For the exports for these components, we are trying to
extract the roleNameKeys COMP1 and COMP2, and the roleNameGroup COMP, because
the external app's exports will have variables like $\{COMP1_HOST} and
$\{COMP_HOST}, without the external-app-name- prefix. In reality, roleNameKey
will always be roleGroupKey plus an integer. Neither will ever be null because
the role name and role group will always begin with the role prefix. The null
checks are just sanity checks.
bq. I think we should not block here, since it will help application owners to
know which variables were not substituted, rather than thinking that exports
don’t work at all. What do you think?
What is happening here is that exports can be specified at the global or the
component level, but they are always exported by individual components. The
tradeoff we are making is that users can put any exports they want at the
global level, and the provider will determine which component is supposed to
export them. I thought this would be easier for the user. (In the old agent
model, the user had to explicitly specify which components published which
exports.)
For example, if http://$\{COMP1_HOST}:port were specified at the global level,
only COMP1 will publish this export in the current patch.
If I make the change you suggest, COMP2 and COMP3 would also export
http://$\{COMP1_HOST}:port without the HOST variable filled in.
If we'd like to move to a model of having components publish all exports they
can see, I think we would have to disallow exports at the global level,
requiring all exports to be component properties. It looks like the services
API only supports global exports at this point (application-level quicklinks)
and ignores any component quicklinks.
bq. Please add null check for containerId in both the methods
bq. Change sortedEntries type from TreeMap to Map
bq. Remove Iterator import
bq. We can change values type from TreeSet to Set
bq. Can you rename entrySet to entries or exportEntries
bq. Can we rename this method to getExportEntries(String key)
bq. Check entry.getValue() for null
Done.
> Allow native services quicklinks to be exported for each component
> ------------------------------------------------------------------
>
> Key: YARN-5993
> URL: https://issues.apache.org/jira/browse/YARN-5993
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Billie Rinaldi
> Assignee: Billie Rinaldi
> Attachments: YARN-5993-yarn-native-services.001.patch
>
>
> The quicklinks export capability changed in switching from the agent provider
> to the docker provider, and currently the docker provider only allows one
> component to export links. We should improve this capability to be more in
> line with the previous agent capability.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
