[
https://issues.apache.org/jira/browse/YARN-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208950#comment-15208950
]
Jonathan Maron commented on YARN-4757:
--------------------------------------
a) A records are more usable from an existing client interaction perspective.
For example, you can use a tool such as nslookup to map from a known name to
its IP. You could potentially leverage an SRV record in that instance, but
you'd have to go into the interactive mode of nslookup, set the type, and then
perform the query - a less intuitive and well known approach.
b) It's not a matter of managing a named.conf file as much as setting up bind
to support the dynamic update protocol (YARN containers will come up and go
down and those record updates may be relatively frequent). In addition, the
stateful complaint has more to do with the need to synch state in multiple
processes rather than rely on one source of truth. Finally, the security needs
for an internal zone server are finite enough that, if security was the primary
driver, would make the BIND selection overkill.
c) Not familiar with manta (even initial web searches didn't seem to bring
anything up)? If there is an open source, available solution I'd be more happy
to evaluate its potential use.
d) I'm not sure the problem is necessarily solved. DNS is well understood,
obviously. But the use case here - mirroring the details of an existing
ZK-based registry or, more accurately, the state of the YARN cluster - present
some requirements that perhaps can be best addressed by a tailored solution.
Given the availability of APIs such as dnsjava etc. the approach is not
necessarily daunting from a development perspective. As such, testing can be
performed to address security and performance concerns, though I'm not naive -
I understand some issues will not manifest till actual deployment.
> [Umbrella] Simplified discovery of services via DNS mechanisms
> --------------------------------------------------------------
>
> Key: YARN-4757
> URL: https://issues.apache.org/jira/browse/YARN-4757
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Jonathan Maron
> Attachments: YARN-4757- Simplified discovery of services via DNS
> mechanisms.pdf
>
>
> [See overview doc at YARN-4692, copying the sub-section (3.2.10.2) to track
> all related efforts.]
> In addition to completing the present story of service-registry (YARN-913),
> we also need to simplify the access to the registry entries. The existing
> read mechanisms of the YARN Service Registry are currently limited to a
> registry specific (java) API and a REST interface. In practice, this makes it
> very difficult for wiring up existing clients and services. For e.g, dynamic
> configuration of dependent endpoints of a service is not easy to implement
> using the present registry-read mechanisms, *without* code-changes to
> existing services.
> A good solution to this is to expose the registry information through a more
> generic and widely used discovery mechanism: DNS. Service Discovery via DNS
> uses the well-known DNS interfaces to browse the network for services.
> YARN-913 in fact talked about such a DNS based mechanism but left it as a
> future task. (Task) Having the registry information exposed via DNS
> simplifies the life of services.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
