[
https://issues.apache.org/jira/browse/YARN-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316435#comment-15316435
]
Jonathan Maron commented on YARN-4757:
--------------------------------------
[~jianhe] - Thanks for the review! Answers:
{quote}
Here, it’s using the ‘description’ filed for constructing the DNS name. is this
expected ? seems not mentioned in the doc
{quote}
The description field is currently the service record field that Slider (the
current primary user of the ZK registry) is leveraging to relate the component
name (role). I assume the reason is because the ZK path to the node relates
the user, application, and component name. [~steve_l] - is that correct? Any
objection to an explicit "name" attribute in the service record?
{quote}
we can close the CuratorService#treeCache when stop the service?
{quote}
Probably makes sense.
{quote}
only readLock is being used. wondering whether these locks are needed.
{quote}
This is probably due to some refactorings - it was leveraged in previous
iterations of the code. I think I will re-introduce it. I see nothing in the
dnsjava code to indicate that Zone implementations are thread safe. Given the
dynamic nature of record registration and deletion in the yarn use case I think
it would be base to synchronize access to the zone object to ensure
deterministic results.
{quote}
question about the dnsEnabled config, if the dnsEnabled is false, what else
does the RegistryDNSServer do ? Asking this because I'm wondering whether this
config is actually needed.
{quote}
I guess this depends on what our expectations are regarding the use of DNS - is
it expected to be available as a default service. If the answer is no, we
could manage the inclusion of the service at this level or perhaps one level
(have the RM not even add the service based on flag value?)
{quote}
RecordCreatorFactory: The RecordCreatroFactory#getRecordCreator is
instantiating the creator instance every time this method gets called. May be
singleton pattern could be useful to avoid creating a new instance every time.
{quote}
Certainly a possibility. The thinking here was to create simple, lightweight,
stateless objects that could be used with little regard to multi-threading
concerns etc. However, if some profiling indicates an issue a singleton
approach may be preferable.
{quote}
DNSManagementOperations class is not used anywhere , can be removed?
{quote}
Yes - probably a leftover from a previous code iteration.
{quote}
a few unused methods in RegistryDNS, e.g. addDSRecord, signZones. is this
intended ?
{quote}
For the time being - yes. DS records appear to play a role in some DNS
negative response processing. Though we have made strides in better support
for negative responses (NXT records), it was still somewhat unclear whether we
ultimately would need to enhance support with full DS record capabilities. So
I have left these methods in place until such time that I could make a better
determination.
{quote}
what does the RegistryDNS#primaryDNS do ?
{quote}
The idea here was to support a use case in which the YARN DNS server was
designated as a resolver for a suite of hosts in the cluster. In those
instances, queries that it itself could not resolve would have to be forwarded
to a "primary" DNS server for resolution. I now think this probably less
likely, so we could certainly look at removing that feature.
> [Umbrella] Simplified discovery of services via DNS mechanisms
> --------------------------------------------------------------
>
> Key: YARN-4757
> URL: https://issues.apache.org/jira/browse/YARN-4757
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Jonathan Maron
> Attachments:
> 0001-YARN-4757-Initial-code-submission-for-DNS-Service.patch, YARN-4757-
> Simplified discovery of services via DNS mechanisms.pdf,
> YARN-4757-YARN-4757.001.patch, YARN-4757-YARN-4757.002.patch,
> YARN-4757-YARN-4757.003.patch
>
>
> [See overview doc at YARN-4692, copying the sub-section (3.2.10.2) to track
> all related efforts.]
> In addition to completing the present story of service-registry (YARN-913),
> we also need to simplify the access to the registry entries. The existing
> read mechanisms of the YARN Service Registry are currently limited to a
> registry specific (java) API and a REST interface. In practice, this makes it
> very difficult for wiring up existing clients and services. For e.g, dynamic
> configuration of dependent endpoints of a service is not easy to implement
> using the present registry-read mechanisms, *without* code-changes to
> existing services.
> A good solution to this is to expose the registry information through a more
> generic and widely used discovery mechanism: DNS. Service Discovery via DNS
> uses the well-known DNS interfaces to browse the network for services.
> YARN-913 in fact talked about such a DNS based mechanism but left it as a
> future task. (Task) Having the registry information exposed via DNS
> simplifies the life of services.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
