[
https://issues.apache.org/jira/browse/ATLAS-512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15163129#comment-15163129
]
Tom Beerbower edited comment on ATLAS-512 at 2/24/16 2:57 PM:
--------------------------------------------------------------
Is there an option 3? ...
The hooks could just use the REST APIs directly for type registration and not
pull in Atlas client and all of its dependencies. I think the argument against
doing this is that the REST API is too difficult to use directly.
It seems to me that an unuseable API is the real problem. Why even have it?
It should be fixed. I know that a new version of the REST API is not something
that can be done quickly but I do think that it should be the long term goal.
Of the two options above, I would prefer Option 1 simply because it's easier (I
think) and does not lock us into anything. If a new easy to use REST API is
provided for type registration down the road, the integrations can continue to
maintain a separate script or they can switch to calling the REST APIs directly.
Of course this does not address the issue of requiring Atlas availability, just
the decoupling.
was (Author: tbeerbower):
Is there an option 3? ...
The hooks could just use the REST APIs directly for type registration and not
pull in Atlas client and all of its dependencies. I think the argument against
doing this is that the REST API is too difficult to use directly.
It seems to me that an unuseable API is the real problem. Why even have it?
It should be fixed. I know that a new version of the REST API is not something
that can be done quickly but I do think that it should be the long term goal.
Of the two options above, I would prefer Option 1 simply because it's easier (I
think) and does not lock us into anything. If a new easy to use REST API is
provided for type registration down the road, the integrations can continue to
maintain a separate script or they can switch to calling the REST APIs directly.
> Decouple currently integrating components from availability of Atlas service
> for raising metadata events
> ---------------------------------------------------------------------------------------------------------
>
> Key: ATLAS-512
> URL: https://issues.apache.org/jira/browse/ATLAS-512
> Project: Atlas
> Issue Type: Sub-task
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
>
> The components that currently integrate with Atlas (Hive, Sqoop, Falcon,
> Storm) all communicate their metadata events using Kafka as a messaging
> layer. This effectively decouples these components from the Atlas server.
> However, all of these components have some initialization that checks if
> their respective models are registered with Atlas. For components that
> integrate on the server, like HiveServer2 and Falcon, this initialization is
> a one time check and hence, is manageable. Others like Sqoop, Storm and the
> Hive CLI are client side components and hence the initialization happens for
> every run or session of these components. Invoking the initialization (and
> the one time check) every time like this effectively means that the Atlas
> server should be always available.
> This JIRA is to try and remove this dependency and thus truly decouple these
> components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
