[
https://issues.apache.org/jira/browse/ATLAS-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16626801#comment-16626801
]
Ayush Nigam commented on ATLAS-2889:
------------------------------------
Also I would like to highlight that S3 objects currently support AVRO types
only according to S3 object data model for Atlas.
We will like to add types such as JSON,XML,CSV to current S3 object type
definitions
[~toopt4] [~barbara]
> S3 object tag import hook
> -------------------------
>
> Key: ATLAS-2889
> URL: https://issues.apache.org/jira/browse/ATLAS-2889
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-core, atlas-intg
> Affects Versions: 1.1.0
> Reporter: t oo
> Priority: Major
>
> https://issues.apache.org/jira/browse/ATLAS-2708 introduced support for s3
> type.
>
> From the comments in the jira, there are some external steps needed to import
> object tags. This Jira is about creating the E2E functionality within Atlas.
> ie functionality in UI or an API that when supplied with argument of an s3
> bucket/slice then Atlas automatically imports all the object names/tags
> recursively from under that s3 bucket/slice.
>
> "It doesn't do it automatically through a listener like the hive hook. We do
> it via lambda functions, triggered, say, on the creation of S3 object or
> pseudodirectory or bucket. We package up the info into AtlasEntities and
> then publish to the ATLAS_HOOK kafka topic.
> You have to create your own Lambda code that creates AtlasEntities on the fly
> as trigerred by Lambda Function(on changes made to s3 object) and then push
> to Kafka Queue. This particular functionality is not part of Atlas tool as of
> now."
> cc: [~barbara] [~ayushmnnit]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
