Joan,
> Is there a way to put the reference without adding the name attribute in the
> values of the "referenced entity"?
Atlas notification module was enhanced recently, with addition of V2 style
notifications (ATLAS-2421). Data structures used in V2 notification would allow
referencing an entity using its unique-name (as shown below), without having to
provide other attributes of the entity. This enhancement was included in Apache
Atlas 0.8.2, which was released on last month.
Please review the sample V2 notification below, for the usecase discussed
earlier in this thread. Hope this helps.
Sorry for the delay in responding to your question.
Regards,
Madhan
{
"message": {
"type": "ENTITY_CREATE_V2",
"user": "user",
"entities": {
"entities": [
{
"guid": "-14568879465268",
"typeName": "database",
"attributes": {
"dbName": "Test7",
"description": "Test db7",
"name": "Testdb7",
"origin_app": "origin",
"origin_username": "externalusername",
"owner": "owner",
"qualifiedName": "Testdb5",
"repository": {
"typeName": "repository",
"qualifiedName": "repository01"
}
}
}
],
"referredEntities": {
}
}
},
"version": {
"version": "1.0.0"
}
}
From: Jo MailingList <[email protected]>
Reply-To: "[email protected]" <[email protected]>
Date: Tuesday, February 6, 2018 at 2:45 AM
To: "[email protected]" <[email protected]>
Subject: Re: Relating to an entity using uniqueAttributes
Hello,
On the same topic,
Now I realised that when refering to an object alredy created, is not enough to
add the qualifiedName, but it also needs to include the name attribute since it
is a required attribute. The point is that no matter which name I put, the
actual referenced entity is updated to have that name.
Is there a way to put the reference without adding the name attribute in the
values of the "referenced entity"?
Thank you very much,
Best regards,
Joan
2018-01-05 14:24 GMT+01:00 Jo MailingList <[email protected]>:
Hello,
Thank you very much,
This has been of great help.
Best regards,
Joan
2018-01-05 9:05 GMT+01:00 Madhan Neethiraj <[email protected]>:
Joan,
A slightly better approach is to provide repository entity as a separate entry
in “entities” and refer to the repository in database entity using its temp-id,
as shown below.
This approach will allow an entity be referenced from multiple entities using
its temp-id (instead of adding the entire entity details in each such
reference).
Madhan
{
"message": {
"type": "ENTITY_FULL_UPDATE",
"user": "user",
"entities": [
{
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id": "-14568879465268",
"typeName": "database",
"state": "ACTIVE",
"version": 0
},
"typeName": "database",
"values": {
"dbName": "Test7",
"description": "Test db7",
"name": "Testdb7",
"origin_app": "origin",
"origin_username": "externalusername",
"owner": "owner",
"qualifiedName": "Testdb5",
"repository": {
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id": "-14568879465269",
"typeName": "repository",
"state": "ACTIVE",
"version": 0
}
},
"traitNames": [],
"traits": {}
},
{
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id": "-14568879465269",
"typeName": "repository",
"state": "ACTIVE",
"version": 0
},
"typeName": "repository",
"values": {
"qualifiedName": "repository01"
},
"traitNames": [],
"traits": {}
}
]
},
"version": {
"version": "1.0.0"
}
}
From: Madhan Neethiraj <[email protected]>
Date: Thursday, January 4, 2018 at 11:18 PM
To: "[email protected]" <[email protected]>
Subject: Re: Relating to an entity using uniqueAttributes
Joan,
To refer to another entity In Atlas hook notifications, add the referred entity
details as shown below. Also, I would suggest using ENTITY_FULL_UPDATE as
message type (instead of ENTITY_CREATE).
{
"message": {
"type": "ENTITY_FULL_UPDATE",
"user": "user",
"entities": [
{
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id": "-14568879465268",
"typeName": "database",
"state": "ACTIVE",
"version": 0
},
"typeName": "database",
"values": {
"dbName": "Test7",
"description": "Test db7",
"name": "Testdb7",
"origin_app": "origin",
"origin_username": "externalusername",
"owner": "owner",
"qualifiedName": "Testdb5",
"repository": {
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id": "-14568879465269",
"typeName": "repository",
"state": "ACTIVE",
"version": 0
},
"typeName": "repository",
"values": {
"qualifiedName": "repository01"
},
"traitNames": [],
"traits": {}
}
},
"traitNames": [],
"traits": {}
}
]
},
"version": {
"version": "1.0.0"
}
}
Hope this helps.
Madhan
From: Jo MailingList <[email protected]>
Reply-To: "[email protected]" <[email protected]>
Date: Tuesday, January 2, 2018 at 9:54 AM
To: "[email protected]" <[email protected]>
Subject: Relating to an entity using uniqueAttributes
Hello everyone,
I am trying to create a database that in my type model has a reference to a
repository. I want to create it by sending a message to the ATLAS_HOOK topic to
the Kafka broker as follows:
{
"version": {
"version": "1.0.0"
},
"message": {
"entities": [{
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
"jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id": "-14568879465268",
"version": 0,
"typeName": "database",
"state": "ACTIVE"
},
"typeName": "database",
"values": {
"qualifiedName": "Testdb5",
"repository": {
"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id"
"version": 0,
"typeName": "repository",
"uniqueAttributes": {
"qualifiedName": "repository01"
},
"state": "ACTIVE"
},
"owner": "owner",
"description": "Test db7",
"name": "Testdb7",
"dbName": "Test7",
"origin_app": "origin",
"origin_username": "externalusername"
},
"traitNames": [],
"traits": {}
}],
"type": "ENTITY_CREATE",
"user": "user"
}
}
The database type expects a repository attribute of the type repository as a
reference. It works when I give the guid of the reference directly but when
trying my example it fails and it seems to me that it does not process the
information I provide as uniqueAttributes. What I see in the application.log is
the following stacktrace:
org.apache.atlas.exception.AtlasBaseException: ObjectId is not valid
AtlasObjectId{guid='null', typeName='repository', uniqueAttributes={}}
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityGraphDiscoveryV1.visitReference(AtlasEntityGraphDiscoveryV1.java:186)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityGraphDiscoveryV1.visitAttribute(AtlasEntityGraphDiscoveryV1.java:234)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityGraphDiscoveryV1.visitEntity(AtlasEntityGraphDiscoveryV1.java:327)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityGraphDiscoveryV1.walkEntityGraph(AtlasEntityGraphDiscoveryV1.java:354)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityGraphDiscoveryV1.discover(AtlasEntityGraphDiscoveryV1.java:142)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityGraphDiscoveryV1.discoverEntities(AtlasEntityGraphDiscoveryV1.java:69)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.preCreateOrUpdate(AtlasEntityStoreV1.java:523)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.createOrUpdate(AtlasEntityStoreV1.java:174)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.createOrUpdate(AtlasEntityStoreV1.java:193)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$$FastClassBySpringCGLIB$$80c00649.invoke(<generated>)
This message makes me suspicious that Atlas is not processing my input message
correctly and not reading my uniqueAttributes information.
Does someone have a hint of what I am doing wrong or if this is a currrent
issue inside Atlas or it is not a supported feature right now?
Thank you very much,
Best regards,
Joan
