[jira] [Updated] (ATLAS-1119) Add retries for edge label creation

[Suma Shivaprasad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated ATLAS-1119:

Attachment: ATLAS-1119.patch

> Add retries for edge label creation
> ---
>
> Key: ATLAS-1119
> URL: https://issues.apache.org/jira/browse/ATLAS-1119
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-1119.patch
>
>
> Concurrent operations will fail when trying to create edge label which tries 
> to define a new property key for the edge label if it doesnt exist. 
> {noformat}
> 2016-08-15 04:08:28,475 DEBUG - [pool-8-thread-1:] ~ Adding a new trait=PII 
> for entity=b1630277-e8dc-4fcf-94af-c0c8b0b51c09 
> (GraphBackedMetadataRepository:222)
> 2016-08-15 04:08:28,476 DEBUG - [pool-8-thread-2:] ~ Adding a new trait=PII 
> for entity=5e335fed-4a74-496d-8521-1678caf85813 
> (GraphBackedMetadataRepository:222)
> 2016-08-15 04:08:28,477 DEBUG - [pool-8-thread-1:] ~ Finding vertex with 
> __guid = b1630277-e8dc-4fcf-94af-c0c8b0b51c09,  (GraphHelper:182)
> 2016-08-15 04:08:28,477 DEBUG - [pool-8-thread-2:] ~ Finding vertex with 
> __guid = 5e335fed-4a74-496d-8521-1678caf85813,  (GraphHelper:182)
> 2016-08-15 04:08:28,477 DEBUG - [pool-8-thread-3:] ~ Executing dsl 
> query=hive_table (GraphBackedDiscoveryService:133)
> 2016-08-15 04:08:28,483 DEBUG - [pool-8-thread-2:] ~ Found a vertex 
> vertex[id=8704 type=hive_database guid=5e335fed-4a74-496d-8521-1678caf85813] 
> with __guid = 5e335fed-4a74-496d-8521-1678caf85813,  (GraphHelper:192)
> 2016-08-15 04:08:28,484 DEBUG - [pool-8-thread-2:] ~ Creating vertex for type 
> PII id null (GraphHelper:97)
> 2016-08-15 04:08:28,485 DEBUG - [pool-8-thread-2:] ~ Setting property 
> __typeName = "PII" to vertex[id=8960 type=null guid=null] (GraphHelper:290)
> 2016-08-15 04:08:28,487 DEBUG - [pool-8-thread-2:] ~ Set property __typeName 
> = "PII" to vertex[id=8960 type=null guid=null] (GraphHelper:300)
> 2016-08-15 04:08:28,487 DEBUG - [pool-8-thread-2:] ~ Setting property __state 
> = "ACTIVE" to vertex[id=8960 type=PII guid=null] (GraphHelper:290)
> 2016-08-15 04:08:28,488 DEBUG - [pool-8-thread-2:] ~ Set property __state = 
> "ACTIVE" to vertex[id=8960 type=PII guid=null] (GraphHelper:300)
> 2016-08-15 04:08:28,488 DEBUG - [pool-8-thread-1:] ~ Found a vertex 
> vertex[id=8448 type=hive_database guid=b1630277-e8dc-4fcf-94af-c0c8b0b51c09] 
> with __guid = b1630277-e8dc-4fcf-94af-c0c8b0b51c09,  (GraphHelper:192)
> 2016-08-15 04:08:28,488 DEBUG - [pool-8-thread-2:] ~ Setting property 
> __timestamp = "1471234108488" to vertex[id=8960 type=PII guid=null] 
> (GraphHelper:290)
> 2016-08-15 04:08:28,489 DEBUG - [pool-8-thread-1:] ~ Creating vertex for type 
> PII id null (GraphHelper:97)
> 2016-08-15 04:08:28,490 DEBUG - [pool-8-thread-2:] ~ Set property __timestamp 
> = "1471234108488" to vertex[id=8960 type=PII guid=null] (GraphHelper:300)
> 2016-08-15 04:08:28,490 DEBUG - [pool-8-thread-2:] ~ Setting property 
> __modificationTimestamp = "1471234108488" to vertex[id=8960 type=PII 
> guid=null] (GraphHelper:290)
> 2016-08-15 04:08:28,490 DEBUG - [pool-8-thread-1:] ~ Setting property 
> __typeName = "PII" to vertex[id=9216 type=null guid=null] (GraphHelper:290)
> 2016-08-15 04:08:28,490 DEBUG - [pool-8-thread-2:] ~ Set property 
> __modificationTimestamp = "1471234108488" to vertex[id=8960 type=PII 
> guid=null] (GraphHelper:300)
> 2016-08-15 04:08:28,491 DEBUG - [pool-8-thread-2:] ~ created vertex 
> vertex[id=8960 type=PII guid=null] for trait PII 
> (TypedInstanceToGraphMapper:665)
> 2016-08-15 04:08:28,492 DEBUG - [pool-8-thread-2:] ~ Mapping instance 
> struct[type=PII] to vertex vertex[id=8960 type=PII guid=null] 
> (TypedInstanceToGraphMapper:181)
> 2016-08-15 04:08:28,492 DEBUG - [pool-8-thread-2:] ~ Setting property 
> __modificationTimestamp = "1471234108488" to vertex[id=8960 type=PII 
> guid=null] (GraphHelper:290)
> 2016-08-15 04:08:28,492 DEBUG - [pool-8-thread-2:] ~ Finding edges for 
> vertex[id=8960 type=PII guid=null] with label hive_database.PII 
> (GraphHelper:201)
> 2016-08-15 04:08:28,493 DEBUG - [pool-8-thread-2:] ~ Adding edge for 
> vertex[id=8704 type=hive_database guid=5e335fed-4a74-496d-8521-1678caf85813] 
> -> label hive_database.PII -> vertex[id=8960 type=PII guid=null] 
> (GraphHelper:122)
> 2016-08-15 04:08:28,493 DEBUG - [pool-8-thread-1:] ~ Set property __typeName 
> = "PII" to vertex[id=9216 type=null guid=null] (GraphHelper:300)
> 2016-08-15 04:08:28,494 DEBUG - [pool-8-thread-1:] ~ Setting property __state 
> = "ACTIVE" to vertex[id=9216 type=PII guid=null] (GraphHelper:290)
> 2016-08-15 04:08:28,495 DEBUG - [pool-8-thread-3:] ~ Query = hive_table limit 
> 10 offset 0  (GraphBackedDiscoverySe

[jira] [Commented] (ATLAS-1119) Add retries for edge label creation

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421820#comment-15421820
 ] 

Suma Shivaprasad commented on ATLAS-1119:
-

Adding retries in GraphHelper.getOrCreateEdge should ensure that the operation 
is retried before failing since on retry it should ideally get the created 
property type for the edge label once the other concurrent transaction is 
committed and should proceed further.

> Add retries for edge label creation
> ---
>
> Key: ATLAS-1119
> URL: https://issues.apache.org/jira/browse/ATLAS-1119
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
> Fix For: 0.8-incubating
>
>
> Concurrent operations will fail when trying to create edge label which tries 
> to define a new property key for the edge label if it doesnt exist. 
> {noformat}
> 2016-08-15 04:08:28,475 DEBUG - [pool-8-thread-1:] ~ Adding a new trait=PII 
> for entity=b1630277-e8dc-4fcf-94af-c0c8b0b51c09 
> (GraphBackedMetadataRepository:222)
> 2016-08-15 04:08:28,476 DEBUG - [pool-8-thread-2:] ~ Adding a new trait=PII 
> for entity=5e335fed-4a74-496d-8521-1678caf85813 
> (GraphBackedMetadataRepository:222)
> 2016-08-15 04:08:28,477 DEBUG - [pool-8-thread-1:] ~ Finding vertex with 
> __guid = b1630277-e8dc-4fcf-94af-c0c8b0b51c09,  (GraphHelper:182)
> 2016-08-15 04:08:28,477 DEBUG - [pool-8-thread-2:] ~ Finding vertex with 
> __guid = 5e335fed-4a74-496d-8521-1678caf85813,  (GraphHelper:182)
> 2016-08-15 04:08:28,477 DEBUG - [pool-8-thread-3:] ~ Executing dsl 
> query=hive_table (GraphBackedDiscoveryService:133)
> 2016-08-15 04:08:28,483 DEBUG - [pool-8-thread-2:] ~ Found a vertex 
> vertex[id=8704 type=hive_database guid=5e335fed-4a74-496d-8521-1678caf85813] 
> with __guid = 5e335fed-4a74-496d-8521-1678caf85813,  (GraphHelper:192)
> 2016-08-15 04:08:28,484 DEBUG - [pool-8-thread-2:] ~ Creating vertex for type 
> PII id null (GraphHelper:97)
> 2016-08-15 04:08:28,485 DEBUG - [pool-8-thread-2:] ~ Setting property 
> __typeName = "PII" to vertex[id=8960 type=null guid=null] (GraphHelper:290)
> 2016-08-15 04:08:28,487 DEBUG - [pool-8-thread-2:] ~ Set property __typeName 
> = "PII" to vertex[id=8960 type=null guid=null] (GraphHelper:300)
> 2016-08-15 04:08:28,487 DEBUG - [pool-8-thread-2:] ~ Setting property __state 
> = "ACTIVE" to vertex[id=8960 type=PII guid=null] (GraphHelper:290)
> 2016-08-15 04:08:28,488 DEBUG - [pool-8-thread-2:] ~ Set property __state = 
> "ACTIVE" to vertex[id=8960 type=PII guid=null] (GraphHelper:300)
> 2016-08-15 04:08:28,488 DEBUG - [pool-8-thread-1:] ~ Found a vertex 
> vertex[id=8448 type=hive_database guid=b1630277-e8dc-4fcf-94af-c0c8b0b51c09] 
> with __guid = b1630277-e8dc-4fcf-94af-c0c8b0b51c09,  (GraphHelper:192)
> 2016-08-15 04:08:28,488 DEBUG - [pool-8-thread-2:] ~ Setting property 
> __timestamp = "1471234108488" to vertex[id=8960 type=PII guid=null] 
> (GraphHelper:290)
> 2016-08-15 04:08:28,489 DEBUG - [pool-8-thread-1:] ~ Creating vertex for type 
> PII id null (GraphHelper:97)
> 2016-08-15 04:08:28,490 DEBUG - [pool-8-thread-2:] ~ Set property __timestamp 
> = "1471234108488" to vertex[id=8960 type=PII guid=null] (GraphHelper:300)
> 2016-08-15 04:08:28,490 DEBUG - [pool-8-thread-2:] ~ Setting property 
> __modificationTimestamp = "1471234108488" to vertex[id=8960 type=PII 
> guid=null] (GraphHelper:290)
> 2016-08-15 04:08:28,490 DEBUG - [pool-8-thread-1:] ~ Setting property 
> __typeName = "PII" to vertex[id=9216 type=null guid=null] (GraphHelper:290)
> 2016-08-15 04:08:28,490 DEBUG - [pool-8-thread-2:] ~ Set property 
> __modificationTimestamp = "1471234108488" to vertex[id=8960 type=PII 
> guid=null] (GraphHelper:300)
> 2016-08-15 04:08:28,491 DEBUG - [pool-8-thread-2:] ~ created vertex 
> vertex[id=8960 type=PII guid=null] for trait PII 
> (TypedInstanceToGraphMapper:665)
> 2016-08-15 04:08:28,492 DEBUG - [pool-8-thread-2:] ~ Mapping instance 
> struct[type=PII] to vertex vertex[id=8960 type=PII guid=null] 
> (TypedInstanceToGraphMapper:181)
> 2016-08-15 04:08:28,492 DEBUG - [pool-8-thread-2:] ~ Setting property 
> __modificationTimestamp = "1471234108488" to vertex[id=8960 type=PII 
> guid=null] (GraphHelper:290)
> 2016-08-15 04:08:28,492 DEBUG - [pool-8-thread-2:] ~ Finding edges for 
> vertex[id=8960 type=PII guid=null] with label hive_database.PII 
> (GraphHelper:201)
> 2016-08-15 04:08:28,493 DEBUG - [pool-8-thread-2:] ~ Adding edge for 
> vertex[id=8704 type=hive_database guid=5e335fed-4a74-496d-8521-1678caf85813] 
> -> label hive_database.PII -> vertex[id=8960 type=PII guid=null] 
> (GraphHelper:122)
> 2016-08-15 04:08:28,493 DEBUG - [pool-8-thread-1:] ~ Set property __typeName 
> = "PII" to vertex[id=9216 type=null guid=null] (GraphHelper:300)
> 2016-08-15 04:0

[jira] [Created] (ATLAS-1119) Add retries for edge label creation

[Suma Shivaprasad (JIRA)]

Suma Shivaprasad created ATLAS-1119:
---

 Summary: Add retries for edge label creation
 Key: ATLAS-1119
 URL: https://issues.apache.org/jira/browse/ATLAS-1119
 Project: Atlas
  Issue Type: Bug
Affects Versions: 0.8-incubating
Reporter: Suma Shivaprasad
Assignee: Suma Shivaprasad
 Fix For: 0.8-incubating


Concurrent operations will fail when trying to create edge label which tries to 
define a new property key for the edge label if it doesnt exist. 

{noformat}

2016-08-15 04:08:28,475 DEBUG - [pool-8-thread-1:] ~ Adding a new trait=PII for 
entity=b1630277-e8dc-4fcf-94af-c0c8b0b51c09 (GraphBackedMetadataRepository:222)
2016-08-15 04:08:28,476 DEBUG - [pool-8-thread-2:] ~ Adding a new trait=PII for 
entity=5e335fed-4a74-496d-8521-1678caf85813 (GraphBackedMetadataRepository:222)
2016-08-15 04:08:28,477 DEBUG - [pool-8-thread-1:] ~ Finding vertex with __guid 
= b1630277-e8dc-4fcf-94af-c0c8b0b51c09,  (GraphHelper:182)
2016-08-15 04:08:28,477 DEBUG - [pool-8-thread-2:] ~ Finding vertex with __guid 
= 5e335fed-4a74-496d-8521-1678caf85813,  (GraphHelper:182)
2016-08-15 04:08:28,477 DEBUG - [pool-8-thread-3:] ~ Executing dsl 
query=hive_table (GraphBackedDiscoveryService:133)
2016-08-15 04:08:28,483 DEBUG - [pool-8-thread-2:] ~ Found a vertex 
vertex[id=8704 type=hive_database guid=5e335fed-4a74-496d-8521-1678caf85813] 
with __guid = 5e335fed-4a74-496d-8521-1678caf85813,  (GraphHelper:192)
2016-08-15 04:08:28,484 DEBUG - [pool-8-thread-2:] ~ Creating vertex for type 
PII id null (GraphHelper:97)
2016-08-15 04:08:28,485 DEBUG - [pool-8-thread-2:] ~ Setting property 
__typeName = "PII" to vertex[id=8960 type=null guid=null] (GraphHelper:290)
2016-08-15 04:08:28,487 DEBUG - [pool-8-thread-2:] ~ Set property __typeName = 
"PII" to vertex[id=8960 type=null guid=null] (GraphHelper:300)
2016-08-15 04:08:28,487 DEBUG - [pool-8-thread-2:] ~ Setting property __state = 
"ACTIVE" to vertex[id=8960 type=PII guid=null] (GraphHelper:290)
2016-08-15 04:08:28,488 DEBUG - [pool-8-thread-2:] ~ Set property __state = 
"ACTIVE" to vertex[id=8960 type=PII guid=null] (GraphHelper:300)
2016-08-15 04:08:28,488 DEBUG - [pool-8-thread-1:] ~ Found a vertex 
vertex[id=8448 type=hive_database guid=b1630277-e8dc-4fcf-94af-c0c8b0b51c09] 
with __guid = b1630277-e8dc-4fcf-94af-c0c8b0b51c09,  (GraphHelper:192)
2016-08-15 04:08:28,488 DEBUG - [pool-8-thread-2:] ~ Setting property 
__timestamp = "1471234108488" to vertex[id=8960 type=PII guid=null] 
(GraphHelper:290)
2016-08-15 04:08:28,489 DEBUG - [pool-8-thread-1:] ~ Creating vertex for type 
PII id null (GraphHelper:97)
2016-08-15 04:08:28,490 DEBUG - [pool-8-thread-2:] ~ Set property __timestamp = 
"1471234108488" to vertex[id=8960 type=PII guid=null] (GraphHelper:300)
2016-08-15 04:08:28,490 DEBUG - [pool-8-thread-2:] ~ Setting property 
__modificationTimestamp = "1471234108488" to vertex[id=8960 type=PII guid=null] 
(GraphHelper:290)
2016-08-15 04:08:28,490 DEBUG - [pool-8-thread-1:] ~ Setting property 
__typeName = "PII" to vertex[id=9216 type=null guid=null] (GraphHelper:290)
2016-08-15 04:08:28,490 DEBUG - [pool-8-thread-2:] ~ Set property 
__modificationTimestamp = "1471234108488" to vertex[id=8960 type=PII guid=null] 
(GraphHelper:300)
2016-08-15 04:08:28,491 DEBUG - [pool-8-thread-2:] ~ created vertex 
vertex[id=8960 type=PII guid=null] for trait PII 
(TypedInstanceToGraphMapper:665)
2016-08-15 04:08:28,492 DEBUG - [pool-8-thread-2:] ~ Mapping instance 
struct[type=PII] to vertex vertex[id=8960 type=PII guid=null] 
(TypedInstanceToGraphMapper:181)
2016-08-15 04:08:28,492 DEBUG - [pool-8-thread-2:] ~ Setting property 
__modificationTimestamp = "1471234108488" to vertex[id=8960 type=PII guid=null] 
(GraphHelper:290)
2016-08-15 04:08:28,492 DEBUG - [pool-8-thread-2:] ~ Finding edges for 
vertex[id=8960 type=PII guid=null] with label hive_database.PII 
(GraphHelper:201)
2016-08-15 04:08:28,493 DEBUG - [pool-8-thread-2:] ~ Adding edge for 
vertex[id=8704 type=hive_database guid=5e335fed-4a74-496d-8521-1678caf85813] -> 
label hive_database.PII -> vertex[id=8960 type=PII guid=null] (GraphHelper:122)
2016-08-15 04:08:28,493 DEBUG - [pool-8-thread-1:] ~ Set property __typeName = 
"PII" to vertex[id=9216 type=null guid=null] (GraphHelper:300)
2016-08-15 04:08:28,494 DEBUG - [pool-8-thread-1:] ~ Setting property __state = 
"ACTIVE" to vertex[id=9216 type=PII guid=null] (GraphHelper:290)
2016-08-15 04:08:28,495 DEBUG - [pool-8-thread-3:] ~ Query = hive_table limit 
10 offset 0  (GraphBackedDiscoveryService:158)
2016-08-15 04:08:28,495 DEBUG - [pool-8-thread-3:] ~ Expression Tree = 
LimitExpression 10, 0
 ClassExpression hive_table
 (GraphBackedDiscoveryService:159)
2016-08-15 04:08:28,496 DEBUG - [pool-8-thread-3:] ~ Gremlin Query = L:{_var_0 
= [] as Set;g.V().has("__typeName", 
"hive_table").fill(_var_0);g.V().has("__superTypeNames", 
"hive_table").fill(_var_