[
https://issues.apache.org/jira/browse/ATLAS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184647#comment-16184647
]
Pierre Padovani edited comment on ATLAS-2178 at 9/29/17 9:00 PM:
-----------------------------------------------------------------
Update: If I set the property 'atlas.ui.editable.entity.types=*' to allow for
custom types to be created from the UI, and create two entities (one on each
side of the CustomerOwned defined relationship type) I can properly define the
relationship on each side, and the data is set correctly.
I would assume that this is due to the UI using the V1 API rather than making a
call to the V2 Relationship API. That being said, my assumption was that if I
set the legacy flag on the RelationshipDef the attributes that correspond to
the field names in each end would be set correctly when the relationship
instance was created. Tracing through the code during a create relationship
call via the REST api, I see nothing that honors the legacy flag. (unless it is
happening under the covers in the titan code.)
was (Author: ppadovani):
Update: If I set the property 'atlas.ui.editable.entity.types=*' to allow for
custom types to be created from the UI, and create two entities (one on each
side of the CustomerOwned defined relationship type) I can properly define the
relationship on each side, and the data is set correctly.
{code:java}
{
"enumDefs": [],
"structDefs": [
{
"name": "FingerprintSpecification",
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "tablesWithMetadataGlobs",
"typeName": "array<string>",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "tablesListGlobs",
"typeName": "array<string>",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "PfKeyDefinition",
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "tableName",
"typeName": "string",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "columns",
"typeName": "array<string>",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "ComparisonFilter",
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "compareColumnNullability",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareColumnRemarks",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareColumnType",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareColumnSize",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "comparePrimaryKey",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareTableRemarks",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareRelationships",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
}
]
},
{
"name": "ScoreSpecification",
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "missingPrimaryKeys",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "changedColumnTypes",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "changedNullability",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "changedColumnRemarks",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "missingTables",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "missingForeignKeys",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "unexpectedTables",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "changedColumnSizes",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "unexpectedColumns",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "changedForeignKeys",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "changedTableRemarks",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "missingColumns",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "changedPrimaryKeys",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
}
],
"classificationDefs": [],
"entityDefs": [
{
"name": "Customer",
"superTypes": [
"Referenceable",
"Asset"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "systems",
"typeName": "array<SorSystem>",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "extractProcesses",
"typeName": "array<SorExtraction>",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "collectedDataSets",
"typeName": "array<CollectedDs>",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "SorSystem",
"superTypes": [
"DataSet"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "customer",
"typeName": "Customer",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "tables",
"typeName": "array<SorTable>",
"cardinality": "SINGLE",
"constraints": [
{
"type": "ownedRef"
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "Table",
"superTypes": [
"DataSet"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "schema",
"typeName": "string",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "fullName",
"typeName": "string",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "tableType",
"typeName": "string",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "columns",
"typeName": "array<Column>",
"cardinality": "SINGLE",
"constraints": [
{
"type": "ownedRef"
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "SorTable",
"superTypes": [
"Table"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "system",
"typeName": "SorSystem",
"cardinality": "SINGLE",
"constraints": [
{
"type": "inverseRef",
"params": {
"attribute": "tables"
}
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
}, {
"name": "DiscoveryPackTable",
"superTypes": [
"Table"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "discoveryPack",
"typeName": "DiscoveryPack",
"cardinality": "SINGLE",
"constraints": [
{
"type": "inverseRef",
"params": {
"attribute": "tables"
}
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
}, {
"name": "ExtractDefinitionTable",
"superTypes": [
"Table"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "extractDefinition",
"typeName": "ExtractDefinition",
"cardinality": "SINGLE",
"constraints": [
{
"type": "inverseRef",
"params": {
"attribute": "tables"
}
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
}, {
"name": "CollectedTable",
"superTypes": [
"Table"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "collectedTableTopic",
"typeName": "CollectedTableTopic",
"cardinality": "SINGLE",
"constraints": [
{
"type": "inverseRef",
"params": {
"attribute": "table"
}
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
}, {
"name": "Column",
"superTypes": [
"DataSet"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "type",
"typeName": "string",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "size",
"typeName": "int",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "nullable",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "table",
"typeName": "Table",
"cardinality": "SINGLE",
"constraints": [
{
"type": "inverseRef",
"params": {
"attribute": "columns"
}
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "CollectedDs",
"superTypes": [
"DataSet"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "customer",
"typeName": "Customer",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "collectedTableTopics",
"typeName": "array<CollectedTableTopic>",
"cardinality": "SINGLE",
"constraints": [
{
"type": "ownedRef"
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "CollectedTableTopic",
"superTypes": [
"kafka_topic",
"DataSet"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "collectedDs",
"typeName": "CollectedDs",
"cardinality": "SINGLE",
"constraints": [
{
"type": "inverseRef",
"params": {
"attribute": "collectedTableTopics"
}
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "table",
"typeName": "CollectedTable",
"cardinality": "SINGLE",
"constraints": [
{
"type": "ownedRef"
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "SorExtraction",
"superTypes": [
"Process"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "customer",
"typeName": "Customer",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "extractDefinition",
"typeName": "ExtractDefinition",
"cardinality": "SINGLE",
"constraints": [
{
"type": "ownedRef"
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "processes",
"typeName": "array<SorTableExtraction>",
"cardinality": "SINGLE",
"constraints": [
{
"type": "ownedRef"
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "SorTableExtraction",
"superTypes": [
"Process"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "watermark_fieldset",
"typeName": "array<string>",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "path",
"typeName": "array<string>",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "sorExtraction",
"typeName": "SorExtraction",
"cardinality": "SINGLE",
"constraints": [
{
"type": "inverseRef",
"params": {
"attribute": "processes"
}
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "ExtractDefinition",
"superTypes": [
"DataSet"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "discoveryPack",
"typeName": "DiscoveryPack",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "sorExtraction",
"typeName": "SorExtraction",
"cardinality": "SINGLE",
"constraints": [
{
"type": "inverseRef",
"params": {
"attribute": "extractDefinition"
}
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "tables",
"typeName": "array<ExtractDefinitionTable>",
"cardinality": "SINGLE",
"constraints": [
{
"type": "ownedRef"
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "DiscoveryPack",
"superTypes": [
"DataSet"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "extractDefinitions",
"typeName": "array<ExtractDefinition>",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "fingerprintSpecification",
"typeName": "FingerprintSpecification",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "scoreSpecification",
"typeName": "ScoreSpecification",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "comparisonFilter",
"typeName": "ComparisonFilter",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "discoveryProcesses",
"typeName": "array<DiscoveryProcess>",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "tables",
"typeName": "array<DiscoveryPackTable>",
"cardinality": "SINGLE",
"constraints": [
{
"type": "ownedRef"
}
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "DiscoveryProcess",
"superTypes": [
"Process"
],
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "discoveryPack",
"typeName": "DiscoveryPack",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
}
],
"relationshipDefs": [
{
"name": "CustomerSystems",
"typeVersion": "1.0",
"relationshipCategory": "AGGREGATION",
"endDef1": {
"type": "Customer",
"name": "systems",
"isContainer": true,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"endDef2": {
"type": "SorSystem",
"name": "customer",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
},
{
"name": "CustomerExtractProcesses",
"typeVersion": "1.0",
"relationshipCategory": "AGGREGATION",
"endDef1": {
"type": "Customer",
"name": "extractProcesses",
"isContainer": true,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"endDef2": {
"type": "SorExtraction",
"name": "customer",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
},
{
"name": "CustomerCollectedDataSets",
"typeVersion": "1.0",
"relationshipCategory": "AGGREGATION",
"endDef1": {
"type": "Customer",
"name": "collectedDataSets",
"isContainer": true,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"endDef2": {
"type": "CollectedDs",
"name": "customer",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
},
{
"name": "SorSystemTables",
"typeVersion": "1.0",
"relationshipCategory": "AGGREGATION",
"endDef1": {
"type": "SorSystem",
"name": "tables",
"isContainer": true,
"cardinality": "SET",
"isLegacyAttribute": true
},
"endDef2": {
"type": "SorTable",
"name": "system",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
},
{
"name": "ExtractDefinitionTables",
"typeVersion": "1.0",
"relationshipCategory": "COMPOSITION",
"endDef1": {
"type": "ExtractDefinition",
"name": "tables",
"isContainer": true,
"cardinality": "SET",
"isLegacyAttribute": true
},
"endDef2": {
"type": "ExtractDefinitionTable",
"name": "extractDefinition",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
},
{
"name": "DiscoveryPackTables",
"typeVersion": "1.0",
"relationshipCategory": "COMPOSITION",
"endDef1": {
"type": "DiscoveryPack",
"name": "tables",
"isContainer": true,
"cardinality": "SET",
"isLegacyAttribute": true
},
"endDef2": {
"type": "DiscoveryPackTable",
"name": "discoveryPack",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
},
{
"name": "SorExtractionExtractDefinition",
"typeVersion": "1.0",
"relationshipCategory": "COMPOSITION",
"endDef1": {
"type": "SorExtraction",
"name": "extractDefinition",
"isContainer": true,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"endDef2": {
"type": "ExtractDefinition",
"name": "sorExtraction",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
},
{
"name": "ExtractDefinitionDiscoveryPack",
"typeVersion": "1.0",
"relationshipCategory": "AGGREGATION",
"endDef1": {
"type": "ExtractDefinition",
"name": "discoveryPack",
"isContainer": true,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"endDef2": {
"type": "DiscoveryPack",
"name": "extractDefinitions",
"isContainer": false,
"cardinality": "SET",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
},
{
"name": "TableColumns",
"typeVersion": "1.0",
"relationshipCategory": "COMPOSITION",
"endDef1": {
"type": "Table",
"name": "columns",
"isContainer": true,
"cardinality": "SET",
"isLegacyAttribute": true
},
"endDef2": {
"type": "Column",
"name": "table",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
},
{
"name": "ExtractProcesses",
"typeVersion": "1.0",
"relationshipCategory": "COMPOSITION",
"endDef1": {
"type": "SorExtraction",
"name": "processes",
"isContainer": true,
"cardinality": "SET",
"isLegacyAttribute": true
},
"endDef2": {
"type": "SorTableExtraction",
"name": "sorExtraction",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
},
{
"name": "CollectedTableTopics",
"typeVersion": "1.0",
"relationshipCategory": "COMPOSITION",
"endDef1": {
"type": "CollectedDs",
"name": "collectedTableTopics",
"isContainer": true,
"cardinality": "SET",
"isLegacyAttribute": true
},
"endDef2": {
"type": "CollectedTableTopic",
"name": "collectedDs",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
},
{
"name": "CollectedTableTopicTable",
"typeVersion": "1.0",
"relationshipCategory": "COMPOSITION",
"endDef1": {
"type": "CollectedTableTopic",
"name": "table",
"isContainer": true,
"cardinality": "SET",
"isLegacyAttribute": true
},
"endDef2": {
"type": "CollectedTable",
"name": "collectedTableTopic",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
},
{
"name": "DiscoveryProcessDiscoveryPack",
"typeVersion": "1.0",
"relationshipCategory": "ASSOCIATION",
"endDef1": {
"type": "DiscoveryProcess",
"name": "discoveryPack",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": true
},
"endDef2": {
"type": "DiscoveryPack",
"name": "discoveryProcesses",
"isContainer": false,
"cardinality": "SET",
"isLegacyAttribute": true
},
"propagateTags": "ONE_TO_TWO"
}
]
}
{code}
I would assume that this is due to the UI using the V1 API rather than making a
call to the V2 Relationship API. That being said, my assumption was that if I
set the legacy flag on the RelationshipDef the attributes that correspond to
the field names in each end would be set correctly when the relationship
instance was created. Tracing through the code during a create relationship
call via the REST api, I see nothing that honors the legacy flag. (unless it is
happening under the covers in the titan code.)
> Type Inheritance in conjunction with RelationshipDef fails to set legacy
> attributes
> -----------------------------------------------------------------------------------
>
> Key: ATLAS-2178
> URL: https://issues.apache.org/jira/browse/ATLAS-2178
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
> Affects Versions: 1.0.0
> Reporter: Pierre Padovani
> Attachments: entities.json, rel1.json, rel2.json, rel3.json,
> rel4.json, rel5.json, types.json
>
>
> If a type inherits from another that has an old style reference attribute,
> and a new style RelationshipDef defined, creating the relationship instance
> fails to set the legacy attributes and in some cases fails to create the
> relationship itself.
> Steps to reproduce:
> * create a type, say my_table, that inherits from the hive_table
> * Create an instance of the new type my_table
> * Create an instance of a hive_column
> * Create the containment relationship between the two models.
> This behavior causes odd and inconsistent behavior and fully breaks lineage
> reports for process types that inherit from the built in Process.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
