----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70600/#review215292 -----------------------------------------------------------
addons/models/1000-Hadoop/1090-spark_model.json Lines 48 (patched) <https://reviews.apache.org/r/70600/#comment301956> Ensure the fields added recently are included. ```json { "enumDefs": [], "structDefs": [], "classificationDefs": [], "entityDefs": [ { "category": "ENTITY", "guid": "a17bded7-bd62-431c-841d-ae34be4f5e10", "createdBy": "admin", "updatedBy": "admin", "createTime": 1557859427155, "updateTime": 1557859427155, "version": 1, "name": "spark_table", "description": "spark_table", "typeVersion": "1.0", "attributeDefs": [ { "name": "qualifiedName", "typeName": "string", "isOptional": false, "cardinality": "SINGLE", "valuesMinCount": 1, "valuesMaxCount": 1, "isUnique": true, "isIndexable": true, "includeInNotification": false }, { "name": "tableType", "typeName": "string", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "schemaDesc", "typeName": "string", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "provider", "typeName": "string", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "partitionProvider", "typeName": "string", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "partitionColumnNames", "typeName": "array<string>", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "bucketSpec", "typeName": "map<string,string>", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "owner", "typeName": "string", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "ownerType", "typeName": "string", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "createTime", "typeName": "date", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "parameters", "typeName": "map<string,string>", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "comment", "typeName": "string", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "unsupportedFeatures", "typeName": "array<string>", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "viewOriginalText", "typeName": "string", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false } ], "superTypes": [ "DataSet" ], "subTypes": [] } ], "relationshipDefs": [] } ``` addons/models/1000-Hadoop/1090-spark_model.json Lines 265 (patched) <https://reviews.apache.org/r/70600/#comment301955> The value I see from querying a local API instance seems to contain other fields like (description, qualifiedName etc) E.g. curl http://localhost:21000/api/atlas/v2/types/typedefs/?name=spark_ml_model' ```json { "enumDefs": [], "structDefs": [], "classificationDefs": [], "entityDefs": [ { "category": "ENTITY", "guid": "05ab97cd-16cc-4c5e-bf12-e23498c82886", "createdBy": "admin", "updatedBy": "admin", "createTime": 1557859427181, "updateTime": 1557859427181, "version": 1, "name": "spark_ml_model", "description": "spark_ml_model", "typeVersion": "1.0", "attributeDefs": [ { "name": "qualifiedName", "typeName": "string", "isOptional": false, "cardinality": "SINGLE", "valuesMinCount": 1, "valuesMaxCount": 1, "isUnique": true, "isIndexable": true, "includeInNotification": false }, { "name": "description", "typeName": "string", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false }, { "name": "extra", "typeName": "string", "isOptional": true, "cardinality": "SINGLE", "valuesMinCount": 0, "valuesMaxCount": 1, "isUnique": false, "isIndexable": false, "includeInNotification": false } ], "superTypes": [ "DataSet" ], "subTypes": [] } ], "relationshipDefs": [] } ``` - Arun Mahadevan On May 15, 2019, 7:39 p.m., Aadarsh Jajodia wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70600/ > ----------------------------------------------------------- > > (Updated May 15, 2019, 7:39 p.m.) > > > Review request for atlas, Ankur Gupta, Arun Mahadevan, Ashutosh Mestry, > Jungtaek Lim, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > ------- > > This is an initial mode file for the SPARK models. Please refer to the file > added here. This file is a GET call on typeDefs on a cluster which had SAC > enabled and SAC has created the models. The SAC version used was > https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev > > > Diffs > ----- > > addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION > > > Diff: https://reviews.apache.org/r/70600/diff/1/ > > > Testing > ------- > > > File Attachments > ---------------- > > SAC Models > > https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt > > > Thanks, > > Aadarsh Jajodia > >
