Dharshana M Krishnamoorthy created ATLAS-4153: -------------------------------------------------
Summary: [Atlas: Spooling] The order of the entities created in atlas is not same as the order created in hive Key: ATLAS-4153 URL: https://issues.apache.org/jira/browse/ATLAS-4153 Project: Atlas Issue Type: Bug Components: atlas-core Reporter: Dharshana M Krishnamoorthy *Steps to re-produce:* * Stop kafka broker * Create a few (20) tables save the prefix (abc_table_1, abc_table_2, ... abc_table_n) * Make sure the data is spooled * Start kafka and create a few more tables (xyz_table_1, xyz_table_2, ... xyz_table_n) * Wait for 5 mins for the tables to reflect in atlas * Fire a basic search with abc_* prefix and verify the tables are created * Collect the createdTime of all the table and verify the order Here tables were created with prefix btwxb_table. Guid: a55ca87c-1122-4ed4-9812-5047cc8a7968 of *btwxb_table_6* has "createTime": *1612959902106*, {code:java} { "referredEntities": {...}, "entity": { "typeName": "hive_table", "attributes": { "owner": "hrt_qa", "temporary": false, "lastAccessTime": 1612959527000, "aliases": null, "replicatedTo": null, "userDescription": null, "replicatedFrom": null, "qualifiedName": "default.btwxb_table_6@cm", "displayName": null, "columns": [ { "guid": "3cf7b33b-7fb7-4de4-a967-7f89410b3a00", "typeName": "hive_column" }, { "guid": "7c5b72ae-e100-467a-b3b2-caade0b7a7a1", "typeName": "hive_column" } ], "description": null, "viewExpandedText": null, "tableType": "MANAGED_TABLE", "sd": { "guid": "7a1e2b04-f786-4f42-a3c4-31a0ad872256", "typeName": "hive_storagedesc" }, "createTime": 1612959527000, "name": "btwxb_table_6", "comment": null, "partitionKeys": [], "parameters": {...}, "retention": 0, "viewOriginalText": null, "db": { "guid": "a8a3227f-ee7a-49c2-8f9e-98534a3146eb", "typeName": "hive_db" } }, "guid": "a55ca87c-1122-4ed4-9812-5047cc8a7968", "isIncomplete": false, "status": "ACTIVE", "createdBy": "hive", "updatedBy": "hrt_qa", "createTime": 1612959902106, "updateTime": 1612959911756, "version": 0, "relationshipAttributes": {...}, "labels": [] } } {code} Guid: 70738a7e-d29f-4234-82e7-adbdc4feaed6 of *btwxb_table_1* has "createTime": *1612959903892*, {code:java} { "referredEntities": {...}, "entity": { "typeName": "hive_table", "attributes": { "owner": "hrt_qa", "temporary": false, "lastAccessTime": 1612959523000, "aliases": null, "replicatedTo": null, "userDescription": null, "replicatedFrom": null, "qualifiedName": "default.btwxb_table_1@cm", "displayName": null, "columns": [ { "guid": "c283ee8c-d292-406a-a062-d81b12614636", "typeName": "hive_column" }, { "guid": "cbb19940-e513-4295-a145-5b2d6691ea13", "typeName": "hive_column" } ], "description": null, "viewExpandedText": null, "tableType": "MANAGED_TABLE", "sd": { "guid": "69d7f6fc-8194-412f-a723-5b142ade1c31", "typeName": "hive_storagedesc" }, "createTime": 1612959523000, "name": "btwxb_table_1", "comment": null, "partitionKeys": [], "parameters": {...}, "retention": 0, "viewOriginalText": null, "db": { "guid": "a8a3227f-ee7a-49c2-8f9e-98534a3146eb", "typeName": "hive_db" } }, "guid": "70738a7e-d29f-4234-82e7-adbdc4feaed6", "isIncomplete": false, "status": "ACTIVE", "createdBy": "hive", "updatedBy": "hrt_qa", "createTime": 1612959903892, "updateTime": 1612959919910, "version": 0, "relationshipAttributes": {...}, "labels": [] } } {code} *btwxb_table_1* has "createTime": *1612959903892*, and *btwxb_table_6* has "createTime": *1612959902106*, which is smaller than *btwxb_table_1's* createTime. Which means it is created before table_6 is created before table_1 in atlas -- This message was sent by Atlassian Jira (v8.3.4#803005)