[ https://issues.apache.org/jira/browse/ATLAS-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17391997#comment-17391997 ]
ASF subversion and git services commented on ATLAS-4164: -------------------------------------------------------- Commit 16759fabb1e397169633fff05b4f4599ac3707b6 in atlas's branch refs/heads/branch-2.0 from Radhika Kundam [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=16759fa ] ATLAS-4164: [Atlas: Spooling] Tables created after spooling are created before the spooled tables when there is multiple frequent restart in kafka brokers Signed-off-by: Sarath Subramanian <sar...@apache.org> (cherry picked from commit 4100684fa3f63cb2a6267ab24051002ce38de017) > [Atlas: Spooling] Tables created after spooling are created before the > spooled tables when there is multiple frequent restart in kafka brokers > ---------------------------------------------------------------------------------------------------------------------------------------------- > > Key: ATLAS-4164 > URL: https://issues.apache.org/jira/browse/ATLAS-4164 > Project: Atlas > Issue Type: Bug > Components: atlas-core > Affects Versions: 2.1.0 > Reporter: Dharshana M Krishnamoorthy > Assignee: Radhika Kundam > Priority: Major > Labels: spool > Fix For: 3.0.0 > > Attachments: ATLAS-4164-Spooling-Status.patch > > > Scenario: > * Stop kafka broker > * Create a few (20) tables save the prefix (abc_table_1, abc_table_2, ... > abc_table_n) > * Make sure the data is spooled > * Start kafka and create a few more tables (xyz_table_1, xyz_table_2, ... > xyz_table_n) > * Wait for 5 mins for the tables to reflect in atlas > In this case we expect all the abc_table_* to be created before xyz_table_1, > meaning all the spooled tables are created before the tables that are created > after spooling. > > Observation: > createTime of some spooled tables is greater than the create time of the > xyz_table_1 > > Sample data: > createTime for tables that are spooled: > {code:java} > [1613573518284, 1613573531470, 1613573531861, 1613573529446, 1613573543253, > 1613573525390, 1613573525950, 1613573517796, 1613573518284, 1613573522629, > 1613573513524, 1613573524856, 1613573518992, 1613573519477, 1613573519947, > 1613573521737, 1613573514066, 1613573514555, 1613573515065, > 1613573515605]{code} > createTime for tables that are created after spooling: > {code:java} > [1613573540582, 1613573541300, 1613573551691, 1613573552628, 1613573553356, > 1613573555478, 1613573556275, 1613573556940, 1613573557763, 1613573558659, > 1613573560673, 1613573561363, 1613573562310, 1613573563096, 1613573564004, > 1613573566533, 1613573567602, 1613573568439, 1613573569379, 1613573570202] > {code} > We expect all spooled tables to have createTime smaller than the tables > created after spooling. > But *1613573543253 (Spooled tabled create time) is greater than 1613573540582 > (table created after spooling)* > which means, the table created after spooling is created before spooled table -- This message was sent by Atlassian Jira (v8.3.4#803005)