[ https://issues.apache.org/jira/browse/ATLAS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17240985#comment-17240985 ]
Deep Singh commented on ATLAS-4043: ----------------------------------- Patch Submitted for review. https://reviews.apache.org/r/73049/ > Atlas import-kafka.sh does not import "__consumer_offsets" topic > ---------------------------------------------------------------- > > Key: ATLAS-4043 > URL: https://issues.apache.org/jira/browse/ATLAS-4043 > Project: Atlas > Issue Type: Bug > Components: atlas-core > Affects Versions: 2.1.0 > Reporter: Umesh Padashetty > Assignee: Deep Singh > Priority: Major > Attachments: Screenshot 2020-11-20 at 11.23.47 PM.png > > > Even though kafka-topics.sh --list, lists the __consumer_offsets topic as > below, this topic alone is not imported into Atlas when import-kafka.sh > script is run > {code:java} > ATLAS_ENTITIES > ATLAS_HOOK > ATLAS_SPARK_HOOK > __consumer_offsets > connect-configs > connect-offsets > connect-status {code} > I did not find anything unusual in atlas/application.log. This was working > fine before, seems like a regression. > As per official Kafka documentation, __consumer_offsets is used to store > information about committed offsets for each topic:partition per group of > consumers (groupID). It is compacted topic, so data will be periodically > compressed and only latest offsets information available. > Since __consumer_offsets has __ in the beginning, I have created a topic > starting with __ and ran the import-kafka script. And it works fine. So that > leaves us with having issue with __consumer_offsets alone. -- This message was sent by Atlassian Jira (v8.3.4#803005)