[
https://issues.apache.org/jira/browse/ATLAS-346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hemanth Yamijala updated ATLAS-346:
-----------------------------------
Attachment: ATLAS-346.patch
The NotificationConsumer thread which consumes messages from Kafka uses the
AtlasClient to connect and send the messages to Atlas. The threads are started
as part of the initialization of the Atlas server, but the server itself
becomes ready to listen to requests only at the end of the initialization. If
the consumer starts consuming messages before, it could try to send messages to
the server, which is not yet listening and thus fail. This is seen from the
ConnectionRefused exception in the stack trace.
The attached patch adds retries to the NotificationConsumer thread with a 1s
sleep interval between attempts.
> Atlas server loses messages sent from Hive hook if restarted after unclean
> shutdown
> -----------------------------------------------------------------------------------
>
> Key: ATLAS-346
> URL: https://issues.apache.org/jira/browse/ATLAS-346
> Project: Atlas
> Issue Type: Bug
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Critical
> Attachments: ATLAS-346.patch, hive_server2_import_bug_snip.txt
>
>
> * Start Atlas server pointed to an external Kafka instance
> * Configure HiveServer2 with Atlas hook pointing to the same Kafka instance.
> * Run a hive DDL script like
> {code}
> for i in `seq 1 25`; do ./bin/beeline -u jdbc:hive2://localhost:10000 -n
> ${user} -p ${pass} -e "create table tbl${i} (col${i}1 int, col${i}2
> string);"; done
> {code}
> * While the script is executing, kill -9 the Atlas server
> * Let the script complete.
> * Verify that all the events are added to the ATLAS_HOOK topic.
> * Verify if the tables are added to Atlas.
> The observation is that the Kafka topic has all relevant messages, but the
> tables aren't added to Atlas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
