Sergio Peña created HIVE-16164:
----------------------------------
Summary: Provide mechanism for passing HMS notification ID between
transactional and non-transactional listeners.
Key: HIVE-16164
URL: https://issues.apache.org/jira/browse/HIVE-16164
Project: Hive
Issue Type: Improvement
Components: Metastore
Reporter: Sergio Peña
Assignee: Sergio Peña
The HMS DB notification listener currently stores an event ID on the HMS
backend DB so that external applications (such as backup apps) can request
incremental notifications based on the last event ID requested.
The HMS DB notification and backup applications are asynchronous. However,
there are sometimes that applications may be required to be in sync with the
latest HMS event in order to process an action. These applications will provide
a listener implementation that is called by the HMS after an HMS transaction
happened.
The problem is that the listener running after the transaction (or during the
non-transactional context) may need the DB event ID in order to sync all events
happened previous to that event ID, but this ID is never passed to the
non-transactional listeners.
We can pass this event information through the EnvironmentContext found on each
ListenerEvent implementations (such as CreateTableEvent), and send the
EnvironmentContext to the non-transactional listeners to get the event ID.
The DbNotificactionListener already knows the event ID after calling the
ObjectStore.addNotificationEvent(). We just need to set this event ID to the
EnvironmentContext from each of the event notifications and make sure that this
EnvironmentContext is sent to the non-transactional listeners.
Here's the code example when creating a table on {{create_table_core}}:
{noformat}
ms.createTable(tbl);
if (transactionalListeners.size() > 0) {
CreateTableEvent createTableEvent = new CreateTableEvent(tbl, true, this);
createTableEvent.setEnvironmentContext(envContext);
for (MetaStoreEventListener transactionalListener : transactionalListeners)
{
transactionalListener.onCreateTable(createTableEvent); // <- Here
the notification ID is generated
}
}
success = ms.commitTransaction();
} finally {
if (!success) {
ms.rollbackTransaction();
if (madeDir) {
wh.deleteDir(tblPath, true);
}
}
for (MetaStoreEventListener listener : listeners) {
CreateTableEvent createTableEvent =
new CreateTableEvent(tbl, success, this);
createTableEvent.setEnvironmentContext(envContext);
listener.onCreateTable(createTableEvent); // <- Here
we would like to consume notification ID
}
{noformat}
We could use a specific key name that will be used on the EnvironmentContext,
such as DB_NOTIFICATION_EVENT_ID.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
