[ https://issues.apache.org/jira/browse/HIVE-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15980864#comment-15980864 ]
Lefty Leverenz commented on HIVE-16164: --------------------------------------- [~spena], this went into branch-2 shortly after branch-2.3 was cut, so it won't be in release 2.3.0 but rather 2.4.0. If you want it in 2.3.0 you'll have to commit the patch to branch-2.3. > Provide mechanism for passing HMS notification ID between transactional and > non-transactional listeners. > -------------------------------------------------------------------------------------------------------- > > Key: HIVE-16164 > URL: https://issues.apache.org/jira/browse/HIVE-16164 > Project: Hive > Issue Type: Improvement > Components: Metastore > Reporter: Sergio Peña > Assignee: Sergio Peña > Fix For: 2.3.0, 3.0.0 > > Attachments: HIVE-16164.1.patch, HIVE-16164.2.patch, > HIVE-16164.3.patch, HIVE-16164.6.patch, HIVE-16164.7.patch, HIVE-16164.8.patch > > > The HMS DB notification listener currently stores an event ID on the HMS > backend DB so that external applications (such as backup apps) can request > incremental notifications based on the last event ID requested. > The HMS DB notification and backup applications are asynchronous. However, > there are sometimes that applications may be required to be in sync with the > latest HMS event in order to process an action. These applications will > provide a listener implementation that is called by the HMS after an HMS > transaction happened. > The problem is that the listener running after the transaction (or during the > non-transactional context) may need the DB event ID in order to sync all > events happened previous to that event ID, but this ID is never passed to the > non-transactional listeners. > We can pass this event information through the EnvironmentContext found on > each ListenerEvent implementations (such as CreateTableEvent), and send the > EnvironmentContext to the non-transactional listeners to get the event ID. > The DbNotificactionListener already knows the event ID after calling the > ObjectStore.addNotificationEvent(). We just need to set this event ID to the > EnvironmentContext from each of the event notifications and make sure that > this EnvironmentContext is sent to the non-transactional listeners. > Here's the code example when creating a table on {{create_table_core}}: > {noformat} > ms.createTable(tbl); > if (transactionalListeners.size() > 0) { > CreateTableEvent createTableEvent = new CreateTableEvent(tbl, true, this); > createTableEvent.setEnvironmentContext(envContext); > for (MetaStoreEventListener transactionalListener : > transactionalListeners) { > transactionalListener.onCreateTable(createTableEvent); // <- > Here the notification ID is generated > } > } > success = ms.commitTransaction(); > } finally { > if (!success) { > ms.rollbackTransaction(); > if (madeDir) { > wh.deleteDir(tblPath, true); > } > } > for (MetaStoreEventListener listener : listeners) { > CreateTableEvent createTableEvent = > new CreateTableEvent(tbl, success, this); > createTableEvent.setEnvironmentContext(envContext); > listener.onCreateTable(createTableEvent); // <- > Here we would like to consume notification ID > } > {noformat} > We could use a specific key name that will be used on the EnvironmentContext, > such as DB_NOTIFICATION_EVENT_ID. -- This message was sent by Atlassian JIRA (v6.3.15#6346)