[
https://issues.apache.org/jira/browse/HIVE-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146600#comment-14146600
]
Sushanth Sowmyan commented on HIVE-7974:
----------------------------------------
1. The Main reason for that is visibility. Sometimes, code changes happen in
ql/ that don't make appropriate changes in hcatalog-webhcat-java-client, for
eg., and we have to run around trying to fix it. Strongly connected tests would
catch these kinds of issues, but when you're a serialize and a deserialize away
across a message format, it's easier to break something without realizing it.
If we are to grow this further, I believe it should belong in hive codebase,
rather than tucked away inside hcatalog-server-extensions. Also, if I now put
ReplicationTask basing on top of hcatalog-server-extensions, writing that alone
in a top level repl/ feels like a cyclic dependency issue waiting to happen. If
I put ReplicationTask inside hcatalog-server-extensions, it risks being seen as
HCat Replication rather than Hive Replication. I thought that moving it out to
a top-level made things cleaner in the long run.
2. I thought about this for a bit. Not breaking backward compatibility is a
huge deal, but if I had to move packages out of hcatalog-server-extensions,
that potentially breaks Jackson auto-deserialization/serialization because the
classnames are different. There is no fundamentally different data in it, and
if we wrote our own serialize/deserialize, we could make it work irrespective
of classnames, and the messages are format compatible. If we fixed
serialization/deserialization by not using automatic
serialization/deserialization, we could stick with one listener, and it
wouldn't matter if it were a HCatEventMessage or an EventMessage.
What can be done easily, however, is that the hive metastore supports having
more than one listener loaded, as a comma separated list, and users can manage
migration development and testing with ReplListener, while still keeping their
existing NotificationListener active, simply by specifying a different prefix.
And they can then code-migrate over to the newer listeners over a couple of
releases, and gain the ability to use replication on top of those events.
I completely understand your question on why it's worth the effort, and it was
a hard call for me, and I eventually wound up in the side of this. If we decide
not to, then we can cancel this patch, and simply do ReplicationTask inside
hcatalog-server-extensions.
3. I've created https://issues.apache.org/jira/browse/HIVE-8165 to mark a
deprecation leading to a removal, rather than an outright removal, as discussed
above.
> Notification Event Listener movement to a new top level repl/ module
> --------------------------------------------------------------------
>
> Key: HIVE-7974
> URL: https://issues.apache.org/jira/browse/HIVE-7974
> Project: Hive
> Issue Type: Sub-task
> Components: Import/Export
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-7974.patch
>
>
> We need to create a new hive module (say hive-repl? ) to subsume the
> NotificationListener from HCatalog.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
