[
https://issues.apache.org/jira/browse/HDFS-8940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348554#comment-15348554
]
Colin Patrick McCabe commented on HDFS-8940:
--------------------------------------------
bq. You mean reading inotify messages from the SbNN? It's a very attractive
idea from scalability angle. But how would we handle the staleness? The SbNN
could be a few mins behind ANN right?
Sorry for the misunderstanding. I wasn't talking about HDFS HA. The point
that I was making is that you don't want a single point of failure in whatever
service you are using to fetch the events from HDFS and put them in Kafka.
Perhaps you could also execute the code which fetches events in the context of
Kafka itself somehow, to avoid creating a new service? I'm not familiar with
the programming model there.
> Support for large-scale multi-tenant inotify service
> ----------------------------------------------------
>
> Key: HDFS-8940
> URL: https://issues.apache.org/jira/browse/HDFS-8940
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Ming Ma
> Attachments: Large-Scale-Multi-Tenant-Inotify-Service.pdf
>
>
> HDFS-6634 provides the core inotify functionality. We would like to extend
> that to provide a large-scale service that ten of thousands of clients can
> subscribe to.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
