[
https://issues.apache.org/jira/browse/HDFS-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17635652#comment-17635652
]
ASF GitHub Bot commented on HDFS-16845:
---------------------------------------
simbadzina commented on PR #5142:
URL: https://github.com/apache/hadoop/pull/5142#issuecomment-1319526622
> > @ZanderXu @tomscut could you please take a look whenever you get a
chance.
>
> @simbadzina I did a quick review. The change looks good to me. I am
curious about that what scenarios will use this feature without `msyc()`. Can
you share me some cases?
Thanks @ZanderXu . Third party readers issuing multiple reads will use msync
if they want consistency, both before and after this path. The msync this flag
eliminates is the one at the initialization of the ObserverReadProxyProvider
here
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/ObserverReadProxyProvider.java#L341-L348
Taking an example of a client just doing a list status.
If the client is using ObserverReadProxyProvider. They'll be two calls.
1) msync (during initialization)
2) getListing (actual call the client wants to do)
When the router gets the msync, it will fan this out to all namespace, which
is expensive.
With this patch, they'll only be one call.
1) getListing
The router will send this call to the active.
For future reads after this, there is no difference.
> Add configuration flag to enable observer reads on routers without using
> ObserverReadProxyProvider
> --------------------------------------------------------------------------------------------------
>
> Key: HDFS-16845
> URL: https://issues.apache.org/jira/browse/HDFS-16845
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Simbarashe Dzinamarira
> Assignee: Simbarashe Dzinamarira
> Priority: Critical
> Labels: pull-request-available
>
> In order for clients to have routers forward their reads to observers, the
> clients must use a proxy with an alignment context. This is currently
> achieved by using the ObserverReadProxyProvider.
> Using ObserverReadProxyProvider allows backward compatible for client
> configurations.
> However, the ObserverReadProxyProvider forces an msync on initialization
> which is not required with routers.
> Performing msync calls is more expensive with routers because the router fans
> out the cal to all namespaces, so we'd like to avoid this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
