[
https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280686#comment-15280686
]
Tsz Wo Nicholas Sze commented on HDFS-9924:
-------------------------------------------
> With regard to error handling, why not handle all errors as exceptions thrown
> from Future#get? ...
For some case like network connection errors, if we do not throw exception
until Future#get, the client could summit a large number of calls and the catch
a lot of exceptions in Future#get. It is fail-fast if the client catch an
exception in the first async call.
> Does the Future#get callback get made without holding any locks? ...
Yes, it does not holding any locks.
> It seems concerning that we would have to make such a large change to the
> synchronous DistributedFileSystem code. ...
I agree.
> Blocking the client seems like it could be problematic for code which expects
> to be asynchronous. There should be an option to throw an exception in this
> case.
Throwing exception is indeed better.
> I also think that we could maintain a queue of async calls that we have not
> submitted to the IPC layer yet, to avoid being limited by issues at the IPC
> layer.
This is a good idea although it may not be easy to implement. Will check that.
> Given that Hadoop 3.x will probably be Java 8 (based on the mailing list
> discussion), why not just make the async API use jdk8's CompletableFuture
> from day 1, rather than hacking it in later?
Because Java 7 does not support it. We would like to have a larger audience
for the new async feature.
Will revise the design doc. Thanks for the comments.
> [umbrella] Asynchronous HDFS Access
> -----------------------------------
>
> Key: HDFS-9924
> URL: https://issues.apache.org/jira/browse/HDFS-9924
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: fs
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Xiaobing Zhou
> Attachments: AsyncHdfs20160510.pdf
>
>
> This is an umbrella JIRA for supporting Asynchronous HDFS Access.
> Currently, all the API methods are blocking calls -- the caller is blocked
> until the method returns. It is very slow if a client makes a large number
> of independent calls in a single thread since each call has to wait until the
> previous call is finished. It is inefficient if a client needs to create a
> large number of threads to invoke the calls.
> We propose adding a new API to support asynchronous calls, i.e. the caller is
> not blocked. The methods in the new API immediately return a Java Future
> object. The return value can be obtained by the usual Future.get() method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
