[
https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328291#comment-15328291
]
Andrew Wang commented on HDFS-9924:
-----------------------------------
[~ashutoshc] I still don't find this argument compelling, for the following
reasons:
* A threadpool in Hive would work across all Hadoop versions, and without
requiring a new release of Hadoop with this API.
* A threadpool is just a few extra MBs of memory.
* A threadpool isn't that much code, particularly with the use of Executors.
* We'd have to carry around a much larger amount of code in HDFS, and an API
which we already know needs to be changed to fit our other downstream usecases.
It pushes a larger maintenance burden onto HDFS compared to a threadpool in
Hive.
* Building on the previous, Hive is the only downstream user asking for this
feature. Other downstreams are saying that they really need callbacks for this
to be useful.
If you still feel differently, please help me understand why the above points
are not applicable. Thanks.
> [umbrella] Asynchronous HDFS Access
> -----------------------------------
>
> Key: HDFS-9924
> URL: https://issues.apache.org/jira/browse/HDFS-9924
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: fs
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Xiaobing Zhou
> Attachments: AsyncHdfs20160510.pdf
>
>
> This is an umbrella JIRA for supporting Asynchronous HDFS Access.
> Currently, all the API methods are blocking calls -- the caller is blocked
> until the method returns. It is very slow if a client makes a large number
> of independent calls in a single thread since each call has to wait until the
> previous call is finished. It is inefficient if a client needs to create a
> large number of threads to invoke the calls.
> We propose adding a new API to support asynchronous calls, i.e. the caller is
> not blocked. The methods in the new API immediately return a Java Future
> object. The return value can be obtained by the usual Future.get() method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
