[
https://issues.apache.org/jira/browse/TEZ-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14261847#comment-14261847
]
Jeff Zhang commented on TEZ-1897:
---------------------------------
Comments:
1. Should remove the public modifier from {code}public AsyncDispatcher(String
name, BlockingQueue<Event> eventQueue, int numThreads) {code}, because as we
discussion at in TEZ-1867, we will lose processing order if we create multiple
threads, And this may cause some unexpected weird issues. So at least there
should be only one thread for the central dispatcher, the multiple thread can
only been used internally in registerAndCreateDispatcher
2. This patch cause TestExceptionPropagation#testExceptionPropagationNonSession
fail. This test case will sleep for 1 second to get the diagnostics from RM (
this is due to YARN issue YARN-2560). And I notice that in the
AsyncDispatcher#serviceStop, you will make it sleep 1 second, is it necessary ?
The test case can pass if I remove it.
3. Another minor issue is that AsyncDispatcher would sleep for 1 second when
draining event, is it too long ? Maybe 100 milliseconds would be better.
{code}
waitForDrained.wait(1000);
LOG.info("Waiting for AsyncDispatcher to drain.");
{code}
> Allow higher concurrency in AsyncDispatcher
> -------------------------------------------
>
> Key: TEZ-1897
> URL: https://issues.apache.org/jira/browse/TEZ-1897
> Project: Apache Tez
> Issue Type: Task
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-1897.1.patch
>
>
> Currently, it processes events on a single thread. For events that can be
> executed in parallel, e.g. vertex manager events, allowing higher concurrency
> may be beneficial.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
