[jira] [Created] (HADOOP-13390) GC pressure of NetworkTopology.isAncestor call can be eliminated
He Tianyi created HADOOP-13390:
--
Summary: GC pressure of NetworkTopology.isAncestor call can be
eliminated
Key: HADOOP-13390
URL: https://issues.apache.org/jira/browse/HADOOP-13390
Project: Hadoop Common
Issue Type: Improvement
Components: net
Affects Versions: 2.7.0, 2.6.0, 2.8.0
Reporter: He Tianyi
Priority: Minor
{{NetworkTopology.isAncestor}} is called in {{NetworkTopology.getLeaf}}, which
is excessively used in block placement policies ({{chooseRandom}} for example).
Currently, the implementation calls {{getPath}} twice. And {{getPath}} performs
string concatenation on the fly. On a busy NameNode, this introduces more GC
pressure and CPU overhead for block allocation.
Given that network location and node name does not generally change frequently,
we can cache path as a properly of {{Node}} and update accordingly while
network location or node name is changing.
Also, one of these {{getPath}} calls in {{getLeaf}} can be eliminated because
we are expecting identical result in both calls.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13057) Async IPC server support
[
https://issues.apache.org/jira/browse/HADOOP-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255911#comment-15255911
]
He Tianyi commented on HADOOP-13057:
BTW, would be really appreciate to see further updates on HADOOP-11552.
> Async IPC server support
>
>
> Key: HADOOP-13057
> URL: https://issues.apache.org/jira/browse/HADOOP-13057
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: He Tianyi
>
> On some application, server may run out of handlers when performing many
> blocking I/O operations during processing of each call (e.g. calling another
> service, etc.). A viable solution is increasing number of handlers.
> But this faces the problem that large amount of threads will consume much
> memory (stack, etc.), and performance issues either.
> After HADOOP-12909, work on asynchronization has been done on caller-side.
> This is a similar proposal on server-side.
> Suggesting the ability to handle requests asynchronously.
> For example, in such server, calls may return a Future object instead of
> immediate value. Then sends response to client in {{onSuccess}} or
> {{onFailed}} callbacks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-13057) Async IPC server support
[
https://issues.apache.org/jira/browse/HADOOP-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255907#comment-15255907
]
He Tianyi commented on HADOOP-13057:
Thanks, [~sseth].
Marked as duplicate.
> Async IPC server support
>
>
> Key: HADOOP-13057
> URL: https://issues.apache.org/jira/browse/HADOOP-13057
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: He Tianyi
>
> On some application, server may run out of handlers when performing many
> blocking I/O operations during processing of each call (e.g. calling another
> service, etc.). A viable solution is increasing number of handlers.
> But this faces the problem that large amount of threads will consume much
> memory (stack, etc.), and performance issues either.
> After HADOOP-12909, work on asynchronization has been done on caller-side.
> This is a similar proposal on server-side.
> Suggesting the ability to handle requests asynchronously.
> For example, in such server, calls may return a Future object instead of
> immediate value. Then sends response to client in {{onSuccess}} or
> {{onFailed}} callbacks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HADOOP-13057) Async IPC server support
[
https://issues.apache.org/jira/browse/HADOOP-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi resolved HADOOP-13057.
Resolution: Duplicate
> Async IPC server support
>
>
> Key: HADOOP-13057
> URL: https://issues.apache.org/jira/browse/HADOOP-13057
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: He Tianyi
>
> On some application, server may run out of handlers when performing many
> blocking I/O operations during processing of each call (e.g. calling another
> service, etc.). A viable solution is increasing number of handlers.
> But this faces the problem that large amount of threads will consume much
> memory (stack, etc.), and performance issues either.
> After HADOOP-12909, work on asynchronization has been done on caller-side.
> This is a similar proposal on server-side.
> Suggesting the ability to handle requests asynchronously.
> For example, in such server, calls may return a Future object instead of
> immediate value. Then sends response to client in {{onSuccess}} or
> {{onFailed}} callbacks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-13057) Async IPC server support
[
https://issues.apache.org/jira/browse/HADOOP-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255868#comment-15255868
]
He Tianyi commented on HADOOP-13057:
Sure, added links and description.
> Async IPC server support
>
>
> Key: HADOOP-13057
> URL: https://issues.apache.org/jira/browse/HADOOP-13057
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: He Tianyi
>
> On some application, server may run out of handlers when performing many
> blocking I/O operations during processing of each call (e.g. calling another
> service, etc.). A viable solution is increasing number of handlers.
> But this faces the problem that large amount of threads will consume much
> memory (stack, etc.), and performance issues either.
> After HADOOP-12909, work on asynchronization has been done on caller-side.
> This is a similar proposal on server-side.
> Suggesting the ability to handle requests asynchronously.
> For example, in such server, calls may return a Future object instead of
> immediate value. Then sends response to client in {{onSuccess}} or
> {{onFailed}} callbacks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-13057) Async IPC server support
[
https://issues.apache.org/jira/browse/HADOOP-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HADOOP-13057:
---
Description:
On some application, server may run out of handlers when performing many
blocking I/O operations during processing of each call (e.g. calling another
service, etc.). A viable solution is increasing number of handlers.
But this faces the problem that large amount of threads will consume much
memory (stack, etc.), and performance issues either.
After HADOOP-12909, work on asynchronization has been done on caller-side. This
is a similar proposal on server-side.
Suggesting the ability to handle requests asynchronously.
For example, in such server, calls may return a Future object instead of
immediate value. Then sends response to client in {{onSuccess}} or {{onFailed}}
callbacks.
was:
On some application, server may run out of handlers when performing many
blocking I/O operations during processing of each call (e.g. calling another
service, etc.). A viable solution is increasing number of handlers.
But this faces the problem that large amount of threads will consume much
memory (stack, etc.), and performance issues either.
Suggesting the ability to handle requests asynchronously.
For example, in such server, calls may return a Future object instead of
immediate value. Then sends response to client in {{onSuccess}} or {{onFailed}}
callbacks.
> Async IPC server support
>
>
> Key: HADOOP-13057
> URL: https://issues.apache.org/jira/browse/HADOOP-13057
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: He Tianyi
>
> On some application, server may run out of handlers when performing many
> blocking I/O operations during processing of each call (e.g. calling another
> service, etc.). A viable solution is increasing number of handlers.
> But this faces the problem that large amount of threads will consume much
> memory (stack, etc.), and performance issues either.
> After HADOOP-12909, work on asynchronization has been done on caller-side.
> This is a similar proposal on server-side.
> Suggesting the ability to handle requests asynchronously.
> For example, in such server, calls may return a Future object instead of
> immediate value. Then sends response to client in {{onSuccess}} or
> {{onFailed}} callbacks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-13057) Async IPC server support
[
https://issues.apache.org/jira/browse/HADOOP-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HADOOP-13057:
---
Description:
On some application, server may run out of handlers when performing many
blocking I/O operations during processing of each call (e.g. calling another
service, etc.). A viable solution is increasing number of handlers.
But this faces the problem that large amount of threads will consume much
memory (stack, etc.), and performance issues either.
Suggesting the ability to handle requests asynchronously.
For example, in such server, calls may return a Future object instead of
immediate value. Then sends response to client in {{onSuccess}} or {{onFailed}}
callbacks.
was:
On some application, server may run out of handlers when performing many
blocking I/O operations during processing of each call (e.g. calling another
service, etc.). A viable solution is increasing number of handlers.
But this faces the problem that large amount of threads will consume much
memory (stack, etc.), and performance issues either.
Suggesting the ability to handle requests asynchronously.
For example, in such server, calls may return a Future object instead of
immediate value. Then sends response to client in {onSuccess} or {onFailed}
callbacks.
> Async IPC server support
>
>
> Key: HADOOP-13057
> URL: https://issues.apache.org/jira/browse/HADOOP-13057
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: He Tianyi
>
> On some application, server may run out of handlers when performing many
> blocking I/O operations during processing of each call (e.g. calling another
> service, etc.). A viable solution is increasing number of handlers.
> But this faces the problem that large amount of threads will consume much
> memory (stack, etc.), and performance issues either.
> Suggesting the ability to handle requests asynchronously.
> For example, in such server, calls may return a Future object instead of
> immediate value. Then sends response to client in {{onSuccess}} or
> {{onFailed}} callbacks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-13057) Async IPC server support
He Tianyi created HADOOP-13057:
--
Summary: Async IPC server support
Key: HADOOP-13057
URL: https://issues.apache.org/jira/browse/HADOOP-13057
Project: Hadoop Common
Issue Type: Improvement
Components: ipc
Affects Versions: 2.6.0
Reporter: He Tianyi
On some application, server may run out of handlers when performing many
blocking I/O operations during processing of each call (e.g. calling another
service, etc.). A viable solution is increasing number of handlers.
But this faces the problem that large amount of threads will consume much
memory (stack, etc.), and performance issues either.
Suggesting the ability to handle requests asynchronously.
For example, in such server, calls may return a Future object instead of
immediate value. Then sends response to client in {onSuccess} or {onFailed}
callbacks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
