[
https://issues.apache.org/jira/browse/ARROW-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755804#comment-16755804
]
Zhijun Fu edited comment on ARROW-4418 at 1/30/19 8:12 AM:
---
In addition to above benefits that Robert mentioned, asio also provides
opportunities for performance improvements, by providing io service, thread
pool .etc.
In our internal testing, which uses 10+ actors on a single machine, I found 50%
of plasma store CPU are spent on receiving messages from plasma clients, using
UNIX domain socket.
I'm thinking that one way to improve perf is like this:
* Use a pool of threads to receive messages from clients. To ensure correct
behavior, we can bind a boost::strand to a single client, so that all the
messages from a given client arrives in order. As this part is CPU consumings,
using multiple threads is going to help.
* After this, the messages are posted into io service of main thread, which
calls ProcessMessages for each of them in order.
* After this, post the replies to a pool of threads, again use boost::strand
for each plasma client to ensure correct order.
I'm thinking this would probably help on cases where there are multiple workers
using plasma store on the same machine, which should be very common. And it
seems implementing this would be hard without asio functionalities.
was (Author: zhijunfu):
In addition to above benefits that Robert mentioned, asio also provides
opportunities for performance improvements, by providing io service, thread
pool .etc.
In our internal testing, which uses 10+ actors on a single machine, I found 50%
of plasma store CPU are spent on receiving messages from plasma clients, using
UNIX domain socket.
I'm thinking that one way to improve perf is like this:
* Use a pool of threads to receive messages from clients. To ensure correct
behavior, we can bind a boost::strand to a single client, so that all the
messages from a given client arrives in order. As this part is CPU consumings,
using multiple threads is going to help.
* After this, the messages are posted into io service of main thread, which
calls ProcessMessages for each of them in order.
* After this, post the replies to a pool of threads, again use boost::strand
for each plasma client to ensure correct order.
I'm thinking this would probably help on cases where there are multiple workers
using plasma store on the same machine, which should be very common. And it
seems implementing this would be hard without asio functionalities.
Thoughts?
> [Plasma] replace event loop with boost::asio for plasma store
> -
>
> Key: ARROW-4418
> URL: https://issues.apache.org/jira/browse/ARROW-4418
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Plasma (C++)
>Reporter: Zhijun Fu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Original text:
> It would be nice to move plasma store from current event loop to boost::asio
> to modernize the code, and more importantly to benefit from the
> functionalities provided by asio, which I think also provides opportunities
> for performance improvement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
