[
https://issues.apache.org/jira/browse/THRIFT-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250216#comment-13250216
]
Dvir Volk commented on THRIFT-1083:
-----------------------------------
@goir you can add shared state when creating your handler class. That's
trivial. But you'll have to use python's dedicated multiprocessing shared
ctypes variables to make them accessible via all the processes. See this for an
example:
http://docs.python.org/library/multiprocessing.html#sharing-state-between-processes
Another option is to use the setPostForkCallback(func) call, and have the
server call a function after the sub processes have spawned. This is good if
you want NOT shared stuff like sockets etc.
I'm doing both strategies in my server and sharing state without a problem. I
hope this answers your question.
> Preforking python process pool server
> -------------------------------------
>
> Key: THRIFT-1083
> URL: https://issues.apache.org/jira/browse/THRIFT-1083
> Project: Thrift
> Issue Type: New Feature
> Components: Python - Library
> Affects Versions: 0.6
> Environment: linux. haven't tested on windows
> Reporter: Dvir Volk
> Assignee: Dvir Volk
> Priority: Minor
> Fix For: 0.7
>
> Attachments: TProcessPoolServer.patch, TProcessPoolServer.py,
> TProcessPoolServer.py
>
>
> This patch adds a new server to the python library: TProcessPoolServer, which
> is a preforking python server.
> this server is sort of a mix between TForkingServer and TThreadPoolServer: It
> has a pool of workers that are preforked subprocesses.
> This approach allows the user to avoid the python GIL single processor limit
> in threading applications, and use a high performance server, and at the same
> time it avoids the need to fork a child process for each connection, as is
> the case with the forking server.
> I've benchmarked it to be about 5-6 times faster than TThreadPoolServer on a
> quad Corei7 CPU, and about the same amount faster than TForkingServer if you
> are not using persistent connections and forking a child for each request.
> The patch also updates the python unit tests to include tests for this server.
> Notes:
> 1. Of course this server has the limitations of forking regarding shared
> state and memory copying.
> 2. You should NOT kill the server with kill -9 - as this will not allow the
> parent process to terminate its children, resulting in orphaned processes
> that keep your socket open. either run stop() in your app or kill the process
> with SIGINT. It will respond to ctrl+C however
> 3. use setNumWorkers(n) before starting the server to determine how many
> processes you want to spawn. the default is 10.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
