Re: [fw-general] XMLRPC Server Performance
-- Matt Hoopes [EMAIL PROTECTED] wrote
(on Thursday, 16 October 2008, 06:51 PM -0400):
I'm planning to use Zend_Xmlrpc_Server for my API. I have it working, and so
far, it's been super. It's made everything incredibly easy to do. My only
issue
so far has been performance, and a weird quirk that I've found through a bit
of
testing. Based on a previous issue (http://framework.zend.com/issues/browse/
ZF-3280), I'm not using Zend_XmlRpc_Server_Cache, and I am using the APC
opcode
cache (which has driven down memory usage considerably, but speed still
remains
an issue).
This kind of leads me down the path of guessing the CPU usage is the main
bottleneck. So, I made my XmlrpcController like so (with other setup omitted,
obv.):
public function indexAction() {
$overall_start = microtime(true);
$start = microtime(true);
$this-setClasses(); // sets about 10 classes into the server, for
about 150 (?) functions total
$elapsed = microtime(true) - $start;
API_Logger::debug(Got server in [$elapsed] seconds);
$start = microtime(true);
$ret = $this-server-handle();
$elapsed = microtime(true) - $start;
API_Logger::debug(Handled response in [$elapsed] seconds);
$overall_elapsed = microtime(true) - $overall_start;
API_Logger::debug(Overall request time [$overall_elapsed]);
API_Logger::debug();
die($ret);
}
Now, the interesting thing that I've discovered is that one client request at
a
time seems to respond in a decent amount of time. My output looks like so:
2008-10-16 22:38:36 INFO (6): [6313] Got server in [0.25338101387] seconds
2008-10-16 22:38:37 INFO (6): [6313] Handled response in [0.103404998779]
seconds
2008-10-16 22:38:37 INFO (6): [6313] Overall request time [0.360664129257]
2008-10-16 22:38:37 INFO (6): [6313]
However, when I make three requests to the same method at the same instant
(using firefox refresh all tabs, ha), each request is quite a bit slower:
2008-10-16 22:41:53 INFO (6): [6724] Got server in [0.860436916351] seconds
2008-10-16 22:41:53 INFO (6): [6465] Got server in [0.796639919281] seconds
2008-10-16 22:41:53 INFO (6): [6736] Got server in [0.816692113876] seconds
2008-10-16 22:41:53 INFO (6): [6465] Handled response in [0.284796953201]
seconds
2008-10-16 22:41:53 INFO (6): [6465] Overall request time [1.13558387756]
2008-10-16 22:41:53 INFO (6): [6465]
2008-10-16 22:41:53 INFO (6): [6724] Handled response in [0.300675868988]
seconds
2008-10-16 22:41:53 INFO (6): [6724] Overall request time [1.1651570797]
2008-10-16 22:41:53 INFO (6): [6724]
2008-10-16 22:41:53 INFO (6): [6736] Handled response in [0.293844938278]
seconds
2008-10-16 22:41:53 INFO (6): [6736] Overall request time [1.17823600769]
2008-10-16 22:41:53 INFO (6): [6736]
As I increase the number of simultaneous requests, the difference is more and
more drastic. As the requests to the API server start to pick up, the box is
brought to its knees pretty quickly. The CPU usage on the box is pegged, while
memory isn't that much of a problem. More hardware would help, but not
forever,
if I want the API server cluster to start getting decent 3rd party usage.
So, my question is, is there something I can do? Can I put the server on a
ramdisk? Can I use memcached to cache $this-server in my example above? I'm
running this out of Apache, and would a more drastic move of trying lighttpd
and fastcgi be a path to explore (b/c php stays in memory)?
Failing all that, this is a plea for help. :)
One of the issues with the various server components is that they all
rely on reflection. One idea I've had is to refactor them to allow
caching all the metadata needed to store method signatures and map them
to class methods so that after the initial request, reflection is not
necessary. I'm still scoping the work necessary to do so, however, and
it may or may not make it into the 1.7.0 release.
--
Matthew Weier O'Phinney
Software Architect | [EMAIL PROTECTED]
Zend Framework | http://framework.zend.com/
Re: [fw-general] XMLRPC Server Performance
One of the issues with the various server components is that they all rely on reflection. One idea I've had is to refactor them to allow caching all the metadata needed to store method signatures and map them to class methods so that after the initial request, reflection is not necessary. I'm still scoping the work necessary to do so, however, and it may or may not make it into the 1.7.0 release. Matthew, Thanks for the response. I guess I was confused about why three simultaneous requests would slow down the response time so much. I would think that they'd each be served in a different apache process (The PID of the process in those log lines is in the brackets after the timestamp), and as such, shouldn't really affect each other. Should they? Or is it simply a CPU capacity / disk IO problem? If all requests took as much time as the single request alone, I think it would be good to go. Is this more of an apache tuning issue? thanks for an advice, - hoopes
[fw-general] XMLRPC Server Performance
Hey guys,
I'm planning to use Zend_Xmlrpc_Server for my API. I have it working, and so
far, it's been super. It's made everything incredibly easy to do. My only
issue so far has been performance, and a weird quirk that I've found through
a bit of testing. Based on a previous issue (
http://framework.zend.com/issues/browse/ZF-3280), I'm not using
Zend_XmlRpc_Server_Cache, and I am using the APC opcode cache (which has
driven down memory usage considerably, but speed still remains an issue).
This kind of leads me down the path of guessing the CPU usage is the main
bottleneck. So, I made my XmlrpcController like so (with other setup
omitted, obv.):
public function indexAction() {
$overall_start = microtime(true);
$start = microtime(true);
$this-setClasses(); // sets about 10 classes into the server, for
about 150 (?) functions total
$elapsed = microtime(true) - $start;
API_Logger::debug(Got server in [$elapsed] seconds);
$start = microtime(true);
$ret = $this-server-handle();
$elapsed = microtime(true) - $start;
API_Logger::debug(Handled response in [$elapsed] seconds);
$overall_elapsed = microtime(true) - $overall_start;
API_Logger::debug(Overall request time [$overall_elapsed]);
API_Logger::debug();
die($ret);
}
Now, the interesting thing that I've discovered is that one client request
at a time seems to respond in a decent amount of time. My output looks like
so:
2008-10-16 22:38:36 INFO (6): [6313] Got server in [0.25338101387] seconds
2008-10-16 22:38:37 INFO (6): [6313] Handled response in [0.103404998779]
seconds
2008-10-16 22:38:37 INFO (6): [6313] Overall request time [0.360664129257]
2008-10-16 22:38:37 INFO (6): [6313]
However, when I make three requests to the same method at the same instant
(using firefox refresh all tabs, ha), each request is quite a bit slower:
2008-10-16 22:41:53 INFO (6): [6724] Got server in [0.860436916351] seconds
2008-10-16 22:41:53 INFO (6): [6465] Got server in [0.796639919281] seconds
2008-10-16 22:41:53 INFO (6): [6736] Got server in [0.816692113876] seconds
2008-10-16 22:41:53 INFO (6): [6465] Handled response in [0.284796953201]
seconds
2008-10-16 22:41:53 INFO (6): [6465] Overall request time [1.13558387756]
2008-10-16 22:41:53 INFO (6): [6465]
2008-10-16 22:41:53 INFO (6): [6724] Handled response in [0.300675868988]
seconds
2008-10-16 22:41:53 INFO (6): [6724] Overall request time [1.1651570797]
2008-10-16 22:41:53 INFO (6): [6724]
2008-10-16 22:41:53 INFO (6): [6736] Handled response in [0.293844938278]
seconds
2008-10-16 22:41:53 INFO (6): [6736] Overall request time [1.17823600769]
2008-10-16 22:41:53 INFO (6): [6736]
As I increase the number of simultaneous requests, the difference is more
and more drastic. As the requests to the API server start to pick up, the
box is brought to its knees pretty quickly. The CPU usage on the box is
pegged, while memory isn't that much of a problem. More hardware would help,
but not forever, if I want the API server cluster to start getting decent
3rd party usage.
So, my question is, is there something I can do? Can I put the server on a
ramdisk? Can I use memcached to cache $this-server in my example above? I'm
running this out of Apache, and would a more drastic move of trying lighttpd
and fastcgi be a path to explore (b/c php stays in memory)?
Failing all that, this is a plea for help. :)
thanks in advance,
- hoopes
