Howdy AxDevers,
Okay, I finally had a chance to install the proposed caching patches
that came from Chris L through the users' list (CC'ing you, Chris since
I'm not sure if you're subbed here or not). The changes proposed touch a fair amount of code and re-organize some of AxKit most basic behaviors in to different classes so it just seems smart to be to review them a bit more closely than we do the usual patch.
First, the hard numbers:
Pentium II 366, 96M RAM (Hey, its my old lap-top...) Running RedHat 7.2
The following reflects performance for 10,000 total requests, 10 concurrent, using ab on the loopback. (see the attached bench.txt for full details) Stripped down XSLT stylesheets were used in effort to localize processing time to AxKit itself, rather than XSLT transformation.
## Current CVS, AxNoCache On, 3-step (trivial) XSLT Requests per second: 50.99 [#/sec] (mean)
## Incremental Patch Applied, AxNoCache On, 3-step (trivial) XSLT Requests per second: 51.79 [#/sec] (mean)
## Current CVS, AxNoCache Off, 3-step (trivial) XSLT Requests per second: 80.45 [#/sec] (mean)
## Incremental Patch Applied, AxNoCache Off, 3-step (trivial) XSLT Requests per second: 68.41 [#/sec] (mean)
Some things to note and random ideas:
* There's small increase in requests per second with caching turned off, but a modest loss with caching turned on.
* The content-length returned from requests to the CVS and patched is different.
The changes to the way data is returned from the LibXSLT Language module in the new Provider/Cache/Language interactions further exposes differences in the document returned from XML::LibXSLT when the result is selected as DOM vs.as a string. That is, given:
my $results = $stylesheet->transform($source);
$results->toString();
is *not* guaranteed to give the same result as
$stylesheet->output_string($results);
hence, if only the DOM is returned, there may be unexpected results.
This is not specifically related to Chris' proposed patches directly, but it does point to the fact that we my have rethink the very idea of passing around a DOM tree between Provider and Language modules and between different Language modules in the processing chain. The sad fact is that if we are to adhere to the notion of least surprise for AxKit users, we may need to fully serialize and re-parse at every processing stage in order to allow each processor the chance to perform whatever serialization magic it needs to to deliver the proper result. .
* Given that the Cache and Provider interfaces are more or less identical, making Cache a subclass of Provider makes a lot of sense. +1 on this part.
* Changing the way Language modules get content from the current "it might be in pnotes('dom_tree'), pnotes('xml_string') or one of the Provider methods and you'll have to check" interface to the "give me what i ask for" approach in Chris' patches is a definite change for the better.
* It wasn't clear to me whether or not the new Language module design allows direct output to the client if the current processor is the last in the chain. If not, this presumption puts limits on what can be integrated as last-in-chain Language modules that may need access to the Apache object to return content appropriately.
* Bolting on the result of the transformation to the return code in the new Language modules (e.g. return (Apache::Constants::OK, \%results); ) seems a smidge hacky to me. They should return one or the other, both is an ugly mix of coding strategies.
So, I guess the sum of my evaluation is "I'm not sure". There are definitely some good things in the proposed patches, but I think they go too far in some places.
The questions that need to be answered are:
In general, does switching to incremental caching give us something that we don't already have, or is it arguably a better generic solution than the current all-or-nothing implementation (especially in light of the fact that the typical use-case seems to put the dynamic parts at the front of the processing chain)?
Do the proposed patches make it easier for users to write custom Cache modules? How or how not?
Thoughts?
-kip
10,000 requests, 10 concurrent
Current CVS, AxNoCache On, 3-step (trivial) XSLT
#####################################
Finished 10000 requests
Server Software: Apache/1.3.27
Server Hostname: localhost
Server Port: 80
Document Path: /axkitbook/samples/chapt04/base.xml
Document Length: 186 bytes
Concurrency Level: 10
Time taken for tests: 196.119 seconds
Complete requests: 10000
Failed requests: 0
Broken pipe errors: 0
Total transferred: 4410441 bytes
HTML transferred: 1860186 bytes
Requests per second: 50.99 [#/sec] (mean)
Time per request: 196.12 [ms] (mean)
Time per request: 19.61 [ms] (mean, across all concurrent requests)
Transfer rate: 22.49 [Kbytes/sec] received
Connnection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 11.7 0 453
Processing: 16 194 450.2 107 17789
Waiting: 0 193 449.9 106 17788
Total: 16 195 451.2 107 17789
Percentage of the requests served within a certain time (ms)
50% 107
66% 228
75% 293
80% 331
90% 427
95% 514
98% 642
99% 753
100% 17789 (last request)
#####################################
Incremental Patch Applied, AxNoCache On, 3-step (trivial) XSLT
#####################################
Finished 10000 requests
Server Software: Apache/1.3.27
Server Hostname: localhost
Server Port: 80
Document Path: /axkitbook/samples/chapt04/base.xml
Document Length: 151 bytes
Concurrency Level: 10
Time taken for tests: 193.074 seconds
Complete requests: 10000
Failed requests: 0
Broken pipe errors: 0
Total transferred: 3231258 bytes
HTML transferred: 1510226 bytes
Requests per second: 51.79 [#/sec] (mean)
Time per request: 193.07 [ms] (mean)
Time per request: 19.31 [ms] (mean, across all concurrent requests)
Transfer rate: 16.74 [Kbytes/sec] received
Connnection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 22.0 0 1861
Processing: 16 192 752.3 71 29363
Waiting: 0 191 752.0 71 29362
Total: 16 192 752.8 72 29363
Percentage of the requests served within a certain time (ms)
50% 72
66% 198
75% 273
80% 317
90% 425
95% 516
98% 651
99% 805
100% 29363 (last request)
#####################################
Current CVS, AxNoCache Off, 3-step (trivial) XSLT
#####################################
Finished 10000 requests
Server Software: Apache/1.3.27
Server Hostname: localhost
Server Port: 80
Document Path: /axkitbook/samples/chapt04/base.xml
Document Length: 186 bytes
Concurrency Level: 10
Time taken for tests: 124.300 seconds
Complete requests: 10000
Failed requests: 0
Broken pipe errors: 0
Total transferred: 4740000 bytes
HTML transferred: 1860000 bytes
Requests per second: 80.45 [#/sec] (mean)
Time per request: 124.30 [ms] (mean)
Time per request: 12.43 [ms] (mean, across all concurrent requests)
Transfer rate: 38.13 [Kbytes/sec] received
Connnection Times (ms)
min mean[+/-sd] median max
Connect: 0 6 14.9 0 338
Processing: 9 117 730.3 50 29672
Waiting: 0 115 730.4 48 29672
Total: 9 123 730.4 59 29672
Percentage of the requests served within a certain time (ms)
50% 59
66% 79
75% 90
80% 99
90% 188
95% 325
98% 612
99% 1028
100% 29672 (last request)
#####################################
Incremental Patch Applied, AxNoCache Off, 3-step (trivial) XSLT
#####################################
Finished 10000 requests
Server Software: Apache/1.3.27
Server Hostname: localhost
Server Port: 80
Document Path: /axkitbook/samples/chapt04/base.xml
Document Length: 134 bytes
Concurrency Level: 10
Time taken for tests: 146.187 seconds
Complete requests: 10000
Failed requests: 0
Broken pipe errors: 0
Total transferred: 4220422 bytes
HTML transferred: 1340134 bytes
Requests per second: 68.41 [#/sec] (mean)
Time per request: 146.19 [ms] (mean)
Time per request: 14.62 [ms] (mean, across all concurrent requests)
Transfer rate: 28.87 [Kbytes/sec] received
Connnection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 19.2 0 1629
Processing: 10 142 953.1 22 36560
Waiting: 0 141 952.7 22 36559
Total: 10 143 953.5 22 36560
Percentage of the requests served within a certain time (ms)
50% 22
66% 71
75% 103
80% 148
90% 325
95% 457
98% 640
99% 913
100% 36560 (last request)
#####################################
