On 09/21/2010 05:45 PM, Adam Heath wrote:
On 09/21/2010 05:15 PM, Joe Schaefer wrote:
I don't recall the breakdown, the 10M figure
counts total daily traffic. You could look
at Vadim's stats for more details- the bottom
line is that no app that doesn't support static
exports will function as a suitable CMS for
Apache, which is why we're rolling our own.
Ok, I've looked. 10M seems to be *all* requests. That number doesn't
scare me. I've got a site live right now, http://www.hailmerry.com/,
that is reporting 190req/s with ab(from apache http) while on localhost.
That's without any fancy supercache sitting in front. This site supports
online live editting of content. Anonymous users have no session, and we
make a point of reducing database access for hot-points.
This is running on a single large shared iscsi disk host. The cpu node
is an 8-way 2.33Ghz cpu, using xen, with the domU given 512M and a
single cpu. The hosting framework is no where near what could be
consider super-fast.
The main feature, that I haven't mentioned before now, that helps with
this, is a thing we have designed called TTLObject. It is designed to
protect method calls, by saving their results, and returning old values
for a certain amount of time. It is non-blocking, uses a state engine
internally. It was overhauled to follow the design patterns in Java
Concurrency in Practice.
Again, I'm not afraid of those numbers. Our cms stuff(which isn't quite
up to the same level of feature integration as confluence), stores most
data as raw files(to allow svn/git tracking of history). The database is
not used for most functions.
I'm still trying to understand the distribution of load. If my
understanding is wrong, then please tell me. It'll give me a target
to strive for. I've looked at more graphs at (1), and I see a
breakdown of page views per major sub-site(host).
Our current framework underwent a major rewrite 4 years ago, and it
was first deployed into a production state 3.5 years ago. That
particular site did 3,500,000 total requests the day after it went
live, 500,000 page views. There was no super-cache in front of it.
The version of the software at the time had a bug, where
If-Modified-Since processing didn't work, so all images were always
fetched by clients. Our filesystem code has been rewritten, to be
non-blocking(no synchronized keywords), the cow/overlay feature has
had a second-order of speedups, plus other speed fixes. If I
'flatten' the cow/overlay system, so it's not used, the system easily
approached 1000req/s(single page at a time).
So, I'm still not afraid of these numbers.
Now, if you were to combine *all* these hosts into one, and then try
to run it, we might have an issue. But, that's not what is currently
happening.
And, with the 10M number you originally gave, that is 115req/s. If
that is for a single page, then our current software handles that
fine. If that is for 115 different pages at the same time, then I
will have to get back to you, to try that test. I don't have a
program that can request 115 different pages at once.
Also, if the problem is with remote clients tying up a
thread/connection slot, then that is a separate problem from the
backend system. The backend system should have a small thread pool,
so that it can run fast, and then the frontend(either catalina itself,
or apache/mod-jk) does the send using non-blocking-io.
1: http://people.apache.org/~vgritsenko/stats/index.html
