On Jul 21, 2010, at 1:33 PM, robert mena wrote:
Thanks for that. I surely need to define those parameters and they
will change what solutions I'll be forced to use. But again let's
say I define:
- 99.95% of uptime,
Okay, this means you can tolerate ~5 minutes of outage per week on
average. That's a pretty tall order for availability.
As you know, availability goes hand in hand with redundancy. You want
redundant systems so that if any one fails, the others take over,
hopefully instantly and transparently.
Theo Schlossnagle talks about IP anycast routing as a way to achieve
this architecture, even if you use multiple data centers for even
greater redundancy. I mentioned his book, but here's his 90-minute
tech talk. Highly recommended. http://www.youtube.com/watch?v=2WuT2rdLK5A
- from 256 to 1024 concurrent http requests
- the html served around 60KB uncompressed (the rest are images) but
a single page with ~150KB -> 300KB
Okay that brings up the simple issue of bandwidth requirements, before
you even get to architecting the PHP code. Let's assume you want
requests to be served in 1 second. So that means you expect an
bandwidth usage of ~37MB/s up to 300MB/s (which is 300-2400Mbit/s).
That's a lot to expect from a single data center. This shows the
advantage, or even the necessity, of using a CDN for your static
content. You could cut down bandwidth requirements by making your app
servers output HTML only, referring to other hosts for resources.
That would cut down your primary bandwidth requirement by a factor of
five.
And of course gzip your compressible output, including HTML, CSS, and
JavaScript.
- the ratio will be difficult but a 1 write for every 10 reads
reasonable? Actually the writes would be for logging so each page
will have one log write and more reads. I would probably use a
separate db for the writes.
But is an OLTP database the right solution for logging? I've seen a
number of articles claiming the answer is no.
Facebook has open-sourced their logging solution, called Scribe. I
haven't used it, but you might want to check it out. Here's a blog
about it: http://highscalability.com/product-scribe-facebooks-scalable-logging-system
Although it might be overkill for your project, because it really
fits huge cluster environments.
Another possible solution is to use a single MongoDB database for your
logs. Writing to a log doesn't need to be ACID-compliant so it makes
sense to use a data store that sacrifices ACID for the sake of speed,
which describes MongoDB. http://blog.mongodb.org/post/172254834/mongodb-is-fantastic-for-logging
Bill, my question was more like this... which components of ZF (or
even others built on top of that) could be used for that starting
point? For example. I've read the usage of queues as a strategy to
handle some situations. Should I use Zend_Queue? etc..
As you say, a queueing solution could be useful in some situations, as
you say, but is it right for your app? Optimization must be tailored
to specific cases. You haven't described the work your application is
doing. So I can't say whether a queueing solution is the best choice
for your app. You're the best expert on the functional requirements
and the usage patterns of your app.
Sorry to say it, but internet resources like this mailing list can
only tell you how to use a given technology, not whether that
technology is right for your app.
Regards,
Bill Karwin
