On Apr 16, 2007, at 3:21 PM, Will Fould wrote:
Hi,
I have a service that is currently running a basic LAMP stack with
mod_perl and life has been good!
The site running has been getting very busy and I've ordered a
second machine with intention to move the database off that machine
and start the growing up process.
I am looking for next steps to growing up from this machine. Can
somebody recommend a good article, presentation or document that
advocates various strategies to growing up the current architecture
(i.e. basic load balancing, network topology, switches, etc. )?
I realize that milage will vary based on the particular service and
demands. Currently, the site does not deliver a lot of static
content that can be cached or cause huge I/O issues (i.e. images,
media, huge pages, etc). Our database is probably 95% read-only.
Thanks a lot
w
I don't have any articles or papers offhand, but I can say what I
have been discussing with friends lately-- it seems like everyone is
clustering their apps this month.
i'm in the process right now too -- scaling one of my apps from 1
server to a 2node cluster with a 1TB mirror raid + 4gb ram postgres
store on the back and modperl/nginx on the front. i'm only running 8
apache children on the front, and bumped memcached up to 700mb.
switch to a lightweight proxy + httpd on port 80. i like nginx
because its had much fewer critical bugs than lighttpd. others like
lighty. either will be fine - they'll free up apache to deal with
content generation and you'll see a ginormous performance boost off
that . you could use squid or pound for similar tasks, but they're a
PITA to configure and maintain
the #1 slowdown i've seen from apache is from using the same server
to handle the perl/php/python interpreter as being used for
transferring a static file. every request not served by apache is
more resources / memory for your app.
check your db memory usage. if its 95% read only , is it full of
complex joins? blocking operations? if so, don't just consider
offloading to a dedicated db machine, but also consider running a
slave read-only version of it to the local machine.
apache sucks a ton of memory, but in comparison to what a db needs
its nothing. when you migrate to a clustered setup you'll have at
least 1gb of 'extra' memory to use. half of that can easily go to
apache, but you'll see the law of diminishing returns weigh in after
N httpd instances -- thats when you toss memory to memcached or a
local replicant.
profile your db requests: if you have a lot of repetitive queries,
you could save a bunch of queries by using memcached
profile your app design for db handle / connection suitability. a
lot of people program for a system with 1db connection that handles
all read/write. its usually fine on 1 box, but it doesn't work in a
clustered system.
question your db / schema. if you're using a 'new' feature in
mysql, you might have a giant performance hit. if you have a badly
planned/indexed query on postgres you're looking at the difference
between 100ms and 10minutes on a select. also check for blocking
operations. if your db does 'select for share', you might get a
performance boost.
question your os. some distros run apps better than others-- memory
management / kqueue v select v poll / io / etc . i have friends who
have been swapping distros like crazy over the past few months trying
to squeeze a little more performance out. its easier changing
distros than add new servers to a cluster.
you can hold off on any real networking until you're at a 3+
cluster. if you've got 2-3 machines, you can just handle everything
with an extra NIC. at 3-4 you'll want a lan.
this is generic info -- i use it with all my projects( 60% mp/pg ,
20% php/pg , 20% python/pg ), and i have friends using similar stuff
in php/mysql , erlang/pg , python/pg , rails/mysql
// Jonathan Vanasco
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
| SyndiClick.com
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
| FindMeOn.com - The cure for Multiple Web Personality Disorder
| Web Identity Management and 3D Social Networking
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
| RoadSound.com - Tools For Bands, Stuff For Fans
| Collaborative Online Management And Syndication Tools
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
