John Madden Storing PHP sessions on Gluster FS really isn't a good option, because session file is locked while scripts access it. You will definetly hit a performance issue. My advice is to setup a memcached server and use memcache (or memcached) PHP module's ability to store session data in memcache server. It has the capability to provide a fault-tolerant service if you setup a few memcached servers. And it's lightning fast.
2010/2/15 John Madden <[email protected]>: > I've made a few swings at using glusterfs for the php session store for a > heavily-used web app (~6 million pages daily) and I've found time and again > that cpu usage and odd load characteristics cause glusterfs to be entirely > unsuitable for this use case at least given my configuration. I posted on > this earlier, but I'm hoping I can get some input on this as things are way > better than they were but still not good enough. I'm on v2.0.9 as the 3.0.x > series doesn't seem to be fully settled yet, though feel free to correct me > on that. > > I have a two-nodes replicate setup and four clients. Configs are below. > What I see is that one brick gets pegged (load avg of 8) and the other > sites much more idle (load avg of 1). The pegged node ends up with high run > queues and i/o blocked processes. CPU usage on the clients for the > glusterfs processes gets pretty high, consuming at least an entire cpu when > not spiking to consume both. I have very high thread counts on the clients > to hopefully avoid thread waits on i/o requests. All six machines are > identical xen instances. > > When one of the bricks is down, cpu usage across the board goes way down, > interactivity goes way up, and things seem overall to be a whole lot better. > Why is that? I would think that having two nodes would at least result in > better read rates. > > I've gone through various caching schemes and tried readahead, writebehind, > quick-read, and stat-prefetch. I found quick-read caused a ton of memory > consumption and didn't help on performance. I didn't see much of a change > at all with stat-prefetch. > > ...Any thoughts? > > ### fsd.vol: > > volume sessions > type storage/posix > option directory /var/glusterfs/sessions > option o-direct off > end-volume > volume data > type storage/posix > option directory /var/glusterfs/data > option o-direct off > end-volume > volume locks0 > type features/locks > option mandatory-locks on > subvolumes data > end-volume > volume locks1 > type features/locks > option mandatory-locks on > subvolumes sessions > end-volume > volume brick0 > type performance/io-threads > option thread-count 32 # default is 16 > subvolumes locks0 > end-volume > volume brick1 > type performance/io-threads > option thread-count 32 # default is 16 > subvolumes locks1 > end-volume > volume server > type protocol/server > option transport-type tcp > option transport.socket.nodelay on > subvolumes brick0 brick1 > option auth.addr.brick0.allow ip's... > option auth.addr.brick1.allow ip's... > end-volume > > > ### client.vol (just one connection shown here) > > volume glusterfs0-hs > type protocol/client > option transport-type tcp > option remote-host "ip" > option ping-timeout 10 > option transport.socket.nodelay on > option remote-subvolume brick1 > end-volume > volume glusterfs1-hs > type protocol/client > option transport-type tcp > option remote-host "ip" > option ping-timeout 10server for each request > option transport.socket.nodelay onspeed > option remote-subvolume brick1 > end-volume > volume replicated > type cluster/replicate > subvolumes glusterfs0-hs glusterfs1-hs > end-volume > volume iocache > type performance/io-cache > option cache-size 512MB > option cache-timeout 10 > subvolumes replicated > end-volume > volume writeback > type performance/write-behind > option cache-size 128MB > option flush-behind off > subvolumes iocache > end-volume > volume iothreads > type performance/io-threads > option thread-count 100 > subvolumes writeback > end-volume > > > > > > -- > John Madden > Sr UNIX Systems Engineer > Ivy Tech Community College of Indiana > [email protected] > _______________________________________________ > Gluster-users mailing list > [email protected] > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > _______________________________________________ Gluster-users mailing list [email protected] http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
