Hello! 2009/4/21 randall <rand...@ciparo.nl>: > dear all, > > second post ;) > > another question here, also in most examples i noticed the infiniband or 10 > GigE recommendation, does this really do any good for the individual server > connection?
IIRC, another recommendation is a RAID-6 array with 8-12 disks. I'd expect 400-600MB/s on linear read with hardware like that. In that case, 1GigE would be a bottleneck so 10 GigE or Infiniband might be a reasonable recommendation. IIRC, some posts on the mailing list point out that the performance of GlusterFS (and most or all other distributed filesystems) is limited by connection latency (e.g. 'replicate' has to ask each of the servers if it has a newer version of a file and wait for the answer). On 100Mbit/s it takes longer to push a packet (of equal size) through the wire than it takes at 1GBit/s or even 10GBit/s. Maybe someone with access to a test setup with GigE, 10GigE and/or Infiniband could benchmark this so that others might have a base line of what to expect? > another assumption on my side was that 1 individual server can never > saturate a full gigabit link due to the disk throughput limitation, (maybe a > few 100 MBps at most) AFAIK, a modern disk (>=1TB SATA) can deliver more than 100MB/s locally on linear read: # hdparm -t /dev/sdc /dev/sdc: Timing buffered disk reads: 306 MB in 3.00 seconds = 101.84 MB/sec # fdisk -l /dev/sdc Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes Even a server with just two of these disks in a 'dht' or 'unify' configuration might be able to saturate a 1GigE network link under specific conditions. Some Solid State Disks are even rated at 250MB/s for read. > so if every server has a 1 Gigabit connection to a switch which in turn has > a 10 GigE uplink it would not be a bottleneck (as long as there are not too > many servers sharing the 10 GigE uplink) > > correct? I'm not sure about that, but after looking at prices I'd do a lot of testing that the 1GigE network adapter really is a bottleneck before buying 10GigE or Infiniband hardware. On a cost/value comparison GigE might win on small systems with only one or two disks. Harald Stürzebecher _______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users