FWIW:
Bruce Allen wrote:
Hi Thomas,
If I understand correctly, your NFS server is connected via a 10Gb/s
link to the switch, and you've spent some effort to tune it. What sort
of aggregate NFS performance are you seeing? Is it at the level of 300
or 400 MG/sec? Could you please provide a few details about the
hardware in your NFS box?
Our 15 and 16 disk JackRabbit units have no trouble with this, using
channel bonded gigabit (no 10GbE). We measured with 4 simultaneous
IOzones being run over NFS and an inexpensive gigabit switch. No jumbo
frames, the fingers never left the hands, yadda yadda yadda. Atop
reported the network IO rate, dstat didn't curiously enough ... the
individual adapters were correct, but it got the channel bond throughput
calculation wrong (multiplied the sum of the individual adapters by 4).
vmstat also returned the disk IO rate.
We were seeing a sustained 420 MB/s in this configuration. About 500
MB/s to disk (journaling overhead). Used RAID6.
FWIW we also uncovered some nasty channel bond crashes under intense
loads. We were able to get it stable and repeatable, at the cost of
some of the more interesting channel bond modes. These crashes didn't
manifest themselves until we started pushing the channel bond hard. We
needed 4 clients banging on the bonded server. 3 would not crash it.
These crashes manifested themselves throughout all kernels we tried
(2.6.9 through 2.6.18). Running 4 separate (non-channel bonded) ports
and doing the same tests did not show this crash. The driver was fine
(e1000, don't even think of trying this with a Broadcom chip/driver) and
well behaved.
Cheers,
Bruce
_______________________________________________
Beowulf mailing list, [email protected]
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: [EMAIL PROTECTED]
web : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax : +1 734 786 8452
cell : +1 734 612 4615
_______________________________________________
Beowulf mailing list, [email protected]
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
