What version of Glusterfs are you using? Though, not sure what's the root cause of your problem, just wanted to point out a bug with read-ahead which would cause read-amplification over network [1][2], which should be fixed in recent versions.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1214489 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1393419 On Wed, Dec 18, 2019 at 2:50 AM David Cunningham <dcunning...@voisonics.com> wrote: > Hello, > > We switched a production system to using GFS instead of NFS at the > weekend, however it didn't go well on Monday when full load hit. The > application started crashing regularly and we had to revert to NFS. It > seems that the problem was high network traffic used by GFS. > > We've two GFS nodes plus one arbiter node, each about 1.3ms latency from > each other. Attached is a chart of network traffic on one of the GFS nodes. > We see that it saturated the 1Gbps link before we reverted to NFS at 15:10. > > The question is, why does GFS use so much network traffic and is there > anything we can do about it? NFS traffic doesn't exceed 4MBps, so 120MBps > for GFS seems awfully high. > > It would also be good to have faster read performance from GFS, but that's > another issue. > > Thanks in advance for any assistance. > > -- > David Cunningham, Voisonics Limited > http://voisonics.com/ > USA: +1 213 221 1092 > New Zealand: +64 (0)28 2558 3782 > ________ > > Community Meeting Calendar: > > APAC Schedule - > Every 2nd and 4th Tuesday at 11:30 AM IST > Bridge: https://bluejeans.com/441850968 > > NA/EMEA Schedule - > Every 1st and 3rd Tuesday at 01:00 PM EDT > Bridge: https://bluejeans.com/441850968 > > Gluster-users mailing list > Gluster-users@gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users >
________ Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/441850968 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/441850968 Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users