Play with the read-ahead mount options for NFS, but it might require
more work with that kind of latency. You need to be able to have
a lot of RPC's in-flight to maintain the pipeline with higher latencies.
At least 16 and possibly more.
It might be easier to investigate why the latency is so high and fix
that first. 10ms is way too high for a LAN.
I remember there was some work in the FreeBSD tree to clean up the
client-side NFS rpc mechanics but if they are still threaded (kernel
thread or user thread, doesn't matter) with one synchronous RPC per
thread then a large amount of read-ahead will cause the requests to be
issued out of order over the wire (for both TCP and UDP NFS mounts),
which really messes up the server-side heuristics. Plus the
client-side threads wind up competing with each other for the
socket lock. So there is a limit to how large a read-ahead you
can specify and still get good results.
If they are using a single kernel thread for socket reading and a
single kernel thread for socket writing (i.e. a 100% async RPC model,
which is what DFly uses), then you can boost the read-ahead to 50+.
At that point the socket buffer becomes the limiting factor in the
pipeline.
Make sure the NFS mount is TCP (It defaults to TCP in FreeBSD 8+). UDP
mounts will not perform well with any read-ahead greater then 3 or 4
RPCs because occassional seek latencies on the server will cause
random UDP RPCs to timeout and retry, which completely destroys
performance. UDP mounts have no understanding of the RPC queue backlog
on the server and treat each RPC independently for timeout/retry
purposes. So one minor stall can blow up every single pending RPC
backed up behind the one that stalled.
-Matt
_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
