Win2K/XP, oplocks, and readahead

[Steven French]

The readahead that the Linux page manager does (see mm/readahead.c) seems
more sensible than what you are seeing with the Windows client - if you are
trying to excercise your server in this way you might try the clients using
the current version of Linux 2.5 kernel mounted with the cifs vfs or smbfs
to your Samba server - this is probably a much more efficient way to
exercise readahead using SMB clients.  The algorithm used by the Linux page
manager for readahead is pretty interesting.   Note (if writebehind is also
being tested) that in the current 2.5 kernel implementation of the cifs vfs
at the moment there is a call missing to flush at oplock break time and
also at sync time (flush is called properly at file close time) but this
should be fixed soon.

Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: a href=/faq.html#spam[EMAIL PROTECTED]/a

Re: Win2K/XP, oplocks, and readahead

[Peter Godman]

Hi there!

Although I am most focused on Windows clients since that is what our
customers use, this would be a great thing to try to attempt to exonerate
samba (not that much suspicion lies there anyway :)).  It would be an
interesting data point.  I can't believe that I can't find *any*
information about someone doing this sort of workload over CIFS with
Windows clients!

Thanks for the advice!

Peter Godman

On Fri, 21 Feb 2003, Steven French wrote:

 
 
 
 
 The readahead that the Linux page manager does (see mm/readahead.c) seems
 more sensible than what you are seeing with the Windows client - if you are
 trying to excercise your server in this way you might try the clients using
 the current version of Linux 2.5 kernel mounted with the cifs vfs or smbfs
 to your Samba server - this is probably a much more efficient way to
 exercise readahead using SMB clients.  The algorithm used by the Linux page
 manager for readahead is pretty interesting.   Note (if writebehind is also
 being tested) that in the current 2.5 kernel implementation of the cifs vfs
 at the moment there is a call missing to flush at oplock break time and
 also at sync time (flush is called properly at file close time) but this
 should be fixed soon.
 
 Steve French
 Senior Software Engineer
 Linux Technology Center - IBM Austin
 phone: 512-838-2294
 email: [EMAIL PROTECTED]
 

Win2K/XP, oplocks, and readahead.

[Peter Godman]

Hi!

I'm forwarding this message, which I orignally posted to the samba list,
in the hopes of reaching a wider audience for my question.  Thanks for
considering my problem.

I'm working with Samba backed by a high performance filesystem.  From a
Windows 2K and Windows XP client I'm trying to achieve very high speed
single file throughput over GigE from the Windows client using either
open/write or CreateFile/ReadFile APIs.  I'd rather not venture into
overlapped IO there so that we don't have to recommend that all our
customers rewrite their applications!

I'm seeing a problem where it appears that windows is not reading data far
enough ahead (or maybe at all) to keep the pipeline full.  From assessing
the load on samba, it is a apparent that much of its time is spent idle.  
The characteristics of the load suggest that WinXP or Win2000 is not
requesting readahead far enough to be useful.  My link roundtrip latency
is around .4 ms beyond the data transmission time, the samba servicing
time for 32K of data (32K is the blocksize I'm using) is about .27 ms, and
the wire time for 32K of data should be around .27ms on GigE.

I understand that it is necessary that oplocking be functional to have the
windows client read ahead.  However, I have verified that oplocks are
being established, yet still the readahead seems either nonexistant or
minimal.  I don't know how to establish which.  FreeBSD is the host OS.  
I have verified that samba is not sleeping on socket buffer space, and
this is borne out by the fact that changing TCP window sizes on client and
server improves performance very little.  In addition, samba isn't
sleeping on the filesystem reads, as readahead on the filesystem ensures
data is always available when the windows client requests it.

I'd be grateful for answers to any of the following questions:

1.  How do Windows clients determine appropriate levels of readahead?  
Are there any caps on this algorithm that I might be hitting? 
2.  Is the client or server responsible for producing readahead data (I'm
assuming this is the client)? 
3.  Any other tips on how to make this work or anecdotal evidence of
single file performance in the half-gigabit/sec ballpark in the read
and write of a single file? 
4.  Is there any way for samba to send unsolicited readahead data to the
Windows client when an oplock has been established?
5.  Any samba tricks for debugging this?

In terms of registry keys I've already changed those for window sizes,
MTU, UseOpportunisticLocking, SizReqBuf (to 64K), EnablePMTUDiscovery,
MaxCmds.

Here's my samba configuration:

[global]
  encrypt passwords = yes
  log file = /var/log/samba.log
  max log size = 100
  local master = no
  read size = 8192
  # below socket sizes have been varied without effect
  socket options = TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=65536
SO_SNDBUF=65536
  dns proxy = no
  change notify timeout = 3000
  disable spoolss = yes
  smb passwd file = /usr/local/private/smbpasswd
  password server = *
  winbind uid = 1-2
  winbind gid = 1-2
  winbind enum users = yes
  winbind enum groups = yes
  workgroup = FOO
  server string = A Samba Server
  hosts allow = 
  security = SHARE
  oplocks = true

[myfs]
  printable = no
  level2 oplocks = true
  guest ok = yes
  path = /myfs
  comment = myfs
  read only = no
  hide dot files = no
  share modes = no