Re: [Samba] Question on how to get Samba to use larger pread/write calls.
Thx James Peach sent something similar to me too.. darn vfs objects :) Thx james :) I remember now on irix we had to do some stuff with these. I do not see the cacheprime vfs object on FC13 but I do see readahead so I guess I will start with that... I also heard that there are some other I/O tuning that I can do in the Linux/FC kernel to maybe increase the read/write sizes... The kicker here is that I am using an Areca 1880-ix-16 as well as an ATTO R680 card in my boxes and they do not work very well with small I/Os. They are both 6Gbit technology and I have 16 X 2TB fast Hitachi drives in the connected raid... No one has issues with reading/writing one fileit is where one wants to read or write say 20-40 files simultaneously it all goes to heck with small I/Os to/from the raid cards on Linux. I use fio as a test vehicle. It runs over 10Gbit port from a client to a 10Gbe port on my FC13 system.using samba or netatalksimulating 20-40 video users to see home many read/pread's I can get under 38ms/1MByte.. It is a requirement we have for Video editing..I have tuned Mac OS using the ATTO R680 card and it has no problems with low latency for 20-30 sessions/users..The most I can get out of FC13 is maybe 8-10. So it looks like I have homework to do to get Linux/FC13 so it handles I/O's bigger :) Thx chris On 1/18/11 12:34 AM, Volker Lendecke wrote: On Mon, Jan 17, 2011 at 10:14:04AM -0600, Chris Duffy wrote: We are testing Samba 3 (and 4) on Fedora Core 13, 10Gbit connection with a Mac OS 10.6.4 system as the client. We will be adding some Windows machines sooner or later with 10Gbit interfaces. We are seeing 100-150MBytes/sec read or write performance between the Mac and the FC13 system over 10Gbit interface but it should be capable of 400-500MBytes/sec. We have a local raid on the FC13 system that runs 1GByte/sec locally using an Areca 1880-ix-16 raid card (6Gbit version). It has 16 fast Hitachi disks in a Raid5 format using xfs filesystem. The problem here is that samba is poking the Areca at 128KByte I/O's on preads and writes, i.e. shown to us using strace on the smbd daemons that are running. Using vmstat/iostat/sar utilities, we see 100% utilization of the Areca card because the average wait time is real high and the average queue length to it is also high..too many small I/O's. This is not the case if I run fio or dd locally to/from the Areca's raid using 1-4MByte I/O's. I see fast I/O... I do not see any way to increase the size of Samba's pread/write's in the smb.conf documentation. I am sure it may be just a matter of getting the source code and making some changes to allow larger sized IO's butand of course I suppose Windows clients may complain but I remember that back in the old Irix days with the group of engineers in Australia I worked with, we had Samba screaming fast but not sure if they tweaked the version of Samba on Irix to do this. Can you guys come up with a way for us to allow the reads/writes to/from the disks to be tunable up to say 4MB in size? You might want to play with write cache size. This will only tune writes though. And it will only work for oplocked files. I'm not 100% sure that OS/X plays nicely wrt oplocks. For reads, we need to take a much closer look at your real workload and see if we need to use some kind of preopen, prefetch or so module. We need to closely work with the kernel buffer cache and potential readahead kernel algorithms. With best regards, Volker Lendecke -- Chris Duffy Technical Support Small Tree www.small-tree.com http://www.small-tree.com Direct 651-209-6509 X305 Mobil 651-303-9613 Yahoo:chris_duffy6288 AIM:chris.j.du...@comcast.net -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Question on how to get Samba to use larger pread/write calls.
On 17 January 2011 08:14, Chris Duffy c...@small-tree.com wrote: We are testing Samba 3 (and 4) on Fedora Core 13, 10Gbit connection with a Mac OS 10.6.4 system as the client. We will be adding some Windows machines sooner or later with 10Gbit interfaces. We are seeing 100-150MBytes/sec read or write performance between the Mac and the FC13 system over 10Gbit interface but it should be capable of 400-500MBytes/sec. We have a local raid on the FC13 system that runs 1GByte/sec locally using an Areca 1880-ix-16 raid card (6Gbit version). It has 16 fast Hitachi disks in a Raid5 format using xfs filesystem. The problem here is that samba is poking the Areca at 128KByte I/O's on preads and writes, i.e. shown to us using strace on the smbd daemons that are running. Using vmstat/iostat/sar utilities, we see 100% utilization of the Areca card because the average wait time is real high and the average queue length to it is also high..too many small I/O's. This is not the case if I run fio or dd locally to/from the Areca's raid using 1-4MByte I/O's. I see fast I/O... I do not see any way to increase the size of Samba's pread/write's in the smb.conf documentation. I am sure it may be just a matter of getting the source code and making some changes to allow larger sized IO's butand of course I suppose Windows clients may complain but I remember that back in the old Irix days with the group of engineers in Australia I worked with, we had Samba screaming fast but not sure if they tweaked the version of Samba on Irix to do this. Hi Chris :) IIRC, we used a carefully tweaked read ahead module to make sure that I/Os were hitting the disks properly aligned and of reasonable size. http://www.samba.org/samba/docs/man/manpages-3/vfs_readahead.8.html Can you guys come up with a way for us to allow the reads/writes to/from the disks to be tunable up to say 4MB in size? Thx chris duffy -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba -- James Peach | jor...@gmail.com -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
[Samba] Question on how to get Samba to use larger pread/write calls.
We are testing Samba 3 (and 4) on Fedora Core 13, 10Gbit connection with a Mac OS 10.6.4 system as the client. We will be adding some Windows machines sooner or later with 10Gbit interfaces. We are seeing 100-150MBytes/sec read or write performance between the Mac and the FC13 system over 10Gbit interface but it should be capable of 400-500MBytes/sec. We have a local raid on the FC13 system that runs 1GByte/sec locally using an Areca 1880-ix-16 raid card (6Gbit version). It has 16 fast Hitachi disks in a Raid5 format using xfs filesystem. The problem here is that samba is poking the Areca at 128KByte I/O's on preads and writes, i.e. shown to us using strace on the smbd daemons that are running. Using vmstat/iostat/sar utilities, we see 100% utilization of the Areca card because the average wait time is real high and the average queue length to it is also high..too many small I/O's. This is not the case if I run fio or dd locally to/from the Areca's raid using 1-4MByte I/O's. I see fast I/O... I do not see any way to increase the size of Samba's pread/write's in the smb.conf documentation. I am sure it may be just a matter of getting the source code and making some changes to allow larger sized IO's butand of course I suppose Windows clients may complain but I remember that back in the old Irix days with the group of engineers in Australia I worked with, we had Samba screaming fast but not sure if they tweaked the version of Samba on Irix to do this. Can you guys come up with a way for us to allow the reads/writes to/from the disks to be tunable up to say 4MB in size? Thx chris duffy -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Question on how to get Samba to use larger pread/write calls.
On Mon, Jan 17, 2011 at 10:14:04AM -0600, Chris Duffy wrote: We are testing Samba 3 (and 4) on Fedora Core 13, 10Gbit connection with a Mac OS 10.6.4 system as the client. We will be adding some Windows machines sooner or later with 10Gbit interfaces. We are seeing 100-150MBytes/sec read or write performance between the Mac and the FC13 system over 10Gbit interface but it should be capable of 400-500MBytes/sec. We have a local raid on the FC13 system that runs 1GByte/sec locally using an Areca 1880-ix-16 raid card (6Gbit version). It has 16 fast Hitachi disks in a Raid5 format using xfs filesystem. The problem here is that samba is poking the Areca at 128KByte I/O's on preads and writes, i.e. shown to us using strace on the smbd daemons that are running. Using vmstat/iostat/sar utilities, we see 100% utilization of the Areca card because the average wait time is real high and the average queue length to it is also high..too many small I/O's. This is not the case if I run fio or dd locally to/from the Areca's raid using 1-4MByte I/O's. I see fast I/O... I do not see any way to increase the size of Samba's pread/write's in the smb.conf documentation. I am sure it may be just a matter of getting the source code and making some changes to allow larger sized IO's butand of course I suppose Windows clients may complain but I remember that back in the old Irix days with the group of engineers in Australia I worked with, we had Samba screaming fast but not sure if they tweaked the version of Samba on Irix to do this. Can you guys come up with a way for us to allow the reads/writes to/from the disks to be tunable up to say 4MB in size? You might want to play with write cache size. This will only tune writes though. And it will only work for oplocked files. I'm not 100% sure that OS/X plays nicely wrt oplocks. For reads, we need to take a much closer look at your real workload and see if we need to use some kind of preopen, prefetch or so module. We need to closely work with the kernel buffer cache and potential readahead kernel algorithms. With best regards, Volker Lendecke -- SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen phone: +49-551-37-0, fax: +49-551-37-9 AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
