RE: [Samba] Slow performance with lots of files in one directory
Have you read the XFS tuning recommendations? The XFS developers bitch because people don't tune their volumes, then they don't understand bad performance... notes from Gentoo install: Note: You may want to add a couple of additional flags to the mkfs.xfs command: -d agcount=3 -l size=32m. The -d agcount=3 command will lower the number of allocation groups. XFS will insist on using at least 1 allocation group per 4 GB of your partition, so, for example, if you hava a 20 GB partition you will need a minimum agcount of 5. The try this w/ XFS mkfs.xfs -d agcount=(numgigs / 4) -l size=32m ===THEN== when you mount, try logbuf=8 and noatime in the mount options. Windows is a killer for atimes. js On Fri, 2003-01-10 at 07:05, Gerald Drouillard wrote: > Unless you are a programmer, I am afraid the only thing you can do is to > modify how the files are stored in that directory. I had the files on a > ext3 RAID5 with lots of memory config and any type of access to that > directory would bring smb to a crawl. I even tried putting the files on a > separate XFS RAID5 server and mount the directory, but seemed to just make > it worse even with a 1Gig connection between the servers. The files that I > store are from our in-house imaging program. Our file names were all > numeric so it was just a case of changing the name structure from 123456.TIF > to /3/2/1/456.TIF. In the new file name format, a directory has no more > than 999+10 directory entries. Now the system is working better than ever. > > Regards > - > Gerald Drouillard > Owner and Consultant > Drouillard & Associates, Inc. > http://www.Drouillard.ca > > > -Original Message- > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On > > Behalf Of Anders Nordby > > Sent: Thursday, January 09, 2003 10:19 AM > > To: [EMAIL PROTECTED] > > Subject: [Samba] Slow performance with lots of files in one directory > > > > > > Hello, > > > > I've got performance problems with copying small files over to a Samba > > share in a directory that has lots of small files (1 to 2 > > files). It takes too long time to copy new files (they drip in at a fast > > pace), and smbd eats a lot of CPU time. > > > > Is there any way to make Samba run faster in this situation? > > > > Cheers, > > > > -- > > Anders Nordby > > Aftenposten AS, Systemteknisk avd. > > > > Tlf.: +47 22864083 > > Fax: +47 22864074 > > -- > > To unsubscribe from this list go to the following URL and read the > > instructions: http://lists.samba.org/mailman/listinfo/samba -- VB programmers ask why no one takes them seriously, it's somewhat akin to a McDonalds manager asking employees why they don't take their 'career' seriously. signature.asc Description: This is a digitally signed message part
RE: [Samba] Slow performance with lots of files in one directory
Unless you are a programmer, I am afraid the only thing you can do is to modify how the files are stored in that directory. I had the files on a ext3 RAID5 with lots of memory config and any type of access to that directory would bring smb to a crawl. I even tried putting the files on a separate XFS RAID5 server and mount the directory, but seemed to just make it worse even with a 1Gig connection between the servers. The files that I store are from our in-house imaging program. Our file names were all numeric so it was just a case of changing the name structure from 123456.TIF to /3/2/1/456.TIF. In the new file name format, a directory has no more than 999+10 directory entries. Now the system is working better than ever. Regards - Gerald Drouillard Owner and Consultant Drouillard & Associates, Inc. http://www.Drouillard.ca > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On > Behalf Of Anders Nordby > Sent: Thursday, January 09, 2003 10:19 AM > To: [EMAIL PROTECTED] > Subject: [Samba] Slow performance with lots of files in one directory > > > Hello, > > I've got performance problems with copying small files over to a Samba > share in a directory that has lots of small files (1 to 2 > files). It takes too long time to copy new files (they drip in at a fast > pace), and smbd eats a lot of CPU time. > > Is there any way to make Samba run faster in this situation? > > Cheers, > > -- > Anders Nordby > Aftenposten AS, Systemteknisk avd. > > Tlf.: +47 22864083 > Fax: +47 22864074 > -- > To unsubscribe from this list go to the following URL and read the > instructions: http://lists.samba.org/mailman/listinfo/samba -- To unsubscribe from this list go to the following URL and read the instructions: http://lists.samba.org/mailman/listinfo/samba
Re: [Samba] Slow performance with lots of files in one directory
On Thu, Jan 09, 2003 at 04:19:27PM -0200, Roberto João Lopes Garcia wrote: > At 15:04 09/01/2003, Chris Palmer wrote: > >Anders Nordby writes: > > > >> I've got performance problems with copying small files over to a Samba > >> share in a directory that has lots of small files (1 to 2 > >> files). It takes too long time to copy new files (they drip in at a > >> fast pace), and smbd eats a lot of CPU time. > > > >This could be not so much a Samba problem as a Unix kernel problem. > >Traditional Unix filesystems (UFS, FFS, ext2, et c.) do not deal well > >with very full directories. See Maurice Bach's book *Design of the Unix > >Operating System* and M. K. McKusick's *Design and Implementation of the > >4.4BSD Operating System*. These are both just great books. > > > >If you are on Linux, try using one of the new filesystems like ReiserFS, > >XFS or JFS. Among other abilities, they can handle extremely full > >directories better. > > What about EXT3 file system? EXT3 uses the same linear directory structure as EXT2. Therefore, it suffers the same performance penalty when dealing with large directories. -- JF -- To unsubscribe from this list go to the following URL and read the instructions: http://lists.samba.org/mailman/listinfo/samba
Re: [Samba] Slow performance with lots of files in one directory
Roberto João Lopes Garcia writes: > What about EXT3 file system? ext3 is ext2 plus journalling, and so not fundamentally different. -- Chris Palmer Systems Programmer GeneEd -- To unsubscribe from this list go to the following URL and read the instructions: http://lists.samba.org/mailman/listinfo/samba
Re: [Samba] Slow performance with lots of files in one directory
At 15:04 09/01/2003, Chris Palmer wrote: >Anders Nordby writes: > >> I've got performance problems with copying small files over to a Samba >> share in a directory that has lots of small files (1 to 2 >> files). It takes too long time to copy new files (they drip in at a >> fast pace), and smbd eats a lot of CPU time. > >This could be not so much a Samba problem as a Unix kernel problem. >Traditional Unix filesystems (UFS, FFS, ext2, et c.) do not deal well >with very full directories. See Maurice Bach's book *Design of the Unix >Operating System* and M. K. McKusick's *Design and Implementation of the >4.4BSD Operating System*. These are both just great books. > >If you are on Linux, try using one of the new filesystems like ReiserFS, >XFS or JFS. Among other abilities, they can handle extremely full >directories better. What about EXT3 file system? Thank you Roberto >-- >Chris Palmer Systems Programmer GeneEd > >-- >To unsubscribe from this list go to the following URL and read the >instructions: http://lists.samba.org/mailman/listinfo/samba -- To unsubscribe from this list go to the following URL and read the instructions: http://lists.samba.org/mailman/listinfo/samba
Re: [Samba] Slow performance with lots of files in one directory
Anders Nordby writes: > I've got performance problems with copying small files over to a Samba > share in a directory that has lots of small files (1 to 2 > files). It takes too long time to copy new files (they drip in at a > fast pace), and smbd eats a lot of CPU time. This could be not so much a Samba problem as a Unix kernel problem. Traditional Unix filesystems (UFS, FFS, ext2, et c.) do not deal well with very full directories. See Maurice Bach's book *Design of the Unix Operating System* and M. K. McKusick's *Design and Implementation of the 4.4BSD Operating System*. These are both just great books. If you are on Linux, try using one of the new filesystems like ReiserFS, XFS or JFS. Among other abilities, they can handle extremely full directories better. -- Chris Palmer Systems Programmer GeneEd -- To unsubscribe from this list go to the following URL and read the instructions: http://lists.samba.org/mailman/listinfo/samba
[Samba] Slow performance with lots of files in one directory
Hello, I've got performance problems with copying small files over to a Samba share in a directory that has lots of small files (1 to 2 files). It takes too long time to copy new files (they drip in at a fast pace), and smbd eats a lot of CPU time. Is there any way to make Samba run faster in this situation? Cheers, -- Anders Nordby Aftenposten AS, Systemteknisk avd. Tlf.: +47 22864083 Fax: +47 22864074 -- To unsubscribe from this list go to the following URL and read the instructions: http://lists.samba.org/mailman/listinfo/samba
