Re: ZFS - poor performance with "large" directories

2015-11-27 Thread krad
I was referring to the quality of the usb sticks that people have lying around, rather than the port speed. The biggest thing that flash based drives have always don is lower latency, as getting throughput out of magnetic arrays has never been a big issue, random reads/writes a different story thou

Re: ZFS - poor performance with "large" directories

2015-11-26 Thread Torfinn Ingolfsen
On Wed, 25 Nov 2015 16:24:48 +0100 Albert Cervin wrote: > Just to close this off, when using Samba with ZFS it seems to be very > important (if you have many files in a directory) to make it case > sensitive as per: > https://wiki.samba.org/index.php/Performance_tuning#Handling_Large_Directories.

Re: ZFS - poor performance with "large" directories

2015-11-26 Thread Freddie Cash
On Thu, Nov 26, 2015 at 2:19 AM, krad wrote: > true, but in my experience usb pen drives are variable in terms of > performance across different sticks and different areas of the same stick. > This can complicate things a little, and is often not worth the effort. You > obviously run the ssd over

Re: ZFS - poor performance with "large" directories

2015-11-26 Thread krad
haven't been able to sort the down time yet. On 25 November 2015 at 12:16, Gerrit Kühn wrote: > On Wed, 25 Nov 2015 12:06:38 + krad wrote about Re: > ZFS - poor performance with "large" directories: > > K> consumer SSDs are cheap enough now not to bother with u

Re: ZFS - poor performance with "large" directories

2015-11-25 Thread Albert Cervin
dly under these conditions... > > Thanks for all the ZFS help though, much appreciated! > > Cheers, > Albert > > On Wed, Nov 25, 2015 at 1:16 PM, Gerrit Kühn wrote: >> On Wed, 25 Nov 2015 12:06:38 + krad wrote about Re: >> ZFS - poor performance with "large&

Re: ZFS - poor performance with "large" directories

2015-11-25 Thread Albert Cervin
1:16 PM, Gerrit Kühn wrote: > On Wed, 25 Nov 2015 12:06:38 + krad wrote about Re: > ZFS - poor performance with "large" directories: > > K> consumer SSDs are cheap enough now not to bother with usb drives I would > K> imagine. > > Sure. I was just suggestin

Re: ZFS - poor performance with "large" directories

2015-11-25 Thread Gerrit Kühn
On Wed, 25 Nov 2015 12:06:38 + krad wrote about Re: ZFS - poor performance with "large" directories: K> consumer SSDs are cheap enough now not to bother with usb drives I would K> imagine. Sure. I was just suggesting a USB drive as a quick way to check if this might help at

Re: ZFS - poor performance with "large" directories

2015-11-25 Thread krad
consumer SSDs are cheap enough now not to bother with usb drives I would imagine. On 25 November 2015 at 07:12, Gerrit Kühn wrote: > On Tue, 24 Nov 2015 17:11:54 +0100 Albert Cervin > wrote about Re: ZFS - poor performance with "large" directories: > > AC> Will try

Re: ZFS - poor performance with "large" directories

2015-11-24 Thread Gerrit Kühn
On Tue, 24 Nov 2015 17:11:54 +0100 Albert Cervin wrote about Re: ZFS - poor performance with "large" directories: AC> Will try a bit with the meta limit. You can also put metadata on a flash device to speed things up. To check if this is really the bottleneck in your case, somethin

Re: ZFS - poor performance with "large" directories

2015-11-24 Thread Albert Cervin
"8G is not that much really. In the thread they suggested increasing the meta limit so that the giant directory can fit into cache." It is not really short on ram judging from the usage though and sure 8Gb is not that much but on the other hand, neither is 8000 files in one directory in my opinion

Re: ZFS - poor performance with "large" directories

2015-11-24 Thread Mike Tancsa
On 11/24/2015 10:26 AM, Albert Cervin wrote: > vfs.zfs.txg.timeout is already verified to be 5 (the default). I have > also turned off atime and vfs.zfs.arc_meta_limit is 1287906304. > > "Do you have any memory pressures on your server ? Have a look at this > thread" > > The server has 4 cores

Re: ZFS - poor performance with "large" directories

2015-11-24 Thread Albert Cervin
Thanks! "I should hope not. ext4 vs zfs comparison isn't fair for either." I do realize that comparing ext4 and ZFS is not really giving anything but it tells us one thing, ext4 would work whereas ZFS would not for our use case, which was unexpected, at least to me. vfs.zfs.txg.timeout is alre

Re: ZFS - poor performance with "large" directories

2015-11-24 Thread Adam Vande More
On Tue, Nov 24, 2015 at 8:00 AM, Albert Cervin wrote: > Hi all, > > Please feel free to direct me to a list that is more suitable. > > We are trying to set up a fileserver solution for a web application that we > are building. This fileserver is running FreeBSD 10.2 and ZFS. Files are > written o

Re: ZFS - poor performance with "large" directories

2015-11-24 Thread Mike Tancsa
On 11/24/2015 9:00 AM, Albert Cervin wrote: > However, we are seeing en exponential decrease in performance to write to > the file server when the number of files in the directory grows (when it > goes up to ~6000 files it becomes unusable and the write time has gone from > a fraction of a second t

Re: ZFS - poor performance with "large" directories

2015-11-24 Thread krad
make sure atime if off for starters on the filesystem On 24 November 2015 at 14:00, Albert Cervin wrote: > Hi all, > > Please feel free to direct me to a list that is more suitable. > > We are trying to set up a fileserver solution for a web application that we > are building. This fileserver is

ZFS - poor performance with "large" directories

2015-11-24 Thread Albert Cervin
Hi all, Please feel free to direct me to a list that is more suitable. We are trying to set up a fileserver solution for a web application that we are building. This fileserver is running FreeBSD 10.2 and ZFS. Files are written over CIFS with Samba running on the fileserver host. However, we are