Re: UFS2 optimization for many small files

2007-07-01 Thread Claus Guttesen
We're going to build a server with some 1Tb of over 500 million small files with size from 0,5k to 4k. I'm wonder if the ufs2 can handle this kind of system well. From newfs(8) the min block size is 4k. This is not optimal in our case, a 1k or 0,5k block is more effective IMHO. I'd be happy if

Re: UFS2 optimization for many small files

2007-07-01 Thread Thomas Herrlin
Nguyen Tam Chinh wrote: Greetings, We're going to build a server with some 1Tb of over 500 million small files with size from 0,5k to 4k. I'm wonder if the ufs2 can handle this kind of system well. From newfs(8) the min block size is 4k. This is not optimal in our case, a 1k or 0,5k block is

Re: UFS2 optimization for many small files

2007-07-01 Thread Wojciech Puchar
approx. 15 partitions ranging from 400 GB to 2 TB in size. If the server for some reason had crashed the webservers were unable to the question is about the reason it crashed... access the nfs-mounted partitions during the period the server did a snapshot of a partition, in order to perform

Re: UFS2 optimization for many small files

2007-07-01 Thread Wojciech Puchar
I have tried using a 4K/0.5K UFS1 filesystem in the past and found the performance was very poor. UFS2 was based on 16K/2K and I would expect it to perform even worse with 4K/0.5K. I would suggest you try 8K/1K. not for small files. you are light with large files but it's not THAT bad as

Re: UFS2 optimization for many small files

2007-07-01 Thread Wojciech Puchar
We're going to build a server with some 1Tb of over 500 million small files with size from 0,5k to 4k. I'm wonder if the ufs2 can handle this kind of system well. From newfs(8) the min block size is 4k. This is not optimal in our case, a 1k or 0,5k block is more effective IMHO. I'd be happy if

Re: UFS2 optimization for many small files

2007-07-01 Thread Wojciech Puchar
Thank you very much. Try zfs on amd64 unless your app doesn't work well with zfs or your does zfs have RELIABLE and USABLE software allowing to efficiently backup large filesystems to other media? (DVD's, tapes, other hard discs) ___

Re: UFS2 optimization for many small files

2007-07-01 Thread Claus Guttesen
approx. 15 partitions ranging from 400 GB to 2 TB in size. If the server for some reason had crashed the webservers were unable to the question is about the reason it crashed... access the nfs-mounted partitions during the period the server did a snapshot of a partition, in order to

Re: UFS2 optimization for many small files

2007-07-01 Thread Claus Guttesen
Try zfs on amd64 unless your app doesn't work well with zfs or your does zfs have RELIABLE and USABLE software allowing to efficiently backup large filesystems to other media? (DVD's, tapes, other hard discs) Zfs has send/receive where you can do snapshots and send them to a different host.

Re: UFS2 optimization for many small files

2007-07-01 Thread Wojciech Puchar
snapshot of a partition, in order to perform a background-fsck and thus our website was down. So ufs2 does not scale well. Reasons not related to the nfs-server itself. FreeBSD itself was rock-solid. It was firmware-related on the storage-side. i always use software mirror concat or both in

Re: UFS2 optimization for many small files

2007-07-01 Thread Nikolay Pavlov
On Wednesday, 27 June 2007 at 14:11:19 +0400, Nguyen Tam Chinh wrote: Greetings, We're going to build a server with some 1Tb of over 500 million small files with size from 0,5k to 4k. I'm wonder if the ufs2 can handle this kind of system well. From newfs(8) the min block size is 4k. This

Re: UFS2 optimization for many small files

2007-07-01 Thread Nguyen Tam Chinh
On 7/2/07, Nikolay Pavlov [EMAIL PROTECTED] wrote: On Wednesday, 27 June 2007 at 14:11:19 +0400, Nguyen Tam Chinh wrote: Greetings, We're going to build a server with some 1Tb of over 500 million small files with size from 0,5k to 4k. I'm wonder if the ufs2 can handle this kind of system

Re: UFS2 optimization for many small files

2007-06-30 Thread Peter Jeremy
On 2007-Jun-27 14:11:19 +0400, Nguyen Tam Chinh [EMAIL PROTECTED] wrote: We're going to build a server with some 1Tb of over 500 million small files with size from 0,5k to 4k. I'm wonder if the ufs2 can handle this kind of system well. Short answer: No. Longer answer: FreeBSD and UFS2 have

UFS2 optimization for many small files

2007-06-27 Thread Nguyen Tam Chinh
Greetings, We're going to build a server with some 1Tb of over 500 million small files with size from 0,5k to 4k. I'm wonder if the ufs2 can handle this kind of system well. From newfs(8) the min block size is 4k. This is not optimal in our case, a 1k or 0,5k block is more effective IMHO. I'd

Re: UFS2 optimization for many small files

2007-06-27 Thread Philipp Ost
Nguyen Tam Chinh wrote: [snipped] Please advice with optimizations or tricks. [...] Did you already looked at 'man 7 tuning'? HTH, Philipp -- www.familie-ost.info/~pj ___ freebsd-stable@freebsd.org mailing list