Re: UFS2 optimization for many small files

[Nikolay Pavlov]

On Wednesday, 27 June 2007 at 14:11:19 +0400, Nguyen Tam Chinh wrote:
 Greetings,
 
 We're going to build a server with some 1Tb of over 500 million small
 files with size from 0,5k to 4k.  I'm wonder if the ufs2 can handle
 this kind of system well. From newfs(8) the min block size is 4k. This
 is not optimal in our case, a 1k or 0,5k block is more effective IMHO.
 I'd be happy if anyone can suggest what does fragment (block/8) in the
 ufs2 mean and how this parameter works. I know It's better to read the
 full ufs2 specification, but hope that someone here can give a hint.
 Please advice with optimizations or tricks.
 Thank you very much.
 
 -- 
 With best regards,| The Power to Serve
 Nguyen Tam Chinh  |  http://www.FreeBSD.org
 ___
 [EMAIL PROTECTED] mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-stable
 To unsubscribe, send any mail to [EMAIL PROTECTED]

I am not aware of any ZFS results on such tasks, may be you will be the
one who share them ;) However RaiserFS whould be the best choise on such
spesific case. It's not available on FreeBSD currently.
I don't think UFS can handle a huge amount of small files effectively.  
Of course gjournal could be an option for fsck problems, but how do you
plan to backup or sync this storage?

-- 
==  
- Best regards, Nikolay Pavlov. ---
==  

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: No buffer space available

[Nikolay Pavlov]

On Monday, 23 April 2007 at  4:06:10 -0700, Jeremy Chadwick wrote:
 On Sat, Apr 07, 2007 at 08:20:58PM -0300, Marc G. Fournier wrote:
  - --On Saturday, April 07, 2007 20:12:00 +0100 Chris [EMAIL PROTECTED] 
  wrote:
   Also to add I now have a 2nd box using 6.2 STABLE few days old code,
   had to use it because of broadcom 5755 nic card, I plan to use large
   tcp window sizes so will be interesting to see if this also suffers
   from the problem.
  
  I've got 8 servers on the same network, 3 are almost identical, but one of 
  them 
  (the one with the problem) is using software RAID vs hardware ... but, if 
  you 
  are seeing it without using software RAID, then that is obviously not the 
  culprit :(
 
 May be a red herring...
 
 I'm able to reproduce the No buffer space available message when
 setting net.inet.tcp.(send|recv)space to non-default values.  All I've
 tried is the following, with a kernel dated 2007/04/22:
 
   # sysctl net.inet.tcp.sendspace=131072
   # sysctl net.inet.tcp.recvspace=262144
 
 Example session:
 
 $ su2
 # sysctl net.inet.tcp.sendspace=131072
 net.inet.tcp.sendspace: 32768 - 131072
 # sysctl net.inet.tcp.recvspace=262144
 net.inet.tcp.recvspace: 65536 - 262144
 # logout
 $ ssh medusa
 socket: No buffer space available
 ssh: connect to host medusa port 22: No buffer space available
 $ su2
 # sysctl net.inet.tcp.sendspace=32768
 net.inet.tcp.sendspace: 131072 - 32768
 # sysctl net.inet.tcp.recvspace=65536
 net.inet.tcp.recvspace: 262144 - 65536
 # logout
 $ ssh medusa
 Last login: Mon Apr 23 03:45:45 2007 from ...
 
 I assume this is because the maximum size of a TCP datagram is 65536
 bytes, but as I'm not familiar enough with TCP on such a low level,
 this may be speculation on my part.
 
 Just something worth checking/tinkering with.
 

Try to adjust kern.ipc.maxsockbuf value.

-- 
==  
- Best regards, Nikolay Pavlov. ---
==  

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]