Re: Optimal UFS parameters
:In message [EMAIL PROTECTED] Matt Dillon writes: :: : -b 16384 -f 4096 -c 159 :: I think Bruce swears by 4K (page-sized) fragments. Not a bad :: way to go. I use 2K because I (and others) put in so much hard work :: to fix all the little niggling bugs in the VM system related to partial :: page validation and, damn it, I intend to use those features! : :At the other end of the spectrum, 32M [sic] and 64M [sic] disks work :well with : -b 4096 -f 512 -c 10 : :But I tend to do what phk has done with the large -c flags on my :insanely-sized, rediculously-cheap XXG IDE drives. : :Warner Well, too-large a C/G will result in greater file fragmentation, because FFS can't manage the file layouts in the cylinder groups as well. The default of 16 is definitely too little. 100+ is probably too much. Something in the middle will be about right. The fragmentation value returned by fsck would be an interesting number to publish. 'fsck -n /dev/...' on an idle fs (you don't have to unmount it). Anything over 3% fragmentation is a problem. Something like /usr will typically be in the 1-3% range. A large partition that is still half empty should be in the 0.0-0.5% range. A combination of a larger C/G (meaning fewer groups on the disk) and fewer inodes (a higher -i value) will dramatically decrease fsck times. After a certain point, though, continuing to increase C/G will not effect the fsck times. So. Previously, FreeBSD had various issues with larger block and fragment sizes - I think Matt was the guy who told me this. A larger B/F size also allows a larger C/G, so I'm wondering if this is still true (both for FreeBSD 3.5 stable and 4.2 stable). Comments? -- ... Joe --- Joe Greco - Systems Administrator [EMAIL PROTECTED] Solaria Public Access UNIX - Milwaukee, WI 414/342-4847 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
[EMAIL PROTECTED] schrieb: How frequently do people fsck? Only at boot time, or when problems surface. Just my $.02 -Christoph Sold To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
On Fri, Dec 08, 2000 at 05:53:18AM +, [EMAIL PROTECTED] wrote: How frequently do people fsck? Once per reboot usually. Joe -- Josef Karthauser[[EMAIL PROTECTED], [EMAIL PROTECTED]] . FreeBSD: The power to change the world To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
[EMAIL PROTECTED] wrote: How frequently do people fsck? Well, that depends on whether I'm attached atm or not. Oh, you mean filesystems? :-) -- Daniel C. Sobral(8-DCS) [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] "The bronze landed last, which canceled that method of impartial choice." To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
:On Fri, Dec 08, 2000 at 05:53:18AM +, [EMAIL PROTECTED] wrote: : : How frequently do people fsck? : :Once per reboot usually. : :Joe :-- :Josef Karthauser [[EMAIL PROTECTED], [EMAIL PROTECTED]] No, that's an fsck -p ... if the filesystem is clean, it doesn't do anything. fsck scans the filesystem only when booting after a crash. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
In message [EMAIL PROTECTED], A G F Keahan writes: What parameters should I choose for a large (say, 60 or 80Gb) filesystem? I remember a while ago someone (phk?) conducted a survey, but nothing seems to have come of it. In the meantime, the capacity of an average hard drive has increased tenfold, and the defaults have become even less reasonable. What's the current consensus of opinion? newfs -b ? -f ? -c ? Right now I tend to use: -b 16384 -f 4096 -c 159 -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
* Poul-Henning Kamp [EMAIL PROTECTED] [001207 00:12] wrote: In message [EMAIL PROTECTED], A G F Keahan writes: What parameters should I choose for a large (say, 60 or 80Gb) filesystem? I remember a while ago someone (phk?) conducted a survey, but nothing seems to have come of it. In the meantime, the capacity of an average hard drive has increased tenfold, and the defaults have become even less reasonable. What's the current consensus of opinion? newfs -b ? -f ? -c ? Right now I tend to use: -b 16384 -f 4096 -c 159 I know you're pretty busy, but any chance of getting this into sysinstall? Maybe hindged on the size of the partition? -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
: : Right now I tend to use: : : -b 16384 -f 4096 -c 159 : :I know you're pretty busy, but any chance of getting this into :sysinstall? Maybe hindged on the size of the partition? : :-- :-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] :"I have the heart of a child; I keep it in a jar on my desk." It would be nice to up the default cylinders/group in sysinstall for larger partitions (anything over 8GB). I wouldn't up it to 159 as a default, but 32 would be a whole lot better then the current default 16, especially for fsck times. I think the default fragment/block should still be 1K/8K, even though I use 2K/16K on my own partitions. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
:In message [EMAIL PROTECTED], A G F Keahan writes: :What parameters should I choose for a large (say, 60 or 80Gb) :filesystem? I remember a while ago someone (phk?) conducted a survey, :but nothing seems to have come of it. In the meantime, the capacity of :an average hard drive has increased tenfold, and the defaults have :become even less reasonable. : :What's the current consensus of opinion? : :newfs -b ? -f ? -c ? : :Right now I tend to use: : : -b 16384 -f 4096 -c 159 : :-- :Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 :[EMAIL PROTECTED] | TCP/IP since RFC 956 I think Bruce swears by 4K (page-sized) fragments. Not a bad way to go. I use 2K because I (and others) put in so much hard work to fix all the little niggling bugs in the VM system related to partial page validation and, damn it, I intend to use those features! -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
In message [EMAIL PROTECTED], Alfred Perlstein writes: Right now I tend to use: -b 16384 -f 4096 -c 159 I know you're pretty busy, but any chance of getting this into sysinstall? Maybe hindged on the size of the partition? sysinstall supports you changing the args to newfs, it has for a long time. The default lives in the options screen. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
* Poul-Henning Kamp [EMAIL PROTECTED] [001207 00:25] wrote: In message [EMAIL PROTECTED], Alfred Perlstein writes: Right now I tend to use: -b 16384 -f 4096 -c 159 I know you're pretty busy, but any chance of getting this into sysinstall? Maybe hindged on the size of the partition? sysinstall supports you changing the args to newfs, it has for a long time. The default lives in the options screen. I know that, I meant making it automagically scale as Matt suggested. I'd do it, but I don't really have a grasp on the optimal parameters to set based on FS size. -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
It would be nice to up the default cylinders/group in sysinstall for larger partitions (anything over 8GB). I wouldn't up it to 159 as a default, but 32 would be a whole lot better then the Well, if somebody wants to figure out the best defaults, they're easily set in sysinstall/label.c:new_part(); just calcuate the values dynamically rather than copying in the contents of VAR_NEWFS_ARGS. I'm sure it'd be a snap for a bunch of bright guys like you. :) - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
In message [EMAIL PROTECTED], Alfred Perlstein writes: I'd do it, but I don't really have a grasp on the optimal parameters to set based on FS size. So far I don't see any indication here (or elsewhere) that anybody has that grasp. I guess that is really a testimony to FFS/UFS's qualites... The main thing is that you significantly reduce your fsck time if you reduce the number of inodes. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
In message [EMAIL PROTECTED], Alfred Perlstein writes: So far I don't see any indication here (or elsewhere) that anybody has that grasp. I guess that is really a testimony to FFS/UFS's qualites... The main thing is that you significantly reduce your fsck time if you reduce the number of inodes. Oh, your tunables just reduce the number of inodes? That may come as a suprise to people that are using the larger disks to store images and web/ftp stuff. No they don't, I mainly reduce the number of cylinder-groups. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
Matt Dillon [EMAIL PROTECTED] wrote: The default filesystem parameters are: newfs -f 1024 -b 8192 -i 8192 -c 16 ... -i 4096 -- Christian "naddy" Weisgerber [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
In message [EMAIL PROTECTED] Matt Dillon writes: : : -b 16384 -f 4096 -c 159 : I think Bruce swears by 4K (page-sized) fragments. Not a bad : way to go. I use 2K because I (and others) put in so much hard work : to fix all the little niggling bugs in the VM system related to partial : page validation and, damn it, I intend to use those features! At the other end of the spectrum, 32M [sic] and 64M [sic] disks work well with -b 4096 -f 512 -c 10 But I tend to do what phk has done with the large -c flags on my insanely-sized, rediculously-cheap XXG IDE drives. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
: :In message [EMAIL PROTECTED] Matt Dillon writes: :: :-b 16384 -f 4096 -c 159 :: I think Bruce swears by 4K (page-sized) fragments. Not a bad :: way to go. I use 2K because I (and others) put in so much hard work :: to fix all the little niggling bugs in the VM system related to partial :: page validation and, damn it, I intend to use those features! : :At the other end of the spectrum, 32M [sic] and 64M [sic] disks work :well with : -b 4096 -f 512 -c 10 : :But I tend to do what phk has done with the large -c flags on my :insanely-sized, rediculously-cheap XXG IDE drives. : :Warner Well, too-large a C/G will result in greater file fragmentation, because FFS can't manage the file layouts in the cylinder groups as well. The default of 16 is definitely too little. 100+ is probably too much. Something in the middle will be about right. The fragmentation value returned by fsck would be an interesting number to publish. 'fsck -n /dev/...' on an idle fs (you don't have to unmount it). Anything over 3% fragmentation is a problem. Something like /usr will typically be in the 1-3% range. A large partition that is still half empty should be in the 0.0-0.5% range. A combination of a larger C/G (meaning fewer groups on the disk) and fewer inodes (a higher -i value) will dramatically decrease fsck times. After a certain point, though, continuing to increase C/G will not effect the fsck times. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
This is a interesting topic (to me, anyway), and is one of the things that often gets overlooked by those of us with less experience. Rather than getting into a long discussion about modifying the newfs defaults across the board, what if the newfs options used were based on the size of the FS? There could be a simple rule in sysinstall that increased the newfs options from their default values if the defaults meant there would be more an x number of inodes and cg's. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Optimal UFS parameters
How frequently do people fsck? -- TJ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Optimal UFS parameters
What parameters should I choose for a large (say, 60 or 80Gb) filesystem? I remember a while ago someone (phk?) conducted a survey, but nothing seems to have come of it. In the meantime, the capacity of an average hard drive has increased tenfold, and the defaults have become even less reasonable. What's the current consensus of opinion? newfs -b ? -f ? -c ? Thanks Alex Keahan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
RE: Optimal UFS parameters
That's because any "consensus" would be inappropriate for mass consumtion. It really depends on a lot of fun things like the average file size and the number of files that the drives will be storing. For example, a mail server might want more inodes than a database server. The mail server will likely have a lot of tiny files where the database server would have a collection of much larger (a few k vs several mb's each). What makes you think the defaults are unreasonable? I set up a 300GB filesystem a few months ago. I ran a few numbers, calculated my average file size, compared it to the defaults and found they were very close to reasonable. When I get a couple hundred gig's of data on there I'll know better but I think my guess-timates are very good. Matt -Original Message- From: A G F Keahan [mailto:[EMAIL PROTECTED]] Sent: Wednesday, December 06, 2000 7:53 PM To: [EMAIL PROTECTED] Subject: Optimal UFS parameters What parameters should I choose for a large (say, 60 or 80Gb) filesystem? I remember a while ago someone (phk?) conducted a survey, but nothing seems to have come of it. In the meantime, the capacity of an average hard drive has increased tenfold, and the defaults have become even less reasonable. What's the current consensus of opinion? newfs -b ? -f ? -c ? Thanks Alex Keahan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: RE: Optimal UFS parameters
:That's because any "consensus" would be inappropriate for mass consumtion. :It really depends on a lot of fun things like the average file size and the :number of files that the drives will be storing. For example, a mail server :might want more inodes than a database server. The mail server will likely :have a lot of tiny files where the database server would have a collection :of much larger (a few k vs several mb's each). : :What makes you think the defaults are unreasonable? I set up a 300GB :filesystem a few months ago. I ran a few numbers, calculated my average file :size, compared it to the defaults and found they were very close to :reasonable. When I get a couple hundred gig's of data on there I'll know :better but I think my guess-timates are very good. : :Matt : : -Original Message- : From: A G F Keahan [mailto:[EMAIL PROTECTED]] : Sent: Wednesday, December 06, 2000 7:53 PM : To: [EMAIL PROTECTED] : Subject: Optimal UFS parameters : : What parameters should I choose for a large (say, 60 or 80Gb) : filesystem? I remember a while ago someone (phk?) conducted a survey, : but nothing seems to have come of it. In the meantime, the capacity of : an average hard drive has increased tenfold, and the defaults have : become even less reasonable. : : What's the current consensus of opinion? : : newfs -b ? -f ? -c ? Well, in general I think the defaults are a little overkill... but that may be a good thing. I don't recall us ever getting more then a handful of complaints about a filesystem running out of inodes. Running out of inodes is really annoying and it is best to avoid it. Still, unless your large partition is being used for something like, oh, /home in a multi-user environment, you can probably optimize the newfs parameters a bit to reduce fsck times and indirect block lookup overhead. The default filesystem parameters are: newfs -f 1024 -b 8192 -i 8192 -c 16 ... If you are not going to have a lot of tiny files I would recommend something like this: newfs -f 2048 -b 16384 -i 16384 -c 32 ... You can play with -c and -i, but for a production system the block size (-b) should be either 8192 (the default), or 16384. The filesystem buffer cache is only tuned well for those two sizes and going larger won't help anyway since the kernel already clusters adjacent blocks. Doubling -i from the default halves the number of inodes available. Doubling the cylinders per group reduces the number of allocation groups. If you reduce the number of groups too much your filesystems will become more prone to fragmentation, so don't go overboard. If you increase the number of bytes/inode (-i) too much the filesystem will not have enough inodes and you will run out. For a general purpose filesystem I would not go above -i 16384 -c 64. If the filesystem is going to house a big database (which has many fewer files), you can use a much larger -i but you still shouldn't go overboard with -c. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
