Re: 9.0 install and journaling
On 12/20/2011 2:53 AM, Da Rock wrote: On 12/20/11 16:08, Allen wrote: On 12/13/2011 11:54 AM, Devin Teske wrote: *Snipping* On 12/13/11 06:00, Eric S Pulley wrote: As for one big / partition- linux may be using it: and its their biggest failing! I've had a system lockup due to lack of space. Never a problem with bsd as logs will only fill up var, a user won't break it with filling up usr, etc. And root always stays protected! Its saved my life a number of times... I can quickly fill TB's of data in no time, and if something goes bang the logs can be a silent killer too. My 2c's anyway... I didn't know there WERE any Linux distros that still used one root partition, and one swap... Even Mandriva doesn't do that anymore. Fedora for one... I haven't used Fedora in a VERY long time. I hate it almost as much as I hate Gentoo. Anyway, one distro out of close to 300 isn't bad. -Allen ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 12/13/2011 11:54 AM, Devin Teske wrote: *Snipping* On 12/13/11 06:00, Eric S Pulley wrote: As for one big / partition- linux may be using it: and its their biggest failing! I've had a system lockup due to lack of space. Never a problem with bsd as logs will only fill up var, a user won't break it with filling up usr, etc. And root always stays protected! Its saved my life a number of times... I can quickly fill TB's of data in no time, and if something goes bang the logs can be a silent killer too. My 2c's anyway... I didn't know there WERE any Linux distros that still used one root partition, and one swap... Even Mandriva doesn't do that anymore. -Allen ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 12/20/11 16:08, Allen wrote: On 12/13/2011 11:54 AM, Devin Teske wrote: *Snipping* On 12/13/11 06:00, Eric S Pulley wrote: As for one big / partition- linux may be using it: and its their biggest failing! I've had a system lockup due to lack of space. Never a problem with bsd as logs will only fill up var, a user won't break it with filling up usr, etc. And root always stays protected! Its saved my life a number of times... I can quickly fill TB's of data in no time, and if something goes bang the logs can be a silent killer too. My 2c's anyway... I didn't know there WERE any Linux distros that still used one root partition, and one swap... Even Mandriva doesn't do that anymore. Fedora for one... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 12/13/2011 10:54 AM, Devin Teske wrote: We're seeing in 8.1-RELEASE that nodev is an invalid option for NFS mounts that causes your system to boot into single-user mode. Is this still the case in 9.0-RC2/3 or has the option been re-added? nodev was a valid option in 4.11-RELEASE, not sure why it was removed (and/or made invalid). Since the advent of devfs, device nodes no longer function anywhere other than a devfs-backed filesystem; so 'nodev' is, in a sense, the default. Try it yourself: 8 amani# dd if=/dev/zero count=1 | hd 1+0 records in 1+0 records out 512 bytes transferred in 0.46 secs (11126858 bytes/sec) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 || * 0200 amani# mknod zero c 0 26 amani# dd if=./zero count=1 | hd dd: ./zero: Inappropriate ioctl for device amani# 8 -- Fuzzy love, -CyberLeo Technical Administrator CyberLeo.Net Webhosting http://www.CyberLeo.Net cyber...@cyberleo.net Furry Peace! - http://.fur.com/peace/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
--On Tuesday, December 13, 2011 09:54:38 AM +1000 Da Rock freebsd-questi...@herveybayaustralia.com.au wrote: On 12/13/11 06:00, Eric S Pulley wrote: As for one big / partition- linux may be using it: and its their biggest failing! I've had a system lockup due to lack of space. Never a problem with bsd as logs will only fill up var, a user won't break it with filling up usr, etc. And root always stays protected! Its saved my life a number of times... I can quickly fill TB's of data in no time, and if something goes bang the logs can be a silent killer too. My 2c's anyway... ___ And along those lines for security of the system, this is the U.S. DoD recommendations (well mandates really) including ZFS. Not that the DoD doesn’t have security problems... but I’m not big fan of the one or two mount point solution either… never understood why other OS packagers think is okay to just dump it all under / Per the DISA STIG (Security Technical Implementation Guide) / (obviously) /home directories) /var /tmp /location of audit files should all be separate mount points The use of separate file systems for different paths can protect the system from failures resulting from a file system becoming full or failing... in addition... All local file systems must employ journaling or another mechanism that ensures file system consistency. Removable media, remote file systems, and any file system that does not contain approved device files must be mounted with the nodev option. Removable media, remote file systems, and any file system that does not contain approved setuid files must be mounted with the nosuid option. The nosuid option must be enabled on all NFS client mounts. and so on... you can find a copy of the UNIX STIG online and some of it is just crazy paranoia and makes your life a pain, but there are a lot of good practices in it too. I don't think any of it crazy paranoia. A PITA, maybe, but not paranoid. Do you have a link to the original of it? Sure, http://iase.disa.mil/stigs/ Lots more there than just UNIX too. I find that the newer SRG xml files are easier to just load into a browsers and read the recommendations rather than pouring through the big sections in the STIGs. http://iase.disa.mil/stigs/downloads/zip/unclassified_os-srg-unix_v1r1_finalsrg.zip Or just do the checklists. There are no *BSD specific ones but the the generic UNIX STIG works good (probably because at this point *BSD is basically the reference implementation of UNIX or at least it should be... damn Linux) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: 9.0 install and journaling
-Original Message- From: owner-freebsd-questi...@freebsd.org [mailto:owner-freebsd- questi...@freebsd.org] On Behalf Of Da Rock Sent: Monday, December 12, 2011 3:55 PM To: freebsd-questions@freebsd.org Subject: Re: 9.0 install and journaling On 12/13/11 06:00, Eric S Pulley wrote: As for one big / partition- linux may be using it: and its their biggest failing! I've had a system lockup due to lack of space. Never a problem with bsd as logs will only fill up var, a user won't break it with filling up usr, etc. And root always stays protected! Its saved my life a number of times... I can quickly fill TB's of data in no time, and if something goes bang the logs can be a silent killer too. My 2c's anyway... ___ And along those lines for security of the system, this is the U.S. DoD recommendations (well mandates really) including ZFS. Not that the DoD doesn't have security problems... but I'm not big fan of the one or two mount point solution either. never understood why other OS packagers think is okay to just dump it all under / Per the DISA STIG (Security Technical Implementation Guide) / (obviously) /home directories) /var /tmp /location of audit files should all be separate mount points The use of separate file systems for different paths can protect the system from failures resulting from a file system becoming full or failing... in addition... All local file systems must employ journaling or another mechanism that ensures file system consistency. Removable media, remote file systems, and any file system that does not contain approved device files must be mounted with the nodev option. We're seeing in 8.1-RELEASE that nodev is an invalid option for NFS mounts that causes your system to boot into single-user mode. Is this still the case in 9.0-RC2/3 or has the option been re-added? nodev was a valid option in 4.11-RELEASE, not sure why it was removed (and/or made invalid). -- Devin Removable media, remote file systems, and any file system that does not contain approved setuid files must be mounted with the nosuid option. The nosuid option must be enabled on all NFS client mounts. and so on... you can find a copy of the UNIX STIG online and some of it is just crazy paranoia and makes your life a pain, but there are a lot of good practices in it too. I don't think any of it crazy paranoia. A PITA, maybe, but not paranoid. Do you have a link to the original of it? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: 9.0 install and journaling
--On Tuesday, December 13, 2011 08:54:23 AM -0800 Devin Teske devin.te...@fisglobal.com wrote: We're seeing in 8.1-RELEASE that nodev is an invalid option for NFS mounts that causes your system to boot into single-user mode. Is this still the case in 9.0-RC2/3 or has the option been re-added? nodev was a valid option in 4.11-RELEASE, not sure why it was removed (and/or made invalid). -- Devin No that was just a guideline for generic unix security practices if nodev isn't support by the filesystem there is nothing you can do about it. Not a FreeBSD specific issue. Sorry for the confusion. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On Sun, 11 Dec 2011 15:42:52 +1000 Da Rock wrote: On 12/11/11 10:23, RW wrote: On Sun, 11 Dec 2011 08:17:41 +1000 Da Rock wrote: SUJ speeds up the check a lot, seconds as opposed to minutes. If something happens to the journal, it falls back to a standard fsck. But fsck needs to be run manually- I have users that can't do that, and the filesystem corrupts. Ergo gjournal; it boots up and fixes on the fly. So SU+J needs a manual fsck before booting proper or can it just boot and be done? It's not very different; gjournal and SU both attempt to leave the filesystem in an coherent state, but both still need a preen to recover lost space. In either case the preen can fail requiring a full fsck. Journalled SU make SU behave more like gjournal in that you can do a fast foreground check which avoids the lengthy background fsck and avoids deferring the handling of unexpected inconsistencies to the next boot. Yes, but I don't do a fsck to recover gjournal- it has a miniscule blurp for a nanosecond and prints a message at boot and thats it. If the filesystem is mounted via fstab the fsck is normally done automatically. You may not have noticed this because if nothing needs doing fsck_ufs can mark a gjournal filesystem clean instantaneously. There are two other possibilities. The first is that it may spend some time recovering orphaned files; this is much faster that a full fsck but it's still seconds or minutes. The second is that the journal sync may have failed in which case fsck terminates with UNEXPECTED INCONSISTENCY which requires a full fsck. This is similar to SU. In either case you only need a full fsck when things haven't worked out in line with the theory. Is it the same with su+j? If it does then I'll drop gjournal (and the performance hit) and I'll use su+j when I jump to 9.0. The SU equivalent of the journal sync is done before the crash happens. With SU you can have an instantaneous foreground fsck by deferring the recovery of lost files until the background check that runs after bootup. Journalling SU eliminates the few minutes of sluggish disk IO that that can cause. I've been disappointed by gjournal, the performance hit isn't as bad as background fsck but it is substantial and permanent, rather than a few minutes hare and there. I was hoping that gjournal would be more robust, but I've seen the occassional UNEXPECTED INCONSISTENCY just like I have with SU. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
As for one big / partition- linux may be using it: and its their biggest failing! I've had a system lockup due to lack of space. Never a problem with bsd as logs will only fill up var, a user won't break it with filling up usr, etc. And root always stays protected! Its saved my life a number of times... I can quickly fill TB's of data in no time, and if something goes bang the logs can be a silent killer too. My 2c's anyway... ___ And along those lines for security of the system, this is the U.S. DoD recommendations (well mandates really) including ZFS. Not that the DoD doesnt have security problems... but Im not big fan of the one or two mount point solution either never understood why other OS packagers think is okay to just dump it all under / Per the DISA STIG (Security Technical Implementation Guide) / (obviously) /home directories) /var /tmp /location of audit files should all be separate mount points The use of separate file systems for different paths can protect the system from failures resulting from a file system becoming full or failing... in addition... All local file systems must employ journaling or another mechanism that ensures file system consistency. Removable media, remote file systems, and any file system that does not contain approved device files must be mounted with the nodev option. Removable media, remote file systems, and any file system that does not contain approved setuid files must be mounted with the nosuid option. The nosuid option must be enabled on all NFS client mounts. and so on... you can find a copy of the UNIX STIG online and some of it is just crazy paranoia and makes your life a pain, but there are a lot of good practices in it too. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 12/13/11 06:00, Eric S Pulley wrote: As for one big / partition- linux may be using it: and its their biggest failing! I've had a system lockup due to lack of space. Never a problem with bsd as logs will only fill up var, a user won't break it with filling up usr, etc. And root always stays protected! Its saved my life a number of times... I can quickly fill TB's of data in no time, and if something goes bang the logs can be a silent killer too. My 2c's anyway... ___ And along those lines for security of the system, this is the U.S. DoD recommendations (well mandates really) including ZFS. Not that the DoD doesn’t have security problems... but I’m not big fan of the one or two mount point solution either… never understood why other OS packagers think is okay to just dump it all under / Per the DISA STIG (Security Technical Implementation Guide) / (obviously) /home directories) /var /tmp /location of audit files should all be separate mount points The use of separate file systems for different paths can protect the system from failures resulting from a file system becoming full or failing... in addition... All local file systems must employ journaling or another mechanism that ensures file system consistency. Removable media, remote file systems, and any file system that does not contain approved device files must be mounted with the nodev option. Removable media, remote file systems, and any file system that does not contain approved setuid files must be mounted with the nosuid option. The nosuid option must be enabled on all NFS client mounts. and so on... you can find a copy of the UNIX STIG online and some of it is just crazy paranoia and makes your life a pain, but there are a lot of good practices in it too. I don't think any of it crazy paranoia. A PITA, maybe, but not paranoid. Do you have a link to the original of it? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 12/13/11 04:09, RW wrote: On Sun, 11 Dec 2011 15:42:52 +1000 Da Rock wrote: On 12/11/11 10:23, RW wrote: On Sun, 11 Dec 2011 08:17:41 +1000 Da Rock wrote: SUJ speeds up the check a lot, seconds as opposed to minutes. If something happens to the journal, it falls back to a standard fsck. But fsck needs to be run manually- I have users that can't do that, and the filesystem corrupts. Ergo gjournal; it boots up and fixes on the fly. So SU+J needs a manual fsck before booting proper or can it just boot and be done? It's not very different; gjournal and SU both attempt to leave the filesystem in an coherent state, but both still need a preen to recover lost space. In either case the preen can fail requiring a full fsck. Journalled SU make SU behave more like gjournal in that you can do a fast foreground check which avoids the lengthy background fsck and avoids deferring the handling of unexpected inconsistencies to the next boot. Yes, but I don't do a fsck to recover gjournal- it has a miniscule blurp for a nanosecond and prints a message at boot and thats it. If the filesystem is mounted via fstab the fsck is normally done automatically. You may not have noticed this because if nothing needs doing fsck_ufs can mark a gjournal filesystem clean instantaneously. There are two other possibilities. The first is that it may spend some time recovering orphaned files; this is much faster that a full fsck but it's still seconds or minutes. The second is that the journal sync may have failed in which case fsck terminates with UNEXPECTED INCONSISTENCY which requires a full fsck. This is similar to SU. In either case you only need a full fsck when things haven't worked out in line with the theory. Is it the same with su+j? If it does then I'll drop gjournal (and the performance hit) and I'll use su+j when I jump to 9.0. The SU equivalent of the journal sync is done before the crash happens. With SU you can have an instantaneous foreground fsck by deferring the recovery of lost files until the background check that runs after bootup. Journalling SU eliminates the few minutes of sluggish disk IO that that can cause. I've been disappointed by gjournal, the performance hit isn't as bad as background fsck but it is substantial and permanent, rather than a few minutes hare and there. I was hoping that gjournal would be more robust, but I've seen the occassional UNEXPECTED INCONSISTENCY just like I have with SU. This is going to sound odd, I know, but what does your fstab look like with gjournal? I've only done /var and /usr like this: /dev/ad4s1e.journal /usrufs rw,async2 2 The only message that comes up for me after a crash is consistent or clean. No wait, no fsck. The performance isn't exactly lightning though... :) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 11/12/2011 6:21 AM, Robison, Dave wrote: I prefer having separate partitions because it's more in line with traditional unix systems, and in particular, I don't like letting users have unlimited access to /tmp. Pardon the noob question: will using Disk Quotas work to limit the damage http://www.freebsd.org/doc/handbook/quotas.html ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 12/12/11 12:05, Foo JH wrote: On 11/12/2011 6:21 AM, Robison, Dave wrote: I prefer having separate partitions because it's more in line with traditional unix systems, and in particular, I don't like letting users have unlimited access to /tmp. Pardon the noob question: will using Disk Quotas work to limit the damage http://www.freebsd.org/doc/handbook/quotas.html No, thats a very astute observation. Although its a lot harder to break limits when its an actual partition and everything shows exactly what there is to work with. So I'd personally still use partitioning myself. Quotas could work though, but its an added layer of admin as well. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On Sat, Dec 10, 2011 at 07:51:50PM +1000, R Skinner wrote: I bit the bullet and installed the rc3, after spending half the day fighting to get atheros 9285 working on a new laptop. I have to do another as well, so... After recovering myself from the shock of the new bsdinstall (not bad. A little confusing after using sysinstall for so long), I installed the system with 1 (thats right One! Ah ha ha) partition - yet another shock to figure through. What I'm staggered about is I was using fdisk to setup journaling on the usr and var partitions. So I went to the handbook. I'm still a little confused though: can one still setup the usr and var (and so forth)? It said you possibly could, but it escaped me as to how. And before I do- I looked up journaling on 9. I couldn't quite get to the bottom of whether it is or isn't available/standard, or how to determine its happening. I'm only interested because of unexpected shutdowns/battery dead on the laptop- I also have 500G which is a while to wait for fsck. Speed I'd like, but I have to consider system integrity first. Little light, please? Cheers I'm unfamiliar with the new bsdinstaller but AFAIK it sets up a UFS2 filesystem for you. This comes with background fsck and softupdates which achieve the objective of not having to wait for a lengthy foreground fsck if you don't shutdown your laptop cleanly. As for filesystem integrity, I've occasionally not shutdown properly and the system has subsequently come up quickly again with the background fsck doing it's stuff. I don't remember anybody posting to this list saying: Help! I didn't shutdown my machine properly and now my filesystem is toast. At the worst you may have to run a foreground fsck. You can set up a proper journal: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/geom-gjournal.html but to be honest, I wouldn't bother in your position: it's just more stuff to go wrong for no appreciable gain to you. Regards, -- Frank Contact info: http://www.shute.org.uk/misc/contact.html pgp5taXd3fBLr.pgp Description: PGP signature
Re: 9.0 install and journaling
On Sat, 10 Dec 2011 11:40:53 + Frank Shute wrote: On Sat, Dec 10, 2011 at 07:51:50PM +1000, R Skinner wrote: possibly could, but it escaped me as to how. And before I do- I looked up journaling on 9. I couldn't quite get to the bottom of whether it is or isn't available/standard, or how to determine its happening. I'm only interested because of unexpected shutdowns/battery dead on the laptop- I also have 500G which is a while to wait for fsck. Speed I'd like, but I have to consider system integrity first. I'm unfamiliar with the new bsdinstaller but AFAIK it sets up a UFS2 filesystem for you. This comes with background fsck and softupdates which achieve the objective of not having to wait for a lengthy foreground fsck if you don't shutdown your laptop cleanly. but to be honest, I wouldn't bother in your position: it's just more stuff to go wrong for no appreciable gain to you. 9.0 also supports soft-update journalling which eliminates the background fsck. If you don't know whether it's on or not you can run tunefs -p / If it's not on then tunefs can turn it on, but you will presumably need to reboot into single user mode. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 12/10/11 22:20, RW wrote: On Sat, 10 Dec 2011 11:40:53 + Frank Shute wrote: On Sat, Dec 10, 2011 at 07:51:50PM +1000, R Skinner wrote: possibly could, but it escaped me as to how. And before I do- I looked up journaling on 9. I couldn't quite get to the bottom of whether it is or isn't available/standard, or how to determine its happening. I'm only interested because of unexpected shutdowns/battery dead on the laptop- I also have 500G which is a while to wait for fsck. Speed I'd like, but I have to consider system integrity first. I'm unfamiliar with the new bsdinstaller but AFAIK it sets up a UFS2 filesystem for you. This comes with background fsck and softupdates which achieve the objective of not having to wait for a lengthy foreground fsck if you don't shutdown your laptop cleanly. but to be honest, I wouldn't bother in your position: it's just more stuff to go wrong for no appreciable gain to you. 9.0 also supports soft-update journalling which eliminates the background fsck. If you don't know whether it's on or not you can run tunefs -p / If it's not on then tunefs can turn it on, but you will presumably need to reboot into single user mode. So how does soft-update journaling compare to gjournal? I'm using gjournal now and it runs a bit of a dog, but it is reliable (until another ufs filesystem turns up at boot) and necessary in my environment. Can I dump it for this new one? Its used on a laptop with heavy load on the disk, the power on the battery can run out too quick for batterymon to shut it down- plus kids that play silly monkeys with daddy's laptop... :) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 10.12.2011 15:44, Da Rock wrote: So how does soft-update journaling compare to gjournal? I'm using gjournal now and it runs a bit of a dog, but it is reliable (until another ufs filesystem turns up at boot) and necessary in my environment. Can I dump it for this new one? Its used on a laptop with heavy load on the disk, the power on the battery can run out too quick for batterymon to shut it down- plus kids that play silly monkeys with daddy's laptop... :) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org I am not an expert, but this is how I currently understand this: Geom-journal is a block level journaling mechanism. It means that all data written on a journaled provider will be first written to the providers journal and then later commited to the provider. Thus data is written twice to provide more redundancy. Soft-updates by itself has nothing to do with any journaling. It simply makes synchronous writes faster. When a filesystem is mounted synchronously, each write to a file first updates the metadata of the file, such a file size, and after that writes the actual data into the disk. When doing operations on thousands of small files, the overhead from synchronous writing is considerable. Bring forth soft-updates, which caches the metadata (file size etc.) updates into ram, and commits these updates into disk later when a delay and some conditions trigger; actual data is written into disk as soon as possible. Thus append/random access operations on gazillion of files are a lot faster. From this I would conclude that soft-updates journaling tries to write the metadata updates into a filesystem spesific soft-update journal (the soft-update journal being a file maybe?) sooner than they are actually committed. I also suspect that soft-updates journaling has some clever way of detecting when it is, performance wise, okay to update the journal and/or commit updates. In short: - gjournal writes both data and metadata, providing more redundancy to the actual data AND filesystem integrity while slowing down write operations (although I do not have data how much it slows down disk i/o) - journaled soft-updates writes metadata updates twice on the disk, providing redundancy to file system integrity, but does not prevent actual data loss. I do not even use FreeBSD 9.0 myself, my information is totally third party, second hand, wise-cracks and guessing. Any expert consultation would be appreciated. -- Arto Pekkanen ksym@IRCnet ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On Sat, 10 Dec 2011, R Skinner wrote: So I went to the handbook. I'm still a little confused though: can one still setup the usr and var (and so forth)? It said you possibly could, but it escaped me as to how. Use the bsdinstall partition editor to manually create the partitions. I documented how to create an old-fashioned MBR layout with bsdinstall on the forums a while back: http://forums.freebsd.org/showpost.php?p=149210postcount=13 The process would be similar for GPT, which is really the way to go now. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 10/12/2011 5:19 μμ, Warren Block wrote: On Sat, 10 Dec 2011, R Skinner wrote: So I went to the handbook. I'm still a little confused though: can one still setup the usr and var (and so forth)? It said you possibly could, but it escaped me as to how. Use the bsdinstall partition editor to manually create the partitions. I documented how to create an old-fashioned MBR layout with bsdinstall on the forums a while back: http://forums.freebsd.org/showpost.php?p=149210postcount=13 The process would be similar for GPT, which is really the way to go now. As Warren says, you can still create /usr and /var and all the other legacy partitions if you so wish - and you may even use the full journaling (gjournal) on them. But the default for bsdinstall is to use gpart, install everything on a big / and create UFS2 partitions with the new soft-updates journaling system (on by default). Compared to gjournal, soft-updates journaling only journals metadata and not everything like gjournal does. This will definitely make it faster although probably less safe than gjournal. It should be good for most purposes though and needs no additional steps after install (unlike gjournal). Since it's the default, the decision to go for one big / seems ok after all. I believe this is more or less what Linux is doing with Ext3/Ext4 filesystems (metadata journaling). ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 12/11/11 02:09, Manolis Kiagias wrote: On 10/12/2011 5:19 μμ, Warren Block wrote: On Sat, 10 Dec 2011, R Skinner wrote: So I went to the handbook. I'm still a little confused though: can one still setup the usr and var (and so forth)? It said you possibly could, but it escaped me as to how. Use the bsdinstall partition editor to manually create the partitions. I documented how to create an old-fashioned MBR layout with bsdinstall on the forums a while back: http://forums.freebsd.org/showpost.php?p=149210postcount=13 The process would be similar for GPT, which is really the way to go now. As Warren says, you can still create /usr and /var and all the other legacy partitions if you so wish - and you may even use the full journaling (gjournal) on them. But the default for bsdinstall is to use gpart, install everything on a big / and create UFS2 partitions with the new soft-updates journaling system (on by default). Compared to gjournal, soft-updates journaling only journals metadata and not everything like gjournal does. This will definitely make it faster although probably less safe than gjournal. It should be good for most purposes though and needs no additional steps after install (unlike gjournal). Since it's the default, the decision to go for one big / seems ok after all. I believe this is more or less what Linux is doing with Ext3/Ext4 filesystems (metadata journaling). GPT is cool - no problems there. The main thing I want to know is if I need to run fsck every time the system dies unexpectedly (which is a higher occurrence on a laptop)? GJournal helps in that it takes care of that. The growing size of drives is another concern given the time it takes to check a 500G disk (my smallest atm), although this is way down on the list for the moment. As for one big / partition- linux may be using it: and its their biggest failing! I've had a system lockup due to lack of space. Never a problem with bsd as logs will only fill up var, a user won't break it with filling up usr, etc. And root always stays protected! Its saved my life a number of times... I can quickly fill TB's of data in no time, and if something goes bang the logs can be a silent killer too. My 2c's anyway... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 10/12/2011 11:41 μμ, Da Rock wrote: On 12/11/11 02:09, Manolis Kiagias wrote: On 10/12/2011 5:19 μμ, Warren Block wrote: On Sat, 10 Dec 2011, R Skinner wrote: So I went to the handbook. I'm still a little confused though: can one still setup the usr and var (and so forth)? It said you possibly could, but it escaped me as to how. Use the bsdinstall partition editor to manually create the partitions. I documented how to create an old-fashioned MBR layout with bsdinstall on the forums a while back: http://forums.freebsd.org/showpost.php?p=149210postcount=13 The process would be similar for GPT, which is really the way to go now. As Warren says, you can still create /usr and /var and all the other legacy partitions if you so wish - and you may even use the full journaling (gjournal) on them. But the default for bsdinstall is to use gpart, install everything on a big / and create UFS2 partitions with the new soft-updates journaling system (on by default). Compared to gjournal, soft-updates journaling only journals metadata and not everything like gjournal does. This will definitely make it faster although probably less safe than gjournal. It should be good for most purposes though and needs no additional steps after install (unlike gjournal). Since it's the default, the decision to go for one big / seems ok after all. I believe this is more or less what Linux is doing with Ext3/Ext4 filesystems (metadata journaling). GPT is cool - no problems there. The main thing I want to know is if I need to run fsck every time the system dies unexpectedly (which is a higher occurrence on a laptop)? GJournal helps in that it takes care of that. The growing size of drives is another concern given the time it takes to check a 500G disk (my smallest atm), although this is way down on the list for the moment. It does the fsck automatically and it seems to be fast. As with other metadata journaled filesystems you will probably have to do a full check occasionally. Can't you give you any times atm, I need to dump /repartition/restore some of my systems to use su+j. Only tested on virtual machines. As for one big / partition- linux may be using it: and its their biggest failing! I've had a system lockup due to lack of space. Never a problem with bsd as logs will only fill up var, a user won't break it with filling up usr, etc. And root always stays protected! Its saved my life a number of times... I can quickly fill TB's of data in no time, and if something goes bang the logs can be a silent killer too. My 2c's anyway... I am used to the separate partitions too, although I realize a single big / would be suitable for more than a few systems. It's nice we have a choice here. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 12/11/11 08:02, Manolis Kiagias wrote: On 10/12/2011 11:41 μμ, Da Rock wrote: On 12/11/11 02:09, Manolis Kiagias wrote: On 10/12/2011 5:19 μμ, Warren Block wrote: On Sat, 10 Dec 2011, R Skinner wrote: So I went to the handbook. I'm still a little confused though: can one still setup the usr and var (and so forth)? It said you possibly could, but it escaped me as to how. Use the bsdinstall partition editor to manually create the partitions. I documented how to create an old-fashioned MBR layout with bsdinstall on the forums a while back: http://forums.freebsd.org/showpost.php?p=149210postcount=13 The process would be similar for GPT, which is really the way to go now. As Warren says, you can still create /usr and /var and all the other legacy partitions if you so wish - and you may even use the full journaling (gjournal) on them. But the default for bsdinstall is to use gpart, install everything on a big / and create UFS2 partitions with the new soft-updates journaling system (on by default). Compared to gjournal, soft-updates journaling only journals metadata and not everything like gjournal does. This will definitely make it faster although probably less safe than gjournal. It should be good for most purposes though and needs no additional steps after install (unlike gjournal). Since it's the default, the decision to go for one big / seems ok after all. I believe this is more or less what Linux is doing with Ext3/Ext4 filesystems (metadata journaling). GPT is cool - no problems there. The main thing I want to know is if I need to run fsck every time the system dies unexpectedly (which is a higher occurrence on a laptop)? GJournal helps in that it takes care of that. The growing size of drives is another concern given the time it takes to check a 500G disk (my smallest atm), although this is way down on the list for the moment. It does the fsck automatically and it seems to be fast. As with other metadata journaled filesystems you will probably have to do a full check occasionally. Can't you give you any times atm, I need to dump /repartition/restore some of my systems to use su+j. Only tested on virtual machines. I'll have to try it out then; give it a chance. As for one big / partition- linux may be using it: and its their biggest failing! I've had a system lockup due to lack of space. Never a problem with bsd as logs will only fill up var, a user won't break it with filling up usr, etc. And root always stays protected! Its saved my life a number of times... I can quickly fill TB's of data in no time, and if something goes bang the logs can be a silent killer too. My 2c's anyway... I am used to the separate partitions too, although I realize a single big / would be suitable for more than a few systems. It's nice we have a choice here. True. But as a new user it was the separate partitions that attracted me, having been burned with linux's megaroot. And a new user would have trouble setting up the partitions. Not to mention the break with tradition (what is happening to this world)! :) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On Sun, 11 Dec 2011, Da Rock wrote: GPT is cool - no problems there. The main thing I want to know is if I need to run fsck every time the system dies unexpectedly (which is a higher occurrence on a laptop)? GJournal helps in that it takes care of that. The growing size of drives is another concern given the time it takes to check a 500G disk (my smallest atm), although this is way down on the list for the moment. SUJ speeds up the check a lot, seconds as opposed to minutes. If something happens to the journal, it falls back to a standard fsck. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 12/11/11 08:14, Warren Block wrote: On Sun, 11 Dec 2011, Da Rock wrote: GPT is cool - no problems there. The main thing I want to know is if I need to run fsck every time the system dies unexpectedly (which is a higher occurrence on a laptop)? GJournal helps in that it takes care of that. The growing size of drives is another concern given the time it takes to check a 500G disk (my smallest atm), although this is way down on the list for the moment. SUJ speeds up the check a lot, seconds as opposed to minutes. If something happens to the journal, it falls back to a standard fsck. But fsck needs to be run manually- I have users that can't do that, and the filesystem corrupts. Ergo gjournal; it boots up and fixes on the fly. So SU+J needs a manual fsck before booting proper or can it just boot and be done? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: 9.0 install and journaling
True. But as a new user it was the separate partitions that attracted me, having been burned with linux's megaroot. And a new user would have trouble setting up the partitions. Not to mention the break with tradition (what is happening to this world)! :) I prefer having separate partitions because it's more in line with traditional unix systems, and in particular, I don't like letting users have unlimited access to /tmp. /tmp isn't a place for people to dump their downloads, large file copies, etc. They should do that in their home directories. Having one big partition only allows people to abuse /tmp, among other things. _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On Sun, 11 Dec 2011 08:17:41 +1000 Da Rock wrote: SUJ speeds up the check a lot, seconds as opposed to minutes. If something happens to the journal, it falls back to a standard fsck. But fsck needs to be run manually- I have users that can't do that, and the filesystem corrupts. Ergo gjournal; it boots up and fixes on the fly. So SU+J needs a manual fsck before booting proper or can it just boot and be done? It's not very different; gjournal and SU both attempt to leave the filesystem in an coherent state, but both still need a preen to recover lost space. In either case the preen can fail requiring a full fsck. Journalled SU make SU behave more like gjournal in that you can do a fast foreground check which avoids the lengthy background fsck and avoids deferring the handling of unexpected inconsistencies to the next boot. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 9.0 install and journaling
On 12/11/11 10:23, RW wrote: On Sun, 11 Dec 2011 08:17:41 +1000 Da Rock wrote: SUJ speeds up the check a lot, seconds as opposed to minutes. If something happens to the journal, it falls back to a standard fsck. But fsck needs to be run manually- I have users that can't do that, and the filesystem corrupts. Ergo gjournal; it boots up and fixes on the fly. So SU+J needs a manual fsck before booting proper or can it just boot and be done? It's not very different; gjournal and SU both attempt to leave the filesystem in an coherent state, but both still need a preen to recover lost space. In either case the preen can fail requiring a full fsck. Journalled SU make SU behave more like gjournal in that you can do a fast foreground check which avoids the lengthy background fsck and avoids deferring the handling of unexpected inconsistencies to the next boot. Yes, but I don't do a fsck to recover gjournal- it has a miniscule blurp for a nanosecond and prints a message at boot and thats it. Is it the same with su+j? If it does then I'll drop gjournal (and the performance hit) and I'll use su+j when I jump to 9.0. I've never done fsck on a gjournal (yet). ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
