NFS Cache
Greetings, We are having problems with Record Based Log Shipping from our PostGreSQL database server that appears to stem from NFS caching the files. PostGreSQL are continously writing to its WAL files located in the pg_xlog directory which is shared via NFS Our program is in turn continously detecting the changes as described in 24.4.4 at http://www.postgresql.org/docs/8.3/static/warm-standby.html and copying a segment of the WAL file to the local harddisk. The changes written by PostGreSQL however doesn't appear to be reflected through the mapped NFS drive until at some later point in time (not sure how long the delay is, but it appears to be 10+ seconds). The delay causes the transferred WAL files to become corrupt. Running the same program directly on the PostGreSQL machine provides the expected result. The following is the NFS configuration on both the Client and Server machines: sysctl -a |grep nfs vfs.nfs.downdelayinitial: 12 vfs.nfs.downdelayinterval: 30 vfs.nfs.skip_wcc_data_onerr: 1 vfs.nfs.nfs3_jukebox_delay: 10 vfs.nfs.reconnects: 0 vfs.nfs.bufpackets: 4 vfs.nfs.realign_count: 0 vfs.nfs.realign_test: 0 vfs.nfs.defect: 0 vfs.nfs.iodmax: 20 vfs.nfs.iodmin: 0 vfs.nfs.iodmaxidle: 120 vfs.nfs.diskless_rootpath: vfs.nfs.diskless_valid: 0 vfs.nfs.nfs_ip_paranoia: 1 vfs.nfs.nfs_directio_allow_mmap: 1 vfs.nfs.nfs_directio_enable: 1 vfs.nfs.clean_pages_on_close: 1 vfs.nfs.nfsv3_commit_on_close: 0 vfs.nfs.access_cache_timeout: 0 vfs.nfs4.access_cache_timeout: 0 vfs.nfsrv.nfs_privport: 0 vfs.nfsrv.commit_miss: 0 vfs.nfsrv.commit_blks: 0 vfs.nfsrv.async: 0 vfs.nfsrv.realign_count: 0 vfs.nfsrv.realign_test: 4069 vfs.nfsrv.gatherdelay_v3: 0 vfs.nfsrv.gatherdelay: 1 Are there other ways of disabling the NFS' cache than using sysctl? Preferably at mount time so caching is only disabled for the PostGreSQL mount point. Alternatively, how is NFS forced to use Direct IO? Appreciate the input Cheers Jona ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD 7.0 fdisk issue during installation
Thank you both for the very detailed description. It's nice to get my suspicion about boot sequencing confirmed :-) When I installed the system yesterday (I think I'll try a re-install today based on your input) I observed however that all the slices I made appeared to be bootable. As originally mentioned I managed to get the system to boot when I only had 4 slices which resulted in the system only wanting to boot through the FreeBSD Boot Manager. The Boot menu listed 4 menu items, each called FreeBSD. If I only used the MBR to boot however, then the machine would fail with Invalid Partition Table during startup. I'm quite confident that I had only specified a single slice (/) as being bootable through fdisk in sysinstall but apparently all 4 of them were made bootable anyway. I took a picture of my fdisk screen which can be found at http://demo.ois-inc.com/freebsd_fdisk.jpg From the picture it appears (to me anyway) that only the first slice should be bootable as indicated by the A flag? Not sure what to make of this observation as you both indicate that I wouldn't have had these problems if I only had made 1 slice bootable. Does sysinstall make all slices bootable automatically? Appreciate any input you may have to this observation. /Jona On Fri, Nov 21, 2008 at 11:58 PM, Jerry McAllister [EMAIL PROTECTED] wrote: On Fri, Nov 21, 2008 at 10:41:07PM +0100, Jonatan Evald Buus wrote: Hi Jerry, Thank you for the swift and very thorough response. If I understand you correctly, then I should only create 1 slice of the entire disk (seeing as FreeBSD will be the only OS) using fdisk and then partition the slice using bsdlabels from sysinstall? Yes. Or you don't have to use sysinstall. You can do it manually. But, using sysinstall makes it easy. You don't absolutely have to slice or bsdlabel it. You can either just newfs the device /dev/da0 or you can create a slice and just newfs that /dev/da0s1. Then you get that 'dangerously dedicated' disk which FreeBSD can use, but nothing else can including some non-FreeBSD boot managers. Some people do that to save a couple thousand bytes of space, but on a multi-gigabyte drive, who cares about a couple thousand bytes. Previously I was aiming for 5 slices, each of which had a single partition as described below. Yup. That won't work. From your explanation I take it that slices are what Windows refers to as Primary Partitions? If that's the case then I understand the behaviour I experienced. Yes. There is that conflict of terminology. But, FreeBSD has called it slices from the beginning. Is it possible to make a slice non-bootable? Yes. Just don't put in an MBR and don't mark it bootable in the fdisk stage. And would there be any benefits (less fragmentation, faster access time etc.) in using slices rather than partitions to layout the harddrive or should slices only be used to represent a physical harddrive? There is no advantage in making a slice non-bootable, except you might be able to save a few bytes of storage - storage that is not normally used anyway. There is no advantage in speed or access time and fragmentation is only a MS worry. It is not an issue in superior UNIX filesystems - at least in FreeBSD's. I don't understand the last line of that paragraph. Pretty much everything is virtual in disk drive addressing nowdays. It doesn't matter which level you refer to. The slice and its limit to 4 is a feature of standard BIOS basically. All the other things, partitions, extended partitions, etc are ways of getting around the limits.The only real reason nowdays to have more than one slice on a drive in FreeBSD is if you want to put more than one bootable system on the drive. For example, the machine I am typing on has MS-XP and FreeBSD, plus a Dell diagnostic slice - so three slices are used. I could squish those slices down and add one more, say for Linux or a different version of FreeBSD if I wanted, but I don't. Generally, when I make a machine intended only for FreeBSD, I put all the disk in one bootable slice. Then I partition that slice to suit me. My pattern is usually: a / (root) b swap (125% of memory size) c defines the slice - not a real partition d /tmp (used as scratch space by many utilities) e /usr f /var (size depends on logging and databases which live here) g /home(user home directories, plus I put some of the things that can grow unexpectedly such as /var/mail, /var/spool /usr/ports, /usr/local here and make symlinks to them) Some people make just one big partition for root plus some for swap. I like the control I have over things my way a little better and I can get by with backing up and restoring more manageable chunks my way. If the machine is to be a dual boot as this one is, I carve it up in to slices - one
FreeBSD 7.0 fdisk issue during installation
Greetings, I tried to install FreeBSD 7.0 on an old server earlier today and ran in to a number of issues related to slicing and labeling the disk using fdisk. The drive in the machine is a 40GB Seagate Barracude (ST34001A) installed as a Secondary Master on the IDE bus using LBA. The BIOS reports that the drive has 16 sectors pr block, but little else. When accessing fdisk during install, fdisk complains that the disk geometry is invalid and sets it to the default geometry for 40GB: Cylinders: 4865 Heads: 255 Sectors: 63 I've tried with the following configuration based on what was reported by the BIOS: Cylinders: 19150 Heads: 255 Sectors: 16 Looking in the manual: http://www.seagate.com/support/disc/manuals/ata/cuda7200pm.pdf, Seagate is specifying the following logical characteristic: Cylinders: 16383 Read / Write heads: 16 Sectors pr track: 63 Which of these settings should be the correct one for the fdisk geometry? Additionally I encountered problems during installation if splitting the disk into more than 4 slices. This would cause the following error to be thrown during prior to the install files being copied (when sysinstall was executing the newfs commands): Error mounting /mnt/dev/X on /mnt/usr. No such file or directory Using only 4 slices seems to have solved this error, however I'd like the disk layout to use 5 slices as follows: / = 512MB swap = 2048MB (the machine has 1024MB RAM) /tmp = 512MB /var = 2048MB /usr = whatever remains I noticed that when having 5 slices, the last slice (/usr) would be named X rather than ad2s5 as I'd expect (the drive was detected as ad2). Is this behaviour related to the error in any way? Also, is the above disk layout good for a server intended to run both a web server (Apache) and a database server (PostGreSQL) ? Finally after installation (using only 4 slices) the system will only boot if the FreeBSD boot manager is used. This in turn causes a 4 menu options, all of them named FreeBSD to appear during startup despite only the / slice having been set as bootable in fdisk which appears to be indicated by an A in the flag column. Selecting the first menu item by pressing F1 will make the system boot as expected. It seems rather silly though to use a boot manager when FreeBSD is the only operating system that is installed (and ever will be installed) on the machine. If the FreeBSD boot manager is not used however and only the MBR is set during installation, the system will fail at startup with error Invalid Partition Table. Is this because the harddrive is installed as the Secondary Master on the IDE bus? Appreciate any input on this Cheers Jona ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 7.0 fdisk issue during installation
Hi Jerry, Thank you for the swift and very thorough response. If I understand you correctly, then I should only create 1 slice of the entire disk (seeing as FreeBSD will be the only OS) using fdisk and then partition the slice using bsdlabels from sysinstall? Previously I was aiming for 5 slices, each of which had a single partition as described below. From your explanation I take it that slices are what Windows refers to as Primary Partitions? If that's the case then I understand the behaviour I experienced. Is it possible to make a slice non-bootable? And would there be any benefits (less fragmentation, faster access time etc.) in using slices rather than partitions to layout the harddrive or should slices only be used to represent a physical harddrive? Appreciate the clarification Cheers Jona On Fri, Nov 21, 2008 at 8:55 PM, Jerry McAllister [EMAIL PROTECTED] wrote: On Fri, Nov 21, 2008 at 08:03:58PM +0100, Jonatan Evald Buus wrote: Greetings, I tried to install FreeBSD 7.0 on an old server earlier today and ran in to a number of issues related to slicing and labeling the disk using fdisk. The drive in the machine is a 40GB Seagate Barracude (ST34001A) installed as a Secondary Master on the IDE bus using LBA. The BIOS reports that the drive has 16 sectors pr block, but little else. When accessing fdisk during install, fdisk complains that the disk geometry is invalid and sets it to the default geometry for 40GB: Cylinders: 4865 Heads: 255 Sectors: 63 I've tried with the following configuration based on what was reported by the BIOS: Cylinders: 19150 Heads: 255 Sectors: 16 Looking in the manual: http://www.seagate.com/support/disc/manuals/ata/cuda7200pm.pdf, Seagate is specifying the following logical characteristic: Cylinders: 16383 Read / Write heads: 16 Sectors pr track: 63 Which of these settings should be the correct one for the fdisk geometry? Let the system set it and just go with what it does. Geometry is virtual nowdays. Except in some unusual situations (on IDE) Cylinders, heads and sectors most often do not mean what they used to. The system drivers have it all figured out. The important thing for you is the total number of blocks/sectors. If that doesn't work, you will have to do some diagnosis, but in about 10 out of 9 times, accepting how FreeBSD sets it is correct and works. Additionally I encountered problems during installation if splitting the disk into more than 4 slices. This would cause the following error to be thrown during prior to the install files being copied (when sysinstall was executing the newfs commands): You cannot have more than 4 slices. The system limits you to 4 slices, identified by numbers 1..4 Once you divide in to slices, each can be further divided in to up to 8 partitions, although it is really 7 because partition 'c' has special meaning and is not really available to be a real partition. Partitions are identified with alpha letters a..h - with 'c' being used to identify the whole slice. You use fdisk to create the slices (and write the MBR and set the bootable flag). Then you use bsdlabel (formerly called disklabel) to create the partitions within a slice (plus write the slice boot block. Typically, you want to make partition 'a' be the root (/) filesystem and 'b' be swap space on a bootable system slice. Some things assume these designations. Then you newfs partitions a, d, e, f, g, h or as many as you use. But don't touch c and don't newfs b if it is to be swap. jerry Error mounting /mnt/dev/X on /mnt/usr. No such file or directory Using only 4 slices seems to have solved this error, however I'd like the disk layout to use 5 slices as follows: / = 512MB swap = 2048MB (the machine has 1024MB RAM) /tmp = 512MB /var = 2048MB /usr = whatever remains I noticed that when having 5 slices, the last slice (/usr) would be named X rather than ad2s5 as I'd expect (the drive was detected as ad2). Is this behaviour related to the error in any way? Also, is the above disk layout good for a server intended to run both a web server (Apache) and a database server (PostGreSQL) ? Finally after installation (using only 4 slices) the system will only boot if the FreeBSD boot manager is used. That is probably because you have created what is referred to in the documentation as a dangerously dedicated disk. You can make it work that way. FreeBSD can handle it. But other systems will not play nicely with it. This in turn causes a 4 menu options, all of them named FreeBSD to appear during startup despite only the / slice having been set as bootable in fdisk which appears to be indicated by an A in the flag column. Again, because you tried to do it the wrong way. You created 4 FreeBSD slices, probably each with an MBR and so the BIOS and the first MBR think they are all bootable. Selecting the first menu
