Re: Locking a file backed mdconfig into memory
John Baldwin j...@freebsd.org writes: On Wednesday 02 June 2010 6:37:59 pm Dave Hayes wrote: John Baldwin j...@freebsd.org writes: Ok, if you are using a stock mfsroot from a release build, that should work fine. If you have built a custom mfsroot that is larger, then you may need to increase NKPT on i386. In very recent 7 and later you can do this by setting it to a new value in your kernel config. In older versions you can do this by manually adding a #define to set a new value of NKPT in opt_global.h or hacking on the source directly. Is this also true for amd64 (which is my particular target)? It might be. What is the panic you are seeing? I can't see the panic as it repeatedly scrolls across the console screen faster than I can read it. In this case the mfsroot is around 275MB. I have noticed that sometimes I can build an mfsroot that does not crash of this size. -- Dave Hayes - Consultant - Altadena CA, USA - d...@jetcafe.org The opinions expressed above are entirely my own People usually oppose things because they are ignorant of them. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Friday 04 June 2010 5:37:14 am Dave Hayes wrote: John Baldwin j...@freebsd.org writes: On Wednesday 02 June 2010 6:37:59 pm Dave Hayes wrote: John Baldwin j...@freebsd.org writes: Ok, if you are using a stock mfsroot from a release build, that should work fine. If you have built a custom mfsroot that is larger, then you may need to increase NKPT on i386. In very recent 7 and later you can do this by setting it to a new value in your kernel config. In older versions you can do this by manually adding a #define to set a new value of NKPT in opt_global.h or hacking on the source directly. Is this also true for amd64 (which is my particular target)? It might be. What is the panic you are seeing? I can't see the panic as it repeatedly scrolls across the console screen faster than I can read it. In this case the mfsroot is around 275MB. I have noticed that sometimes I can build an mfsroot that does not crash of this size. Hmmm, I would just try increasing NKPT then. You might have to poke around in sys/amd64 to see what the default size is and how to tune it. -- John Baldwin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Fri, Jun 04, 2010 at 08:20:49AM -0400, John Baldwin wrote: Hmmm, I would just try increasing NKPT then. You might have to poke around in sys/amd64 to see what the default size is and how to tune it. When Isilon did the stable/7 merge and amd64 default NKPT changed from 240 to 32 amd64 started having weird pmap issues during boot. At panic time the stack wasn't very useful, and I didn't finish debugging the issue since eventually I just had to get something working. We just reverted NKPT to 240 and it worked for us. I didn't see an anything in optsions.amd64 so I hard-coded it in amd64/include/pmap.h. Supposedly amd64 can deal with a small NKPT and grow dynamically, but it didn't seem to work for us. :-( Perhaps when we do the next merge project I'll have a few days to devote to debugging the root cause. Thanks, matthew ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
Matthew D Fleming wrote: On Fri, Jun 04, 2010 at 08:20:49AM -0400, John Baldwin wrote: Hmmm, I would just try increasing NKPT then. You might have to poke around in sys/amd64 to see what the default size is and how to tune it. When Isilon did the stable/7 merge and amd64 default NKPT changed from 240 to 32 amd64 started having weird pmap issues during boot. At panic time the stack wasn't very useful, and I didn't finish debugging the issue since eventually I just had to get something working. We just reverted NKPT to 240 and it worked for us. I didn't see an anything in optsions.amd64 so I hard-coded it in amd64/include/pmap.h. Supposedly amd64 can deal with a small NKPT and grow dynamically, but it didn't seem to work for us. :-( Perhaps when we do the next merge project I'll have a few days to devote to debugging the root cause. NKPT controls the number of page table pages that are initially allocated at the bottom of the top 2GB of the kernel address space. However, the vast majority of the kernel address space, 510GB in FreeBSD =7.3, is below these page table pages. The page table pages for this region are dynamically allocated as needed. If you're booting a kernel and modules greater than 64GB in size, then I can certainly see why you would need to increase NKPT. John, is there some way to know at boot time how big the kernel and modules were? Then, we could probably eliminate NKPT. Alan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Friday 04 June 2010 1:58:13 pm Alan Cox wrote: Matthew D Fleming wrote: On Fri, Jun 04, 2010 at 08:20:49AM -0400, John Baldwin wrote: Hmmm, I would just try increasing NKPT then. You might have to poke around in sys/amd64 to see what the default size is and how to tune it. When Isilon did the stable/7 merge and amd64 default NKPT changed from 240 to 32 amd64 started having weird pmap issues during boot. At panic time the stack wasn't very useful, and I didn't finish debugging the issue since eventually I just had to get something working. We just reverted NKPT to 240 and it worked for us. I didn't see an anything in optsions.amd64 so I hard-coded it in amd64/include/pmap.h. Supposedly amd64 can deal with a small NKPT and grow dynamically, but it didn't seem to work for us. :-( Perhaps when we do the next merge project I'll have a few days to devote to debugging the root cause. NKPT controls the number of page table pages that are initially allocated at the bottom of the top 2GB of the kernel address space. However, the vast majority of the kernel address space, 510GB in FreeBSD =7.3, is below these page table pages. The page table pages for this region are dynamically allocated as needed. If you're booting a kernel and modules greater than 64GB in size, then I can certainly see why you would need to increase NKPT. 64GB seems like a lot of address space, I would not expect that to be completely used by kernel and modules. I think earlier in the thread someone said they had problems with a mere 295MB mfsroot. John, is there some way to know at boot time how big the kernel and modules were? Then, we could probably eliminate NKPT. I think the loader knows, so it could pass that info to the kernel. -- John Baldwin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On 6/4/2010 1:53 PM, John Baldwin wrote: On Friday 04 June 2010 1:58:13 pm Alan Cox wrote: Matthew D Fleming wrote: On Fri, Jun 04, 2010 at 08:20:49AM -0400, John Baldwin wrote: Hmmm, I would just try increasing NKPT then. You might have to poke around in sys/amd64 to see what the default size is and how to tune it. When Isilon did the stable/7 merge and amd64 default NKPT changed from 240 to 32 amd64 started having weird pmap issues during boot. At panic time the stack wasn't very useful, and I didn't finish debugging the issue since eventually I just had to get something working. We just reverted NKPT to 240 and it worked for us. I didn't see an anything in optsions.amd64 so I hard-coded it in amd64/include/pmap.h. Supposedly amd64 can deal with a small NKPT and grow dynamically, but it didn't seem to work for us. :-( Perhaps when we do the next merge project I'll have a few days to devote to debugging the root cause. NKPT controls the number of page table pages that are initially allocated at the bottom of the top 2GB of the kernel address space. However, the vast majority of the kernel address space, 510GB in FreeBSD =7.3, is below these page table pages. The page table pages for this region are dynamically allocated as needed. If you're booting a kernel and modules greater than 64GB in size, then I can certainly see why you would need to increase NKPT. 64GB seems like a lot of address space, I would not expect that to be completely used by kernel and modules. I think earlier in the thread someone said they had problems with a mere 295MB mfsroot. Oops. I meant to say 64MB. :-) John, is there some way to know at boot time how big the kernel and modules were? Then, we could probably eliminate NKPT. I think the loader knows, so it could pass that info to the kernel. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Wednesday 02 June 2010 6:37:59 pm Dave Hayes wrote: John Baldwin j...@freebsd.org writes: Ok, if you are using a stock mfsroot from a release build, that should work fine. If you have built a custom mfsroot that is larger, then you may need to increase NKPT on i386. In very recent 7 and later you can do this by setting it to a new value in your kernel config. In older versions you can do this by manually adding a #define to set a new value of NKPT in opt_global.h or hacking on the source directly. Is this also true for amd64 (which is my particular target)? It might be. What is the panic you are seeing? -- John Baldwin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
John Baldwin j...@freebsd.org writes: Ok, if you are using a stock mfsroot from a release build, that should work fine. If you have built a custom mfsroot that is larger, then you may need to increase NKPT on i386. In very recent 7 and later you can do this by setting it to a new value in your kernel config. In older versions you can do this by manually adding a #define to set a new value of NKPT in opt_global.h or hacking on the source directly. Is this also true for amd64 (which is my particular target)? -- Dave Hayes - Consultant - Altadena CA, USA - d...@jetcafe.org The opinions expressed above are entirely my own Be who you are and say what you feel, because those who mind don't matter, and those who matter don't mind. - Dr. Seuss ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Monday 31 May 2010 2:45:25 pm Jeremy Chadwick wrote: On Mon, May 31, 2010 at 11:25:51AM -0700, Dave Hayes wrote: Jeremy Chadwick free...@jdc.parodius.com writes: Is the mfsroot file compressed (.gz extension)? Reason I ask is that the OP states he's using RELENG_7... Yes it is compressed. http://jdc.parodius.com/freebsd/pxeboot_serial_install_7.html#step7 Thanks much for this. I did a simple test, I rebuilt a DVD that wasn't booting to use a lower level of compression (gzip -9 to gzip -6) on mfsroot without changing anything else. This caused it to boot normally. I'm not sure it's conclusive evidence, but it certainly looks like a weak datapoint supporting this kernel bug being the source of my problem. Is this problem fixed in 8.0 or by a patch? With regards to said bug, gzip compression seems to work fine on RELENG_8, at least in my experiences: http://jdc.parodius.com/freebsd/pxeboot_serial_install_8.html I'm not sure the level of compression is what triggers the bug though; I haven't tested all levels (1 through 9). CC'ing jhb@ since he last updated PR kern/120127 (which I would say is still a problem on RELENG_7 :-) ). John, are you aware of any gzip decompression / mfsroot changes which might explain the problem on RELENG_7? I haven't done a thorough series of tests, but on my testbed boxes RELENG_8 works fine with a gzip'd mfsroot. Ok, if you are using a stock mfsroot from a release build, that should work fine. If you have built a custom mfsroot that is larger, then you may need to increase NKPT on i386. In very recent 7 and later you can do this by setting it to a new value in your kernel config. In older versions you can do this by manually adding a #define to set a new value of NKPT in opt_global.h or hacking on the source directly. -- John Baldwin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Tue, 1 Jun 2010, John Baldwin wrote: On Monday 31 May 2010 2:45:25 pm Jeremy Chadwick wrote: On Mon, May 31, 2010 at 11:25:51AM -0700, Dave Hayes wrote: Jeremy Chadwick free...@jdc.parodius.com writes: Is the mfsroot file compressed (.gz extension)? Reason I ask is that the OP states he's using RELENG_7... Yes it is compressed. http://jdc.parodius.com/freebsd/pxeboot_serial_install_7.html#step7 Thanks much for this. I did a simple test, I rebuilt a DVD that wasn't booting to use a lower level of compression (gzip -9 to gzip -6) on mfsroot without changing anything else. This caused it to boot normally. I'm not sure it's conclusive evidence, but it certainly looks like a weak datapoint supporting this kernel bug being the source of my problem. Is this problem fixed in 8.0 or by a patch? With regards to said bug, gzip compression seems to work fine on RELENG_8, at least in my experiences: http://jdc.parodius.com/freebsd/pxeboot_serial_install_8.html I'm not sure the level of compression is what triggers the bug though; I haven't tested all levels (1 through 9). CC'ing jhb@ since he last updated PR kern/120127 (which I would say is still a problem on RELENG_7 :-) ). John, are you aware of any gzip decompression / mfsroot changes which might explain the problem on RELENG_7? I haven't done a thorough series of tests, but on my testbed boxes RELENG_8 works fine with a gzip'd mfsroot. Ok, if you are using a stock mfsroot from a release build, that should work fine. If you have built a custom mfsroot that is larger, then you may need to increase NKPT on i386. In very recent 7 and later you can do this by setting it to a new value in your kernel config. In older versions you can do this by manually adding a #define to set a new value of NKPT in opt_global.h or hacking on the source directly. This is the original post I found confirming all this: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/hackers/2004-02/0241.html I can also confirm your changes that allow it to be a kernel config option works as well. :) It's nice to essentially have a full livefs as your mfsroot for non-sysinstall installs. Charles -- John Baldwin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
Jeremy Chadwick free...@jdc.parodius.com writes: Is the mfsroot file compressed (.gz extension)? Reason I ask is that the OP states he's using RELENG_7... Yes it is compressed. http://jdc.parodius.com/freebsd/pxeboot_serial_install_7.html#step7 Thanks much for this. I did a simple test, I rebuilt a DVD that wasn't booting to use a lower level of compression (gzip -9 to gzip -6) on mfsroot without changing anything else. This caused it to boot normally. I'm not sure it's conclusive evidence, but it certainly looks like a weak datapoint supporting this kernel bug being the source of my problem. Is this problem fixed in 8.0 or by a patch? -- Dave Hayes - Consultant - Altadena CA, USA - d...@jetcafe.org The opinions expressed above are entirely my own Suggested definition of sin: Trading aliveness for survival. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Mon, May 31, 2010 at 11:25:51AM -0700, Dave Hayes wrote: Jeremy Chadwick free...@jdc.parodius.com writes: Is the mfsroot file compressed (.gz extension)? Reason I ask is that the OP states he's using RELENG_7... Yes it is compressed. http://jdc.parodius.com/freebsd/pxeboot_serial_install_7.html#step7 Thanks much for this. I did a simple test, I rebuilt a DVD that wasn't booting to use a lower level of compression (gzip -9 to gzip -6) on mfsroot without changing anything else. This caused it to boot normally. I'm not sure it's conclusive evidence, but it certainly looks like a weak datapoint supporting this kernel bug being the source of my problem. Is this problem fixed in 8.0 or by a patch? With regards to said bug, gzip compression seems to work fine on RELENG_8, at least in my experiences: http://jdc.parodius.com/freebsd/pxeboot_serial_install_8.html I'm not sure the level of compression is what triggers the bug though; I haven't tested all levels (1 through 9). CC'ing jhb@ since he last updated PR kern/120127 (which I would say is still a problem on RELENG_7 :-) ). John, are you aware of any gzip decompression / mfsroot changes which might explain the problem on RELENG_7? I haven't done a thorough series of tests, but on my testbed boxes RELENG_8 works fine with a gzip'd mfsroot. -- | Jeremy Chadwick j...@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
Jeremy Chadwick free...@jdc.parodius.com writes: CC'ing jhb@ since he last updated PR kern/120127 (which I would say is still a problem on RELENG_7 :-) ). John, are you aware of any gzip decompression / mfsroot changes which might explain the problem on RELENG_7? I haven't done a thorough series of tests, but on my testbed boxes RELENG_8 works fine with a gzip'd mfsroot. I can get you an iso that crashes when you try to boot it, if that will help? -- Dave Hayes - Consultant - Altadena CA, USA - d...@jetcafe.org The opinions expressed above are entirely my own Your ego is a failed API for a system that doesn't need your input. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
Dave Hayes d...@jetcafe.org writes: Jeremy Chadwick free...@jdc.parodius.com writes: CC'ing jhb@ since he last updated PR kern/120127 (which I would say is still a problem on RELENG_7 :-) ). John, are you aware of any gzip decompression / mfsroot changes which might explain the problem on RELENG_7? I haven't done a thorough series of tests, but on my testbed boxes RELENG_8 works fine with a gzip'd mfsroot. I can get you an iso that crashes when you try to boot it, if that will help? Well this doesn't seem to be the compression issue, since I just tried a test without compressing the ramdisk image. It still crashes. I'm back to thinking it's the size of the ramdisk image itself, but really I've no clue. -- Dave Hayes - Consultant - Altadena CA, USA - d...@jetcafe.org The opinions expressed above are entirely my own In theory, there is no difference between theory and practice. In practice, there is. -- Jan L. A. Van De Snepscheut ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
Garrett Cooper yanef...@gmail.com writes: This is how I do it in my quickie loader.rc: include /boot/loader.4th set vfs.root.mountfrom=ufs:/dev/md0 load /kernel load -t mfs_root /mfsroot start I used to do exactly this back at FreeBSD 4.11 to boot off a cdrom. Nice to know it still works this way. Jeremy Chadwick free...@jdc.parodius.com writes: However, what I'm having trouble understanding is what exactly preload_search_info() looks for and how all this actually connects and works. It appears to me that there are specific drivers located in src/sys/dev that are KLD-supported and others which are expected to be included in the kernel statically. Maybe this confusion explains why /dev/md0c is giving me random crashes at the moment? Of course, another theory might be the size of my initial ramdisk (300Mb). Would there any known bug where booting off a large ramdisk causes unreadable panics to flash past the console too rapidly to view? -- Dave Hayes - Consultant - Altadena CA, USA - d...@jetcafe.org The opinions expressed above are entirely my own By and large, language is a tool for concealing the truth. -George Carlin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On 2010-05-30 07:42:47PM -0700, Dave Hayes wrote: Garrett Cooper yanef...@gmail.com writes: This is how I do it in my quickie loader.rc: include /boot/loader.4th set vfs.root.mountfrom=ufs:/dev/md0 load /kernel load -t mfs_root /mfsroot start I used to do exactly this back at FreeBSD 4.11 to boot off a cdrom. Nice to know it still works this way. Jeremy Chadwick free...@jdc.parodius.com writes: However, what I'm having trouble understanding is what exactly preload_search_info() looks for and how all this actually connects and works. It appears to me that there are specific drivers located in src/sys/dev that are KLD-supported and others which are expected to be included in the kernel statically. Maybe this confusion explains why /dev/md0c is giving me random crashes at the moment? Of course, another theory might be the size of my initial ramdisk (300Mb). Would there any known bug where booting off a large ramdisk causes unreadable panics to flash past the console too rapidly to view? Run memtest86 on the first 300mb of ram lately? Does your ramdisk crash at any size? -- === Peter C. Lai | Bard College at Simon's Rock Systems Administrator| 84 Alford Rd. Information Technology Svcs. | Gt. Barrington, MA 01230 USA peter AT simons-rock.edu | (413) 528-7428 === ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Sun, May 30, 2010 at 07:42:47PM -0700, Dave Hayes wrote: Garrett Cooper yanef...@gmail.com writes: This is how I do it in my quickie loader.rc: include /boot/loader.4th set vfs.root.mountfrom=ufs:/dev/md0 load /kernel load -t mfs_root /mfsroot start I used to do exactly this back at FreeBSD 4.11 to boot off a cdrom. Nice to know it still works this way. Jeremy Chadwick free...@jdc.parodius.com writes: However, what I'm having trouble understanding is what exactly preload_search_info() looks for and how all this actually connects and works. It appears to me that there are specific drivers located in src/sys/dev that are KLD-supported and others which are expected to be included in the kernel statically. Maybe this confusion explains why /dev/md0c is giving me random crashes at the moment? Of course, another theory might be the size of my initial ramdisk (300Mb). Would there any known bug where booting off a large ramdisk causes unreadable panics to flash past the console too rapidly to view? Is the mfsroot file compressed (.gz extension)? Reason I ask is that the OP states he's using RELENG_7... http://jdc.parodius.com/freebsd/pxeboot_serial_install_7.html#step7 -- | Jeremy Chadwick j...@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
Clifton Royston clift...@lava.net writes: It sounds like what you really want is to load the contents of the specified file as a memory system? That's not part of mdconfig's repertoire, to the best of my recollection. So if, in /etc/loader.conf, I do this: rootfs_load=YES rootfs_type=mfs_root rootfs_name=/mfsboot vfs.root.mountfrom=ufs:md0 and /mfsboot comes from a bootable DVD, am I to assume that this mount is under the same constraint? Specifically, this constraint is that the /mfsboot file (and hence the DVD) will be read repeatedly if the system is under memory pressure. If that's what you want, you need to use a different tool; the purpose of mdconfig is to provide scratch disk. The backing store is to specify a region its contents can be swapped out to if the system is under memory pressure (which certainly won't work with a DVD) Heh. Certainly I am using tools for purposes which may not match the original intention. If I wanted to run the rootfs on a memory device which did -not- swap to the backing store of /mfsboot, how would I go about doing that? -- Dave Hayes - Consultant - Altadena CA, USA - d...@jetcafe.org The opinions expressed above are entirely my own Faith (n): The quality by which we believe what we would otherwise think was false. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
on 28/05/2010 20:57 Dave Hayes said the following: Clifton Royston clift...@lava.net writes: It sounds like what you really want is to load the contents of the specified file as a memory system? That's not part of mdconfig's repertoire, to the best of my recollection. So if, in /etc/loader.conf, I do this: rootfs_load=YES rootfs_type=mfs_root rootfs_name=/mfsboot vfs.root.mountfrom=ufs:md0 and /mfsboot comes from a bootable DVD, am I to assume that this mount is under the same constraint? Specifically, this constraint is that the /mfsboot file (and hence the DVD) will be read repeatedly if the system is under memory pressure. No. Contents of mfsboot is loaded into physical memory and always stays there. -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Fri, May 28, 2010 at 10:57:29AM -0700, Dave Hayes wrote: Clifton Royston clift...@lava.net writes: It sounds like what you really want is to load the contents of the specified file as a memory system? That's not part of mdconfig's repertoire, to the best of my recollection. So if, in /etc/loader.conf, I do this: rootfs_load=YES rootfs_type=mfs_root rootfs_name=/mfsboot vfs.root.mountfrom=ufs:md0 I think you mean /boot/loader.conf? And I think you meant this for variable names, in addition to what vfs.root.mountfrom should be (specific to RELENG_8): mfsroot_load=YES mfsroot_type=mfs_root mfsroot_name=/some/path/mfsroot vfs.root.mountfrom=ufs:/dev/md0 If using RELENG_7 and the mfsroot was made on RELENG_7, replace /dev/md0 with /dev/md0c. -- | Jeremy Chadwick j...@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
Jeremy Chadwick free...@jdc.parodius.com writes: On Fri, May 28, 2010 at 10:57:29AM -0700, Dave Hayes wrote: Clifton Royston clift...@lava.net writes: It sounds like what you really want is to load the contents of the specified file as a memory system? That's not part of mdconfig's repertoire, to the best of my recollection. So if, in /etc/loader.conf, I do this: rootfs_load=YES rootfs_type=mfs_root rootfs_name=/mfsboot vfs.root.mountfrom=ufs:md0 I think you mean /boot/loader.conf? Yes, you are correct. And I think you meant this for variable names, in addition to what vfs.root.mountfrom should be (specific to RELENG_8): mfsroot_load=YES mfsroot_type=mfs_root mfsroot_name=/some/path/mfsroot I'm using RELENG_7, but it seems rootfs_* works just like mfsroot_* ... is the former deprecated? vfs.root.mountfrom=ufs:/dev/md0 Hm, 'ufs:md0' currently works. What trouble can be had from using the abbreviated device name? If using RELENG_7 and the mfsroot was made on RELENG_7, replace /dev/md0 with /dev/md0c. Is there a reason for doing this? -- Dave Hayes - Consultant - Altadena CA, USA - d...@jetcafe.org The opinions expressed above are entirely my own If you want to strengthen an enemy -- hate them. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Fri, May 28, 2010 at 12:26:12PM -0700, Dave Hayes wrote: Jeremy Chadwick free...@jdc.parodius.com writes: And I think you meant this for variable names, in addition to what vfs.root.mountfrom should be (specific to RELENG_8): mfsroot_load=YES mfsroot_type=mfs_root mfsroot_name=/some/path/mfsroot I'm using RELENG_7, but it seems rootfs_* works just like mfsroot_* ... is the former deprecated? I spent a small bit of time looking through both RELENG_7 and RELENG_8 source (/usr/src) as well as /boot (for the Forth portion) and I was unable to find any reference to rootfs aside from actual variable names (not loader variables) in C code or include files. None were strings. There's a strong possibility I'm not looking the right way (grep -ri ...). I can assure you that mfsroot_xxx works on both platforms because I wrote the below documentation: http://jdc.parodius.com/freebsd/pxeboot_serial_install.html vfs.root.mountfrom=ufs:/dev/md0 Hm, 'ufs:md0' currently works. What trouble can be had from using the abbreviated device name? I have no idea; probably none. This is just the first time I've seen the variable set to something shorthand. I would have to review the parser code to see how it works. If using RELENG_7 and the mfsroot was made on RELENG_7, replace /dev/md0 with /dev/md0c. Is there a reason for doing this? Yes, the c slice is gone with present-day GEOM. I'm fairly certain this is also the reason for the geometry does not match label (255h,63s != 16h,63s) message seen when running RELENG_8 with a disk which was labelled using an older GEOM (e.g. RELENG_7). This also applies to mfsroot images because all they are is raw disk images that are literally loaded into memory. -- | Jeremy Chadwick j...@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
on 29/05/2010 00:58 Jeremy Chadwick said the following: On Fri, May 28, 2010 at 12:26:12PM -0700, Dave Hayes wrote: Jeremy Chadwick free...@jdc.parodius.com writes: And I think you meant this for variable names, in addition to what vfs.root.mountfrom should be (specific to RELENG_8): mfsroot_load=YES mfsroot_type=mfs_root mfsroot_name=/some/path/mfsroot I'm using RELENG_7, but it seems rootfs_* works just like mfsroot_* ... is the former deprecated? I spent a small bit of time looking through both RELENG_7 and RELENG_8 source (/usr/src) as well as /boot (for the Forth portion) and I was unable to find any reference to rootfs aside from actual variable names (not loader variables) in C code or include files. None were strings. There's a strong possibility I'm not looking the right way (grep -ri ...). I can assure you that mfsroot_xxx works on both platforms because I wrote the below documentation: Guys, you could have used donotboot_ prefix there and it would still work :-) _type, _name and _load suffixes are what is important in this particular case. See description of these suffixes in loader.conf(5). -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Sat, May 29, 2010 at 01:34:24AM +0300, Andriy Gapon wrote: on 29/05/2010 00:58 Jeremy Chadwick said the following: On Fri, May 28, 2010 at 12:26:12PM -0700, Dave Hayes wrote: Jeremy Chadwick free...@jdc.parodius.com writes: And I think you meant this for variable names, in addition to what vfs.root.mountfrom should be (specific to RELENG_8): mfsroot_load=YES mfsroot_type=mfs_root mfsroot_name=/some/path/mfsroot I'm using RELENG_7, but it seems rootfs_* works just like mfsroot_* ... is the former deprecated? I spent a small bit of time looking through both RELENG_7 and RELENG_8 source (/usr/src) as well as /boot (for the Forth portion) and I was unable to find any reference to rootfs aside from actual variable names (not loader variables) in C code or include files. None were strings. There's a strong possibility I'm not looking the right way (grep -ri ...). I can assure you that mfsroot_xxx works on both platforms because I wrote the below documentation: Guys, you could have used donotboot_ prefix there and it would still work :-) _type, _name and _load suffixes are what is important in this particular case. See description of these suffixes in loader.conf(5). Andriy, I'm still having a bit of trouble understanding how all of this works. No, not the loader part -- thanks for pointing that out, yeah, I get that piece (mostly). This is all I've managed to figure out, and it's probably wrong, but I'm trying. :-) It appears the key piece is the xxx_type=mfs_root declaration; the string passed there (the value mfs_root) is important. One could also use md_image. I'm basing that statement off of code in src/sys/dev/md/md.c which appears to look for + permit either string. However, what I'm having trouble understanding is what exactly preload_search_info() looks for and how all this actually connects and works. It appears to me that there are specific drivers located in src/sys/dev that are KLD-supported and others which are expected to be included in the kernel statically. The ones which appear to be required static are md, loop, ether, and pty; they have no *_MODULE macros in use. random and firmware, however, are built as KLDs and do call DEV_MODULE() and DECLARE_MODULE() macros respectively. All that said, I'm still not sure xxx_type=mfs_root tickles the use of the md code. I thought maybe the metadata was just a symbol lookup (and maybe it is), but objdump on /boot/kernel/kernel and /boot/kernel/kernel.symbols doesn't return anything. strings on /boot/kernel/kernel does return mfs_root, but I'm almost certain that's because of the strncmp() in src/sys/dev/md/md.c. Actually, as I dig through it more, it looks like the md stuff is loaded by GEOM? I see DECLARE_GEOM_CLASS() in use which refers to g_md_class that provides a series of addresses to initialisation and startup routines, amongst other stuff. ...except there's /boot/kernel/geom_md.ko, so now I'm even more confused. Aaugh! :-) -- | Jeremy Chadwick j...@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Fri, May 28, 2010 at 2:58 PM, Jeremy Chadwick free...@jdc.parodius.com wrote: On Fri, May 28, 2010 at 12:26:12PM -0700, Dave Hayes wrote: Jeremy Chadwick free...@jdc.parodius.com writes: And I think you meant this for variable names, in addition to what vfs.root.mountfrom should be (specific to RELENG_8): mfsroot_load=YES mfsroot_type=mfs_root mfsroot_name=/some/path/mfsroot I'm using RELENG_7, but it seems rootfs_* works just like mfsroot_* ... is the former deprecated? I spent a small bit of time looking through both RELENG_7 and RELENG_8 source (/usr/src) as well as /boot (for the Forth portion) and I was unable to find any reference to rootfs aside from actual variable names (not loader variables) in C code or include files. None were strings. There's a strong possibility I'm not looking the right way (grep -ri ...). I vaguely remember reading about rootfs in the past, but it's definitely mfs_root now. This is how I do it in my quickie loader.rc: include /boot/loader.4th set vfs.root.mountfrom=ufs:/dev/md0 load /kernel load -t mfs_root /mfsroot start I can assure you that mfsroot_xxx works on both platforms because I wrote the below documentation: http://jdc.parodius.com/freebsd/pxeboot_serial_install.html vfs.root.mountfrom=ufs:/dev/md0 Hm, 'ufs:md0' currently works. What trouble can be had from using the abbreviated device name? I have no idea; probably none. This is just the first time I've seen the variable set to something shorthand. I would have to review the parser code to see how it works. All depends on how you create the mfsroot. makefs(8) style mfsroots for instance are like /dev/md0a blah. It also depends on your version of FreeBSD IIRC, because (again IIRC) GEOM changed the status quo in either 7.x or 8.x as to how md(4) devices represented. Again, this is IIRC :P! HTH, -Garrett ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Fri, May 28, 2010 at 5:48 PM, Garrett Cooper yanef...@gmail.com wrote: On Fri, May 28, 2010 at 2:58 PM, Jeremy Chadwick free...@jdc.parodius.com wrote: On Fri, May 28, 2010 at 12:26:12PM -0700, Dave Hayes wrote: Jeremy Chadwick free...@jdc.parodius.com writes: And I think you meant this for variable names, in addition to what vfs.root.mountfrom should be (specific to RELENG_8): mfsroot_load=YES mfsroot_type=mfs_root mfsroot_name=/some/path/mfsroot I'm using RELENG_7, but it seems rootfs_* works just like mfsroot_* ... is the former deprecated? I spent a small bit of time looking through both RELENG_7 and RELENG_8 source (/usr/src) as well as /boot (for the Forth portion) and I was unable to find any reference to rootfs aside from actual variable names (not loader variables) in C code or include files. None were strings. There's a strong possibility I'm not looking the right way (grep -ri ...). I vaguely remember reading about rootfs in the past, but it's definitely mfs_root now. This is how I do it in my quickie loader.rc: include /boot/loader.4th set vfs.root.mountfrom=ufs:/dev/md0 load /kernel load -t mfs_root /mfsroot start I can assure you that mfsroot_xxx works on both platforms because I wrote the below documentation: http://jdc.parodius.com/freebsd/pxeboot_serial_install.html vfs.root.mountfrom=ufs:/dev/md0 Hm, 'ufs:md0' currently works. What trouble can be had from using the abbreviated device name? I have no idea; probably none. This is just the first time I've seen the variable set to something shorthand. I would have to review the parser code to see how it works. All depends on how you create the mfsroot. makefs(8) style mfsroots for instance are like /dev/md0a blah. Ugh. Please ignore this statement. It's wrong.. It also depends on your version of FreeBSD IIRC, because (again IIRC) GEOM changed the status quo in either 7.x or 8.x as to how md(4) devices represented. Again, this is IIRC :P! ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Locking a file backed mdconfig into memory
On FreeBSD 7.3-STABLE I'm mounting a DVD and doing something like this: mdconfig -a -t vnode -o reserve -o readonly -f /dvd/file so that /dvd/file becomes the backing storage for my memory disk. Now if the system is under severe memory pressure, will this memory get swapped out, causing a read from the DVD? How would I tell the system to never swap this file out of ram, even under severe memory pressure? The idea is to load this backing storage once and only once from the DVD into memory and leave it there. Thanks in advance. -- Dave Hayes - Consultant - Altadena CA, USA - d...@jetcafe.org The opinions expressed above are entirely my own Have you noticed how economical the human race is with it's idols? It sets them up, enjoys them, then falls upon them and devours them until nothing is left. Even the complete consumption of the idol, if it is another human being, is not the end. There are then hundreds of years worth of argument and analysis to be worked through... ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Thu, May 27, 2010 at 4:48 PM, Dave Hayes d...@jetcafe.org wrote: On FreeBSD 7.3-STABLE I'm mounting a DVD and doing something like this: mdconfig -a -t vnode -o reserve -o readonly -f /dvd/file so that /dvd/file becomes the backing storage for my memory disk. Now if the system is under severe memory pressure, will this memory get swapped out, causing a read from the DVD? What swap? Need more details... How would I tell the system to never swap this file out of ram, even under severe memory pressure? You might be misunderstanding the purpose of the -t option for mdconfig(8): -t type Select the type of the memory disk. malloc Storage for this type of memory disk is allocated with malloc(9). This limits the size to the malloc bucket limit in the kernel. If the -o reserve option is not set, creating and filling a large malloc-backed memory disk is a very easy way to panic a system. vnodeA file specified with -f file becomes the backing store for this memory disk. swap Storage for this type of memory disk is allocated from buffer memory. Pages get pushed out to the swap when the system is under memory pressure, otherwise they stay in the operating memory. Using swap backing is gener- ally preferable over malloc backing. -t vnode points to a file, not to memory. I have no idea how that file is being backed though on the machine.. The idea is to load this backing storage once and only once from the DVD into memory and leave it there. I think you wanted -t malloc -o reserve, maybe based on your description above. HTH, -Garrett ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Locking a file backed mdconfig into memory
On Thu, May 27, 2010 at 04:48:42PM -0700, Dave Hayes wrote: On FreeBSD 7.3-STABLE I'm mounting a DVD and doing something like this: mdconfig -a -t vnode -o reserve -o readonly -f /dvd/file so that /dvd/file becomes the backing storage for my memory disk. It sounds like what you really want is to load the contents of the specified file as a memory system? That's not part of mdconfig's repertoire, to the best of my recollection. If that's what you want, you need to use a different tool; the purpose of mdconfig is to provide scratch disk. The backing store is to specify a region its contents can be swapped out to if the system is under memory pressure (which certainly won't work with a DVD) -- Clifton -- Clifton Royston -- clift...@iandicomputing.com / clift...@lava.net President - I and I Computing * http://www.iandicomputing.com/ Custom programming, network design, systems and network consulting services ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
