Re: FreeBSD Boot Times
Greetings, I was just wondering what it is that FreeBSD does that makes it take so long to boot. Booting into Ubuntu minimal or my own custom Linux distro, literally takes 0.5-2 seconds to boot up to shell, where FreeBSD takes about 10-20 seconds. I'm not sure if anything could be parallelized in the boot process, mostly kernel time. Note: This isn't really an issue, moreso a curiosity. true. system that never crash are not often booted ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On Wed, Jun 13, 2012 at 09:10:04AM +0200, Wojciech Puchar wrote: mostly kernel time. Note: This isn't really an issue, moreso a curiosity. true. system that never crash are not often booted ... I'd rather not get into a long discussion about this, but the above reflects a perception that I do not share, for the simple reason that I track FreeBSD stable/8, stable/9, and head daily on my laptop. And since each of these is an in-place source upgrade, I reboot to get into each of those environments. And since I actually track stable/9 twice (one slice has stable/9 built with gcc; the other with clang) -- well, my mornings tend to be busy. (Just installing the stable/8 freshly-built kernel as I type.) In addition, I normally power the laptop off when I put it in my backpack cycle to or from the train station. (Well, if I'm running stable/9 or head at the time, I can suspend it resume once I've arrived.) Granted, neither of the above may be especially common, but diversity [in this case, of experience] is a wonderful thing. Let us not assume that others' experiences and perceptions are limited to our own -- please? :-) Peace, david -- David H. Wolfskill da...@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. pgpMSL26mqKiE.pgp Description: PGP signature
Re: FreeBSD Boot Times
In message alpine.bsf.2.00.1206130909310.73...@wojtek.tensor.gdynia.pl, Wojci ech Puchar writes: One of the major slowdowns is that we do all the device drivers serially synchronously. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: wired memory - again!
On Tue, 2012-06-12 at 23:45 +0300, Konstantin Belousov wrote: On Tue, Jun 12, 2012 at 08:51:34AM -0600, Ian Lepore wrote: On Sat, 2012-06-09 at 22:45 +0200, Wojciech Puchar wrote: First, all memory allocated by UMA and consequently malloc(9) is wired. In other words, almost all memory used by kernel is accounted as wired. yes i understand this. still i found no way how to find out what allocated that much. Second, the buffer cache wires the pages which are inserted into VMIO buffers. So your observation is basically right, cached buffers means what are exactly VMIO buffers. i understand that page must be wired WHEN doing I/O. But i have too much wired memory even when doing no I/O at all. I agree, this is The Big Question for me. Why does the system keep wired writable mappings of the buffers in kva after the IO operations are completed? Read about buffer cache, e.g. in the Design and Implementation of the FreeBSD OS book. If it did not do so, it would fix the instruction-cache-disabled bug that kills performance on VIVT cache architectures (arm and mips) and it would reduce the amount of wired memory (that apparently doesn't need to be wired, unless I've missed the implications of a previous reply in this thread). I have no idea what is the bug you are talking about. If my guess is right, and it specifically references unability of some processors to correctly handle several mappings of the same physical page into different virtual addresses due to cache tagging using virtual address instead of physical, then this is a hardware bug, not software. This bug: http://lists.freebsd.org/pipermail/freebsd-arm/2012-January/003288.html The bug isn't the VIVT cache hardware, it's the fact that the way we handle the requirements of the hardware has the side effect of leaving the instruction cache bit disabled on executable pages because the kernel keeps writable mappings of the pages even after the IO is done. AFAIR, at least HP PA and MIPS have different instantiation of this problem. Our kernel uses multi-mapping quite often, and buffers is only one example. Also, why do you think that the pages entered into buffers shall not be wired, it is completely beyond my understanding. What's beyond my understanding is why a page has to remain wired after the IO is complete. That question seems to me to be tangentially related to the above question of why the kernel needs to keep a writable mapping of the buffer after it's done writing into the page (either via DMA or via uiomove() depending on the direction of the IO). -- Ian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On Wed, 2012-06-13 at 09:10 +0200, Wojciech Puchar wrote: Greetings, I was just wondering what it is that FreeBSD does that makes it take so long to boot. Booting into Ubuntu minimal or my own custom Linux distro, literally takes 0.5-2 seconds to boot up to shell, where FreeBSD takes about 10-20 seconds. I'm not sure if anything could be parallelized in the boot process, mostly kernel time. Note: This isn't really an issue, moreso a curiosity. true. system that never crash are not often booted An embedded system may be booted or powered cycled dozens of times a day, and boot time can be VERY important. Don't assume that the way you use FreeBSD is the only way. -- Ian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: boot menu option to disable graphics mode
on 09/06/2012 19:17 Doug Barton said the following: If this were a problem we didn't already have a solution for, I'd be much more interested in what you're proposing. I wonder if you were in the same mindset when you worked on service(8). This is not to doubt service(8) usefulness, of course. Just drawing a parallel. But in no particular order ... 1. This is not something most users would have to do very often, if at all. 1. Let's not generalize. 2. It is not a coincidence that I started this thread on this mailing list. 2. We have a variety of different login managers, several of which do things subtly differently, all of which would need ongoing support. The solution as proposed of now does not require any support or modifications. If people would be willing to implement additional support, then probably they would be doing that because they would want to have that, and to support that. 3. While the changes you're proposing sound simple, the startup stuff has some subtle interactions that we don't like to disrupt without good reason. This is too vague to comment. It's also worth pointing out that if all you need is a shell at boot time, you can still do Ctrl-Alt-F1 to get that, without having to change anything. Thank you for opening my eyes. And sorry for using sarcasm again. No, that's not what I want. I want X to not start. And if you find yourself needing to prevent the login manager from starting more often than not, just disable it by default and start it with 'service blah onestart', or use startx. I do need it that often that I'd have to inconvenience each boot. But I also want convenience those time when I need it. My point being that this doesn't come with zero costs, and has very little benefit. That usually spells no in my book. I understand your point. On the other hand, I find the proposed change to have measurable benefit and insignificant cost. This is yes in my book. Please also note that I am not asking you to do any work. -- Andriy Gapon ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On 06/12/12 18:00, Richard Yao wrote: On 06/11/12 18:51, Garrett Cooper wrote: On Mon, Jun 11, 2012 at 3:21 PM, Brandon Falkbfalk_...@brandonfa.lk wrote: Greetings, I was just wondering what it is that FreeBSD does that makes it take so long to boot. Booting into Ubuntu minimal or my own custom Linux distro, literally takes 0.5-2 seconds to boot up to shell, where FreeBSD takes about 10-20 seconds. I'm not sure if anything could be parallelized in the boot process, but Linux somehow manages to do it. The Ubuntu install I do pretty much consists of a shell and developers tools, but it still has a generic kernel. There must be some sort of polling done in the FreeBSD boot process that could be parallelized or eliminated. Anyone have any suggestions? Note: This isn't really an issue, moreso a curiosity. The single process nature of rc is a big part of the problem, as is the single AP bootup of FreeBSD right before multiuser mode. There are a number of threads that discuss this (look for parallel rc bootup or something like that in the current, hacker, and rc archives -- the most recent discussion was probably 6~9 months ago). Given past experience, a big part of getting past the parallelized rc mess would be to make services fail/wait gracefully for all their resources to come up before proceeding. It's not easy, but it's possible with enough resources. HTH, -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org Gentoo FreeBSD shares OpenRC with Gentoo Linux. OpenRC is a BSD 2-clause licensed System V init system replacement that supports parallel boot. Its boot performance is competitive with systemd and Ubuntu's upstart. If FreeBSD's init system is serializing the boot process, it might be worthwhile to consider importing OpenRC. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org Please don't change any of the user-facing aspects of the RC system. One of the things that brought me (and many others I know) to FreeBSD, besides working sound, was having an rc.conf that was easy to configure instead of the nightmare that is System V init. -Nathan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Upcoming release schedule - 8.4 ?
On Tuesday, June 12, 2012 8:01:00 pm Adrian Chadd wrote: hi, You don't need to change the FreeBSD culture. We'd love to do an 8.4 release. And an 8.5 release, and 8.6 release, etc. The problem is one of resources and time, not of culture/desire. I disagree. The pace of X.0 releases is a deliberate choice FreeBSD has made and directly impacts the number of live branches in existence. Given our developer base, we can't really support 3 branches concurrently (head + 2 stable like we have now with head, 9, and 8). Having longer lived stable branches requires either increasing resources to support exising releases longer, or slowing the pace of X.0 releases (but more aggressively merging things from HEAD back). The latter case, especially, is part of the culture and would be a choice we as a Project would have to make. -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Upcoming release schedule - 8.4 ?
On Wed, Jun 13, 2012 at 5:53 AM, John Baldwin j...@freebsd.org wrote:
On Tuesday, June 12, 2012 8:01:00 pm Adrian Chadd wrote:
hi,
You don't need to change the FreeBSD culture. We'd love to do an 8.4
release. And an 8.5 release, and 8.6 release, etc. The problem is one
of resources and time, not of culture/desire.
I disagree. The pace of X.0 releases is a deliberate choice FreeBSD
has made and directly impacts the number of live branches in existence.
Given our developer base, we can't really support 3 branches concurrently
(head + 2 stable like we have now with head, 9, and 8). Having longer lived
stable branches requires either increasing resources to support exising
releases longer, or slowing the pace of X.0 releases (but more aggressively
merging things from HEAD back). The latter case, especially, is part of
the culture and would be a choice we as a Project would have to make.
The only way that this would really work is if there were
dedicated sustaining engineers working on actively backporting code,
testing it, committing it, etc as the current paradigm requires the
developer (or another stand-in developer in the event that the
original developer failed to MFC the code) to do the work (which is
sort of what the OP is doing in this case, and what I've seen a few
different groups do that don't run bleeding edge code). That concept
doesn't really exist today. Maybe this would be a good idea for
improving the longevity of release cycles and maybe that's what
ultimately needs to be done as it would reduce distractions on
developers actively churning away on {[CURRENT-1],}[CURRENT], and
maybe that's what should be proposed.
As good as BSDi was from what I hear, that business model alone
won't probably work as many people take the support piece (companies
that just want the software and might develop/support apps/services on
top of it) and the longevity piece (companies that develop with/on the
software as a product) separately. They're typically (but not always)
mutually exclusive from what I've seen in my limited experience.
Thanks,
-Garrett
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Upcoming release schedule - 8.4 ?
On 13 June 2012 05:53, John Baldwin j...@freebsd.org wrote: You don't need to change the FreeBSD culture. We'd love to do an 8.4 release. And an 8.5 release, and 8.6 release, etc. The problem is one of resources and time, not of culture/desire. I disagree. The pace of X.0 releases is a deliberate choice FreeBSD has made and directly impacts the number of live branches in existence. Given our developer base, we can't really support 3 branches concurrently (head + 2 stable like we have now with head, 9, and 8). Having longer lived stable branches requires either increasing resources to support exising releases longer, or slowing the pace of X.0 releases (but more aggressively merging things from HEAD back). The latter case, especially, is part of the culture and would be a choice we as a Project would have to make. Right, but I don't think the freebsd project would really mind or change much if more people came on board to handle legacy releases and support them. If you're a company that uses FreeBSD stable releases, please consider contributing engineering resources and/or donations to the Foundation to improve the support of said stable releases. :) Adrian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On 6/13/2012 10:19 AM, Nathan Whitehorn wrote: On 06/12/12 18:00, Richard Yao wrote: On 06/11/12 18:51, Garrett Cooper wrote: On Mon, Jun 11, 2012 at 3:21 PM, Brandon Falkbfalk_...@brandonfa.lk wrote: Greetings, I was just wondering what it is that FreeBSD does that makes it take so long to boot. Booting into Ubuntu minimal or my own custom Linux distro, literally takes 0.5-2 seconds to boot up to shell, where FreeBSD takes about 10-20 seconds. I'm not sure if anything could be parallelized in the boot process, but Linux somehow manages to do it. The Ubuntu install I do pretty much consists of a shell and developers tools, but it still has a generic kernel. There must be some sort of polling done in the FreeBSD boot process that could be parallelized or eliminated. Anyone have any suggestions? Note: This isn't really an issue, moreso a curiosity. The single process nature of rc is a big part of the problem, as is the single AP bootup of FreeBSD right before multiuser mode. There are a number of threads that discuss this (look for parallel rc bootup or something like that in the current, hacker, and rc archives -- the most recent discussion was probably 6~9 months ago). Given past experience, a big part of getting past the parallelized rc mess would be to make services fail/wait gracefully for all their resources to come up before proceeding. It's not easy, but it's possible with enough resources. HTH, -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org Gentoo FreeBSD shares OpenRC with Gentoo Linux. OpenRC is a BSD 2-clause licensed System V init system replacement that supports parallel boot. Its boot performance is competitive with systemd and Ubuntu's upstart. If FreeBSD's init system is serializing the boot process, it might be worthwhile to consider importing OpenRC. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org Please don't change any of the user-facing aspects of the RC system. One of the things that brought me (and many others I know) to FreeBSD, besides working sound, was having an rc.conf that was easy to configure instead of the nightmare that is System V init. -Nathan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org Sorry guys, I was very busy last night. The issue is not the init system, rather the kernel. Maybe for a server does the init system start to get lengthy, but at least on my desktop environment where I have almost nothing for RC to do, the kernel bootup takes about 95% of the boot process. If I were to really delve into this (as it would require extensive changes in the kernel [I think]), I would probably end up branching FreeBSD either to a new OS, or if I got permission, to just another branch of FreeBSD. -Brandon ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Upcoming release schedule - 8.4 ?
On Wednesday, June 13, 2012 11:52:28 am Adrian Chadd wrote: On 13 June 2012 05:53, John Baldwin j...@freebsd.org wrote: You don't need to change the FreeBSD culture. We'd love to do an 8.4 release. And an 8.5 release, and 8.6 release, etc. The problem is one of resources and time, not of culture/desire. I disagree. The pace of X.0 releases is a deliberate choice FreeBSD has made and directly impacts the number of live branches in existence. Given our developer base, we can't really support 3 branches concurrently (head + 2 stable like we have now with head, 9, and 8). Having longer lived stable branches requires either increasing resources to support exising releases longer, or slowing the pace of X.0 releases (but more aggressively merging things from HEAD back). The latter case, especially, is part of the culture and would be a choice we as a Project would have to make. Right, but I don't think the freebsd project would really mind or change much if more people came on board to handle legacy releases and support them. If you're a company that uses FreeBSD stable releases, please consider contributing engineering resources and/or donations to the Foundation to improve the support of said stable releases. :) No, that doesn't actually work. Having additional support on a stable branch requires someone able to 1) commit changes to stable branches and 2) be able to cut newer releases from said branches (i.e. doing the work of re@). You cannot get that as an outside entity. It requires buy-in from the Project itself. -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Upcoming release schedule - 8.4 ?
Hey, I'm a Desktop User and I wish FreeBSD v8.3 worked for me. I can't get a Dialup Internet Connection without setting up a complicated script. And my Porn Videos crash halfway through. Yours frustratedly: Frank Mitchell On Wednesday 13 June 2012 00:08:08 Jerry McAllister wrote: Well, 8.3 is working fine for me. It is being well maintained. You sound like the people who can't decide to get something because a new version is going to come out sometime before they die. jerry ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: wired memory - again!
On Wed, Jun 13, 2012 at 07:14:09AM -0600, Ian Lepore wrote: On Tue, 2012-06-12 at 23:45 +0300, Konstantin Belousov wrote: On Tue, Jun 12, 2012 at 08:51:34AM -0600, Ian Lepore wrote: On Sat, 2012-06-09 at 22:45 +0200, Wojciech Puchar wrote: First, all memory allocated by UMA and consequently malloc(9) is wired. In other words, almost all memory used by kernel is accounted as wired. yes i understand this. still i found no way how to find out what allocated that much. Second, the buffer cache wires the pages which are inserted into VMIO buffers. So your observation is basically right, cached buffers means what are exactly VMIO buffers. i understand that page must be wired WHEN doing I/O. But i have too much wired memory even when doing no I/O at all. I agree, this is The Big Question for me. Why does the system keep wired writable mappings of the buffers in kva after the IO operations are completed? Read about buffer cache, e.g. in the Design and Implementation of the FreeBSD OS book. If it did not do so, it would fix the instruction-cache-disabled bug that kills performance on VIVT cache architectures (arm and mips) and it would reduce the amount of wired memory (that apparently doesn't need to be wired, unless I've missed the implications of a previous reply in this thread). I have no idea what is the bug you are talking about. If my guess is right, and it specifically references unability of some processors to correctly handle several mappings of the same physical page into different virtual addresses due to cache tagging using virtual address instead of physical, then this is a hardware bug, not software. This bug: http://lists.freebsd.org/pipermail/freebsd-arm/2012-January/003288.html The bug isn't the VIVT cache hardware, it's the fact that the way we handle the requirements of the hardware has the side effect of leaving the instruction cache bit disabled on executable pages because the kernel keeps writable mappings of the pages even after the IO is done. Can you point me at the exact code in arm pmap ? I remember an issue on PPC which Nathan discussed, that sounds somewhat similar (but I still do not understand what exactly happens on ARM). On PowerPC, icache needs to be explicitely flushed if write happen to the executable mapping. See r233949 for current solution. There were some followups, but I believe the core change is still valid. AFAIR, at least HP PA and MIPS have different instantiation of this problem. Our kernel uses multi-mapping quite often, and buffers is only one example. Also, why do you think that the pages entered into buffers shall not be wired, it is completely beyond my understanding. What's beyond my understanding is why a page has to remain wired after the IO is complete. That question seems to me to be tangentially related to the above question of why the kernel needs to keep a writable mapping of the buffer after it's done writing into the page (either via DMA or via uiomove() depending on the direction of the IO). Because the buffer is cached. pgpgg7akZmC0h.pgp Description: PGP signature
Re: Upcoming release schedule - 8.4 ?
I'll share my 2 cents here, as someone who maintains a decent sided FreeBSD install. 1. FreeBSD needs to make end users more comfortable with using a Dot-Ohh release; and at the time of the dot-ohh release a timeline for the next point releases should be made. * 2. Having three supported releases is showing issues , and brings up the point of why was 9.0 not released as 8.3 ? ** 3. The end users appear to want less releases, and for them to be supported longer . * A rough outline would do and it should be on the main release page http://www.freebsd.org/releases/ ** Yes I understand that 9.0 had tons of new features that were added and its not exactly a point release upgrade from 8.2 , however one can argue that if it were there would be less yelling about when version X is going to be EOL'd and when will version Y be released. -- mark saad | nones...@longcount.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: wired memory - again!
On 06/13/12 13:22, Konstantin Belousov wrote: On Wed, Jun 13, 2012 at 07:14:09AM -0600, Ian Lepore wrote: On Tue, 2012-06-12 at 23:45 +0300, Konstantin Belousov wrote: On Tue, Jun 12, 2012 at 08:51:34AM -0600, Ian Lepore wrote: On Sat, 2012-06-09 at 22:45 +0200, Wojciech Puchar wrote: First, all memory allocated by UMA and consequently malloc(9) is wired. In other words, almost all memory used by kernel is accounted as wired. yes i understand this. still i found no way how to find out what allocated that much. Second, the buffer cache wires the pages which are inserted into VMIO buffers. So your observation is basically right, cached buffers means what are exactly VMIO buffers. i understand that page must be wired WHEN doing I/O. But i have too much wired memory even when doing no I/O at all. I agree, this is The Big Question for me. Why does the system keep wired writable mappings of the buffers in kva after the IO operations are completed? Read about buffer cache, e.g. in the Design and Implementation of the FreeBSD OS book. If it did not do so, it would fix the instruction-cache-disabled bug that kills performance on VIVT cache architectures (arm and mips) and it would reduce the amount of wired memory (that apparently doesn't need to be wired, unless I've missed the implications of a previous reply in this thread). I have no idea what is the bug you are talking about. If my guess is right, and it specifically references unability of some processors to correctly handle several mappings of the same physical page into different virtual addresses due to cache tagging using virtual address instead of physical, then this is a hardware bug, not software. This bug: http://lists.freebsd.org/pipermail/freebsd-arm/2012-January/003288.html The bug isn't the VIVT cache hardware, it's the fact that the way we handle the requirements of the hardware has the side effect of leaving the instruction cache bit disabled on executable pages because the kernel keeps writable mappings of the pages even after the IO is done. Can you point me at the exact code in arm pmap ? I remember an issue on PPC which Nathan discussed, that sounds somewhat similar (but I still do not understand what exactly happens on ARM). On PowerPC, icache needs to be explicitely flushed if write happen to the executable mapping. See r233949 for current solution. There were some followups, but I believe the core change is still valid. The general algorithm I used on PPC (which is PIPT, but still has an incoherent icache) is based on the following guarantees/observations, which seem to be sufficient for FreeBSD to run correctly: 1. Executable kernel memory never contains new data except after a module load, so only do kernel icache flushes in elf_machdep.c after a module load. 2. Executable pages are never mapped into userland until the kernel is finished writing to them. Thus, userland icache consistency is maintained with respect to all kernel operations (executable loading, swap, etc.) if icaches are made coherent once at the time that the page is entered into its first non-kernel pmap. If your chip has an NX feature, this only need be done for the first executable user mapping -- otherwise it needs to be done for the first overall mapping to prevent information leakage via the icache. I guess for VIVT, first may mean every here. 3. I-cache coherency with respect to userland modifications is the responsibility of the user program. All self-modifying code knows, or should know, what to do here. Otherwise the only time this comes up is in RTLD, which is easily modified to do an icache flush after load. The general point is that even if the kernel maintains its writable mapping after the IO is complete, it will never actually write to that mapping after the page has been mapped into its first user process and therefore it is safe to maintain cacheability at all times and do a single invalidation when it is mapped into the user pmap. -Nathan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Rtld object tasting [Was: Re: wired memory - again!]
On Wed, Jun 13, 2012 at 07:14:09AM -0600, Ian Lepore wrote:
http://lists.freebsd.org/pipermail/freebsd-arm/2012-January/003288.html
The map_object.c patch is step in the almost right direction, I wanted
to remove the static page-sized buffer from get_elf_header for long time.
It works because rtld always holds bind_lock exclusively while loading
an object. There is no need to copy the first page after it is mapped.
commit 0f6f8629af1345acded7c0c685d3ff7b4d9180d6
Author: Konstantin Belousov k...@freebsd.org
Date: Wed Jun 13 22:04:18 2012 +0300
Eliminate the static buffer used to read the first page of the mapped
object, and eliminate the pread(2) call as well. Mmap the first page
of the object temporaly, and unmap it on error or last use.
Fix several cases were the whole mapping of the object leaked on error.
Potentially, this leaves one-page gap between succeeding dlopen(3),
but there are other mmap(2) consumers as well.
diff --git a/libexec/rtld-elf/map_object.c b/libexec/rtld-elf/map_object.c
index d74ef15..7068433 100644
--- a/libexec/rtld-elf/map_object.c
+++ b/libexec/rtld-elf/map_object.c
@@ -38,7 +38,7 @@
#include debug.h
#include rtld.h
-static Elf_Ehdr *get_elf_header (int, const char *);
+static Elf_Ehdr *get_elf_header(int, const char *);
static int convert_prot(int); /* Elf flags - mmap protection */
static int convert_flags(int); /* Elf flags - mmap flags */
@@ -121,7 +121,7 @@ map_object(int fd, const char *path, const struct stat *sb)
if ((segs[nsegs]-p_align (PAGE_SIZE - 1)) != 0) {
_rtld_error(%s: PT_LOAD segment %d not page-aligned,
path, nsegs);
- return NULL;
+ goto error;
}
break;
@@ -161,12 +161,12 @@ map_object(int fd, const char *path, const struct stat
*sb)
}
if (phdyn == NULL) {
_rtld_error(%s: object is not dynamically-linked, path);
- return NULL;
+ goto error;
}
if (nsegs 0) {
_rtld_error(%s: too few PT_LOAD segments, path);
- return NULL;
+ goto error;
}
/*
@@ -183,13 +183,12 @@ map_object(int fd, const char *path, const struct stat
*sb)
if (mapbase == (caddr_t) -1) {
_rtld_error(%s: mmap of entire address space failed: %s,
path, rtld_strerror(errno));
- return NULL;
+ goto error;
}
if (base_addr != NULL mapbase != base_addr) {
_rtld_error(%s: mmap returned wrong address: wanted %p, got %p,
path, base_addr, mapbase);
- munmap(mapbase, mapsize);
- return NULL;
+ goto error1;
}
for (i = 0; i = nsegs; i++) {
@@ -204,7 +203,7 @@ map_object(int fd, const char *path, const struct stat *sb)
data_flags, fd, data_offset) == (caddr_t) -1) {
_rtld_error(%s: mmap of data failed: %s, path,
rtld_strerror(errno));
- return NULL;
+ goto error1;
}
/* Do BSS setup */
@@ -221,7 +220,7 @@ map_object(int fd, const char *path, const struct stat *sb)
mprotect(clear_page, PAGE_SIZE, data_prot|PROT_WRITE)) {
_rtld_error(%s: mprotect failed: %s, path,
rtld_strerror(errno));
- return NULL;
+ goto error1;
}
memset(clear_addr, 0, nclear);
@@ -240,7 +239,7 @@ map_object(int fd, const char *path, const struct stat *sb)
data_flags | MAP_ANON, -1, 0) == (caddr_t)-1) {
_rtld_error(%s: mmap of bss failed: %s, path,
rtld_strerror(errno));
- return NULL;
+ goto error1;
}
}
}
@@ -273,7 +272,7 @@ map_object(int fd, const char *path, const struct stat *sb)
if (obj-phdr == NULL) {
obj_free(obj);
_rtld_error(%s: cannot allocate program header, path);
-return NULL;
+ goto error1;
}
memcpy((char *)obj-phdr, (char *)hdr + hdr-e_phoff, phsize);
obj-phdr_alloc = true;
@@ -293,63 +292,72 @@ map_object(int fd, const char *path, const struct stat
*sb)
obj-relro_page = obj-relocbase + trunc_page(relro_page);
obj-relro_size = round_page(relro_size);
-return obj;
+munmap(hdr, PAGE_SIZE);
+return (obj);
+
+error1:
+munmap(mapbase, mapsize);
+error:
+munmap(hdr, PAGE_SIZE);
+return (NULL);
}
static Elf_Ehdr *
-get_elf_header (int fd, const char *path)
+get_elf_header(int fd, const char *path)
{
-static union {
- Elf_Ehdr hdr;
- char buf[PAGE_SIZE];
-} u;
-ssize_t nbytes;
-
-if ((nbytes = pread(fd, u.buf, PAGE_SIZE, 0)) == -1) {
- _rtld_error(%s: read error: %s, path, rtld_strerror(errno));
- return NULL;
-}
+ Elf_Ehdr *hdr;
-/* Make sure the file is valid */
-if (nbytes
Re: FreeBSD Boot Times
On Wednesday 13 June 2012 15:21:29 Ian Lepore wrote: On Wed, 2012-06-13 at 09:10 +0200, Wojciech Puchar wrote: Greetings, I was just wondering what it is that FreeBSD does that makes it take so long to boot. Booting into Ubuntu minimal or my own custom Linux distro, literally takes 0.5-2 seconds to boot up to shell, where FreeBSD takes about 10-20 seconds. I'm not sure if anything could be parallelized in the boot process, mostly kernel time. Note: This isn't really an issue, moreso a curiosity. true. system that never crash are not often booted An embedded system may be booted or powered cycled dozens of times a day, and boot time can be VERY important. Don't assume that the way you use FreeBSD is the only way. -- Ian Try setting: sysctl hw.usb.no_boot_wait=1 Might help a bit :-) --HPS ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Rtld object tasting [Was: Re: wired memory - again!]
On Wed, Jun 13, 2012 at 2:12 PM, Konstantin Belousov kostik...@gmail.comwrote: On Wed, Jun 13, 2012 at 07:14:09AM -0600, Ian Lepore wrote: http://lists.freebsd.org/pipermail/freebsd-arm/2012-January/003288.html The map_object.c patch is step in the almost right direction, I wanted to remove the static page-sized buffer from get_elf_header for long time. It works because rtld always holds bind_lock exclusively while loading an object. There is no need to copy the first page after it is mapped. commit 0f6f8629af1345acded7c0c685d3ff7b4d9180d6 Author: Konstantin Belousov k...@freebsd.org Date: Wed Jun 13 22:04:18 2012 +0300 Eliminate the static buffer used to read the first page of the mapped object, and eliminate the pread(2) call as well. Mmap the first page of the object temporaly, and unmap it on error or last use. Fix several cases were the whole mapping of the object leaked on error. Potentially, this leaves one-page gap between succeeding dlopen(3), but there are other mmap(2) consumers as well. I suggest adding MAP_PREFAULT_READ to the mmap(2) call. A heuristic in vm_map_pmap_enter() would trigger automatic mapping for small files, but if the object file is larger than 96 pages then you need to explicitly specific MAP_PREFAULT_READ. Alan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Rtld object tasting [Was: Re: wired memory - again!]
On Wed, Jun 13, 2012 at 03:17:56PM -0500, Alan Cox wrote:
On Wed, Jun 13, 2012 at 2:12 PM, Konstantin Belousov
kostik...@gmail.comwrote:
On Wed, Jun 13, 2012 at 07:14:09AM -0600, Ian Lepore wrote:
http://lists.freebsd.org/pipermail/freebsd-arm/2012-January/003288.html
The map_object.c patch is step in the almost right direction, I wanted
to remove the static page-sized buffer from get_elf_header for long time.
It works because rtld always holds bind_lock exclusively while loading
an object. There is no need to copy the first page after it is mapped.
commit 0f6f8629af1345acded7c0c685d3ff7b4d9180d6
Author: Konstantin Belousov k...@freebsd.org
Date: Wed Jun 13 22:04:18 2012 +0300
Eliminate the static buffer used to read the first page of the mapped
object, and eliminate the pread(2) call as well. Mmap the first page
of the object temporaly, and unmap it on error or last use.
Fix several cases were the whole mapping of the object leaked on error.
Potentially, this leaves one-page gap between succeeding dlopen(3),
but there are other mmap(2) consumers as well.
I suggest adding MAP_PREFAULT_READ to the mmap(2) call. A heuristic in
vm_map_pmap_enter() would trigger automatic mapping for small files, but if
the object file is larger than 96 pages then you need to explicitly
specific MAP_PREFAULT_READ.
Thank you for the suggestion.
commit 77de77b3124fb2742db1db72e9dfc47050c5ac36
Author: Konstantin Belousov k...@freebsd.org
Date: Wed Jun 13 23:22:27 2012 +0300
Use MAP_PREFAULT_READ for mmap(2) calls which map real object pages.
Suggested by: alc
diff --git a/libexec/rtld-elf/map_object.c b/libexec/rtld-elf/map_object.c
index 2afc88c..437e7c2 100644
--- a/libexec/rtld-elf/map_object.c
+++ b/libexec/rtld-elf/map_object.c
@@ -200,7 +200,7 @@ map_object(int fd, const char *path, const struct stat *sb)
data_prot = convert_prot(segs[i]-p_flags);
data_flags = convert_flags(segs[i]-p_flags) | MAP_FIXED;
if (mmap(data_addr, data_vlimit - data_vaddr, data_prot,
- data_flags, fd, data_offset) == (caddr_t) -1) {
+ data_flags | MAP_PREFAULT_READ, fd, data_offset) == (caddr_t) -1) {
_rtld_error(%s: mmap of data failed: %s, path,
rtld_strerror(errno));
goto error1;
@@ -307,7 +307,8 @@ get_elf_header(int fd, const char *path)
{
Elf_Ehdr *hdr;
- hdr = mmap(NULL, PAGE_SIZE, PROT_READ, MAP_PRIVATE, fd, 0);
+ hdr = mmap(NULL, PAGE_SIZE, PROT_READ, MAP_PRIVATE | MAP_PREFAULT_READ,
+ fd, 0);
if (hdr == (Elf_Ehdr *)MAP_FAILED) {
_rtld_error(%s: read error: %s, path, rtld_strerror(errno));
return NULL;
pgpLNjRFFnoAL.pgp
Description: PGP signature
Re: FreeBSD Boot Times
On 13 June 2012 12:55, Hans Petter Selasky hsela...@c2i.net wrote: On Wednesday 13 June 2012 15:21:29 Ian Lepore wrote: On Wed, 2012-06-13 at 09:10 +0200, Wojciech Puchar wrote: Greetings, I was just wondering what it is that FreeBSD does that makes it take so long to boot. Booting into Ubuntu minimal or my own custom Linux distro, literally takes 0.5-2 seconds to boot up to shell, where FreeBSD takes about 10-20 seconds. I'm not sure if anything could be parallelized in the boot process, mostly kernel time. Note: This isn't really an issue, moreso a curiosity. true. system that never crash are not often booted An embedded system may be booted or powered cycled dozens of times a day, and boot time can be VERY important. Don't assume that the way you use FreeBSD is the only way. -- Ian Try setting: sysctl hw.usb.no_boot_wait=1 Can you explain in a bit more detail what this does and why it isn't default? -- Eitan Adler ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On Wed, Jun 13, 2012 at 10:58 PM, Eitan Adler li...@eitanadler.com wrote: On 13 June 2012 12:55, Hans Petter Selasky hsela...@c2i.net wrote: On Wednesday 13 June 2012 15:21:29 Ian Lepore wrote: On Wed, 2012-06-13 at 09:10 +0200, Wojciech Puchar wrote: Greetings, I was just wondering what it is that FreeBSD does that makes it take so long to boot. Booting into Ubuntu minimal or my own custom Linux distro, literally takes 0.5-2 seconds to boot up to shell, where FreeBSD takes about 10-20 seconds. I'm not sure if anything could be parallelized in the boot process, mostly kernel time. Note: This isn't really an issue, moreso a curiosity. true. system that never crash are not often booted An embedded system may be booted or powered cycled dozens of times a day, and boot time can be VERY important. Don't assume that the way you use FreeBSD is the only way. -- Ian Try setting: sysctl hw.usb.no_boot_wait=1 Can you explain in a bit more detail what this does and why it isn't default? -- Eitan Adler ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org If you simplky do sysctl -d hw.usb.no_boot_wait you will see the explanation ;) -- Best regards, Claudiu Vasadi ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
The OpenRC is sysvinit compatible, but it has few of sysvinit's flaws. It has named runlevels, the presence of an init script does not cause it to start and it is in my opinion a joy to use. I suggest that you try OpenRC before drawing conclusions. You can install Gentoo FreeBSD in a jail. There are instructions for this on the Gentoo wiki: http://wiki.gentoo.org/wiki/Gentoo_FreeBSD#Howto_run_G.2FFBSD_in_vanilla_FreeBSD.27s_jail If you find deficiencies, I am certain that the OpenRC developers would appreciate feedback regarding them. On 06/13/12 10:19, Nathan Whitehorn wrote: On 06/12/12 18:00, Richard Yao wrote: On 06/11/12 18:51, Garrett Cooper wrote: On Mon, Jun 11, 2012 at 3:21 PM, Brandon Falkbfalk_...@brandonfa.lk wrote: Greetings, I was just wondering what it is that FreeBSD does that makes it take so long to boot. Booting into Ubuntu minimal or my own custom Linux distro, literally takes 0.5-2 seconds to boot up to shell, where FreeBSD takes about 10-20 seconds. I'm not sure if anything could be parallelized in the boot process, but Linux somehow manages to do it. The Ubuntu install I do pretty much consists of a shell and developers tools, but it still has a generic kernel. There must be some sort of polling done in the FreeBSD boot process that could be parallelized or eliminated. Anyone have any suggestions? Note: This isn't really an issue, moreso a curiosity. The single process nature of rc is a big part of the problem, as is the single AP bootup of FreeBSD right before multiuser mode. There are a number of threads that discuss this (look for parallel rc bootup or something like that in the current, hacker, and rc archives -- the most recent discussion was probably 6~9 months ago). Given past experience, a big part of getting past the parallelized rc mess would be to make services fail/wait gracefully for all their resources to come up before proceeding. It's not easy, but it's possible with enough resources. HTH, -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org Gentoo FreeBSD shares OpenRC with Gentoo Linux. OpenRC is a BSD 2-clause licensed System V init system replacement that supports parallel boot. Its boot performance is competitive with systemd and Ubuntu's upstart. If FreeBSD's init system is serializing the boot process, it might be worthwhile to consider importing OpenRC. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org Please don't change any of the user-facing aspects of the RC system. One of the things that brought me (and many others I know) to FreeBSD, besides working sound, was having an rc.conf that was easy to configure instead of the nightmare that is System V init. -Nathan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On 13 June 2012 14:16, claudiu vasadi claudiu.vas...@gmail.com wrote:
If you simplky do sysctl -d hw.usb.no_boot_wait you will see the
explanation ;)
No, you see a one liner that only explains things if you already
understand what is going on:
hw.usb.no_boot_wait: No USB device enumerate waiting at boot.
Does setting this sysctl mean that my { printer, phone, keyboard }
won't function?***
--
Eitan Adler
*** I'm being rhetorical here, but we don't have good sysctl documentation.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On Wed, Jun 13, 2012 at 8:35 PM, Richard Yao r...@gentoo.org wrote: The OpenRC is sysvinit compatible, but it has few of sysvinit's flaws. It has named runlevels, the presence of an init script does not cause it to start and it is in my opinion a joy to use. I suggest that you try OpenRC before drawing conclusions. You can install Gentoo FreeBSD in a jail. There are instructions for this on the Gentoo wiki: http://wiki.gentoo.org/wiki/Gentoo_FreeBSD#Howto_run_G.2FFBSD_in_vanilla_FreeBSD.27s_jail If you find deficiencies, I am certain that the OpenRC developers would appreciate feedback regarding them. On 06/13/12 10:19, Nathan Whitehorn wrote: On 06/12/12 18:00, Richard Yao wrote: On 06/11/12 18:51, Garrett Cooper wrote: On Mon, Jun 11, 2012 at 3:21 PM, Brandon Falkbfalk_...@brandonfa.lk wrote: Greetings, I was just wondering what it is that FreeBSD does that makes it take so long to boot. Booting into Ubuntu minimal or my own custom Linux distro, literally takes 0.5-2 seconds to boot up to shell, where FreeBSD takes about 10-20 seconds. I'm not sure if anything could be parallelized in the boot process, but Linux somehow manages to do it. The Ubuntu install I do pretty much consists of a shell and developers tools, but it still has a generic kernel. There must be some sort of polling done in the FreeBSD boot process that could be parallelized or eliminated. Anyone have any suggestions? Note: This isn't really an issue, moreso a curiosity. The single process nature of rc is a big part of the problem, as is the single AP bootup of FreeBSD right before multiuser mode. There are a number of threads that discuss this (look for parallel rc bootup or something like that in the current, hacker, and rc archives -- the most recent discussion was probably 6~9 months ago). Given past experience, a big part of getting past the parallelized rc mess would be to make services fail/wait gracefully for all their resources to come up before proceeding. It's not easy, but it's possible with enough resources. HTH, -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org Gentoo FreeBSD shares OpenRC with Gentoo Linux. OpenRC is a BSD 2-clause licensed System V init system replacement that supports parallel boot. Its boot performance is competitive with systemd and Ubuntu's upstart. If FreeBSD's init system is serializing the boot process, it might be worthwhile to consider importing OpenRC. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org Please don't change any of the user-facing aspects of the RC system. One of the things that brought me (and many others I know) to FreeBSD, besides working sound, was having an rc.conf that was easy to configure instead of the nightmare that is System V init. -Nathan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org @Eitan: I think is simply does what it says (does not wait for the usb devices to be ready, instead just activates the usb hub and then the device will signal when it;s ready). My low level programming sucks and this is just a hunch. PS: I tried it on my laptop and as far as I've seen, it behaves like that. corrections are welcomed. -- Best regards, Claudiu Vasadi ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On 13-6-2012 23:16, claudiu vasadi wrote: If you simplky do sysctl -d hw.usb.no_boot_wait you will see the explanation ;) Probably why Eitan asked as that description: a) means nothing to people unfamiliar with device enumerations b) does not point to a manual page that explains how USB does device enumerations and why it would account for a significant chunk of the boot process. -- Mel ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Upcoming release schedule - 8.4 ?
On 13 Jun 2012, at 19:27, John Baldwin j...@freebsd.org wrote: On Wednesday, June 13, 2012 11:52:28 am Adrian Chadd wrote: On 13 June 2012 05:53, John Baldwin j...@freebsd.org wrote: You don't need to change the FreeBSD culture. We'd love to do an 8.4 release. And an 8.5 release, and 8.6 release, etc. The problem is one of resources and time, not of culture/desire. I disagree. The pace of X.0 releases is a deliberate choice FreeBSD has made and directly impacts the number of live branches in existence. Given our developer base, we can't really support 3 branches concurrently (head + 2 stable like we have now with head, 9, and 8). Having longer lived stable branches requires either increasing resources to support exising releases longer, or slowing the pace of X.0 releases (but more aggressively merging things from HEAD back). The latter case, especially, is part of the culture and would be a choice we as a Project would have to make. Right, but I don't think the freebsd project would really mind or change much if more people came on board to handle legacy releases and support them. If you're a company that uses FreeBSD stable releases, please consider contributing engineering resources and/or donations to the Foundation to improve the support of said stable releases. :) No, that doesn't actually work. Having additional support on a stable branch requires someone able to 1) commit changes to stable branches and 2) be able to cut newer releases from said branches (i.e. doing the work of re@). You cannot get that as an outside entity. It requires buy-in from the Project itself. Jumping in. I for one, as a fbsd admin on corporate servers ( read not commiter), would dearly like less releases but a more aggressive MFC approach. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On Tue, 12 Jun 2012, Warren Block wrote: On Tue, 12 Jun 2012, Richard Yao wrote: Gentoo FreeBSD shares OpenRC with Gentoo Linux. OpenRC is a BSD 2-clause licensed System V init system replacement that supports parallel boot. Its boot performance is competitive with systemd and Ubuntu's upstart. If FreeBSD's init system is serializing the boot process, it might be worthwhile to consider importing OpenRC. There have been at least two attempts to parallelize the rc scripts. The more recent one was posted on the forums. AFAIR, one advantage it had was no or few changes to the rc scripts, and something like 40% improvement in startup time. I'll post a link if I can find it. Startup time is a big deal for notebooks. Thanks to pointers from forum members and moderators, links to various attempts to improve startup time: An older attempt in -current: http://freebsd.1045724.n5.nabble.com/rc-improvements-wanted-td3911390.html The forums post: http://forums.freebsd.org/showthread.php?t=25822 And a newer effort from the PC-BSD folks which sounds very promising: http://comments.gmane.org/gmane.os.pcbsd.testing/6225 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: boot menu option to disable graphics mode
On 06/13/2012 06:50 AM, Andriy Gapon wrote: on 09/06/2012 19:17 Doug Barton said the following: If this were a problem we didn't already have a solution for, I'd be much more interested in what you're proposing. I wonder if you were in the same mindset when you worked on service(8). This is not to doubt service(8) usefulness, of course. Just drawing a parallel. But in no particular order ... 1. This is not something most users would have to do very often, if at all. 1. Let's not generalize. In order to make intelligent decisions about what changes to include and what changes to exclude we have to measure both the costs, and the benefits. Part of measuring the latter is knowing how many of our users will benefit from the proposed change. The percentage of desktop users that we have is (regrettably) a very small percentage of our total demographic. The percentage of those users who would benefit from this proposed change is smaller still. OTOH, every single FreeBSD user is affected by changes to the boot process. Furthermore, we already have a non-trivial public perception that FreeBSD is a hobbyist OS, by and for the developers first and foremost. I don't want to do anything that contributes to that perception without good reason. 2. It is not a coincidence that I started this thread on this mailing list. I get that. But changes we make to the boot process aren't restricted to the members of this mailing list. 2. We have a variety of different login managers, several of which do things subtly differently, all of which would need ongoing support. The solution as proposed of now does not require any support or modifications. Which solution are you discussing? Marcin's? I think his idea has a lot of merit, but in reference to your particular use case (inhibiting X from starting) it requires the user to know which particular knob (or knobs) is responsible. That's not necessarily a show-stopper, and people who are likely to need this are likely to be able to figure that out. If you're talking about a different proposed solution, please clarify. If people would be willing to implement additional support, then probably they would be doing that because they would want to have that, and to support that. The latter bit is what I'm most concerned about, especially long term. 3. While the changes you're proposing sound simple, the startup stuff has some subtle interactions that we don't like to disrupt without good reason. This is too vague to comment. That doesn't make the point invalid. :) If we have a specific solution that people are excited about, with patches, I'm happy to give it a more detailed review (along with freebsd-rc@ of course). It's also worth pointing out that if all you need is a shell at boot time, you can still do Ctrl-Alt-F1 to get that, without having to change anything. Thank you for opening my eyes. And sorry for using sarcasm again. FWIW I realize that *you* know that already. What I'm trying to do is to get an idea of what people want to accomplish, and make sure that we're not reinventing the wheel. No, that's not what I want. I want X to not start. Thanks for clarifying. And if you find yourself needing to prevent the login manager from starting more often than not, just disable it by default and start it with 'service blah onestart', or use startx. I do need it that often that I'd have to inconvenience each boot. But I also want convenience those time when I need it. My point being that this doesn't come with zero costs, and has very little benefit. That usually spells no in my book. I understand your point. On the other hand, I find the proposed change to have measurable benefit and insignificant cost. This is yes in my book. I think that we disagree on both the relative costs and the relative benefits. That's why I wanted to express my concerns so that others could weigh in. Please also note that I am not asking you to do any work. That sounds great in theory. However given the amount of time that I've spent on refining the boot process, as well as trying to get the boot time down as low as possible; and given the overwhelming importance of the boot process to the OS generally, I have concerns about what you're proposing. Just to be clear, I'm not saying, NO!, I'm saying that if we're going to make changes in this area that we need to understand the landscape very well before we move forward. My other concern, to be perfectly blunt, is that this project not become something where changes are made, and then when users report problems with those changes they are told that they are on their own to come up with the debugging, analysis, and/or fixes for those problems. If you're saying that resources exist to support the design, implementation, testing, commit to HEAD, support in HEAD, MFC(s), and support in the older branches; then I'm glad to hear that. If we don't have those resources, that's a factor we
Re: Upcoming release schedule - 8.4 ?
Mark Saad wrote: I'll share my 2 cents here, as someone who maintains a decent sided FreeBSD install. 1. FreeBSD needs to make end users more comfortable with using a Dot-Ohh release; and at the time of the dot-ohh release a timeline for the next point releases should be made. * 2. Having three supported releases is showing issues , and brings up the point of why was 9.0 not released as 8.3 ? ** 3. The end users appear to want less releases, and for them to be supported longer . * A rough outline would do and it should be on the main release page http://www.freebsd.org/releases/ ** Yes I understand that 9.0 had tons of new features that were added and its not exactly a point release upgrade from 8.2 , however one can argue that if it were there would be less yelling about when version X is going to be EOL'd and when will version Y be released. One thought here might be to revisit the Kernel APIs can only change on a major release rule. It seems to me that some KPIs could be frozen for longer periods than others, maybe? For example: - If device driver KPIs were frozen for a longer period of time, there wouldn't be the challenge of backporting drivers for newer hardware to the older systems. vs - The VFS/VOP interface. As far as I know, there are currently 2 out of source tree file systems (OpenAFS and FUSE) and there are FreeBSD committers involved in both of these. As such, making a VFS change within a minor release cycle might not be a big problem, so long as all the file systems in the source tree are fixed and the maintainers for the above 2 file systems were aware of the change and when they needed to release a patch/rebuild their module. - Similarily, are there any out of source tree network stacks? It seems that this rule is where the controversy of major vs minor release changes comes in? Just a thought, rick -- mark saad | nones...@longcount.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On 6/13/12 2:16 AM, Poul-Henning Kamp wrote: In message alpine.bsf.2.00.1206130909310.73...@wojtek.tensor.gdynia.pl, Wojci ech Puchar writes: One of the major slowdowns is that we do all the device drivers serially synchronously. Yes definitely. I have been looking into how to potentially defer or parallelize device_attach'es. Defer is turning out to be hard enough since each system is has different requirements to reach a state where it can run /sbin/init. I've started with the John Baldwin's multipass work and have a system stops probing/attaching devices and allows the boot to continue on. The remaining passes I'm triggering from userspace once the system is up. This is all very crude at this point and has been an some work just to understand how the kernel startup code all links together. Note systemd looks interesting from from a demand based startup scheme much like apples launchd. (note systemd uses linux process groups so porting it would take some effort) Ideally it would be nice to get to the point where many devices are only attached once there is a demand for it. Say network interfaces for example: attach it once the init scripts need to config it and then hopefully in an async fashion. Unfortunately that will require locking a bit more fine grain than the current Giant lock. -Russell signature.asc Description: OpenPGP digital signature
Re: FreeBSD Boot Times
On 12-6-2012 0:51, Garrett Cooper wrote: On Mon, Jun 11, 2012 at 3:21 PM, Brandon Falk bfalk_...@brandonfa.lk wrote: Greetings, I was just wondering what it is that FreeBSD does that makes it take so long to boot. Booting into Ubuntu minimal or my own custom Linux distro, literally takes 0.5-2 seconds to boot up to shell, where FreeBSD takes about 10-20 seconds. I'm not sure if anything could be parallelized in the boot process, but Linux somehow manages to do it. The Ubuntu install I do pretty much consists of a shell and developers tools, but it still has a generic kernel. There must be some sort of polling done in the FreeBSD boot process that could be parallelized or eliminated. Anyone have any suggestions? Note: This isn't really an issue, moreso a curiosity. The single process nature of rc is a big part of the problem, as is the single AP bootup of FreeBSD right before multiuser mode. There are a number of threads that discuss this (look for parallel rc bootup or something like that in the current, hacker, and rc archives -- the most recent discussion was probably 6~9 months ago). Given past experience, a big part of getting past the parallelized rc mess would be to make services fail/wait gracefully for all their resources to come up before proceeding. It's not easy, but it's possible with enough resources. I realize people are working on this and that it's generally a good thing, however - please don't underestimate the importance of getting an accurate list of what boots when and equally important how it shuts down. rcorder is vary valuable in diagnosing why certain services fail to start or throw fits, but you have to be able to match it with output from the rc script (something that not all scripts do I might add). -- Mel ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On Wed, Jun 13, 2012 at 3:49 PM, Russell Cattelan catte...@thebarn.comwrote: On 6/13/12 2:16 AM, Poul-Henning Kamp wrote: In message alpine.bsf.2.00.1206130909310.73...@wojtek.tensor.gdynia.pl, Wojci ech Puchar writes: One of the major slowdowns is that we do all the device drivers serially synchronously. Yes definitely. I have been looking into how to potentially defer or parallelize device_attach'es. Defer is turning out to be hard enough since each system is has different requirements to reach a state where it can run /sbin/init. I've started with the John Baldwin's multipass work and have a system stops probing/attaching devices and allows the boot to continue on. The remaining passes I'm triggering from userspace once the system is up. This is all very crude at this point and has been an some work just to understand how the kernel startup code all links together. Note systemd looks interesting from from a demand based startup scheme much like apples launchd. (note systemd uses linux process groups so porting it would take some effort) Ideally it would be nice to get to the point where many devices are only attached once there is a demand for it. Say network interfaces for example: attach it once the init scripts need to config it and then hopefully in an async fashion. Unfortunately that will require locking a bit more fine grain than the current Giant lock. -Russell To reduce the boot time , my opinion is as follows : During install or by using a program , generate a Hardware Profile File . By editing it , mark some devices No check ( for example , a network card or PS/2 mouse or key board , is not connected , RS-232 , Firewire , unused SATA ports , unused IDE ports , etc. , then it is not necessary to check them . ) During boot , first read that Hardware Profile File . Only check ports marked as Check . After completion of boot , the other ports may checked to update Hardware Profile File if it is requested in Hardware Profile File . Later on , assume a new device is attached . Run the Hardware Profile program to regenerate the Hardware Profile File , or by using dmesg , manually add this device into Hardware Profile File . For removable devices , if some USB , etc. ports are not used , they all may be marked as No Check , for example internal USB ports , unused back panel ports . I do not know such a scheme is useful or not , or usable or not . If I were a boot manager program writer , I would try it . To my knowledge which I may be wrong , at present there is no such a facility . Thank you very much . Mehmet Erol Sanliturk ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On 2012-06-13 14:18, Eitan Adler wrote: On 13 June 2012 14:16, claudiu vasadiclaudiu.vas...@gmail.com wrote: If you simplky do sysctl -d hw.usb.no_boot_wait you will see the explanation ;) No, you see a one liner that only explains things if you already understand what is going on: I believe it pertains to mounting root from a USB device. When set to 0, usb_attach tells VFS to wait on the USB device being attached. This results in not mounting root until the USB busses are all fully explored. If you don't rely on any USB devices for multi-user start-up, set this to 1 and let the kernel launch multi-user without waiting for USB probing. This is nice for those systems where the BIOS takes a long time to release control of any hubs with keyboards attached. IMO I'd rather have my gear do the boot process in a well-ordered fashion and get everything right, than try to shave off a handful of seconds at the risk of unreliable booting. For servers, the baseboard, OOB management, and SCSI BIOSes can mean minutes between power-on and OS bootstrap, so an extra 20 seconds is nothing. :) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: BIO_DELETE equivalent for file on FFS filesystem
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 05/26/12 08:06, Wojciech Puchar wrote: is it possible. suppose i have 1GB file with my data and 100 1 megabyte parts of it is no longer needed. i could reorganize that file to take 900MB or... can i call some system function to punch holes? I think you can only truncate the file at this time, pretty much like brk() works for memory. Cheers, - -- Xin LI delp...@delphij.nethttps://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBCAAGBQJP2UJAAAoJEG80Jeu8UPuzvaMH/Rb0IwPZV1xyapMoSv71FQPz 0yhAMLamNBEIRf2mvBZHp9ASLBRDYZsDtU+EAFelS56hBkuYywSb//m+cTeQpqT3 Wz5713DRtrpi6X7KrGCFmtpzhCiYyS11YLBGWIe6PjBFNdF+dveNGh5TPy4bKmVy j4cx+a3mHEdXOinayUzfPI1wpxF1eI/6cIiP0G5wy0VAApbk5qgE4PVlqZa8zKFF 4sePD6vsYTQ+3PVMwkfeJiX8E1ZMKAo/boD8Hw3jU24kY5n4bC+Qcqvm/JCFArGr QfA0K+TZ7R86lfs7yyjhmnf3fSBZVTOG4anYOAeOqgghORL43JhGjKZcDRw2CjM= =Y8H3 -END PGP SIGNATURE- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Upcoming release schedule - 8.4 ?
Realized my earlier related post was a bit misplaced in questions@. So I just refer to it here by link, ok then that is all. http://docs.freebsd.org/cgi/getmsg.cgi?fetch=968504+0+current/freebsd-questions ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Upcoming release schedule - 8.4 ?
On Wed, Jun 13, 2012 at 08:50:24AM -0700, Garrett Cooper wrote: The only way that this would really work is if there were dedicated sustaining engineers working on actively backporting code, testing it, committing it, etc. I'm going to agree with Garrett here. IMHO we've reached (or surpassed) the limit of what is reasonable to ask volunteers to commit their spare time to. This is doubly true when we have more than one stable branch. mcl ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Upcoming release schedule - 8.4 ?
On 13 June 2012 21:26, Mark Linimon lini...@lonesome.com wrote: On Wed, Jun 13, 2012 at 08:50:24AM -0700, Garrett Cooper wrote: The only way that this would really work is if there were dedicated sustaining engineers working on actively backporting code, testing it, committing it, etc. I'm going to agree with Garrett here. IMHO we've reached (or surpassed) the limit of what is reasonable to ask volunteers to commit their spare time to. This is doubly true when we have more than one stable branch. I totally concur. Adrian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: BIO_DELETE equivalent for file on FFS filesystem
file to take 900MB or... can i call some system function to punch holes? I think you can only truncate the file at this time, pretty much like brk() works for memory. BAD. suppose i keep windoze VM image on filesystem which takes 10GB but uses 5GB. i could write simple program to find out what blocks are unused and then...do nothing. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Upcoming release schedule - 8.4 ?
Friends, I am looking at the upcoming release schedule, and I only see 9.1 listed - can anyone confirm or deny 8.4 ? does it matter. cvsup RELENG_8 and you see updates are done constantly. just sometime somebody decide to change number :) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On Wednesday 13 June 2012 23:35:17 Mel Flynn wrote: On 13-6-2012 23:16, claudiu vasadi wrote: If you simplky do sysctl -d hw.usb.no_boot_wait you will see the explanation ;) Probably why Eitan asked as that description: a) means nothing to people unfamiliar with device enumerations b) does not point to a manual page that explains how USB does device enumerations and why it would account for a significant chunk of the boot process. It's not dangerous. It only means that you can't boot off a USB disk for example, because the USB stack will then not wait for the BOOT device at mount-root time, if you set this options. --HPS ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On 13 June 2012 22:13, Hans Petter Selasky hsela...@c2i.net wrote: It's not dangerous. It only means that you can't boot off a USB disk for example, because the USB stack will then not wait for the BOOT device at mount-root time, if you set this options. This is what I expected, but it should be documented somewhere. Thanks! -- Eitan Adler ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Upcoming release schedule - 8.4 ?
On Wed, Jun 13, 2012 at 8:30 PM, Adrian Chadd adr...@freebsd.org wrote: On 13 June 2012 21:26, Mark Linimon lini...@lonesome.com wrote: On Wed, Jun 13, 2012 at 08:50:24AM -0700, Garrett Cooper wrote: The only way that this would really work is if there were dedicated sustaining engineers working on actively backporting code, testing it, committing it, etc. I'm going to agree with Garrett here. IMHO we've reached (or surpassed) the limit of what is reasonable to ask volunteers to commit their spare time to. This is doubly true when we have more than one stable branch. I totally concur. Ah, but you can get the same effect by freeing up those engineers to work on the hard stuff. This is my usual soapbox (see [1], [2]): Push more of the mundane work out to the edges, so that the developers can focus more on the core (like more releases/features/testing/projects). Here are some ideas. Only developers can implement them, but they would start paying for themselves immediately ... in developer time. - Frequent snapshots, with tools to automatically apply them and roll them back (freebsd-update + ZFS snapshots?). - Tools to do binary walks of snapshots to pinpoint when a bug appeared. (Think 'git bisect' + freebsd-update.) - A taggable FAQ that supports faceted search, and a quick way to add entries (or propose them for approval). - A way to search for known fixes to transient bugs and hardware issues [1]. - General debugging and testing tools for non-developers, including tools for filing smarter bug reports. - A way to automatically upload crash dumps for bulk analysis (like Windows does). - A dmesg analyzer that downloads a list during install, and looks for known issues (or workarounds) with your hardware for that version of FreeBSD (or recommend a different version!). Tools like these would also help more people achieve the I tried it, and it Just Worked moment. This can keep people's interest long enough to give FreeBSD a serious try. Some of them might enter the volunteer pool. I'm not a developer, but if some of the above could be tackled, they might free up enough Developer Equivalents (DEs, a term which I have just made up) to be more than worth the effort. Royce [1]. http://lists.freebsd.org/pipermail/freebsd-doc/2011-September/018865.html [2]. http://lists.freebsd.org/pipermail/freebsd-hackers/2012-January/037310.html ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD Boot Times
On Thu, Jun 14, 2012 at 07:13:23AM +0200, Hans Petter Selasky wrote: On Wednesday 13 June 2012 23:35:17 Mel Flynn wrote: On 13-6-2012 23:16, claudiu vasadi wrote: If you simplky do sysctl -d hw.usb.no_boot_wait you will see the explanation ;) Probably why Eitan asked as that description: a) means nothing to people unfamiliar with device enumerations b) does not point to a manual page that explains how USB does device enumerations and why it would account for a significant chunk of the boot process. It's not dangerous. It only means that you can't boot off a USB disk for example, because the USB stack will then not wait for the BOOT device at mount-root time, if you set this options. Can we disable that by default and add a boot menu entry like Boot from USB Disk to enable it? pgpPxWPragKoo.pgp Description: PGP signature
