Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On Saturday, 25 August 2012 at 01:33, Glen Barber wrote: > On Sat, Aug 25, 2012 at 01:25:15AM +0100, Jonathan Anderson wrote: > > On 24 Aug 2012, at 23:38, Doug Barton > (mailto:do...@freebsd.org)> wrote: > > > Let me rephrase that more simply ... very few users are ever going to > > > need the bootstrapping tool that will be in the base. > > > > > So, then they won't use it. I fail to see the problem here. I also fail to see the problem. :) Just to be clear, my post was arguing against Doug's assertion that few will use pkg's bootstrapper (and that this is a problem): I hope that pkgng and package sets will vastly increase the use of binary packages by FreeBSD consumers. > /usr/sbin/pkg installs /usr/local/sbin/pkg without requiring the Ports > Collection to be available locally. Which is exactly the behaviour that I want: I view the ports tree as a last resort to be used only if binary packages fail to fulfil my needs. Sometimes I don't even bother fetching it. Once again, we may be in violent agreement here. :) Jon -- Jonathan Anderson jonat...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 8/24/2012 5:33 PM, Glen Barber wrote: > On Sat, Aug 25, 2012 at 01:25:15AM +0100, Jonathan Anderson wrote: >> On 24 Aug 2012, at 23:38, Doug Barton wrote: >>> Let me rephrase that more simply ... very few users are ever going to >>> need the bootstrapping tool that will be in the base. >> > > So, then they won't use it. I fail to see the problem here. That's because you're not paying attention. :) Which comes first in your PATH, /usr/sbin, or /usr/local/sbin? Which comes first in the default PATH? What Baptiste said is that the way /usr/sbin/pkg works is to take arguments handed to it and pass them through to /usr/local/sbin/pkg. That means that every user who has their PATH configured in the default manner (which is what every security text on Unix has recommended for 30 years) will be using /usr/sbin/pkg every time they type the pkg command. >> But surely the whole point of pkgng is that people *will* use pkg >> as the default method of acquiring third-party software, so they'll >> want to "pkg install foo" and have it Just Work. To say either "you >> must download the ports tree in order to use binary packages" or >> "you must use pkg_add to install pkg" seems to miss the point... >> > > /usr/sbin/pkg installs /usr/local/sbin/pkg without requiring the Ports > Collection to be available locally. It does much more than that. Go read the code. As to the security related problems, they should be obvious. Having 1 binary that is always executed to pass arguments to another binary at minimum doubles your attack surface. Given what /usr/sbin/pkg does, it more than doubles it. Not to mention the flat out wrong-headed design of having a binary that will be run as root whose primary purpose is to pass arguments to another binary. The reason this defeats the purpose of putting pkg in the ports tree is that if there is a bug in /usr/sbin/pkg (which of course, there will be) then it has to be fixed in the base, with all of the consequent drama and delays that this will entail. If there is a bug in /usr/local/bin/pkg, it gets fixed in the ports tree and instantly updated, which is part of the virtue of having it in the ports tree in the first place. Given that if we do the rollout properly the bootstrap function will be limited to a very small percentage of users, it makes sense to split that function out into a separate, limited binary; and not pollute the pkg stream with extra cruft it does not need. > What I would like to know, is why all the anti-progress emails[1] have > to wait until the Last Minute(tm) when information on pkgng availability > has been available for quite some time now. First off, I resent being told that because I'm raising legitimate issues with something that I am being "obstructionist," or "anti-progress." And my concerns are certainly not "last minute." I've been raising concerns about pkg since day 1, and given that there is still no coherent, comprehensive project plan about the migration it's not at all surprising that others are also starting to discover daemons in the details. It's also part and parcel of life in an open source project. Most people don't pay attention about most things until they feel that it will be affecting them. This is doubly true in open source. Given how well-known this issue is, it should be planned for in any kind of big project such as this. It's probably also worth mentioning that there are only so many hours in the day, so one has to prioritize. Doug - -- I am only one, but I am one. I cannot do everything, but I can do something. And I will not let what I cannot do interfere with what I can do. -- Edward Everett Hale, (1822 - 1909) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (MingW32) iQEcBAEBCAAGBQJQOCwUAAoJEFzGhvEaGryEcpgH/2CAPBCldr7HlTjIzErqtbTO S0ZaI0RabwEk85+HuFCmBLTbdKqVjGYcLqIbz7l6wOa20N1rPARtBDy5DkrMrL6s 5YAgWiZ43FyKQ4826VDVBvhPqxXMD0O+sETs2kskFUkV73u/r1/8EpfZgwCDk9F9 G8hqMVTRyoWgoh1HIaBba5/m4D7+UGPYE2w8M3QAGSULePYJLgaRdu+jd2aNBrJD NFjY4lyLbitbIH1/fYHDR90KqlBVP6vr+bWUvdoHFJQ0W0HQw7wMtamo418SlORI qfTaoHL4sA1sggHrlUVvxjgWbAtIcYT2F3K+u34yTaWAoqxoN9pzRy3GWXyFRzM= =PNr3 -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On 24 Aug 2012, at 23:38, Doug Barton wrote: > Let me rephrase that more simply ... very few users are ever going to > need the bootstrapping tool that will be in the base. But surely the whole point of pkgng is that people *will* use pkg as the default method of acquiring third-party software, so they'll want to "pkg install foo" and have it Just Work. To say either "you must download the ports tree in order to use binary packages" or "you must use pkg_add to install pkg" seems to miss the point... Jon -- Jonathan Anderson jonat...@freebsd.org http://freebsd.org/~jonathan/___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On Sat, Aug 25, 2012 at 01:47:26AM +0100, Jonathan Anderson wrote: > On Saturday, 25 August 2012 at 01:33, Glen Barber wrote: > > On Sat, Aug 25, 2012 at 01:25:15AM +0100, Jonathan Anderson wrote: > > > On 24 Aug 2012, at 23:38, Doug Barton > > (mailto:do...@freebsd.org)> wrote: > > > > Let me rephrase that more simply ... very few users are ever going to > > > > need the bootstrapping tool that will be in the base. > > > > > > > > > So, then they won't use it. I fail to see the problem here. > > I also fail to see the problem. :) Just to be clear, my post was > arguing against Doug's assertion that few will use pkg's bootstrapper > (and that this is a problem): I hope that pkgng and package sets > will vastly increase the use of binary packages by FreeBSD consumers. > I was avoiding writing two separate emails about this thread - yes, I did realize we had a similar point of view on this. Sorry if it came out differently. > > /usr/sbin/pkg installs /usr/local/sbin/pkg without requiring the Ports > > Collection to be available locally. > > Which is exactly the behaviour that I want: I view the ports tree > as a last resort to be used only if binary packages fail to fulfil > my needs. Sometimes I don't even bother fetching it. Once again, > we may be in violent agreement here. :) > Understood. I misinterpreted your last sentence. What I would like to know, is why all the anti-progress emails[1] have to wait until the Last Minute(tm) when information on pkgng availability has been available for quite some time now. Welcome to 2012. [1] - Doing the same things we've done keeps us in the same place we've been. Glen pgpn7shOVODrE.pgp Description: PGP signature
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On Sat, Aug 25, 2012 at 01:25:15AM +0100, Jonathan Anderson wrote: > On 24 Aug 2012, at 23:38, Doug Barton wrote: > > Let me rephrase that more simply ... very few users are ever going to > > need the bootstrapping tool that will be in the base. > So, then they won't use it. I fail to see the problem here. > But surely the whole point of pkgng is that people *will* use pkg > as the default method of acquiring third-party software, so they'll > want to "pkg install foo" and have it Just Work. To say either "you > must download the ports tree in order to use binary packages" or > "you must use pkg_add to install pkg" seems to miss the point... > /usr/sbin/pkg installs /usr/local/sbin/pkg without requiring the Ports Collection to be available locally. Glen pgpkjHK846k4Q.pgp Description: PGP signature
Re: Time to bump default VM_SWZONE_SIZE_MAX?
On 08/24/12 07:13, John Baldwin wrote: > On Friday, August 24, 2012 8:45:43 am Dag-Erling Smørgrav wrote: >> John Baldwin writes: >>> Note that on i386 you can't get more than 4GB of RAM without PAE, and if you >>> have any modern x86 box with > 4GB of RAM, you are most likely running amd64 >>> on it, not i386. I think i386 would be fine to just keep the limit it had. >> >> The limit we had was insufficient for 8 GB of swap. > > In absolute or practical terms? Not all swap blocks are fully utilized. At > Y! the install script we used would compute the maximum theoretical swap zone > needed and then cut it in half, and this worked quite well. Also, keep in > mind, > this is for i386, not amd64. At this point i386 is going to be used on > smaller > systems (e.g. netbooks, etc.), not servers that have lots of swap. I'd like to see i386 bumped slightly, just so that the rule of "allocate swap space equal to max(RAM, min(2*RAM, 8 GB))" (which I've seen in lots of places) is more likely to be safe. If I'm understanding things correctly, bumping from 32 MB up to 34.5 MB should give us a theoretical 16 GiB or a "safe" 8 GiB limit on swap usage (2^17 structures which are 276 bytes each on i386). But I agree that the real issue was with amd64, not i386. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On Fri, Aug 24, 2012 at 03:38:33PM -0700, Doug Barton wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> On 8/24/2012 1:15 AM, Baptiste Daroussin wrote:
> > BTW for people who haven't tested and want to share their opinion,
> > here is how work /usr/sbin/pkg:
> >
> > it first checks if ${LOCALBASE}/sbin/pkg is there - if yes it
> > directly execute ${LOCALBASE}/sbin/pkg with arguments passed to
> > /usr/sbin/pkg
>
> As others have already pointed out, this is a bad idea for a variety
> of reasons, not the least of which is security related. It also
> removes one of the primary benefits of pkg, that it be (fully) hosted
> in the ports tree.
Can anyone give me he details on the security related problem?
Can I also have the details on why it would remove the benefits of being fully
hosted in the ports, I have no plan to remove it, currently the ports tree is
also able to bootstrap itself pkg without needing /usr/sbin/pkg.
the bootstrap tool is currently just a transparent way to bootstrap pkgng, it is
not mandatory at all, one can leave without it, and it doesn't prevent pkgng to
fully leave in the ports tree?
What do I miss here?
Once again I'm not opposed at all to remove it in favour of pkg-bootstrap, but
it currently seems to lacks a bit of detailed arguments.
> Let me rephrase that more simply ... very few users are ever going to
> need the bootstrapping tool that will be in the base. Making it
> mandatory for *every* user is therefore not only a bad idea, it's
> contrary to one of the primary goals of the project.
>
Why would it be mandatory? it is just a transparent kind of helper tool
regards,
Bapt
pgpUNRkiuW2p0.pgp
Description: PGP signature
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On 8/24/2012 1:15 AM, Baptiste Daroussin wrote:
> BTW for people who haven't tested and want to share their opinion,
> here is how work /usr/sbin/pkg:
>
> it first checks if ${LOCALBASE}/sbin/pkg is there - if yes it
> directly execute ${LOCALBASE}/sbin/pkg with arguments passed to
> /usr/sbin/pkg
As others have already pointed out, this is a bad idea for a variety
of reasons, not the least of which is security related. It also
removes one of the primary benefits of pkg, that it be (fully) hosted
in the ports tree.
The bootstrap procedure does not need to be simple or transparent
because it's only going to exist for a very short period while users
are bringing pkg into already-installed systems where pkg is not
already the default; and they don't have an existing ports tree. The
way that you solve the bootstrap problem for systems where pkg IS the
default is to install the pkg package at system install time.
Let me rephrase that more simply ... very few users are ever going to
need the bootstrapping tool that will be in the base. Making it
mandatory for *every* user is therefore not only a bad idea, it's
contrary to one of the primary goals of the project.
Doug
- --
I am only one, but I am one. I cannot do everything, but I can do
something. And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (MingW32)
iQEcBAEBCAAGBQJQOAJpAAoJEFzGhvEaGryEY2QH/2Hv+cW20htODBTrFNScd7qS
NxBc7YHr3ddoyoui+Qwtl3ErjMOJr+kA3nRSsZ5ewGN5HVQ4gCAWp76bJn75BM71
q2Qmgx7HCnBMJKrmRTvDAA1nJcTKAgXFKK8hYQgiTOhWFaIjJxhDlln5llFcXwBa
VN0ErF421FkD8oyHpcQLga1BHLtldOL5itt/4Tp9cKaC5l2P9dBNbyCTxVa4XBiZ
tsK+x7XJqwj0NvXLk+b3icLIEeyopa3TJAoAtXZ27igg65norCoh2EPo7aJqP2zY
T75PKdbRJprwCpeJXulC02oZu/UERoIMLeaH1JbCZRcLAqaCJQuEGSP95as3hSY=
=BEya
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On 8/23/2012 8:03 PM, Eitan Adler wrote: > On 23 August 2012 22:59, Doug Barton wrote: >> I tend to agree with Steve here ... we can't be responsible for other >> people's poorly written docs. > > This isn't about poorly written docs. This is the user expecting a > tool to exist, which doesn't. Take another example of a sysadmin which > rarely installs new systems, installs FreeBSD for the third time, and > then gets confused when "pkg install vim" fails. Aren't we going to install the pkg package on new systems when they are installed? Isn't that (shouldn't that be?) part of the project plan? It would be insane for us not to do that, at least for the releases where pkg is the default. >> You bring up a valid point that we should >> keep in mind for our own however. The bootstrapping issue will be the >> smallest possible annoyance on a long road of the migration process. > > The bootstrapping issue is a factor even after the migration :) I think that the point I'm trying to make is that it shouldn't be. > note that I'm not talking about the mechanism here, I'm trying to > avoid "pkg doesn't seem to be installed on my fresh system" becoming a > FAQ. The way that we avoid that problem is not to create it for ourselves in the first place. :) The role of pkg-bootstrap is for those users who have already-installed systems that need to do the conversion, or if somehow pkg becomes corrupted on the user's system and needs to be reinstalled. That's it. I like that you're thinking through the related issues, but in this particular case I think you're overthinking it. Doug -- I am only one, but I am one. I cannot do everything, but I can do something. And I will not let what I cannot do interfere with what I can do. -- Edward Everett Hale, (1822 - 1909) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: less aggressive contigmalloc ?
On 08/24/2012 11:54, Luigi Rizzo wrote: On Fri, Aug 24, 2012 at 11:12:51AM -0500, Alan Cox wrote: On 08/24/2012 09:57, Luigi Rizzo wrote: On Fri, Aug 24, 2012 at 12:43:33AM -0500, Alan Cox wrote: On 08/23/2012 12:45, Luigi Rizzo wrote: On Thu, Aug 23, 2012 at 12:08:40PM -0500, Alan Cox wrote: ... yes i do see that. Maybe less aggressive with M_NOWAIT but still kills processes. Are you compiling world with MALLOC_PRODUCTION? The latest version of whatever the default is. But: jemalloc uses significantly more memory when debugging options are enabled. This first came up in a thread titled "10-CURRENT and swap usage" back in June. Even at its most aggressive, M_WAITOK, contigmalloc() does not directly kill processes. If process death coincides with the use of contigmalloc(), then it is simply the result of earlier, successful contigmalloc() calls, or for that matter any other physical memory allocation calls, having depleted the pool of free pages to the point that the page daemon runs and invokes vm_pageout_oom(). does it mean that those previous allocations relied on memory overbooking ? Yes. Is there a way to avoid that, then ? I believe that malloc()'s default minimum allocation size is 4MB. You could reduce that. Alternatively, you can enable MALLOC_PRODUCTION. i tried this, and as others mentioned it makes life better and reduces the problem but contigmalloc still triggers random process kills. I would be curious to see a stack backtrace when vm_pageout_oom() is called. you mean a backtrace of the process(es) that get killed ? No, a backtrace showing who called vm_pageout_oom(). Simply add a kdb_backtrace() call at the start of vm_pageout_oom(). There are two possibilities. I want to know which it is. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: less aggressive contigmalloc ?
On Fri, Aug 24, 2012 at 11:12:51AM -0500, Alan Cox wrote: > On 08/24/2012 09:57, Luigi Rizzo wrote: > >On Fri, Aug 24, 2012 at 12:43:33AM -0500, Alan Cox wrote: > >>On 08/23/2012 12:45, Luigi Rizzo wrote: > >>>On Thu, Aug 23, 2012 at 12:08:40PM -0500, Alan Cox wrote: > >>>... > >yes i do see that. > > > >Maybe less aggressive with M_NOWAIT but still kills processes. > Are you compiling world with MALLOC_PRODUCTION? The latest version of > >>>whatever the default is. But: > >>> > jemalloc uses significantly more memory when debugging options are > enabled. This first came up in a thread titled "10-CURRENT and swap > usage" back in June. > > Even at its most aggressive, M_WAITOK, contigmalloc() does not directly > kill processes. If process death coincides with the use of > contigmalloc(), then it is simply the result of earlier, successful > contigmalloc() calls, or for that matter any other physical memory > allocation calls, having depleted the pool of free pages to the point > that the page daemon runs and invokes vm_pageout_oom(). > >>>does it mean that those previous allocations relied on memory > >>>overbooking ? > >>Yes. > >> > >>>Is there a way to avoid that, then ? > >>I believe that malloc()'s default minimum allocation size is 4MB. You > >>could reduce that. > >> > >>Alternatively, you can enable MALLOC_PRODUCTION. > >i tried this, and as others mentioned it makes life > >better and reduces the problem but contigmalloc still triggers > >random process kills. > > I would be curious to see a stack backtrace when vm_pageout_oom() is called. you mean a backtrace of the process(es) that get killed ? I'll see if i manage to generate something, not easy because this is a picobsd image and there is barely anything available. I am trying to see if i can remove the memory overcommit, but limited success so far. I have built a kernel with "options NO_SWAPPING" but while it makes vm.swap_enabled: 0 i still see vm.swap_reserved grow beyond available RAM size. As an example, on a VM with 128MB of RAM, of which about 28M used for a MFS root filesystem, i get the following. vm.vmtotal: System wide totals computed every five seconds: (values in kilobytes) === Processes: (RUNQ: 1 Disk Wait: 1 Page Wait: 0 Sleep: 9) Virtual Memory: (Total: 1074551344K Active: 738328K) Real Memory:(Total: 17208K Active: 13368K) Shared Virtual Memory: (Total: 7788K Active: 7788K) Shared Real Memory: (Total: 5024K Active: 5024K) Free Memory:68160K # sysctl vm | grep swap vm.swap_enabled: 0 vm.nswapdev: 0 vm.swap_async_max: 4 vm.swap_reserved: 149917696 <--- varies from ~140795904 to 160e6 and up vm.swap_total: 0 vm.stats.vm.v_swappgsout: 0 vm.stats.vm.v_swappgsin: 0 vm.stats.vm.v_swapout: 0 vm.stats.vm.v_swapin: 0 vm.disable_swapspace_pageouts: 0 vm.defer_swapspace_pageouts: 0 vm.swap_idle_enabled: 0 vm.overcommit: 0 -- cheers luigi ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: less aggressive contigmalloc ?
On 08/24/2012 09:57, Luigi Rizzo wrote: On Fri, Aug 24, 2012 at 12:43:33AM -0500, Alan Cox wrote: On 08/23/2012 12:45, Luigi Rizzo wrote: On Thu, Aug 23, 2012 at 12:08:40PM -0500, Alan Cox wrote: ... yes i do see that. Maybe less aggressive with M_NOWAIT but still kills processes. Are you compiling world with MALLOC_PRODUCTION? The latest version of whatever the default is. But: jemalloc uses significantly more memory when debugging options are enabled. This first came up in a thread titled "10-CURRENT and swap usage" back in June. Even at its most aggressive, M_WAITOK, contigmalloc() does not directly kill processes. If process death coincides with the use of contigmalloc(), then it is simply the result of earlier, successful contigmalloc() calls, or for that matter any other physical memory allocation calls, having depleted the pool of free pages to the point that the page daemon runs and invokes vm_pageout_oom(). does it mean that those previous allocations relied on memory overbooking ? Yes. Is there a way to avoid that, then ? I believe that malloc()'s default minimum allocation size is 4MB. You could reduce that. Alternatively, you can enable MALLOC_PRODUCTION. i tried this, and as others mentioned it makes life better and reduces the problem but contigmalloc still triggers random process kills. I would be curious to see a stack backtrace when vm_pageout_oom() is called. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On 8/24/2012 11:02 AM, Baptiste Daroussin wrote: > On Fri, Aug 24, 2012 at 10:50:30AM -0500, Bryan Drewery wrote: >> On 8/24/2012 9:31 AM, Baptiste Daroussin wrote: >>> On Fri, Aug 24, 2012 at 02:23:47PM +0200, Alex Dupre wrote: Baptiste Daroussin ha scritto: > On of the thing I forgot and kan@ has added is a prompt for the user in > case it > is going to bootstrap. So, removing the prompt will make everybody happy? :-) What about a prompt with timeout? "This is the first time pkg is run, I'll start bootstrapping in 10 seconds. Press CTRL-C now to abort" >>> >>> Imho the best is to prompt only if not on a tty, otherwise fail, except if a >>> environment variable is set. >>> >>> regards, >>> Bapt >>> >> >> Maybe an ENV variable to auto confirm the prompt too? I'm thinking about >> scripts using pkg to bootstrap an entire system install. Maybe I'm on a >> TTY and just want it to confirm auto anyway. > > This is already the case (thanks kan@) >> Perfect. I see it now, ALWAYS_ASSUME_YES. Bryan ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On Fri, Aug 24, 2012 at 10:50:30AM -0500, Bryan Drewery wrote: > On 8/24/2012 9:31 AM, Baptiste Daroussin wrote: > > On Fri, Aug 24, 2012 at 02:23:47PM +0200, Alex Dupre wrote: > >> Baptiste Daroussin ha scritto: > >> > >>> On of the thing I forgot and kan@ has added is a prompt for the user in > >>> case it > >>> is going to bootstrap. > >> > >> So, removing the prompt will make everybody happy? :-) > >> > >> What about a prompt with timeout? "This is the first time pkg is run, > >> I'll start bootstrapping in 10 seconds. Press CTRL-C now to abort" > >> > > > > Imho the best is to prompt only if not on a tty, otherwise fail, except if a > > environment variable is set. > > > > regards, > > Bapt > > > > Maybe an ENV variable to auto confirm the prompt too? I'm thinking about > scripts using pkg to bootstrap an entire system install. Maybe I'm on a > TTY and just want it to confirm auto anyway. This is already the case (thanks kan@) > > Bryan pgpe1Gfob2VIC.pgp Description: PGP signature
Re: [HEADSUP] geli(4) weak master key generation on -CURRENT
On Tue, Aug 21, 2012 at 1:05 PM, Ulrich Spörlein wrote: > On Mon, 2012-08-20 at 22:24:56 +0100, Simon L. B. Nielsen wrote: >> Hello, >> >> If you are not using geli(4) on -CURRENT (AKA FreeBSD 10) you can safely >> ignore this mail. If you are, please read on! >> >> -CURRENT users of geli(4) should be advised that, a geli(4) device may >> have weak master key, if the provider is created on -CURRENT system >> built against source code between r238116 (Jul 4 17:54:17 2012 UTC) >> and r239184 (non-inclusive, Aug 10 18:43:29 2012 UTC). >> >> One can verify if its provider was created with weak keys by running: >> >> # geli dump | grep version >> >> If the version is 7 and the system did not include this fix (r239184) >> when provider was initialized, then the data has to be backed up, >> underlying provider overwritten with random data, system upgraded and >> provider recreated. >> >> Thanks to Fabian Keil for reporting the issue, Pawel Jakub Dawidek for >> fixing it, and Xin Li for drafting this text. >> >> PS. This only affects FreeBSD 10 / -CURRENT, and as -CURRENT isn't >> supported by the FreeBSD Security Team, we are not releasing an >> advisory, just this heads up. > > I haven't read commit mails in a very long time, but is there code in > place that will issue a warning upon geli attach if version 7 is > detected? While -CURRENT is not supported, there might be a lot of disks > initialized with version 7 and they'll eventually be upgraded to > 10.0-RELEASE (the OS, not necessarily the geli volumes). No, the bad code was only in head for about a month. I'm fine with having a warning, but somebody has to code it. -- Simon L. B. Nielsen ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On 8/24/2012 9:31 AM, Baptiste Daroussin wrote: > On Fri, Aug 24, 2012 at 02:23:47PM +0200, Alex Dupre wrote: >> Baptiste Daroussin ha scritto: >> >>> On of the thing I forgot and kan@ has added is a prompt for the user in >>> case it >>> is going to bootstrap. >> >> So, removing the prompt will make everybody happy? :-) >> >> What about a prompt with timeout? "This is the first time pkg is run, >> I'll start bootstrapping in 10 seconds. Press CTRL-C now to abort" >> > > Imho the best is to prompt only if not on a tty, otherwise fail, except if a > environment variable is set. > > regards, > Bapt > Maybe an ENV variable to auto confirm the prompt too? I'm thinking about scripts using pkg to bootstrap an entire system install. Maybe I'm on a TTY and just want it to confirm auto anyway. Bryan ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: less aggressive contigmalloc ?
On Fri, Aug 24, 2012 at 12:43:33AM -0500, Alan Cox wrote: > On 08/23/2012 12:45, Luigi Rizzo wrote: > >On Thu, Aug 23, 2012 at 12:08:40PM -0500, Alan Cox wrote: > >... > >>>yes i do see that. > >>> > >>>Maybe less aggressive with M_NOWAIT but still kills processes. > >>Are you compiling world with MALLOC_PRODUCTION? The latest version of > >whatever the default is. But: > > > >>jemalloc uses significantly more memory when debugging options are > >>enabled. This first came up in a thread titled "10-CURRENT and swap > >>usage" back in June. > >> > >>Even at its most aggressive, M_WAITOK, contigmalloc() does not directly > >>kill processes. If process death coincides with the use of > >>contigmalloc(), then it is simply the result of earlier, successful > >>contigmalloc() calls, or for that matter any other physical memory > >>allocation calls, having depleted the pool of free pages to the point > >>that the page daemon runs and invokes vm_pageout_oom(). > >does it mean that those previous allocations relied on memory overbooking ? > > Yes. > > >Is there a way to avoid that, then ? > > I believe that malloc()'s default minimum allocation size is 4MB. You > could reduce that. > > Alternatively, you can enable MALLOC_PRODUCTION. i tried this, and as others mentioned it makes life better and reduces the problem but contigmalloc still triggers random process kills. Thanks for the hints. cheers luigi ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On Fri, Aug 24, 2012 at 02:23:47PM +0200, Alex Dupre wrote: > Baptiste Daroussin ha scritto: > > > On of the thing I forgot and kan@ has added is a prompt for the user in > > case it > > is going to bootstrap. > > So, removing the prompt will make everybody happy? :-) > > What about a prompt with timeout? "This is the first time pkg is run, > I'll start bootstrapping in 10 seconds. Press CTRL-C now to abort" > Imho the best is to prompt only if not on a tty, otherwise fail, except if a environment variable is set. regards, Bapt pgpxwGXk6wbM7.pgp Description: PGP signature
Re: Time to bump default VM_SWZONE_SIZE_MAX?
On Friday, August 24, 2012 8:45:43 am Dag-Erling Smørgrav wrote: > John Baldwin writes: > > Note that on i386 you can't get more than 4GB of RAM without PAE, and if you > > have any modern x86 box with > 4GB of RAM, you are most likely running amd64 > > on it, not i386. I think i386 would be fine to just keep the limit it had. > > The limit we had was insufficient for 8 GB of swap. In absolute or practical terms? Not all swap blocks are fully utilized. At Y! the install script we used would compute the maximum theoretical swap zone needed and then cut it in half, and this worked quite well. Also, keep in mind, this is for i386, not amd64. At this point i386 is going to be used on smaller systems (e.g. netbooks, etc.), not servers that have lots of swap. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: build error at rpc code
On 2012-08-24 15:07, Dima Panov wrote: 24.08.2012 19:12, Dimitry Andric пишет: ... You are most likely setting CPP as follows: CPP=clang -E Don't do that, use the following instead: CPP=clang-cpp If there is no clang-cpp symlink in /usr/bin yet, just make it manually for now. It will also be created by installworld. Oh, indeed. Thanks for advice. However, why such different results in preprocessor invocation style? When you invoke either clang or gcc with -E, files with unknown extensions are not preprocessed, and considered to be linker input files. When you invoke clang-cpp or cpp, files with unknown extensions are considered to be C instead. Since our RPC-related files use the .x extension, -E doesn't work as expected, and the resulting files will give a compile error later on. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: build error at rpc code
24.08.2012 19:12, Dimitry Andric пишет: On 2012-08-24 05:55, Dima Panov wrote: Does anybody get same error at buildworld with recent -current? ... In file included from /usr/src/lib/libc/../../include/rpc/rpc.h:76: /usr/src/lib/libc/../../include/rpc/rpcb_clnt.h:69:8: error: unknown type name 'rpcblist' extern rpcblist *rpcb_getmaps(const struct netconfig *, const char *); You are most likely setting CPP as follows: CPP=clang -E Don't do that, use the following instead: CPP=clang-cpp If there is no clang-cpp symlink in /usr/bin yet, just make it manually for now. It will also be created by installworld. Oh, indeed. Thanks for advice. However, why such different results in preprocessor invocation style? -- Dima Panov (flu...@freebsd.org) (KDE, Office)@FreeBSD team Facebook: http://www.facebook.com/fluffy.khv IRC: fluffy@EFNet, fluffykhv@FreeNode twitter: fluffy_khv | skype: dima.panov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Time to bump default VM_SWZONE_SIZE_MAX?
John Baldwin writes: > Note that on i386 you can't get more than 4GB of RAM without PAE, and if you > have any modern x86 box with > 4GB of RAM, you are most likely running amd64 > on it, not i386. I think i386 would be fine to just keep the limit it had. The limit we had was insufficient for 8 GB of swap. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Time to bump default VM_SWZONE_SIZE_MAX?
On Friday, August 24, 2012 5:44:48 am Dag-Erling Smørgrav wrote: > John Baldwin writes: > > Hmm, this is not true on i386 where the problem is not just the physical > > RAM required, but also address space. (The swap zone is all mapped into > > KVA > > even if it isn't used.) This is why Alan's e-mail specifically > > mentioned amd64, ia64, etc. but not i386 in his list. I think i386 still > > needs this limit, and I think your commit jumped the gun a bit. > > How about we reinstate the limit on i386, but increase it to 64 MB? > That would increase the theoretical maximum to ~15 GB. People with 8 GB > swap would get a warning, but would be unlikely to run into trouble. > > (or we could increase the limit to 72351744 bytes, which is the precise > amount required to support 16 GB) Note that on i386 you can't get more than 4GB of RAM without PAE, and if you have any modern x86 box with > 4GB of RAM, you are most likely running amd64 on it, not i386. I think i386 would be fine to just keep the limit it had. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
Baptiste Daroussin ha scritto: > On of the thing I forgot and kan@ has added is a prompt for the user in case > it > is going to bootstrap. So, removing the prompt will make everybody happy? :-) What about a prompt with timeout? "This is the first time pkg is run, I'll start bootstrapping in 10 seconds. Press CTRL-C now to abort" -- Alex Dupre ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
RE: pkgng default release schedule (contd...)
On 24 Aug 2012 11:08, "Jeffrey Bouquet" wrote: > > A few more reasons (unless I have not seen some relevant documentation to the contrary) to not mandate pkgng as the default... Why don't you phrase this as "How can one ..." so you sound less negative? > Nowadays, one can save time by installing two ports which officially or unofficially conflict, and have /var/db/pkg entries for both, and even > local workarounds (for instance, moving the duplicate binary elsewhere before the second install) (Perchance removing the line in the Makefile). Currently you can still do this, at least until STAGEDIR. > A failed "make install (register)", one can check the /var/db/pkg/ directory(ies) to double check visually to what extent it did NOT fail. > Similarly for a failed "make deinstall (unregister)"... The error messages are perfectly clear. > pkgdb -F to fixup stale dependencies and resolve dependency information. Unnecessary with pkgng. > A proven method in the portmaster manpage to reinstall all ports. You want to talk to the portmaster author about that. Chris ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
RE: pkgng default release schedule (contd...)
A few more reasons (unless I have not seen some relevant documentation to the contrary) to not mandate pkgng as the default... Nowadays, one can save time by installing two ports which officially or unofficially conflict, and have /var/db/pkg entries for both, and even local workarounds (for instance, moving the duplicate binary elsewhere before the second install) (Perchance removing the line in the Makefile). ... A failed "make install (register)", one can check the /var/db/pkg/ directory(ies) to double check visually to what extent it did NOT fail. Similarly for a failed "make deinstall (unregister)"... ... pkgdb -F to fixup stale dependencies and resolve dependency information. A proven method in the portmaster manpage to reinstall all ports. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Time to bump default VM_SWZONE_SIZE_MAX?
John Baldwin writes: > Hmm, this is not true on i386 where the problem is not just the physical > RAM required, but also address space. (The swap zone is all mapped into KVA > even if it isn't used.) This is why Alan's e-mail specifically > mentioned amd64, ia64, etc. but not i386 in his list. I think i386 still > needs this limit, and I think your commit jumped the gun a bit. How about we reinstate the limit on i386, but increase it to 64 MB? That would increase the theoretical maximum to ~15 GB. People with 8 GB swap would get a warning, but would be unlikely to run into trouble. (or we could increase the limit to 72351744 bytes, which is the precise amount required to support 16 GB) DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On 08/24/2012 10:15, Baptiste Daroussin wrote:
> On Thu, Aug 23, 2012 at 06:19:57PM -0400, Steve Wills wrote:
>> Hi,
>>
>> It seems to me that renaming the pkg binary in /usr/sbin/pkg to
>> /usr/sbin/pkg-bootstrap would make sense. From a user standpoint, it is
>> confusing that running the command gets different results the second time it
>> is run vs. the first time. I can imagine a user saying "I ran pkg, but it
>> didn't do what they said it would. Now I run it again, and it does do what
>> it is supposed to." Also, it would enable setting up a pkg-bootstrap man
>> page separate from the pkg man page, without confusion about which one
>> you're looking at.
>>
>> So, opinions? There may still be time to fix it for 9.1 if we can decide
>> quickly.
>>
>> Thanks,
>> Steve
>>
>
> BTW for people who haven't tested and want to share their opinion, here is how
> work /usr/sbin/pkg:
>
> it first checks if ${LOCALBASE}/sbin/pkg is there
> - if yes it directly execute ${LOCALBASE}/sbin/pkg with arguments passed to
> /usr/sbin/pkg
> - if no then it will determine you ABI (or take the one in environnement
>variable), and fetch the last available pkgng version from
> http://pkgbeta...
>it will extract pkg-static and use it to install pkgng with itself.
>on installation is done: it executes ${LOCALBASE}/sbin/pkg with arguments
>passed to /usr/sbin/pkg.
>
> Lots of people having ask in the early days of pkgng for a transparent
> bootstrap
> I have done it that way.
>
> On of the thing I forgot and kan@ has added is a prompt for the user in case
> it
> is going to bootstrap.
>
> So that mean that for a normal user, on a fresh vanilla FreeBSD
> "pkg install vim-lite" will prompt the user asking if he wants to bootstrap
> pkgng, and once bootstraped proceed to the installation of vim-lite
>
> if pkgng is already there then it will just install vim-lite.
>
> It was just to clarify, so that anyone understand was this is about.
>
> I tend to like the bootstrap like it is now (I find it transparent, and
> straight
> forward) but as I said earlier I have no strong opinion on this, so it most
> people prefers a separate pkg-bootstrap tools then I'll do it :)
Having installed a few 9.1-BETA1 boxes recently, i have to say i
absolutely like this behavior, it's totally transparent and you can just
start installing packages like you could with pkg_add -r. I don't see
the need to introduce an additional step. I actually think the current
behavior is user friendly, and renaming it would make it more difficult.
If people think the current behavior is misleading we could still
clarify the confirmation message.
Florian
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On Thu, 23 Aug 2012 23:00:25 -0400 Eitan Adler wrote: > On 23 August 2012 22:55, Steve Wills wrote: > > > As far as I understand it, POLA is about changing existing things: > > okay, so forget POLA. My point is that a user following a how to or > even *our* documentation on how to install something unrelated, like > say, apache, will be very confused when the documentation tells them > to run "pkg install apache" and finds out that pkg doesn't exist. A > similar example is needing "rehash" in tcsh. > what about autorehash, it didn't work ? you commit that change into default tcsh config long time ago :) -- wbr, tiger ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On Thu, Aug 23, 2012 at 06:19:57PM -0400, Steve Wills wrote:
> Hi,
>
> It seems to me that renaming the pkg binary in /usr/sbin/pkg to
> /usr/sbin/pkg-bootstrap would make sense. From a user standpoint, it is
> confusing that running the command gets different results the second time it
> is run vs. the first time. I can imagine a user saying "I ran pkg, but it
> didn't do what they said it would. Now I run it again, and it does do what
> it is supposed to." Also, it would enable setting up a pkg-bootstrap man page
> separate from the pkg man page, without confusion about which one you're
> looking at.
>
> So, opinions? There may still be time to fix it for 9.1 if we can decide
> quickly.
>
> Thanks,
> Steve
>
BTW for people who haven't tested and want to share their opinion, here is how
work /usr/sbin/pkg:
it first checks if ${LOCALBASE}/sbin/pkg is there
- if yes it directly execute ${LOCALBASE}/sbin/pkg with arguments passed to
/usr/sbin/pkg
- if no then it will determine you ABI (or take the one in environnement
variable), and fetch the last available pkgng version from http://pkgbeta...
it will extract pkg-static and use it to install pkgng with itself.
on installation is done: it executes ${LOCALBASE}/sbin/pkg with arguments
passed to /usr/sbin/pkg.
Lots of people having ask in the early days of pkgng for a transparent bootstrap
I have done it that way.
On of the thing I forgot and kan@ has added is a prompt for the user in case it
is going to bootstrap.
So that mean that for a normal user, on a fresh vanilla FreeBSD
"pkg install vim-lite" will prompt the user asking if he wants to bootstrap
pkgng, and once bootstraped proceed to the installation of vim-lite
if pkgng is already there then it will just install vim-lite.
It was just to clarify, so that anyone understand was this is about.
I tend to like the bootstrap like it is now (I find it transparent, and straight
forward) but as I said earlier I have no strong opinion on this, so it most
people prefers a separate pkg-bootstrap tools then I'll do it :)
regards,
Bapt
pgpdtHjkmpoJl.pgp
Description: PGP signature
Re: build error at rpc code
On 2012-08-24 05:55, Dima Panov wrote: Does anybody get same error at buildworld with recent -current? ... In file included from /usr/src/lib/libc/../../include/rpc/rpc.h:76: /usr/src/lib/libc/../../include/rpc/rpcb_clnt.h:69:8: error: unknown type name 'rpcblist' extern rpcblist *rpcb_getmaps(const struct netconfig *, const char *); You are most likely setting CPP as follows: CPP=clang -E Don't do that, use the following instead: CPP=clang-cpp If there is no clang-cpp symlink in /usr/bin yet, just make it manually for now. It will also be created by installworld. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap
On 8/24/2012 3:57 AM, Eitan Adler wrote: On 23 August 2012 18:19, Steve Wills wrote: Hi, It seems to me that renaming the pkg binary in /usr/sbin/pkg to /usr/sbin/pkg-bootstrap would make sense. From a user standpoint, it is confusing that running the command gets different results the second time it is run vs. the first time. I can imagine a user saying "I ran pkg, but it didn't do what they said it would. Now I run it again, and it does do what it is supposed to." Also, it would enable setting up a pkg-bootstrap man page separate from the pkg man page, without confusion about which one you're looking at. So, opinions? There may still be time to fix it for 9.1 if we can decide quickly. no opinion on the name, but imho there should be *something* called "pkg" on a fresh system. Users will install a new system, follow some random how-to, and not realize they missed a step. If the default package errors with exit code 1 and says "run pkgbootstrap first" that is okay too. Ideally, pkgng bootstrap process will be part of the bsdinstall steps (using the pkg package tarball on the installation media). The bootstrap tool is in base (instead of only on the install media) essentially for installations without the installer (like creating jails). So its true that in these case you have to think to run the bootstrap command. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
