Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/31 Andriy Gapon : > on 29/03/2011 23:29 Baptiste Daroussin said the following: >> ok let's try to say it simpler :) the main goal is to keep it simple >> for now, simple and rock solid, so that we can replace pkg_install and >> do some cleanup in the ports tree, add the "must have" features while >> doing that. And only when we will be ready for that and that portmgr >> have decided that it is mature enough to replace pkg_install, only >> after that we will start improving with new features and new changes. >> >> I thinks changing the package name scheme is not a "must have" >> feature, it for sure is and intresting feature, but what about pushing >> to after the first stable release? managing architecture as we plan to >> do it is enough imho. > > Oh, yes, I realize all this and totally agree with it. > Given how huge and how visible our ports and packages systems are, it's > better to > be slow and cautious. > All the ideas that I suggested were more for the "next step" than for now. > And noted in my personnal TODO list :) > Thank you for the work! > -- > Andriy Gapon > ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
on 29/03/2011 23:29 Baptiste Daroussin said the following: > ok let's try to say it simpler :) the main goal is to keep it simple > for now, simple and rock solid, so that we can replace pkg_install and > do some cleanup in the ports tree, add the "must have" features while > doing that. And only when we will be ready for that and that portmgr > have decided that it is mature enough to replace pkg_install, only > after that we will start improving with new features and new changes. > > I thinks changing the package name scheme is not a "must have" > feature, it for sure is and intresting feature, but what about pushing > to after the first stable release? managing architecture as we plan to > do it is enough imho. Oh, yes, I realize all this and totally agree with it. Given how huge and how visible our ports and packages systems are, it's better to be slow and cautious. All the ideas that I suggested were more for the "next step" than for now. Thank you for the work! -- Andriy Gapon ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/29 Baptiste Daroussin : > 2011/3/29 Andriy Gapon : >> on 28/03/2011 21:22 Julien Laffaye said the following: >>> On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper wrote: > III. Package naming that includes architecture, major OS version (for > API/ABI), > maybe more. This could be provided in the manifest. Doing it in the filename sort of turns into a mess, as I've discovered working at Cisco :). >>> >>> Actually, it *is* in the +MANIFEST of pkgng packages archives :-) >> >> Well, by the package name I meant not only a package file name. >> Let's imagine that we do support installing i386 packages on amd64 in >> parallel to >> amd64 packages. And for some reason I want to have both 32-bit and 64-bit >> versions of, say, firefox; e.g. for benchmarking. If the packages would >> have the >> same name, then that would be impossible. >> >> I think that having some thing in package name in addition to package >> metadata >> could have certain benefits. >> >> -- >> Andriy Gapon >> > > I understand but I think pkgng is already quite radical changement. > More change is taking the risk that it would be rejected in the end, > we still do not have any reply from portmgr, there is no insurance > pkgng will in the end replace pkg_install. Currently pkgng requires > only very few changes from the ports infrastruture, I don't know the > cost of changing the name scheme. > > If I'm not clear enough, supporting both 32bits and 64bits packages at > the same time on amd64 or arches that could support this kind of > installation, is a large change we don't want to take the > responsability of :) and implementing this in pkgng would significate > we already choose how it should work. > > regards, > Bapt > seems it was not clear :) ok let's try to say it simpler :) the main goal is to keep it simple for now, simple and rock solid, so that we can replace pkg_install and do some cleanup in the ports tree, add the "must have" features while doing that. And only when we will be ready for that and that portmgr have decided that it is mature enough to replace pkg_install, only after that we will start improving with new features and new changes. I thinks changing the package name scheme is not a "must have" feature, it for sure is and intresting feature, but what about pushing to after the first stable release? managing architecture as we plan to do it is enough imho. But I can be wrong. regards, Bapt ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/29 Andriy Gapon : > on 28/03/2011 21:22 Julien Laffaye said the following: >> On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper wrote: III. Package naming that includes architecture, major OS version (for API/ABI), maybe more. >>> >>> This could be provided in the manifest. Doing it in the filename sort >>> of turns into a mess, as I've discovered working at Cisco :). >>> >> >> Actually, it *is* in the +MANIFEST of pkgng packages archives :-) > > Well, by the package name I meant not only a package file name. > Let's imagine that we do support installing i386 packages on amd64 in > parallel to > amd64 packages. And for some reason I want to have both 32-bit and 64-bit > versions of, say, firefox; e.g. for benchmarking. If the packages would have > the > same name, then that would be impossible. > > I think that having some thing in package name in addition to package metadata > could have certain benefits. > > -- > Andriy Gapon > I understand but I think pkgng is already quite radical changement. More change is taking the risk that it would be rejected in the end, we still do not have any reply from portmgr, there is no insurance pkgng will in the end replace pkg_install. Currently pkgng requires only very few changes from the ports infrastruture, I don't know the cost of changing the name scheme. If I'm not clear enough, supporting both 32bits and 64bits packages at the same time on amd64 or arches that could support this kind of installation, is a large change we don't want to take the responsability of :) and implementing this in pkgng would significate we already choose how it should work. regards, Bapt ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
on 28/03/2011 21:22 Julien Laffaye said the following: > On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper wrote: >>> III. Package naming that includes architecture, major OS version (for >>> API/ABI), >>> maybe more. >> >> This could be provided in the manifest. Doing it in the filename sort >> of turns into a mess, as I've discovered working at Cisco :). >> > > Actually, it *is* in the +MANIFEST of pkgng packages archives :-) Well, by the package name I meant not only a package file name. Let's imagine that we do support installing i386 packages on amd64 in parallel to amd64 packages. And for some reason I want to have both 32-bit and 64-bit versions of, say, firefox; e.g. for benchmarking. If the packages would have the same name, then that would be impossible. I think that having some thing in package name in addition to package metadata could have certain benefits. -- Andriy Gapon ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
I'm just going to clarify a statement I made earlier on this thread in order to remove some possible misconceptions. One can only boot 32bit PPC on a 32bit PPC machines and have it work properly. The same applies for 64bit ppc machines. On Tue, Mar 29, 2011 at 8:11 AM, Julien Laffaye wrote: > On Tue, Mar 29, 2011 at 5:15 AM, Tim Kientzle > wrote: > > II. Package signing. > > That would be really nice. > >>> > >>> Right know we only planned to sign the repo database, so we can trust > >>> the sah256 of the packages stored in the database. Then if the package > >>> has the same sha256 as the one in the repo database it is considered > >>> trusted. > >>> If we want a per-package signing, we would have a tarball in a tarball. > >> > >> I really expected this to have been mentioned already, but this approach > (tarball in a tarball) is taken by Debian packages, and I don't remember > hearing of any issues related to it. I don't think it's worth discounting > from the start without giving some considerationg, but I will defer to the > people actually doing the work. > > > > If you use libarchive-style streaming, it's even > > pretty straightforward to read and extract such > > things without having to create a bunch of > > temporary files. > > > > You just need to be careful about compression. > > Agreed, if we dont want to verify the signature, we can extract the > tarball in the tarball efficiently. > > But to verify the signature, we have to read the tarball in the > tarball twice: the first time to compute the digest and verify the > signature, the second time to do the real extraction. > So I guess that the tarball containing the real package archive and > the signature should be uncompressed. The real package archive would > be compressed, though. > ___ > freebsd-hack...@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Tue, Mar 29, 2011 at 5:15 AM, Tim Kientzle wrote: > II. Package signing. That would be really nice. >>> >>> Right know we only planned to sign the repo database, so we can trust >>> the sah256 of the packages stored in the database. Then if the package >>> has the same sha256 as the one in the repo database it is considered >>> trusted. >>> If we want a per-package signing, we would have a tarball in a tarball. >> >> I really expected this to have been mentioned already, but this approach >> (tarball in a tarball) is taken by Debian packages, and I don't remember >> hearing of any issues related to it. I don't think it's worth discounting >> from the start without giving some considerationg, but I will defer to the >> people actually doing the work. > > If you use libarchive-style streaming, it's even > pretty straightforward to read and extract such > things without having to create a bunch of > temporary files. > > You just need to be careful about compression. Agreed, if we dont want to verify the signature, we can extract the tarball in the tarball efficiently. But to verify the signature, we have to read the tarball in the tarball twice: the first time to compute the digest and verify the signature, the second time to do the real extraction. So I guess that the tarball containing the real package archive and the signature should be uncompressed. The real package archive would be compressed, though. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/29 Tim Kientzle : > II. Package signing. That would be really nice. >>> >>> Right know we only planned to sign the repo database, so we can trust >>> the sah256 of the packages stored in the database. Then if the package >>> has the same sha256 as the one in the repo database it is considered >>> trusted. >>> If we want a per-package signing, we would have a tarball in a tarball. >> >> I really expected this to have been mentioned already, but this approach >> (tarball in a tarball) is taken by Debian packages, and I don't remember >> hearing of any issues related to it. I don't think it's worth discounting >> from the start without giving some considerationg, but I will defer to the >> people actually doing the work. > > If you use libarchive-style streaming, it's even > pretty straightforward to read and extract such > things without having to create a bunch of > temporary files. > > You just need to be careful about compression. > > Tim > > ok but what is the problem with signing only the repository then rely on digest? I am not sure we need more that this. second question howto sign? pgp? ssl? First would be the easiest way to go but we don't have in base anything to check signatures (maybe we should in that case investigating to import netpgp), ssl why not? but which algorithm? what security officer would prefer? We are ok to investigate that part, but we need more information about what is expected. regards, Bapt ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
II. Package signing. >>> >>> That would be really nice. >> >> Right know we only planned to sign the repo database, so we can trust >> the sah256 of the packages stored in the database. Then if the package >> has the same sha256 as the one in the repo database it is considered >> trusted. >> If we want a per-package signing, we would have a tarball in a tarball. > > I really expected this to have been mentioned already, but this approach > (tarball in a tarball) is taken by Debian packages, and I don't remember > hearing of any issues related to it. I don't think it's worth discounting > from the start without giving some considerationg, but I will defer to the > people actually doing the work. If you use libarchive-style streaming, it's even pretty straightforward to read and extract such things without having to create a bunch of temporary files. You just need to be careful about compression. Tim ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Mon, 28 Mar 2011, Julien Laffaye wrote: On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper wrote: On Mon, Mar 28, 2011 at 10:44 AM, Andriy Gapon wrote: II. Package signing. That would be really nice. Right know we only planned to sign the repo database, so we can trust the sah256 of the packages stored in the database. Then if the package has the same sha256 as the one in the repo database it is considered trusted. If we want a per-package signing, we would have a tarball in a tarball. I really expected this to have been mentioned already, but this approach (tarball in a tarball) is taken by Debian packages, and I don't remember hearing of any issues related to it. I don't think it's worth discounting from the start without giving some considerationg, but I will defer to the people actually doing the work. -Ben Kaduk ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper wrote: > On Mon, Mar 28, 2011 at 10:44 AM, Andriy Gapon wrote: >> on 25/03/2011 12:11 Baptiste Daroussin said the following: >>> Hi all, >>> >>> miwi@ launched the new thing called Experimental Call For Testing, >>> it's our turn :) >>> >>> Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge >>> contributor) have been working since the end of the last GSoC on a >>> rewrite of pkg_install. >>> >>> pkgng is a binary package manager written from scratch for FreeBSD. >>> >>> After a long period of technology testing, (json, tinycdb, bdb, etc) >>> and we now have achieved to implement the basic functionnality. We >>> would greatly approciate to have some feedback, wider testing, >>> patching, documenting etc, before implementing the higher level >>> features. >>> >>> pkgng is built on top of a new libpkg, which allow to deal with the >>> database of >>> installed packages, to deal with remote repositories, manage packages: >>> creation, installation gathering informations, registering new ports. >>> >>> features supported are or will be : >>> >>> - smooth integration with bsd.port.mk (including bsd.pkg.mk line 2486) >>> which allow to have a bsd.port.mk which deal with both pkg_install >>> and pkgng. (done in alpha) >>> >>> - the register command can analyse elf files when registering a new port to >>> discover forgotten dependencies if necessary. (done in alpha using libelf) >>> >>> - the register command has two mode available : when dealing with old >>> fashion ports it just registers the package, in new mode it does >>> everything that would >>> have been done by pkg add when installing the package : should display >>> messages, execute post-install, execute @exec etc. (old fashion done >>> in the alpha) >>> >>> - pkg add supports two mode : the old fashion one (no real upgrade >>> support) and new one: upgrade scripts supported. (old fashion in the >>> alpha) >>> >>> - new scripts supported +PREINSTALL +POSTINSTALL, +PREDEINSTALL, >>> +POSTDEINSTALL, +PREUPGRADE, +POSTUPGRADE as well as the old fashion >>> scripts : +INSTALL +DEINSTALL +UPGRADE (all supported *UPGRADES aren't >>> supported in the alpha) >>> >>> - new +MANIFEST (plist-like format) with new metadatas : options, arch, os >>> version, etc. (done in the alpha) >>> >>> - pkgng supports checking arch of the package which means that users >>> won't be able to install sparc64 binary package into amd64 machines. >>> (not done yet) >>> >>> - a special architecture "all" allows to specify when a package can be used >>> on every architecture. (not done yet) >>> >>> - @dirrm and @dirrmtry are now deprecated, pkgng can discover itself >>> which directory has to be removed. (done in the alpha but needs love >>> :)) >>> >>> - new repository (apt-like feature) (only the repository generation is done) >>> >>> - real support for reverse dependency (no ugly +REQUIRED_BY) (done in the >>> alpha) >>> >>> - test unit (libcheck) on libpkg. (done in the alpha needs some more love) >>> >>> - many more >> >> Perhaps I am too impatient :) but I would like to inquire about the following >> features: >> >> I. A provides/requires interface for packages. >> Each package specified a list of files (and perhaps other entities) that it >> provides and requires. At the initial stage, without ports modifications, >> these >> could be: (1) a list of all files installed by package for provides; (2) for >> requires - an auto-generated list of dependencies based on required shared >> libraries plus dependency specifications in ports. >> I think that this kind of interface should help with using alternatives that >> provide the same interface (e.g. like gamin vs fam). Adding require/provide support in pkgng is kind of trivial. But ports have to support it first. >> >> II. Package signing. > > That would be really nice. Right know we only planned to sign the repo database, so we can trust the sah256 of the packages stored in the database. Then if the package has the same sha256 as the one in the repo database it is considered trusted. If we want a per-package signing, we would have a tarball in a tarball. > >> III. Package naming that includes architecture, major OS version (for >> API/ABI), >> maybe more. > > This could be provided in the manifest. Doing it in the filename sort > of turns into a mess, as I've discovered working at Cisco :). > Actually, it *is* in the +MANIFEST of pkgng packages archives :-) >> IV. "Convenient" support for i386 packages on amd64. >> Distinct installation directories, proper installation conflict >> detection/avoidance between i386 and amd64 packages, proper library paths, >> etc. > > There are other architectures that would benefit from this as well, > like powerpc 32-bit on 64-bit, MIPs 32-bit on n32, etc. > > This involves more work than pkgng could provide IIRC as build > infrastructure would need to be fixed to look at and link against > usr/lib32 instead of usr/lib, u
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Mon, Mar 28, 2011 at 10:44 AM, Andriy Gapon wrote: > on 25/03/2011 12:11 Baptiste Daroussin said the following: >> Hi all, >> >> miwi@ launched the new thing called Experimental Call For Testing, >> it's our turn :) >> >> Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge >> contributor) have been working since the end of the last GSoC on a >> rewrite of pkg_install. >> >> pkgng is a binary package manager written from scratch for FreeBSD. >> >> After a long period of technology testing, (json, tinycdb, bdb, etc) >> and we now have achieved to implement the basic functionnality. We >> would greatly approciate to have some feedback, wider testing, >> patching, documenting etc, before implementing the higher level >> features. >> >> pkgng is built on top of a new libpkg, which allow to deal with the database >> of >> installed packages, to deal with remote repositories, manage packages: >> creation, installation gathering informations, registering new ports. >> >> features supported are or will be : >> >> - smooth integration with bsd.port.mk (including bsd.pkg.mk line 2486) >> which allow to have a bsd.port.mk which deal with both pkg_install >> and pkgng. (done in alpha) >> >> - the register command can analyse elf files when registering a new port to >> discover forgotten dependencies if necessary. (done in alpha using libelf) >> >> - the register command has two mode available : when dealing with old >> fashion ports it just registers the package, in new mode it does >> everything that would >> have been done by pkg add when installing the package : should display >> messages, execute post-install, execute @exec etc. (old fashion done >> in the alpha) >> >> - pkg add supports two mode : the old fashion one (no real upgrade >> support) and new one: upgrade scripts supported. (old fashion in the >> alpha) >> >> - new scripts supported +PREINSTALL +POSTINSTALL, +PREDEINSTALL, >> +POSTDEINSTALL, +PREUPGRADE, +POSTUPGRADE as well as the old fashion >> scripts : +INSTALL +DEINSTALL +UPGRADE (all supported *UPGRADES aren't >> supported in the alpha) >> >> - new +MANIFEST (plist-like format) with new metadatas : options, arch, os >> version, etc. (done in the alpha) >> >> - pkgng supports checking arch of the package which means that users >> won't be able to install sparc64 binary package into amd64 machines. >> (not done yet) >> >> - a special architecture "all" allows to specify when a package can be used >> on every architecture. (not done yet) >> >> - @dirrm and @dirrmtry are now deprecated, pkgng can discover itself >> which directory has to be removed. (done in the alpha but needs love >> :)) >> >> - new repository (apt-like feature) (only the repository generation is done) >> >> - real support for reverse dependency (no ugly +REQUIRED_BY) (done in the >> alpha) >> >> - test unit (libcheck) on libpkg. (done in the alpha needs some more love) >> >> - many more > > Perhaps I am too impatient :) but I would like to inquire about the following > features: > > I. A provides/requires interface for packages. > Each package specified a list of files (and perhaps other entities) that it > provides and requires. At the initial stage, without ports modifications, > these > could be: (1) a list of all files installed by package for provides; (2) for > requires - an auto-generated list of dependencies based on required shared > libraries plus dependency specifications in ports. > I think that this kind of interface should help with using alternatives that > provide the same interface (e.g. like gamin vs fam). > > II. Package signing. That would be really nice. > III. Package naming that includes architecture, major OS version (for > API/ABI), > maybe more. This could be provided in the manifest. Doing it in the filename sort of turns into a mess, as I've discovered working at Cisco :). > IV. "Convenient" support for i386 packages on amd64. > Distinct installation directories, proper installation conflict > detection/avoidance between i386 and amd64 packages, proper library paths, > etc. There are other architectures that would benefit from this as well, like powerpc 32-bit on 64-bit, MIPs 32-bit on n32, etc. This involves more work than pkgng could provide IIRC as build infrastructure would need to be fixed to look at and link against usr/lib32 instead of usr/lib, unless you mean to rewrite the linker stuff at install-time. > And finally some exotic ideas - support for multiple package sources (when > different people build packages in different ways (e.g. with different > options, or > different optimizations) from the same ports tree; support for multiple ports > sources.(when people maintain different ports tree (e.g. kde or gnome > development > ports tree)). Perhaps, with some compatibility/hierarchy support for > packages and > ports sources. But that's almost a pipe dream, so don't take it seriously :) It would be nice. That's a request in the same general area that Gent
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
on 25/03/2011 12:11 Baptiste Daroussin said the following: > Hi all, > > miwi@ launched the new thing called Experimental Call For Testing, > it's our turn :) > > Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge > contributor) have been working since the end of the last GSoC on a > rewrite of pkg_install. > > pkgng is a binary package manager written from scratch for FreeBSD. > > After a long period of technology testing, (json, tinycdb, bdb, etc) > and we now have achieved to implement the basic functionnality. We > would greatly approciate to have some feedback, wider testing, > patching, documenting etc, before implementing the higher level > features. > > pkgng is built on top of a new libpkg, which allow to deal with the database > of > installed packages, to deal with remote repositories, manage packages: > creation, installation gathering informations, registering new ports. > > features supported are or will be : > > - smooth integration with bsd.port.mk (including bsd.pkg.mk line 2486) > which allow to have a bsd.port.mk which deal with both pkg_install > and pkgng. (done in alpha) > > - the register command can analyse elf files when registering a new port to > discover forgotten dependencies if necessary. (done in alpha using libelf) > > - the register command has two mode available : when dealing with old > fashion ports it just registers the package, in new mode it does > everything that would > have been done by pkg add when installing the package : should display > messages, execute post-install, execute @exec etc. (old fashion done > in the alpha) > > - pkg add supports two mode : the old fashion one (no real upgrade > support) and new one: upgrade scripts supported. (old fashion in the > alpha) > > - new scripts supported +PREINSTALL +POSTINSTALL, +PREDEINSTALL, > +POSTDEINSTALL, +PREUPGRADE, +POSTUPGRADE as well as the old fashion > scripts : +INSTALL +DEINSTALL +UPGRADE (all supported *UPGRADES aren't > supported in the alpha) > > - new +MANIFEST (plist-like format) with new metadatas : options, arch, os > version, etc. (done in the alpha) > > - pkgng supports checking arch of the package which means that users > won't be able to install sparc64 binary package into amd64 machines. > (not done yet) > > - a special architecture "all" allows to specify when a package can be used > on every architecture. (not done yet) > > - @dirrm and @dirrmtry are now deprecated, pkgng can discover itself > which directory has to be removed. (done in the alpha but needs love > :)) > > - new repository (apt-like feature) (only the repository generation is done) > > - real support for reverse dependency (no ugly +REQUIRED_BY) (done in the > alpha) > > - test unit (libcheck) on libpkg. (done in the alpha needs some more love) > > - many more Perhaps I am too impatient :) but I would like to inquire about the following features: I. A provides/requires interface for packages. Each package specified a list of files (and perhaps other entities) that it provides and requires. At the initial stage, without ports modifications, these could be: (1) a list of all files installed by package for provides; (2) for requires - an auto-generated list of dependencies based on required shared libraries plus dependency specifications in ports. I think that this kind of interface should help with using alternatives that provide the same interface (e.g. like gamin vs fam). II. Package signing. III. Package naming that includes architecture, major OS version (for API/ABI), maybe more. IV. "Convenient" support for i386 packages on amd64. Distinct installation directories, proper installation conflict detection/avoidance between i386 and amd64 packages, proper library paths, etc. And finally some exotic ideas - support for multiple package sources (when different people build packages in different ways (e.g. with different options, or different optimizations) from the same ports tree; support for multiple ports sources.(when people maintain different ports tree (e.g. kde or gnome development ports tree)). Perhaps, with some compatibility/hierarchy support for packages and ports sources. But that's almost a pipe dream, so don't take it seriously :) -- Andriy Gapon ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Fri, Mar 25, 2011 at 10:11 AM, Baptiste Daroussin wrote: > Developpement site: http://git.etoilebsd.net/pkgng/ FYI, we moved to github[1] in order to have a bug tracker, pull request and code review. Also, I recommend to build from the HEAD of the git repository, to not report fixed compilation warnings/bugs. [1]: https://github.com/pkgng/pkgng ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
> This can still be discussed but I don't really like the idea that > users may alter packages an installation/extraction time, that would > lead to lots of potential buggy installation and report. > If user aren't happy with the packaging, they can poke the maintainer, > send PR, patch etc. The whole point of *binary* packages is that people don't mess changing options, changing what is installed, etc. so that port maintainers know exactly what people have on their machine, and be able to test it. This is a prerequisite for a system which is not a complete mess. Those you like to change things are on their own. -- Michel TALON ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Sat, 26 Mar 2011 10:22:50 +, Baptiste Daroussin wrote: > This can still be discussed but I don't really like the idea that users > may alter packages an installation/extraction time, that would lead to > lots of potential buggy installation and report. That's why list of alterations have to be recorded and relevant tools would mark such packages appropriately. Using filters would be considered unsupported (unless maintainer of particular port/package says otherwise). It's no different than doing unsupported things right now, just giving user more tools. Besides, examples mentioned by me like removing docs and unused translations are rather safe and save significant amount of disk space. > If user aren't happy > with the packaging, they can poke the maintainer, send PR, patch etc. That's not possible unless user is willing to build packages himself. Actually I would rather not have build time options for things that can just as well be performed during installation. Hacking makefiles to implement NOPORTDOCS is quite more complicated than setting a glob pattern. There is also one very important use case I didn't mention before. Such filters could serve as a hooking point for configuration management system. It makes it possible to capture config files as they are installed and perform backups, merges, check-in to vcs etc. > regards, > Bapt > ___ > freebsd-ports@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, > send any mail to "freebsd-ports-unsubscr...@freebsd.org" ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Fri, Mar 25, 2011 at 12:11 PM, Baptiste Daroussin wrote: > Hi all, > Hi, Great news! I'll try to contribute with what I can for this project! I've mirrored the pkgng Git repo here as well: - http://git.unix-heaven.org/cgit.cgi/pkgng/ And attached is my first patch :) Could you please review and if possible, apply it? Regards, Marin -- Marin Atanasov Nikolov dnaeon AT gmail DOT com daemon AT unix-heaven DOT org http://www.unix-heaven.org/ From bd17ad1dfa336717fcc924217552dadd5236ed5f Mon Sep 17 00:00:00 2001 From: Marin Atanasov Nikolov Date: Sat, 26 Mar 2011 14:19:04 +0200 Subject: [PATCH] Clarify a bit more the GOALS and some other minor changes - Clarify GOALS a bit more - Added main.h which contains CMD_MAX_LEN - Use strnlen() instead of strlen() in main.c - Use float numbers in fetch_status() --- docs/GOALS | 178 ++-- pkg/add.c |2 +- pkg/main.c |9 ++-- pkg/main.h |7 +++ 4 files changed, 126 insertions(+), 70 deletions(-) create mode 100644 pkg/main.h diff --git a/docs/GOALS b/docs/GOALS index 1c381d2..ccf3414 100644 --- a/docs/GOALS +++ b/docs/GOALS @@ -1,104 +1,152 @@ -comamnds: +pkgng - New version of FreeBSD's pkg_install + +Table of Contents: + +1. Available commands of pkgng +2. Available subcommands of pkgng +3. General behaviour +4. New manifest format + 4.1. During new install + 4.2. During package deletion + 4.3. During upgrade of a package +5. Additional resources + + +1. Available commands of pkgng: +--- + - pkg: - - one command to rule them all - - each time pkg has to query the installed database it will try to first - read the cache is the cache is too old (check but last mtime of the - directory) it will regenerate the cache from installed package + + - One command to rule them all + - Each time `pkg' has to query the package database it will try to first + read the cache and if the cache is too old (check by last mtime of the + directory) it will regenerate the cache for the installed packages. - pkg_*: - this will be shell script their goal is to be 100% compatible with - pkg_install they will call pkg which the good options (not sure it will - be really done in the end) -subcommands: + - These will be shell scripts, which goal is to be 100% compatible with + pkg_install. They will call `pkg' with the appropriate options (not sure + it will be really done in the end). + +2. Available subcommands of pkgng: +-- + - add: - - installs a package if local package is given - - manage http/ftp url to download throught libfetch a package and try to - install it. - - in the two previous case, if the package as not installed dependencies - it will try to fetch/find them in the same place the original package - was. - - it will manage a completely new repository format pkg add bla will - check in the database of the repository if the is a bla package and - install it as well as it's dependencies but the dependencies will be - compute before begining the fetch install process, to be sure - everything is ok. - - should be able to only take a plist or a manifest in argument + + - Installs a package if a local package is given + - Manages ftp/http downloads of packages through libfetch and + tries to install them. + - In the two previous cases, if a package does not have it's + dependencies installed, `pkgng' will try to fetch/find them in the same + place as the original package was. + - It will manage a completely new repository format. + `pkg add ' will check in the database of the repository for + package and try to install it as well as it's dependencies. + Dependencies will be computed before beginning the fetch/install + process to ensure that everything will be OK. + - Should be able to only take a plist or a manifest in argument considering the files are already installed - repo: - - takes only one argument and generate two databases files (maybe more for - the time we will be willing to work with UPDATING and MOVED) the first - repo contain every informations concerning a package except files - (which will be in the second one) + it will compute and add a sha256 - sum for each package. - the database will be a sqlite file compressed with the xz format. - the database will be signed so we can trust the sha256 of the - packages, so if a package has the expected hash, it is considered + + - Takes only one argument and generates two database files (maybe more for + the time we will be willing to work with UPDATING and MOVED). + The first repo contains all information about a package, except for + the files (which will be in the second one) + it will compute and + add a sha256 sum for each package. + - The database will be a sqlite file compressed with the xz format. + - The database will be signed so we can trust the sha256 of the + packa
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/25 Marcin Wisnicki : > On Fri, 25 Mar 2011 11:11:11 +0100, Baptiste Daroussin wrote: > >> pkgng is a binary package manager written from scratch for FreeBSD. >> > > Fantastic! > > I know it is quite too early but I already have one feature request ;) > Perhaps it could be added to the TODO as a post-1.0 goal. > > = Generic extraction filters = > > Allow registration of custom filters that can alter/exclude/add? files > during package extraction (installation). > > Examples of possible filters: > - strip debug info > - exclude development files (headers, static libs) > - exclude unused translations > - exclude documentation (all or just unknown languages) > - generic glob/regex path filters > - optional file groups defined in package (install time OPTIONS) ? > > Some sort of configuration mechanism with list of enabled filters and > their options (like a list of languages to keep). > > Most of this can be implemented as a simple glob/regex matching but there > are edge cases where packages have some non-standard layout or have to > keep certain file in which case a package metadata should contain a list > of exclusions/additions from/to above categories. > > Package manager should register only actually installed files but list of > alterations should be also kept somewhere in database. > > > What do you think ? > > ___ > freebsd-ports@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ports > To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org" > This can still be discussed but I don't really like the idea that users may alter packages an installation/extraction time, that would lead to lots of potential buggy installation and report. If user aren't happy with the packaging, they can poke the maintainer, send PR, patch etc. regards, Bapt ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Fri, 25 Mar 2011 11:11:11 +0100, Baptiste Daroussin wrote: > pkgng is a binary package manager written from scratch for FreeBSD. > Fantastic! I know it is quite too early but I already have one feature request ;) Perhaps it could be added to the TODO as a post-1.0 goal. = Generic extraction filters = Allow registration of custom filters that can alter/exclude/add? files during package extraction (installation). Examples of possible filters: - strip debug info - exclude development files (headers, static libs) - exclude unused translations - exclude documentation (all or just unknown languages) - generic glob/regex path filters - optional file groups defined in package (install time OPTIONS) ? Some sort of configuration mechanism with list of enabled filters and their options (like a list of languages to keep). Most of this can be implemented as a simple glob/regex matching but there are edge cases where packages have some non-standard layout or have to keep certain file in which case a package metadata should contain a list of exclusions/additions from/to above categories. Package manager should register only actually installed files but list of alterations should be also kept somewhere in database. What do you think ? ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On 03/25/2011 03:11, Baptiste Daroussin wrote: Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge contributor) have been working since the end of the last GSoC on a rewrite of pkg_install. pkgng is a binary package manager written from scratch for FreeBSD. How does it relate to portmaster and portupgrade packages, which both have (or include) supposedly the same functionality? Yuri ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Fri, 25 Mar 2011 13:47:13 -0700 Garrett Cooper wrote: > On Fri, Mar 25, 2011 at 1:14 PM, Alexander Leidinger > wrote: > > On Fri, 25 Mar 2011 16:35:21 +0100 Pietro Cerutti > > wrote: > > > >> On 2011-Mar-25, 15:03, Julien Laffaye wrote: > >> > >>> What about DB corruption/loss? Do you keep > >> > >>> the /var/db/pkg//xxx files even with pkgng and only > >> > >>> use the DB as a way to speed up some work (so > >> > >>> the DB corruption just requires to run pkg2ng), or are you > >> > >>> lost of the DB is > >> > >>> lost? > >> > >>> > >> > >> > >> > >> Nothing is done about DB corruption/loss, I am not sure we > >> > >> need to do something. > >> > >> Maybe. > >> > > > >> > > I would say "for sure". Info: In Solaris 10 sqlite is used for > >> > > the service managenemt framework (SMF). It is possible that the > >> > > DB is corrupt in some bad situations. In this case you have to > >> > > rebuild the DB (script provided, been there, had to use it). > >> > > >> > If sqlite is properly used with transactions, it is very hard to > >> > corrupt the database. But if hardware lies to us and say that the > > > > And as I told above, I even had such a case (more than once), and > > the hardware was not buggy. What do you want to tell in this case, > > "life sucks, reinstall everything"? > > If you use binary packages, pulling down everything should be trivial, > fast, and easy to install. If you're using ports, well then things are > going to be slow as expected. And if there is a fast way to cut down the slow part... why not? > >> > data is on disk whereas it isnt... what can we do? > > > > Sometimes you have to stay with broken hardware. > > Sometimes you have to go buy new parts? Yes, but if we talk e.g. about aging hardware, having the luck to get hit directly in the parts which hurt is not nice. You want to have the time to find a suitable replacement. > Playing with broken hardware is like playing with fire -- sometimes > you'll get burned if it goes out of commission during critical > operations. I would be more concerned about overall system operation > than having a packaging system that can handle all error conditions > that should be rightfully handled by various kernel subsystems. If the > kernel's doing it's job, then the packaging manager can do its job as > well. You know that the world is not an ideal one. Shit happens and Murphy visits you. > >> > Another potential problem is fsync(), but if it is broken on > >> > FreeBSD we want to fix it! > >> > > >> > BTW, the goal is to only have the database and not the flat > >> > files. If you are paranoid about power outage, use something > >> > like zfs snapshots... > > > > There are more FS than only ZFS (personally I use ZFS, and I have > > snapshots, but this is not a good solution for this problem). > > A lot of filesystems feature snapshot'ing, including UFS. If you > aren't smart enough to back up your data you're toast if the data is > gone. So... why do we have /var/backups/master.passwd.bak then? To make life easy. > I would be more concerned about the program getting killed, not > getting properly cleaned up, etc as this is something that the package > manager frontend (or whatever the official name is) should catch and > fail gracefully with. Things need to follow an ACID methodology and be > recoverable in the event that it can't be ACID, or it's no better than > pkg_install/ports currently is if it's caught in the middle of a > critical operation today installing or removing software. I agree. > If SQLite can't deliver this level of ACID-like capability, then > pkg_install needs to be redesigned. AFAIK it can. > > As I told already, if it isn't automatic, nearly nobody will use it. > > And the package management stuff has to be automatic, no freshman > > will think about setting up a snapshot script when he starts to use > > packages/ports. > > I'd just provide an export command to print out a (JSON?) > version of the information, and move on. None of the other major > packaging systems out there that I know of use flat files for this > data, and I would rather not make it automatic because it's an > unnecessary performance hit. If the user feels the need for backing up > his/her data they will. If not, they're SoL in the event of a crash. - It does not need to be done with every change. - You do not know if it is a performance hit or not, we do not have numbers. - If making an automatic export after X modifications is not expensive, I say: why not? It would make more easy in case of fire. > >> No need to look for strange scenarios, I'm surely going to sudo rm > >> -f the file more sooner than later, so... maybe just save a copy? > > > > A copy or two would be enough, but it has to be done automatically, > > and once a day is not enough. A copy after each X modifications > > maybe (for suitable definitions of X and 'modifications'). > > Please see my comment above. There's no reason why this belongs in a > packag
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/25 Yuri : > On 03/25/2011 03:11, Baptiste Daroussin wrote: >> >> Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge >> contributor) have been working since the end of the last GSoC on a >> rewrite of pkg_install. >> >> pkgng is a binary package manager written from scratch for FreeBSD. >> > > How does it relate to portmaster and portupgrade packages, which both have > (or include) supposedly the same functionality? > > Yuri > both have to be adapted, portupgrade throught maybe some ruby bindings to libpkg, portmaster by patching it to use pkg frontend instead of pkg_* tools (as I did for the ports (see ports/bsd.pkgng.mk in the git tree) regards, Bapt ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Fri, Mar 25, 2011 at 1:14 PM, Alexander Leidinger wrote: > On Fri, 25 Mar 2011 16:35:21 +0100 Pietro Cerutti > wrote: > >> On 2011-Mar-25, 15:03, Julien Laffaye wrote: >> > >>> What about DB corruption/loss? Do you keep >> > >>> the /var/db/pkg//xxx files even with pkgng and only >> > >>> use the DB as a way to speed up some work (so >> > >>> the DB corruption just requires to run pkg2ng), or are you lost >> > >>> of the DB is >> > >>> lost? >> > >>> >> > >> >> > >> Nothing is done about DB corruption/loss, I am not sure we need >> > >> to do something. >> > >> Maybe. >> > > >> > > I would say "for sure". Info: In Solaris 10 sqlite is used for >> > > the service managenemt framework (SMF). It is possible that the >> > > DB is corrupt in some bad situations. In this case you have to >> > > rebuild the DB (script provided, been there, had to use it). >> > >> > If sqlite is properly used with transactions, it is very hard to >> > corrupt the database. But if hardware lies to us and say that the > > And as I told above, I even had such a case (more than once), and the > hardware was not buggy. What do you want to tell in this case, "life > sucks, reinstall everything"? If you use binary packages, pulling down everything should be trivial, fast, and easy to install. If you're using ports, well then things are going to be slow as expected. >> > data is on disk whereas it isnt... what can we do? > > Sometimes you have to stay with broken hardware. Sometimes you have to go buy new parts? Playing with broken hardware is like playing with fire -- sometimes you'll get burned if it goes out of commission during critical operations. I would be more concerned about overall system operation than having a packaging system that can handle all error conditions that should be rightfully handled by various kernel subsystems. If the kernel's doing it's job, then the packaging manager can do its job as well. >> > Another potential problem is fsync(), but if it is broken on FreeBSD >> > we want to fix it! >> > >> > BTW, the goal is to only have the database and not the flat files. >> > If you are paranoid about power outage, use something like zfs >> > snapshots... > > There are more FS than only ZFS (personally I use ZFS, and I have > snapshots, but this is not a good solution for this problem). A lot of filesystems feature snapshot'ing, including UFS. If you aren't smart enough to back up your data you're toast if the data is gone. I would be more concerned about the program getting killed, not getting properly cleaned up, etc as this is something that the package manager frontend (or whatever the official name is) should catch and fail gracefully with. Things need to follow an ACID methodology and be recoverable in the event that it can't be ACID, or it's no better than pkg_install/ports currently is if it's caught in the middle of a critical operation today installing or removing software. If SQLite can't deliver this level of ACID-like capability, then pkg_install needs to be redesigned. > As I told already, if it isn't automatic, nearly nobody will use it. > And the package management stuff has to be automatic, no freshman will > think about setting up a snapshot script when he starts to use > packages/ports. I'd just provide an export command to print out a (JSON?) version of the information, and move on. None of the other major packaging systems out there that I know of use flat files for this data, and I would rather not make it automatic because it's an unnecessary performance hit. If the user feels the need for backing up his/her data they will. If not, they're SoL in the event of a crash. >> No need to look for strange scenarios, I'm surely going to sudo rm -f >> the file more sooner than later, so... maybe just save a copy? > > A copy or two would be enough, but it has to be done automatically, and > once a day is not enough. A copy after each X modifications maybe > (for suitable definitions of X and 'modifications'). Please see my comment above. There's no reason why this belongs in a packaging system (you can add it as an external tool, but the point is to avoid architectural mistakes that leaked into the old pkg_install over the period of 10 years or so). Thanks, -Garrett PS Sorry for being so hardnosed on this, but I want something that's fast and correct, instead of something bloated, slow, half-baked, harder to test, etc. pkg_install gets executed enough times during a port upgrade that having something more streamlined for most usecases is the only way to go, and there's enough code that doesn't get executed on a regular basis that has no business being in pkg_install. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
As far as package managers go, yum, which is used widely, uses SQLite. -- Peace cannot be achieved through violence, it can only be attained through understanding. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Fri, 25 Mar 2011 16:35:21 +0100 Pietro Cerutti wrote: > On 2011-Mar-25, 15:03, Julien Laffaye wrote: > > >>> What about DB corruption/loss? Do you keep > > >>> the /var/db/pkg//xxx files even with pkgng and only > > >>> use the DB as a way to speed up some work (so > > >>> the DB corruption just requires to run pkg2ng), or are you lost > > >>> of the DB is > > >>> lost? > > >>> > > >> > > >> Nothing is done about DB corruption/loss, I am not sure we need > > >> to do something. > > >> Maybe. > > > > > > I would say "for sure". Info: In Solaris 10 sqlite is used for > > > the service managenemt framework (SMF). It is possible that the > > > DB is corrupt in some bad situations. In this case you have to > > > rebuild the DB (script provided, been there, had to use it). > > > > If sqlite is properly used with transactions, it is very hard to > > corrupt the database. But if hardware lies to us and say that the And as I told above, I even had such a case (more than once), and the hardware was not buggy. What do you want to tell in this case, "life sucks, reinstall everything"? > > data is on disk whereas it isnt... what can we do? Sometimes you have to stay with broken hardware. > > Another potential problem is fsync(), but if it is broken on FreeBSD > > we want to fix it! > > > > BTW, the goal is to only have the database and not the flat files. > > If you are paranoid about power outage, use something like zfs > > snapshots... There are more FS than only ZFS (personally I use ZFS, and I have snapshots, but this is not a good solution for this problem). As I told already, if it isn't automatic, nearly nobody will use it. And the package management stuff has to be automatic, no freshman will think about setting up a snapshot script when he starts to use packages/ports. > No need to look for strange scenarios, I'm surely going to sudo rm -f > the file more sooner than later, so... maybe just save a copy? A copy or two would be enough, but it has to be done automatically, and once a day is not enough. A copy after each X modifications maybe (for suitable definitions of X and 'modifications'). Bye, Alexander. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Fri, Mar 25, 2011 at 11:54:35AM -0500, Eitan Adler wrote: > On Fri, Mar 25, 2011 at 11:35 AM, Andriy Gapon wrote: > > on 25/03/2011 17:35 Pietro Cerutti said the following: > >> No need to look for strange scenarios, I'm surely going to sudo rm -f the > >> file > >> more sooner than later, so... maybe just save a copy? > > > > I even can rm -rf / by accident. > > What's your solution to this? :) > > rm -rf / > rm: "/" may not be removed referring to the CVS, this should improve the approach. cd /tmp rm -rf ../* ymmv -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net pgp5D40nqAsVs.pgp Description: PGP signature
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
> Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge > contributor) have been working since the end of the last GSoC on a > rewrite of pkg_install. Fantastic! it has been necessary since a looong time. > - the register command can analyse elf files when registering a new port > to discover forgotten dependencies if necessary. (done in alpha using > libelf) Fantastic! the dependencies as mentioned directly in Makefiles by ports maintainers were not always perfect. > In term of technology we decided to use a sqlite3 database, and to > prevent potential trolling, sqlite3 is used in it's amalgamation form > which means it is incorporated in the code sources (as recommanded by > sqlite developpers like a statically linked library) on build we only > activate the features we need in sqlite. Fantastic! at least using something fast and tested instead of some half-brewed solutions. I have just taken a look at the table "packages", it seems that it does not contain dependency information, but you can discover it through analyze_elf, where do you store it? This project is the thing i had dreamed about. -- Michel TALON ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Fri, Mar 25, 2011 at 11:35 AM, Andriy Gapon wrote: > on 25/03/2011 17:35 Pietro Cerutti said the following: >> No need to look for strange scenarios, I'm surely going to sudo rm -f the >> file >> more sooner than later, so... maybe just save a copy? > > I even can rm -rf / by accident. > What's your solution to this? :) rm -rf / rm: "/" may not be removed -- Eitan Adler ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
Sorry i have just seen a table for dependencies in pkg_repo.c
"CREATE TABLE deps ("
"origin TEXT,"
"name TEXT,"
"version TEXT,"
"package_id INTEGER REFERENCES packages(id),"
"PRIMARY KEY (package_id, origin)"
So this seems fine.
--
Michel TALON
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
on 25/03/2011 17:35 Pietro Cerutti said the following: > No need to look for strange scenarios, I'm surely going to sudo rm -f the file > more sooner than later, so... maybe just save a copy? I even can rm -rf / by accident. What's your solution to this? :) P.S. one solution would be a subcase of the other -- Andriy Gapon ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/25 Pietro Cerutti : > On 2011-Mar-25, 15:03, Julien Laffaye wrote: >> >>> What about DB corruption/loss? Do you keep the /var/db/pkg//xxx >> >>> files even with pkgng and only use the DB as a way to speed up some work >> >>> (so >> >>> the DB corruption just requires to run pkg2ng), or are you lost of the DB >> >>> is >> >>> lost? >> >>> >> >> >> >> Nothing is done about DB corruption/loss, I am not sure we need to do >> >> something. >> >> Maybe. >> > >> > I would say "for sure". Info: In Solaris 10 sqlite is used for the service >> > managenemt framework (SMF). It is possible that the DB is corrupt in some >> > bad situations. In this case you have to rebuild the DB (script provided, >> > been there, had to use it). >> >> If sqlite is properly used with transactions, it is very hard to >> corrupt the database. But if hardware lies to us and say that the data >> is on disk whereas it isnt... what can we do? >> Another potential problem is fsync(), but if it is broken on FreeBSD >> we want to fix it! >> >> BTW, the goal is to only have the database and not the flat files. >> If you are paranoid about power outage, use something like zfs snapshots... > > No need to look for strange scenarios, I'm surely going to sudo rm -f the file > more sooner than later, so... maybe just save a copy? > > -- > Pietro Cerutti > The FreeBSD Project > g...@freebsd.org > > PGP Public Key: > http://gahr.ch/pgp > I think we can provide a periodic script activable by users (I let other decide if it has to be activated by default or not) that does a pkg backup /path/to/file/backup and xz it. because copying can be a huge. 40Mo for the database here, corresponding to 70Mo in the old format and to 600 packages. the dump xzed is only 3Mo regards, Bapt ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On 2011-Mar-25, 15:03, Julien Laffaye wrote: > >>> What about DB corruption/loss? Do you keep the /var/db/pkg//xxx > >>> files even with pkgng and only use the DB as a way to speed up some work > >>> (so > >>> the DB corruption just requires to run pkg2ng), or are you lost of the DB > >>> is > >>> lost? > >>> > >> > >> Nothing is done about DB corruption/loss, I am not sure we need to do > >> something. > >> Maybe. > > > > I would say "for sure". Info: In Solaris 10 sqlite is used for the service > > managenemt framework (SMF). It is possible that the DB is corrupt in some > > bad situations. In this case you have to rebuild the DB (script provided, > > been there, had to use it). > > If sqlite is properly used with transactions, it is very hard to > corrupt the database. But if hardware lies to us and say that the data > is on disk whereas it isnt... what can we do? > Another potential problem is fsync(), but if it is broken on FreeBSD > we want to fix it! > > BTW, the goal is to only have the database and not the flat files. > If you are paranoid about power outage, use something like zfs snapshots... No need to look for strange scenarios, I'm surely going to sudo rm -f the file more sooner than later, so... maybe just save a copy? -- Pietro Cerutti The FreeBSD Project g...@freebsd.org PGP Public Key: http://gahr.ch/pgp pgp4AYvylMp4W.pgp Description: PGP signature
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Fri, Mar 25, 2011 at 2:38 PM, Alexander Leidinger wrote: > Quoting Baptiste Daroussin (from Fri, 25 Mar 2011 > 15:14:52 +0100): > >> 2011/3/25 Alexander Leidinger : >>> >>> Quoting Baptiste Daroussin (from Fri, 25 Mar 2011 >>> 11:11:11 +0100): >>> pkgng is a binary package manager written from scratch for FreeBSD. >>> >>> I didn't had a look at it, just some comments about some parts you >>> explained. >>> features supported are or will be : > - a special architecture "all" allows to specify when a package can be used on every architecture. (not done yet) >>> >>> What if a package is able to install on a subset, e.g. the linuxulator >>> ports >>> are for amd64 and i386? >>> >> >> No clue for that at the moment but we are open to suggestions. > > The suggestion is easy, allow a way to specify a set of valid architectures. That looks reasonable and easy to implement. > >>> What about DB corruption/loss? Do you keep the /var/db/pkg//xxx >>> files even with pkgng and only use the DB as a way to speed up some work >>> (so >>> the DB corruption just requires to run pkg2ng), or are you lost of the DB >>> is >>> lost? >>> >> >> Nothing is done about DB corruption/loss, I am not sure we need to do >> something. >> Maybe. > > I would say "for sure". Info: In Solaris 10 sqlite is used for the service > managenemt framework (SMF). It is possible that the DB is corrupt in some > bad situations. In this case you have to rebuild the DB (script provided, > been there, had to use it). If sqlite is properly used with transactions, it is very hard to corrupt the database. But if hardware lies to us and say that the data is on disk whereas it isnt... what can we do? Another potential problem is fsync(), but if it is broken on FreeBSD we want to fix it! BTW, the goal is to only have the database and not the flat files. If you are paranoid about power outage, use something like zfs snapshots... > >> Currently a filesystem corruption/loss on /var/db/pkg would do the same. > > Put a corruption of /var/db/pkg/xyz-1/+REQUIRED_BY would only affect a small > part, and this part could be even recovered from (pkgdb from portupgrade is > able to do it). With sqlite we have atomicity! And locks! > >> but it is sqlite so we can perhaps provide a way to get compressed >> dump so user can periodically backup their database. > > It needs to be automated. Maybe periodic daily... but maybe this is not > often enough after a day of a lot of changes (think about it this way: do > you want to lose a day of changes?). The current FS based DB is very robust, > partly because there is redundant data, pertly because losing a file just > means that the very limited subset of information is lost (and a reinstall > of one port will fix it). > > Bye, > Alexander. Regards, Julien ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
Quoting Baptiste Daroussin (from Fri, 25 Mar 2011 15:14:52 +0100): 2011/3/25 Alexander Leidinger : Quoting Baptiste Daroussin (from Fri, 25 Mar 2011 11:11:11 +0100): pkgng is a binary package manager written from scratch for FreeBSD. I didn't had a look at it, just some comments about some parts you explained. features supported are or will be : - a special architecture "all" allows to specify when a package can be used on every architecture. (not done yet) What if a package is able to install on a subset, e.g. the linuxulator ports are for amd64 and i386? No clue for that at the moment but we are open to suggestions. The suggestion is easy, allow a way to specify a set of valid architectures. What about DB corruption/loss? Do you keep the /var/db/pkg//xxx files even with pkgng and only use the DB as a way to speed up some work (so the DB corruption just requires to run pkg2ng), or are you lost of the DB is lost? Nothing is done about DB corruption/loss, I am not sure we need to do something. Maybe. I would say "for sure". Info: In Solaris 10 sqlite is used for the service managenemt framework (SMF). It is possible that the DB is corrupt in some bad situations. In this case you have to rebuild the DB (script provided, been there, had to use it). Currently a filesystem corruption/loss on /var/db/pkg would do the same. Put a corruption of /var/db/pkg/xyz-1/+REQUIRED_BY would only affect a small part, and this part could be even recovered from (pkgdb from portupgrade is able to do it). but it is sqlite so we can perhaps provide a way to get compressed dump so user can periodically backup their database. It needs to be automated. Maybe periodic daily... but maybe this is not often enough after a day of a lot of changes (think about it this way: do you want to lose a day of changes?). The current FS based DB is very robust, partly because there is redundant data, pertly because losing a file just means that the very limited subset of information is lost (and a reinstall of one port will fix it). Bye, Alexander. -- Programming is an unnatural act. http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
Quoting Baptiste Daroussin (from Fri, 25 Mar 2011 11:11:11 +0100): pkgng is a binary package manager written from scratch for FreeBSD. I didn't had a look at it, just some comments about some parts you explained. features supported are or will be : - the register command can analyse elf files when registering a new port to discover forgotten dependencies if necessary. (done in alpha using libelf) This will probably fail if LD_LIBRARY_PATH is used, or if we are installing linuxulator ports. - new +MANIFEST (plist-like format) with new metadatas : options, arch, os version, etc. (done in the alpha) - pkgng supports checking arch of the package which means that users won't be able to install sparc64 binary package into amd64 machines. (not done yet) - a special architecture "all" allows to specify when a package can be used on every architecture. (not done yet) What if a package is able to install on a subset, e.g. the linuxulator ports are for amd64 and i386? In term of technology we decided to use a sqlite3 database, and to prevent potential trolling, sqlite3 is used in it's amalgamation form which means it is incorporated in the code sources (as recommanded by sqlite developpers like a statically linked library) on build we only activate the features we need in sqlite. The alpha release come with an experimental tool "pkg2ng" to convert an existing package database to the new pkgng database format. So one can test pkgng without rebuild all its packages. What about DB corruption/loss? Do you keep the /var/db/pkg//xxx files even with pkgng and only use the DB as a way to speed up some work (so the DB corruption just requires to run pkg2ng), or are you lost of the DB is lost? Bye, Alexander. -- Real computer scientists don't comment their code. The identifiers are so long they can't afford the disk space. http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/25 Alexander Leidinger : > Quoting Baptiste Daroussin (from Fri, 25 Mar 2011 > 11:11:11 +0100): > >> pkgng is a binary package manager written from scratch for FreeBSD. > > I didn't had a look at it, just some comments about some parts you > explained. > >> features supported are or will be : > >> - the register command can analyse elf files when registering a new port >> to >> discover forgotten dependencies if necessary. (done in alpha using libelf) > > This will probably fail if LD_LIBRARY_PATH is used, or if we are installing > linuxulator ports. > this isn't activated by default, and if activated is only intended to work on freebsd elf files. This is done to workaround some bugguy ports not to be used in production, pkg register shows in warning in that case so that user/maintainers are warned they have something to fix. >> >> - a special architecture "all" allows to specify when a package can be >> used >> on every architecture. (not done yet) > > What if a package is able to install on a subset, e.g. the linuxulator ports > are for amd64 and i386? > No clue for that at the moment but we are open to suggestions. > What about DB corruption/loss? Do you keep the /var/db/pkg//xxx > files even with pkgng and only use the DB as a way to speed up some work (so > the DB corruption just requires to run pkg2ng), or are you lost of the DB is > lost? > Nothing is done about DB corruption/loss, I am not sure we need to do something. Maybe. Currently a filesystem corruption/loss on /var/db/pkg would do the same. but it is sqlite so we can perhaps provide a way to get compressed dump so user can periodically backup their database. regards, Bapt ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
