[CFT] pkgng alpha2
Hi all, We are releasing pkgng (the next pkg_install) alpha2 to the world and we want you to test it! There is no "good" method to test it: use it as you would in the real world. Of course, you are encouraged to backup your data or test it in some kind of virtualized environment. After using it for some time, you will certainly find bugs. You can report them on the issues tracker [1]. If you find missing features, that is things you can't do with pkgng but can with pkg_install, you can also report them. New features are not the expected outcome of this call, as we want to release a final version ASAP. FYI, an alpha3 should follow shortly to fix issues in alpha3 and test additional features. After that, there will be a feature freeze with beta1. Getting started: You can download or git clone the source code of pkgng on the github page [2]. Then, a boring `make' followed by `make install' will do it. If you have some packages installed by pkg_add, you can convert the old database to the pkgng database with the 'pkg2ng' shell script in the ports/ folder. You can also add packages from the ports tree (with bsd.pkgng.mk) or with a pkgng repository. All is documented in the README and manpages. If you are a newcomer to pkgng, this doc reading step is also valuable to us. Indeed, if you fight to get the right infos, or if some things feel counter intuitive, we should improve it! Which brings me to the topic of contributing to pkgng. The best thing you can do is to write down the documentation you would have loved to read while testing pkgng! And of course, if you have a patch with your bug report, it is much appreciated. If you read this entire mail and wonder what is this pkgng thing, you can read the wiki page [3], bapt's presentation from BSDCan [4], EuroBSDCon [5] [6] and browse the source code. Regards, Julien, on behalf of the pkgng team. And remember, we _do_ want to hear back from you! Please also note that it is still alpha code and it can kill kitten and puppies. You are warned ;-) [1] : https://github.com/pkgng/pkgng/issues [2] : https://github.com/pkgng/pkgng [3] : http://wiki.freebsd.org/pkgng [4] : http://people.freebsd.org/~bapt/pkgng-bsdcan2011.pdf [5] : http://wiki.freebsd.org/201110DevSummit/Ports?action=AttachFile&do=get&target=pkgng-devsummit.pdf [6] : http://wiki.freebsd.org/201110DevSummit?action=AttachFile&do=get&target=pkgng-devsummit-track.pdf ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [RFC] rcexecr: rcorder in parallel
On Sat, Jun 4, 2011 at 10:10 AM, Buganini wrote: > https://github.com/buganini/rcexecr > > Currently it is able to determine the exec/wait order > > There are something I haven't digged in deeply in the "self modification" > part. > > patches/ideas are welcome. Hello, Thanks for doing that! You should use kqueue(2) instead of waitpid(2) so that you can efficiently monitor a pool of processes. See pwait(1) for an example. Regards, Julien ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Tue, Mar 29, 2011 at 5:15 AM, Tim Kientzle wrote: > II. Package signing. That would be really nice. >>> >>> Right know we only planned to sign the repo database, so we can trust >>> the sah256 of the packages stored in the database. Then if the package >>> has the same sha256 as the one in the repo database it is considered >>> trusted. >>> If we want a per-package signing, we would have a tarball in a tarball. >> >> I really expected this to have been mentioned already, but this approach >> (tarball in a tarball) is taken by Debian packages, and I don't remember >> hearing of any issues related to it. I don't think it's worth discounting >> from the start without giving some considerationg, but I will defer to the >> people actually doing the work. > > If you use libarchive-style streaming, it's even > pretty straightforward to read and extract such > things without having to create a bunch of > temporary files. > > You just need to be careful about compression. Agreed, if we dont want to verify the signature, we can extract the tarball in the tarball efficiently. But to verify the signature, we have to read the tarball in the tarball twice: the first time to compute the digest and verify the signature, the second time to do the real extraction. So I guess that the tarball containing the real package archive and the signature should be uncompressed. The real package archive would be compressed, though. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper wrote: > On Mon, Mar 28, 2011 at 10:44 AM, Andriy Gapon wrote: >> on 25/03/2011 12:11 Baptiste Daroussin said the following: >>> Hi all, >>> >>> miwi@ launched the new thing called Experimental Call For Testing, >>> it's our turn :) >>> >>> Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge >>> contributor) have been working since the end of the last GSoC on a >>> rewrite of pkg_install. >>> >>> pkgng is a binary package manager written from scratch for FreeBSD. >>> >>> After a long period of technology testing, (json, tinycdb, bdb, etc) >>> and we now have achieved to implement the basic functionnality. We >>> would greatly approciate to have some feedback, wider testing, >>> patching, documenting etc, before implementing the higher level >>> features. >>> >>> pkgng is built on top of a new libpkg, which allow to deal with the >>> database of >>> installed packages, to deal with remote repositories, manage packages: >>> creation, installation gathering informations, registering new ports. >>> >>> features supported are or will be : >>> >>> - smooth integration with bsd.port.mk (including bsd.pkg.mk line 2486) >>> which allow to have a bsd.port.mk which deal with both pkg_install >>> and pkgng. (done in alpha) >>> >>> - the register command can analyse elf files when registering a new port to >>> discover forgotten dependencies if necessary. (done in alpha using libelf) >>> >>> - the register command has two mode available : when dealing with old >>> fashion ports it just registers the package, in new mode it does >>> everything that would >>> have been done by pkg add when installing the package : should display >>> messages, execute post-install, execute @exec etc. (old fashion done >>> in the alpha) >>> >>> - pkg add supports two mode : the old fashion one (no real upgrade >>> support) and new one: upgrade scripts supported. (old fashion in the >>> alpha) >>> >>> - new scripts supported +PREINSTALL +POSTINSTALL, +PREDEINSTALL, >>> +POSTDEINSTALL, +PREUPGRADE, +POSTUPGRADE as well as the old fashion >>> scripts : +INSTALL +DEINSTALL +UPGRADE (all supported *UPGRADES aren't >>> supported in the alpha) >>> >>> - new +MANIFEST (plist-like format) with new metadatas : options, arch, os >>> version, etc. (done in the alpha) >>> >>> - pkgng supports checking arch of the package which means that users >>> won't be able to install sparc64 binary package into amd64 machines. >>> (not done yet) >>> >>> - a special architecture "all" allows to specify when a package can be used >>> on every architecture. (not done yet) >>> >>> - @dirrm and @dirrmtry are now deprecated, pkgng can discover itself >>> which directory has to be removed. (done in the alpha but needs love >>> :)) >>> >>> - new repository (apt-like feature) (only the repository generation is done) >>> >>> - real support for reverse dependency (no ugly +REQUIRED_BY) (done in the >>> alpha) >>> >>> - test unit (libcheck) on libpkg. (done in the alpha needs some more love) >>> >>> - many more >> >> Perhaps I am too impatient :) but I would like to inquire about the following >> features: >> >> I. A provides/requires interface for packages. >> Each package specified a list of files (and perhaps other entities) that it >> provides and requires. At the initial stage, without ports modifications, >> these >> could be: (1) a list of all files installed by package for provides; (2) for >> requires - an auto-generated list of dependencies based on required shared >> libraries plus dependency specifications in ports. >> I think that this kind of interface should help with using alternatives that >> provide the same interface (e.g. like gamin vs fam). Adding require/provide support in pkgng is kind of trivial. But ports have to support it first. >> >> II. Package signing. > > That would be really nice. Right know we only planned to sign the repo database, so we can trust the sah256 of the packages stored in the database. Then if the package has the same sha256 as the one in the repo database it is considered trusted. If we want a per-package signing, we would have a tarball in a tarball. > >> III. Package naming that includes architecture, majo
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Fri, Mar 25, 2011 at 10:11 AM, Baptiste Daroussin wrote: > Developpement site: http://git.etoilebsd.net/pkgng/ FYI, we moved to github[1] in order to have a bug tracker, pull request and code review. Also, I recommend to build from the HEAD of the git repository, to not report fixed compilation warnings/bugs. [1]: https://github.com/pkgng/pkgng ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
On Fri, Mar 25, 2011 at 2:38 PM, Alexander Leidinger wrote: > Quoting Baptiste Daroussin (from Fri, 25 Mar 2011 > 15:14:52 +0100): > >> 2011/3/25 Alexander Leidinger : >>> >>> Quoting Baptiste Daroussin (from Fri, 25 Mar 2011 >>> 11:11:11 +0100): >>> pkgng is a binary package manager written from scratch for FreeBSD. >>> >>> I didn't had a look at it, just some comments about some parts you >>> explained. >>> features supported are or will be : > - a special architecture "all" allows to specify when a package can be used on every architecture. (not done yet) >>> >>> What if a package is able to install on a subset, e.g. the linuxulator >>> ports >>> are for amd64 and i386? >>> >> >> No clue for that at the moment but we are open to suggestions. > > The suggestion is easy, allow a way to specify a set of valid architectures. That looks reasonable and easy to implement. > >>> What about DB corruption/loss? Do you keep the /var/db/pkg//xxx >>> files even with pkgng and only use the DB as a way to speed up some work >>> (so >>> the DB corruption just requires to run pkg2ng), or are you lost of the DB >>> is >>> lost? >>> >> >> Nothing is done about DB corruption/loss, I am not sure we need to do >> something. >> Maybe. > > I would say "for sure". Info: In Solaris 10 sqlite is used for the service > managenemt framework (SMF). It is possible that the DB is corrupt in some > bad situations. In this case you have to rebuild the DB (script provided, > been there, had to use it). If sqlite is properly used with transactions, it is very hard to corrupt the database. But if hardware lies to us and say that the data is on disk whereas it isnt... what can we do? Another potential problem is fsync(), but if it is broken on FreeBSD we want to fix it! BTW, the goal is to only have the database and not the flat files. If you are paranoid about power outage, use something like zfs snapshots... > >> Currently a filesystem corruption/loss on /var/db/pkg would do the same. > > Put a corruption of /var/db/pkg/xyz-1/+REQUIRED_BY would only affect a small > part, and this part could be even recovered from (pkgdb from portupgrade is > able to do it). With sqlite we have atomicity! And locks! > >> but it is sqlite so we can perhaps provide a way to get compressed >> dump so user can periodically backup their database. > > It needs to be automated. Maybe periodic daily... but maybe this is not > often enough after a day of a lot of changes (think about it this way: do > you want to lose a day of changes?). The current FS based DB is very robust, > partly because there is redundant data, pertly because losing a file just > means that the very limited subset of information is lost (and a reinstall > of one port will fix it). > > Bye, > Alexander. Regards, Julien ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: txt-sysinstall scrapped
On Sun, Nov 7, 2010 at 10:49 AM, Joel Dahl wrote: > On 07-11-2010 11:40, Daniel Gerzo wrote: >> On 6.11.2010 6:17, Josh Paetzel wrote: >> > Without putting much analysis into it, we talked about using lighttpd, >> > which >> > is BSDL. As far as another service, it would be running for the install >> > only >> > which is in most circumstances something that happens locally. >> > >> >> NGiNX might a worth a shoot as well. Seems like the development of >> lighty has been somewhat "stalled", and Igor Sysoev seems like a FreeBSD >> user himself too... > > I was about to suggest the same thing myself. I use nginx everywhere > nowadays. It's both small and powerful, easy to setup and BSD licensed. > Mongoose [1] is a very small HTTP server. It is designed to be embedded in C programs. Basically it is a lib, and a daemon. I think it would perfectly replace Lighttpd/Nginx/Apache for the tasks described. [1]: http://code.google.com/p/mongoose/ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: GSoC:Complete Package support in the pkg_install tools and cleanup
On Wed, May 5, 2010 at 7:34 PM, Wesley Shields wrote: > On Tue, May 04, 2010 at 10:30:13AM -0700, Julien Laffaye wrote: >> On Tue, May 4, 2010 at 3:15 AM, Andrew Brampton >> wrote: >> > >> > Hi Julien, >> > >> > Glad you got onto the GSoC programme. I'm curious, what benefit is a >> > complete package over many individual ones? >> >> Hi Andrew, >> >> If you cant or dont want to use the remote feature of of pkg_add (ex. >> your packages are built with non default options and you think its >> overkill to setup a server to distribute them) then you make a >> complete package. You only have to copy one file (say on an usb >> device), which is less error prone than 150 files. >> The global idea is to write a meta port which depends on the desired >> ports, type `make complete-package`, copy the output file on the >> machine to bootstrap, pkg_add /path/to/complete-pkg and voila! > > Do you intend to add the "complete-package" target also or is that out > of scope? > Yes, it is scheduled to add the ability to build complete packages from the ports tree. The name of the target is not yet endorsed but I think that "complete-package" is pretty straight forward. Regards, Julien ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: GSoC:Complete Package support in the pkg_install tools and cleanup
On Tue, May 4, 2010 at 3:15 AM, Andrew Brampton wrote: > > Hi Julien, > > Glad you got onto the GSoC programme. I'm curious, what benefit is a > complete package over many individual ones? Hi Andrew, If you cant or dont want to use the remote feature of of pkg_add (ex. your packages are built with non default options and you think its overkill to setup a server to distribute them) then you make a complete package. You only have to copy one file (say on an usb device), which is less error prone than 150 files. The global idea is to write a meta port which depends on the desired ports, type `make complete-package`, copy the output file on the machine to bootstrap, pkg_add /path/to/complete-pkg and voila! Regards, Julien ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
GSoC:Complete Package support in the pkg_install tools and cleanup
Hello, During this summer, I'll work on the pkg_install tools to add complete package support. A complete package is a package which includes all the required dependencies in its tarball. Unlike the PBI package format of PC-BSD, once a complete package is installed, it appears as every package contained in the complete package was installed separately. As a side effect, I'll add libarchive support to the pkg_add tool to make installation as efficient as possible and do some refactoring to allow code re usability. I'll also surely write automated tests which would benefit to the pkg_install tools. My page on the wiki is http://wiki.freebsd.org/SOC2010JulienLaffaye You can email me on or off list is you have any questions, comments or suggestions. Best regards, Julien Laffaye ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: SoC2009: libpkg, pkg tools rewrite
On Sat, Apr 25, 2009 at 9:20 PM, David Forsythe wrote: > Hi, > I'm David Forsythe, 3rd year student at the University of Maryland, College > Park. For SoC2008 I worked on added parallel build support and database > locks to ports. I've been using FreeBSD for a while, and have taken > particular interest in ports and packages. > This summer I'll be working on creating a package library and using that > library to rewrite the pkg tools. A package library has been discussed and > even started before, but FreeBSD still does not have one. This summer I'd > like to get enough of the library done to atleast have a new set of pkg > tools completed with the current features, but ideally I'd like to get far > enough to splice in some of the ideas I have for new features. > Here's the wiki page: http://wiki.freebsd.org/SoC2009DavidForsythe > Dave Hi, It'll be nice if libpkg can deal with all the infos in the INDEX with an elegant API. You have to read the packages names in the INDEX for `pkg_version -I` but maybe other tools will enjoy to get other infos (especially the one line description, the categories and the run time dependencies). So basically, an API which understand "give me all the packages names you have in the INDEX", "give me the description for _this_ package", "give me all the names plus the categories", "give me..." Regards, Julien Laffaye ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
