pkgupgrade
Hello, this is to report a revised version of my program, intended to upgrade a machine using mainly precompiled packages. All problems that i have seen or have been reported to me have been fixed. So i think it is reasonably suitable for use, and i have sticked a 1.0 label. I have also fixed all problems i have encountered with save_pkg.py, but Cyrille Szymanski will perform a better cleanup, so i have appended a 0.5 tag. So fresh copies can be downloaded from: http://www.lpthe.jussieu.fr/~talon/pkgupgrade-1.0 http://www.lpthe.jussieu.fr/~talon/save_pkg.py-0.5 They will be accessible as soon as the administrator will have fixed the apache configuration :-( (broken after a crash), or by email from me. This is an example of running it: niobe% ./pkgupgrade There are now 621 packages installed. Collecting installed packages. Building the updated index. Downloading INDEX from ftp server ftp.freebsd.org in directory /pub/FreeBSD/ports/i386/packages-6.2-release INDEX downloading finished. Building and filling the dependency DAG. Printing the upgrade list in UpgradeLog Total time spent in analysis: 01 minutes 20 seconds. Second phase, downloads and backups. Total time spent in downloads: 00 minutes 01 seconds. Total time spent in backups: 00 minutes 20 seconds. Writing upgrade shell script. Will remove 391 old packages. Will install 465 new binary packages. Will compile 7 ports. All tasks completed. ** Total time: 02 minutes 30 seconds. Here of course downloads and backups had been performed in previous runs, except for a small part of backups which are redone. So one can say that 2mn30s is the overhead of the program itself. Out of the binary packages, to be installed, 245 have been found on the FreeBSD-6.2 disk2 and so are not downloaded. Downloads take of the order of 10 minutes on a high speed connection, more on a low speed one, of course, but have to be done anyways. Backups take less time, and are performed during downloads. It is to be noted that, out of 600 installed ports, 200 will not be upgraded, because there is no more recent precompiled version, 400 will be removed and replaced by more recent versions, and only 7 ports will need recompilation. In the present case they are, as shown by the upgrade shell script: TOBUILD='print/pdflib graphics/blender-devel editors/vim emulators/kqemu-kmod devel/py-urwid audio/lame audio/faac' These are indeed not present on the FreeBSD-6.2 repository. Clearly these compilations will take very little time, and the quasi totality of ports will be upgraded through binary packages, as intended, including openoffice which is now found directly in the repository. Hoping that this will help maintaining FreeBSD boxes without spending hours doing it. -- Michel TALON ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
pkgupgrade
Hello, this is to announce version 1.2 of pkgupgrade, a python tool aimed at upgrading freebsd ports installations mainly using binary packages. It has a companion program pkg_save.py by Cyrille Szymanski which performs backups prior to upgrades, which has been upgraded to version 0.7 and a small documentation. You can find them at: http://www.lpthe.jussieu.fr/~talon/pkgupgrade http://www.lpthe.jussieu.fr/~talon/pkg-save.py http://www.lpthe.jussieu.fr/~talon/README.pkgupgrade The main difference with previous version: - for homogeneity, Cyrille likes to call his program pkg_save.py, so i have changed the name accordingly in pkgupgrade. This requires changing the name in /usr/local/sbin. - there is a flag "pkg_release" such that, if True, it works as before, but if False, pkgupgrade will use the Latest packages from pointyhat. - There is a list COMPILE where you can put ports you insist of compiling, instead of using binary packages. This is to cope with the problem of people having special flags in /etc/make.conf for programs they use a lot. - When running UpgradeShell, if some postinstall scripts answer questions, they will not be screened out. There are some user unfriendliness, for example you have to edit the begnning of the script to modify its behavior, you need to put correct origins in the lists HOLD and COMPILE, no globbing has been implemented here, but otherwise the program has been tested on several machines and by several people, and it works. You can expect rather important time gains with respect to using portupgrade -aP to do the same job, and most importantly, you will know beforehand exactly what will be removed and upgraded or compiled, before changing anything on disk. The programs above are under BSD licence, and anyone is welcome to do anything he wants with them, including modifying, improving, rewriting in C++, whatever. Personnally i have no intention to change them in the short term. Perhaps rewrite in C++ later on, if nobody does, and if it appears useful. -- Michel TALON ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: pkgupgrade
On Sat, Feb 10, 2007 at 04:40:28PM +0100, Michel Talon wrote: > Hello, > > this is to report a revised version of my program, intended to upgrade > a machine using mainly precompiled packages. All problems that i have > seen or have been reported to me have been fixed. So i think it is > reasonably suitable for use, and i have sticked a 1.0 label. > I have also fixed all problems i have encountered with save_pkg.py, but > Cyrille Szymanski will perform a better cleanup, so i have appended a > 0.5 tag. So fresh copies can be downloaded from: > http://www.lpthe.jussieu.fr/~talon/pkgupgrade-1.0 > http://www.lpthe.jussieu.fr/~talon/save_pkg.py-0.5 Hi Michel. Good work, this script is pretty cool! One thing that might be nice to add is an option to download new packages from the packages-X-stable directory, if a newer version is found there. I think you only ever look in packages-X.Y-release at the moment, and those never change after the release as far as I know. Best regards, Scott -- === Scott Mitchell | PGP Key ID | "Eagles may soar, but weasels Cambridge, England | 0x54B171B9 | don't get sucked into jet engines" scott at fishballoon.org | 0xAA775B8B | -- Anon ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: pkgupgrade
In <[EMAIL PROTECTED]>, Scott Mitchell <[EMAIL PROTECTED]> typed: > On Sat, Feb 10, 2007 at 04:40:28PM +0100, Michel Talon wrote: > > Hello, > > > > this is to report a revised version of my program, intended to upgrade > > a machine using mainly precompiled packages. All problems that i have > > seen or have been reported to me have been fixed. So i think it is > > reasonably suitable for use, and i have sticked a 1.0 label. > > I have also fixed all problems i have encountered with save_pkg.py, but > > Cyrille Szymanski will perform a better cleanup, so i have appended a > > 0.5 tag. So fresh copies can be downloaded from: > > http://www.lpthe.jussieu.fr/~talon/pkgupgrade-1.0 > > http://www.lpthe.jussieu.fr/~talon/save_pkg.py-0.5 > Good work, this script is pretty cool! One thing that might be nice to add > is an option to download new packages from the packages-X-stable > directory, if a newer version is found there. I think you only ever look > in packages-X.Y-release at the moment, and those never change after the > release as far as I know. While the option might be useful, you may want to build from source in that case. While ports that postdate X.Y are supposed to build and run correctly on X.Y (for supported values of X & Y), and packages built on systems that predate X.Y are supposed to run correctly on X.Y, packages built on systems that postdate X.Y may not run correctly on X.Y. If you're already doing thorough testing of the system after you upgrade the packages, this doesn't really make much difference. But if you're expecting that the testing was done by someone else, your expectations don't match reality. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: pkgupgrade
On Sun, Feb 11, 2007 at 01:31:59PM -0500, Mike Meyer wrote: > In <[EMAIL PROTECTED]>, Scott Mitchell <[EMAIL PROTECTED]> typed: > > Good work, this script is pretty cool! One thing that might be nice to add > > is an option to download new packages from the packages-X-stable > > directory, if a newer version is found there. I think you only ever look > > in packages-X.Y-release at the moment, and those never change after the > > release as far as I know. > > While the option might be useful, you may want to build from source in > that case. While ports that postdate X.Y are supposed to build and run > correctly on X.Y (for supported values of X & Y), and packages built > on systems that predate X.Y are supposed to run correctly on X.Y, > packages built on systems that postdate X.Y may not run correctly on > X.Y. If you're already doing thorough testing of the system after you > upgrade the packages, this doesn't really make much difference. But if > you're expecting that the testing was done by someone else, your > expectations don't match reality. Yup, I know all that. I wouldn't do that kind of upgrade on a server, but on my workstation, with copies of the old packages around just in case, I'm willing to take that (in practice very small) risk, if only to avoid spending the half a day it takes to rebuild KDE from source :( Scott -- === Scott Mitchell | PGP Key ID | "Eagles may soar, but weasels Cambridge, England | 0x54B171B9 | don't get sucked into jet engines" scott at fishballoon.org | 0xAA775B8B | -- Anon ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: pkgupgrade
Dear Michel, On Thu, Mar 15, 2007 at 01:59:25PM +0100, Michel Talon wrote: > this is to announce version 1.2 of pkgupgrade, a python tool aimed at > upgrading freebsd ports installations mainly using binary packages. > It has a companion program pkg_save.py by Cyrille Szymanski which > performs backups prior to upgrades, which has been upgraded to version > 0.7 and a small documentation. You can find them at: > http://www.lpthe.jussieu.fr/~talon/pkgupgrade > http://www.lpthe.jussieu.fr/~talon/pkg-save.py > http://www.lpthe.jussieu.fr/~talon/README.pkgupgrade > The main difference with previous version: > - for homogeneity, Cyrille likes to call his program pkg_save.py, so i > have changed the name accordingly in pkgupgrade. This requires > changing the name in /usr/local/sbin. > - there is a flag "pkg_release" such that, if True, it works as before, > but if False, pkgupgrade will use the Latest packages from pointyhat. > - There is a list COMPILE where you can put ports you insist of > compiling, instead of using binary packages. This is to cope with the > problem of people having special flags in /etc/make.conf for programs > they use a lot. > - When running UpgradeShell, if some postinstall scripts answer > questions, they will not be screened out. > > There are some user unfriendliness, for example you have to edit the > begnning of the script to modify its behavior, you need to put correct > origins in the lists HOLD and COMPILE, no globbing has been implemented > here, but otherwise the program has been tested on several machines and > by several people, and it works. You can expect rather important time > gains with respect to using portupgrade -aP to do the same job, and most > importantly, you will know beforehand exactly what will be removed and > upgraded or compiled, before changing anything on disk. > > The programs above are under BSD licence, and anyone is welcome to do > anything he wants with them, including modifying, improving, rewriting > in C++, whatever. Personnally i have no intention to change them in the > short term. Perhaps rewrite in C++ later on, if nobody does, and if it > appears useful. Thank you for your work. I know that writting a manual page is not the most exciting task in a project. However, I strongly recommend you to write at least a small one and wrap the whole bunch into a port. People will likely be more willing to test it and you will get more feedback, including bug reports and enhancements. Best regards, -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: portupgrade O(n^m)? and pkgupgrade
As a tentative solution to the problem in the subject, i have corrected still more bugs in my program pkguprade. The last version is at http://www.lpthe.jussieu.fr/~talon/pkgupgrade I had observed on my own machine two problems, one being that the topological sort was scrambled, this i have discovered was due to a bad use of the python sort() function (although i don't understand clearly why the syntax was bad), the other had to do with ports being moved in the MOVED files and whose moves were accounted incorrectly. This is a hairy problem with the FreeBSD ports sytem. As an example of running pkgupgrade, here is what i get after having upgraded the port tree from 6.2-RELEASE to current: niobe% ./pkgupgrade There are now 711 packages installed. Collecting installed packages. Building the updated index. Building and filling the dependency DAG. Printing the upgrade list in UpgradeLog Total time spent in analysis: 01 minutes 29 seconds. Second phase, downloads and backups. Total time spent in downloads: 00 minutes 13 seconds. Total time spent in backups: 00 minutes 15 seconds. Writing upgrade shell script. Will remove 3 old packages. Will install 17 new binary packages. Will compile 8 ports. All tasks completed. ** Total time: 01 minutes 53 seconds. At first sight, the shell script which has been produced, UpgradeShell seems reasonable. Of course such a program can only gain very good reliability by testing, but testing is always dangerous ..., except perhaps on testing machines, but i don't have any. -- Michel TALON ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: portupgrade O(n^m)? and pkgupgrade
On Thu, 15 Feb 2007 17:52:29 +0100 Michel Talon <[EMAIL PROTECTED]> wrote: > As a tentative solution to the problem in the subject, i have corrected > still more bugs in my program pkguprade. The last version is at > http://www.lpthe.jussieu.fr/~talon/pkgupgrade > > I had observed on my own machine two problems, one being that the > topological sort was scrambled, this i have discovered was due to a bad > use of the python sort() function (although i don't understand clearly > why the syntax was bad), the other had to do with ports being moved in > the MOVED files and whose moves were accounted incorrectly. This is > a hairy problem with the FreeBSD ports sytem. > > As an example of running pkgupgrade, here is what i get after having > upgraded the port tree from 6.2-RELEASE to current: > niobe% ./pkgupgrade > There are now 711 packages installed. > Collecting installed packages. > Building the updated index. > Building and filling the dependency DAG. > Printing the upgrade list in UpgradeLog > Total time spent in analysis: 01 minutes 29 seconds. > Second phase, downloads and backups. > Total time spent in downloads: 00 minutes 13 seconds. > Total time spent in backups: 00 minutes 15 seconds. > Writing upgrade shell script. > Will remove 3 old packages. > Will install 17 new binary packages. > Will compile 8 ports. > All tasks completed. > ** > Total time: 01 minutes 53 seconds. > At first sight, the shell script which has been produced, UpgradeShell > seems reasonable. Of course such a program can only gain very good > reliability by testing, but testing is always dangerous ..., except > perhaps on testing machines, but i don't have any. You could use a jail for this, although a separate system is always nice. > -- > > Michel TALON > > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
