Re: How did upgrading applications happen before portupgrade etc?
Quoting RW [EMAIL PROTECTED] (Tue, 14 Aug 2007 15:35:15 +0100): On Sun, 12 Aug 2007 14:20:59 +0200 Alexander Leidinger [EMAIL PROTECTED] wrote: Quoting RW [EMAIL PROTECTED] (Sat, 11 Aug 2007 22:58:58 +0100): Actually having dependencies package version mismatches needn't cause any significant problems. It does cause problems. You have no +REQUIRED_BY file anymore, so pkg_delete allows you to remove it without a warning even if it is still needed. I didn't say it doesn't, I said it needn't. When I have to remove a It will. No doubts about this, sorry. It may be ok for some hobbyist installation, but in the generic case you want to do it right (and the OP asked for the generic case which includes production quality ports management). ... It's not about removing a port when updating. It's about correct leaf ports detection which may be necessary from time to time. Detecting leaves by +REQUIRED_BY alone is unreliable unless you can guarantee that the entries have *always* been updated to reflect the installed packages - by everybody that worked on the server. +REQUIRED_BY is not only used in leaf ports detection, but also to prevent unwanted removal of ports which other ports depend upon (there's some implicit dependency here, yes). And as the file is handled by pkg_* programs and not by hand, it is a procedural error (or a bug in our programs) if it is not updated correctly. It's more forgiving to work downwards through the dependencies to find leaves,and fairly easy to script. Are you sure? I would say it is more easy to do it right in the beginning. To be able to do it your way you need to update the +CONTENT files to point to the updated version instead to the old version. When you do this, it is not much more work to keep the +REQUIRED_BY file up-to-date. doesn't rely on +REQUIRED_BY, and provides additional protection against build-dependency deletion which you don't get from pkg_delete. Sidenote: I also don't see a benefit when pruning to keep the obscure build dependencies of a port you don't want anymore. Just like there's no point in keeping the obscure run dependencies of a port you don't want anymore. Leaf deletion is an iterative I agree. process, when you delete one leaf you may create more. I agree. Portmanager -slid prompts you to delete a build dependency when the last package that needs it for a rebuild is removed. Nice. Bye, Alexander. -- Rule of Life #1 -- Never get separated from your luggage. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
Quoting RW [EMAIL PROTECTED] (from Mon, 13 Aug 2007 12:38:37 +0100): On Mon, 13 Aug 2007 08:14:46 +0200 Alexander Leidinger [EMAIL PROTECTED] wrote: Quoting RW [EMAIL PROTECTED] (from Sun, 12 Aug 2007 18:18:10 +0100): On Sun, 12 Aug 2007 14:20:59 +0200 Alexander Leidinger [EMAIL PROTECTED] wrote: Quoting RW [EMAIL PROTECTED] (Sat, 11 Aug 2007 22:58:58 +0100): Actually having dependencies package version mismatches needn't cause any significant problems. It does cause problems. You have no +REQUIRED_BY file anymore, so pkg_delete allows you to remove it without a warning even if it is still needed. I didn't say it doesn't, I said it needn't. When I have to remove a It will. No doubts about this, sorry. It may be ok for some hobbyist installation, but in the generic case you want to do it right (and the OP asked for the generic case which includes production quality ports management). package manually as part of an upgrade, I use pkg_delete -f, so it doesn't matter. When I prune ports I use portmanager -slid which It's not about removing a port when updating. It's about correct leaf ports detection which may be necessary from time to time. On a busy (as in ports are (de-)installed often) system this may be needed more often than on a idle system. doesn't rely on +REQUIRED_BY, and provides additional protection against build-dependency deletion which you don't get from pkg_delete. portmanager ist very recent, it is even not as old as portupgrade. So a discussion about what you do with current tools does not fit to the topic of the original question which started this thread. Sidenote: I also don't see a benefit when pruning to keep the obscure build dependencies of a port you don't want anymore. In the good old days(TM) you searched for ports without a +REQUYIRED_BY field. Those are the leaf ports. And then you used your brain to decide what to remove or not. What we didn't had was an easy procedure for pruning old distfiles. I was excited when I discovered this feature in portupgrade. Bye, Alexander. -- Marriage is a lot like the army, everyone complains, but you'd be surprised at the large number that re-enlist. -- James Garner http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
On Sun, 12 Aug 2007 14:20:59 +0200 Alexander Leidinger [EMAIL PROTECTED] wrote: Quoting RW [EMAIL PROTECTED] (Sat, 11 Aug 2007 22:58:58 +0100): Actually having dependencies package version mismatches needn't cause any significant problems. It does cause problems. You have no +REQUIRED_BY file anymore, so pkg_delete allows you to remove it without a warning even if it is still needed. I didn't say it doesn't, I said it needn't. When I have to remove a It will. No doubts about this, sorry. It may be ok for some hobbyist installation, but in the generic case you want to do it right (and the OP asked for the generic case which includes production quality ports management). ... It's not about removing a port when updating. It's about correct leaf ports detection which may be necessary from time to time. Detecting leaves by +REQUIRED_BY alone is unreliable unless you can guarantee that the entries have *always* been updated to reflect the installed packages - by everybody that worked on the server. It's more forgiving to work downwards through the dependencies to find leaves,and fairly easy to script. doesn't rely on +REQUIRED_BY, and provides additional protection against build-dependency deletion which you don't get from pkg_delete. Sidenote: I also don't see a benefit when pruning to keep the obscure build dependencies of a port you don't want anymore. Just like there's no point in keeping the obscure run dependencies of a port you don't want anymore. Leaf deletion is an iterative process, when you delete one leaf you may create more. Portmanager -slid prompts you to delete a build dependency when the last package that needs it for a rebuild is removed. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
Quoting RW [EMAIL PROTECTED] (from Sun, 12 Aug 2007 18:18:10 +0100): On Sun, 12 Aug 2007 14:20:59 +0200 Alexander Leidinger [EMAIL PROTECTED] wrote: Quoting RW [EMAIL PROTECTED] (Sat, 11 Aug 2007 22:58:58 +0100): On Sat, 11 Aug 2007 13:33:22 -0700 Jeremy Chadwick [EMAIL PROTECTED] wrote: On Sat, Aug 11, 2007 at 03:02:53PM +0400, Rakhesh Sasidharan wrote: 5. pkg_delete port I see. In step 5, pkg_delete port wont work if port is required by others right? So you delete those apps too? Could be a lot of stuff to uninstall, right? Absolutely correct. That might seem like a nightmare to most people, but to me it's not. It's not correct, pkg_delete -f can force the deletion. I would manually upgrade a port like this: cd /usr/ports/misc/foo make ; do the build pkg_info -qO misc/foo ; get old package name pkg_create -b old-package-name ; backup existing package /usr/local/etc/rc.d/foo stop ; stop the daemon if needed pkg_delete -f old-package-name ; force removal make install At this point your /var/db/pkg/ directory does not reflect reality anymore, I know, I was just pointing out that it is possible to upgrade a port manually without removing every single package that depends on it. Actually having dependencies package version mismatches needn't cause any significant problems. And massaging them into self-consistency is It does cause problems. You have no +REQUIRED_BY file anymore, so pkg_delete allows you to remove it without a warning even if it is still needed. If you make sure to keep the file, you still need to change the dependencies (in +CONTENTS and in +REQUIRED_BY), else the entries in +REQUIRED_BY are not removed on deinstall of a port. itself a form of corruption, since you lose information about what was built against what. No, it's not a corruption as long as the changes are ABI compatible (if it isn't compatible, you need to update some more ports, but this is not the subject of the discussion). The dependency tracking in /var/db is meant to reflect the current dependencies of the installed software, not the dependencies when a port was first build. It doesn't matter for e.g. a viruschecker if rar is at version 1.1 or 1.2, as long as the command line options used still have the same syntax and semantic. Bye, Alexander. -- I'll burn my books. -- Christopher Marlowe http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
On Mon, 13 Aug 2007 08:14:46 +0200 Alexander Leidinger [EMAIL PROTECTED] wrote: Quoting RW [EMAIL PROTECTED] (from Sun, 12 Aug 2007 18:18:10 +0100): On Sun, 12 Aug 2007 14:20:59 +0200 Alexander Leidinger [EMAIL PROTECTED] wrote: Quoting RW [EMAIL PROTECTED] (Sat, 11 Aug 2007 22:58:58 +0100): Actually having dependencies package version mismatches needn't cause any significant problems. It does cause problems. You have no +REQUIRED_BY file anymore, so pkg_delete allows you to remove it without a warning even if it is still needed. I didn't say it doesn't, I said it needn't. When I have to remove a package manually as part of an upgrade, I use pkg_delete -f, so it doesn't matter. When I prune ports I use portmanager -slid which doesn't rely on +REQUIRED_BY, and provides additional protection against build-dependency deletion which you don't get from pkg_delete. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
On Sat, Aug 11, 2007 at 02:07:17PM -0700 I heard the voice of Jeremy Chadwick, and lo! it spake thus: I'm aware it manages all of the regular /var/db/pkg files just like the pkg_* tools do. It has to for it to play nice. But it also maintains a separate database for (I think) versioning and dependency matching. I've been told the reason it does this is due to shortcomings in /var/db/pkg or the pkg_* tools. As far as I can tell, there's nothing in the DB files that isn't in the /var/db/pkg files (or at least derivable from them), as I've never seen anything unusual happen when I blow away the .db files. About the only times I've ever seen them go wonky is around a bdb/ruby/portupgrade upgrade. Sure, it'd be nice if that didn't happen, but if that's the biggest flaw I have to contend with, I'm buying a lottery ticket in the morning. Also, if you use portupgrade, you pretty much HAVE to use it exclusively for everything, e.g. you can't go mix-and-matching use of portupgrade and the base pkg_* tools without there being some inconsistency induced and thus breaking portupgrade. Am I correct? I sure hope not, or all these years of installing ports manually is has been biting me without me knowing it. I almost never use portinstall (xref slow). I use pkg_delete to remove stuff. And I upgrade stuff manually with some regularity (like portupgrade, just for my own sanity, and ruby upgrade usually involve me doing a mass pkg_delete and reinstall. AFAIK, those would work just fine with portupgrade, but I like the net). All that happens is that portupgrade notices its db is out of date, and updates it. Where the only solution is to rm the pkgdb.db file and then run pkgdb -u, correct? That class of things, yes. It's happened to me in ways I couldn't attribute to upgrades, but I could count the number of times on one hand and have fingers left over. But if someone asks me Why don't you use portupgrade I'm going to tell them why. Well, that's why I asked 8-} I don't develop portupgrade, and I don't get paid for people using it (I do accept certified checks, however), and I certainly wish that many of its long-time shortcomings would get cleaned up. But this thing you hold up as constant dealbreakers, my experience slots as minor, trivially deal-with-able, and extremely rare hiccups. I see claims like the above (you just nuke the db and move on. What's the problem?) and I think What the f***?. Removing a file and ignoring the problem because the problem then goes away (until the next time it happens...) makes me think of a Windows-like solution. I can run a 10 line shell script and lock a filesystem up so it's dirty and can't be unmounted. I can upgrade a system and make it lose the ability to see its hard drives. I think What the f***?. Who would use this FreeBSD crap? That sort of thing makes me think of Windows. Don't upgrade your system, and treat your filesystems with kid gloves, and the problem goes away. Both of these have been reported, multiple times over time, and they're still there. In fact, neither of them has that I recall even elicited a single response from anybody. *THAT* sure as hell makes me uncomfortable. I managed systems manually for years before portupgrade came along (and for that matter, for years after, because I don't like changing methods that works, however badly). It was miserable, and everything was always out of date. Portupgrade turned that into very minor effort to keep everything always up to date (or firmly out of date, if that's my intention for something). A peripheral and totally optional tool that very occasionally ends up with a broken cache that takes a minute to rebuild, and otherwise turns that part of my admin life from a living hell to a blissful joy, is a long way down my list of things to worry about. -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
Quoting RW [EMAIL PROTECTED] (Sat, 11 Aug 2007 22:58:58 +0100): On Sat, 11 Aug 2007 13:33:22 -0700 Jeremy Chadwick [EMAIL PROTECTED] wrote: On Sat, Aug 11, 2007 at 03:02:53PM +0400, Rakhesh Sasidharan wrote: 5. pkg_delete port I see. In step 5, pkg_delete port wont work if port is required by others right? So you delete those apps too? Could be a lot of stuff to uninstall, right? Absolutely correct. That might seem like a nightmare to most people, but to me it's not. It's not correct, pkg_delete -f can force the deletion. I would manually upgrade a port like this: cd /usr/ports/misc/foo make ; do the build pkg_info -qO misc/foo ; get old package name pkg_create -b old-package-name ; backup existing package /usr/local/etc/rc.d/foo stop ; stop the daemon if needed pkg_delete -f old-package-name ; force removal make install At this point your /var/db/pkg/ directory does not reflect reality anymore, e.g. all ports which depend upon this port still list the old version of it. Before I used portupgrade I had some scripts which dealed with parts of this problem. You also lose the +REQUIRED_BY file of the updated port. My scripts solved the urgent dependency problems. When portupgrade came along and was stable it had all those nice features I would have been happy to see in my own scripts but never had the pressure to implement because they where not that important for me. I also have to do less work by hand when using portupgrade. It may be not the best implementation for such a tool, but it's a huge step forward compared to what we had before. /usr/local/etc/rc.d/foo start ; start the daemon if needed make clean rm back-up package Bye, Alexander. -- Change is the essential process of all existence. -- Spock, Let That Be Your Last Battlefield, stardate 5730.2 http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
On Sun, 12 Aug 2007 14:20:59 +0200 Alexander Leidinger [EMAIL PROTECTED] wrote: Quoting RW [EMAIL PROTECTED] (Sat, 11 Aug 2007 22:58:58 +0100): On Sat, 11 Aug 2007 13:33:22 -0700 Jeremy Chadwick [EMAIL PROTECTED] wrote: On Sat, Aug 11, 2007 at 03:02:53PM +0400, Rakhesh Sasidharan wrote: 5. pkg_delete port I see. In step 5, pkg_delete port wont work if port is required by others right? So you delete those apps too? Could be a lot of stuff to uninstall, right? Absolutely correct. That might seem like a nightmare to most people, but to me it's not. It's not correct, pkg_delete -f can force the deletion. I would manually upgrade a port like this: cd /usr/ports/misc/foo make ; do the build pkg_info -qO misc/foo ; get old package name pkg_create -b old-package-name ; backup existing package /usr/local/etc/rc.d/foo stop ; stop the daemon if needed pkg_delete -f old-package-name ; force removal make install At this point your /var/db/pkg/ directory does not reflect reality anymore, I know, I was just pointing out that it is possible to upgrade a port manually without removing every single package that depends on it. Actually having dependencies package version mismatches needn't cause any significant problems. And massaging them into self-consistency is itself a form of corruption, since you lose information about what was built against what. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
How did upgrading applications happen before portupgrade etc?
Hi, Just a question that struck me today. Before there were the portupgrade and other tools for upgrading installed applications to their newer versions, how did things work out? Did one upgrade applications through a series of make deinstall reinstall commands (I wonder if these commands take care of dependencies too) or was there any other way? I happened to be reading up on how pkgsrc packages are upgraded. And it seemed to be sooo complicated. (See this link for some info: http://wiki.netbsd.se/How_to_upgrade_packages). The ports system never appeared so complicated to me, which is when it struck to inquire how things were before portupgrade and other wonderful third party tools came into place ... Thanks, Rakhesh ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
On Sat, Aug 11, 2007 at 12:03:33PM +0400, Rakhesh Sasidharan wrote: Just a question that struck me today. Before there were the portupgrade and other tools for upgrading installed applications to their newer versions, how did things work out? Did one upgrade applications through a series of make deinstall reinstall commands (I wonder if these commands take care of dependencies too) or was there any other way? Along those lines, yes. I'm one of those administrators who does not want something like portupgrade on his systems; I do not believe in having two separate databases maintaining dependencies and what's installed (portupgrade vs. base system pkg_* tools). Lots of people love portupgrade, and that's great. But the number of times I have seen reports of database corruption or database sync mismatches (portupgrade thinks A, while pkg_* thinks B) is astounding. There are script-based tools now such as Doug Barton's portmaster which I haven't used or looked into yet, but the fact that they rely on the stock base pkg_* tools is fantastic. Those are something I'm open to. Anyways, opinions aside and to answer your question -- the procedure I go through is something along these lines: 1. Update ports tree (cd /usr/ports make update) 2. pkg_version -v and look for stuff that's reported outdated or Comparison failed (likely the port relocated to another section) 3. Examine the CVS commit log for the port in question. A lot of the time I'll find that the upgrade simply isn't worth it, so no harm done in leaving it outdated. 4. Shut down services/daemons which might be affected by below steps. 5. pkg_delete port 6. cd /usr/ports/whatever make clean 7. make config, make make install 8. Make sure configuration file symlinks didn't get nuked. 9. Restart services/daemons. 10. Pay close attention to /var/log/all.log for quite some time. :-) Every 4-6 months, I go through the below process. I'll also do this when there's something major in the dependency tree which gets updated that lots of things depend upon. The most common examples are gettext, libiconv, libtool, and the automake/autoconf tools. 1. Do all of above steps 2. Make a backup of /usr/local (usually using rsync) 3. pkg_version or pkg_info and save a list of what's installed. 4. pkg_delete -a -f 5. rm -fr /usr/local/* 6. rm -fr /var/db/ports/* 7. Go through the process of manually: cd /usr/ports/whatever make clean make config Select applicable options pertaining to our setup make make install Or for things I don't want to build (like perl and python): pkg_add -r perl (or whatever) 8. Re-add symlinks for applicable configuration files, etc. 9. Restart services/daemons. 10. Pay close attention to /var/log/all.log for quite some time. :-) ...I think I got those right. I might be missing a step or two. Note that we keep all of our configuration files in a directory called /conf/ME and in /usr/local simply use symlinks. (Matt Dillon might be grinning over /conf/ME, since it's a Best Internet-ism :) ). I don't trust that all ports will play nice with existing configuration files in /usr/local, and I have seen much evidence of this in the past (nuking files without your knowledge, mainly. Oh crap! It nuked our entire configuration for the apache/mail/whatever server!!!). In the case that I can't, I might ask on #bsdports, or mailing lists. Hope this gives you some insight into the mind of one admin who prefers to do things the old way rather than let automated tools try and emulate my brain. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
Kris Kennaway wrote: On Sat, Aug 11, 2007 at 12:03:33PM +0400, Rakhesh Sasidharan wrote: Hi, Just a question that struck me today. Before there were the portupgrade and other tools for upgrading installed applications to their newer versions, how did things work out? Did one upgrade applications through a series of make deinstall reinstall commands (I wonder if these commands take care of dependencies too) or was there any other way? Basically that, plus a lot of other manual steps that were always somewhat hard to get exactly right but which more or less worked back in the days when Gnomes lived in your garden and Java was a place on the map. It was a simpler, more innocent age. There was pkg_version -c that printed a sequence of cd /usr/ports/foo/bar make make deinstall make install make clean or something like that. Whatever broke was fixed manually afterwards :-) Oh, and there was no UPDATING in /usr/ports/ as well I think. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
On Sat, Aug 11, 2007 at 01:33:57AM -0700 I heard the voice of Jeremy Chadwick, and lo! it spake thus: I'm one of those administrators who does not want something like portupgrade on his systems; I do not believe in having two separate databases maintaining dependencies and what's installed (portupgrade vs. base system pkg_* tools). Lots of people love portupgrade, and that's great. But the number of times I have seen reports of database corruption or database sync mismatches (portupgrade thinks A, while pkg_* thinks B) is astounding. I'm always a little surprised at statements like this... portupgrade manages all the regular /var/db/pkg/*/* files just like any other tool. It's got its db files, sure, but they're caches, not alternate masters. I've never seen a sync mismatch (or rather, I see them all the time, when portupgrade sees that the source is newer than its cache and updates the cache). I've seen them go wonky a time or two when I've upgrade bdb or ruby-bdb, but so what? You blow 'em away and recreate them. I blow 'em away every time I upgrade ruby or ruby-bdb or bdb just out of GP to head off potential troubles. With the growing number of installed ports, rebuilding the pkgdb.db files takes a long time, but it's what, a minute? Minute and a half? There are certainly reasons to dislike portupgrade (like that it's slow. Godawful long. Where's-War-And-Peace-I-need-something-to-read slow. It's almost as slow as yum is on a machine twice as fast), but I don't understand this one. The db's dont go wonky with any notable regularity IME, and when they do you just rm 'em and move on. What's the big deal? -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
Jeremy Chadwick wrote: Along those lines, yes. I'm one of those administrators who does not want something like portupgrade on his systems; I do not believe in having two separate databases maintaining dependencies and what's installed (portupgrade vs. base system pkg_* tools). Lots of people love portupgrade, and that's great. But the number of times I have seen reports of database corruption or database sync mismatches (portupgrade thinks A, while pkg_* thinks B) is astounding. Actually, doesn't portupgrade use the same databases as the base tools? It just creates other databases based on these existing ones, for faster lookups etc, but they are expendable in case of problems afaik. Just delete and it will get recreated again ... (I am new to all this, so I could be wrong in some way I don't know; this is my understanding). There are script-based tools now such as Doug Barton's portmaster which I haven't used or looked into yet, but the fact that they rely on the stock base pkg_* tools is fantastic. Those are something I'm open to. Afaik portmaster's beauty is that it doesn't have any dependencies like portuprade does. Doesn't require ruby etc, so upgrades to them don't cause problems like they could with portupgrade. (Again, haven't used portmaster, so I don't know; this is my understanding). Anyways, opinions aside and to answer your question -- the procedure I go through is something along these lines: 1. Update ports tree (cd /usr/ports make update) 2. pkg_version -v and look for stuff that's reported outdated or Comparison failed (likely the port relocated to another section) 3. Examine the CVS commit log for the port in question. A lot of the time I'll find that the upgrade simply isn't worth it, so no harm done in leaving it outdated. 4. Shut down services/daemons which might be affected by below steps. 5. pkg_delete port 6. cd /usr/ports/whatever make clean 7. make config, make make install 8. Make sure configuration file symlinks didn't get nuked. 9. Restart services/daemons. 10. Pay close attention to /var/log/all.log for quite some time. :-) I see. In step 5, pkg_delete port wont work if port is required by others right? So you delete those apps too? Could be a lot of stuff to uninstall, right? Note that we keep all of our configuration files in a directory called /conf/ME and in /usr/local simply use symlinks. (Matt Dillon might be grinning over /conf/ME, since it's a Best Internet-ism :) ). I don't trust that all ports will play nice with existing configuration files in /usr/local, and I have seen much evidence of this in the past (nuking files without your knowledge, mainly. Oh crap! It nuked our entire configuration for the apache/mail/whatever server!!!). That's a smashing idea! Thanks for mentioning that here. :-) What does the ME stand for in /conf/ME btw? Thanks for sharing your traditional steps with here btw. Regards, Rakhesh ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
On Sat, Aug 11, 2007 at 11:37:22AM +0200, Tobias Roth wrote: Kris Kennaway wrote: On Sat, Aug 11, 2007 at 12:03:33PM +0400, Rakhesh Sasidharan wrote: Hi, Just a question that struck me today. Before there were the portupgrade and other tools for upgrading installed applications to their newer versions, how did things work out? Did one upgrade applications through a series of make deinstall reinstall commands (I wonder if these commands take care of dependencies too) or was there any other way? Basically that, plus a lot of other manual steps that were always somewhat hard to get exactly right but which more or less worked back in the days when Gnomes lived in your garden and Java was a place on the map. It was a simpler, more innocent age. There was pkg_version -c that printed a sequence of cd /usr/ports/foo/bar make make deinstall make install make clean or something like that. Whatever broke was fixed manually afterwards :-) Oh, and there was no UPDATING in /usr/ports/ as well I think. Old-timers will tell you that pkg_version is a new-fangled invention, and back in the day they had to slave for hours over a hot keyboard to run all those make commands by hand :) Kris pgpkQZ83MN9n1.pgp Description: PGP signature
Re: How did upgrading applications happen before portupgrade etc?
Kris Kennaway wrote: On Sat, Aug 11, 2007 at 11:37:22AM +0200, Tobias Roth wrote: Kris Kennaway wrote: On Sat, Aug 11, 2007 at 12:03:33PM +0400, Rakhesh Sasidharan wrote: Hi, Just a question that struck me today. Before there were the portupgrade and other tools for upgrading installed applications to their newer versions, how did things work out? Did one upgrade applications through a series of make deinstall reinstall commands (I wonder if these commands take care of dependencies too) or was there any other way? Basically that, plus a lot of other manual steps that were always somewhat hard to get exactly right but which more or less worked back in the days when Gnomes lived in your garden and Java was a place on the map. It was a simpler, more innocent age. There was pkg_version -c that printed a sequence of cd /usr/ports/foo/bar make make deinstall make install make clean or something like that. Whatever broke was fixed manually afterwards :-) Oh, and there was no UPDATING in /usr/ports/ as well I think. Old-timers will tell you that pkg_version is a new-fangled invention, and back in the day they had to slave for hours over a hot keyboard to run all those make commands by hand :) I see. That bad eh ... no pkg_version even! I guess then the only way was to track cvs updates to see if anything of interest has been updated recently and then upgrade it manually. :-/ I'm curious now -- how does portupgrade (that's the tool I know/ use so I'll use that as an example) do its upgrading? I have seen that in case of an upgrade in builds the newer version, uninstalls the previous one (even though it might be required by other apps), and then installs the newer version. How does it do that -- by some magic of its own or does it use the usual ports commands etc? Any place where I can get more info on these 3rd party tools? Not too techie, but a bird's eye view of things ... TIA, Rakhesh ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
On Sat, Aug 11, 2007 at 09:01:27PM +0400, Rakhesh Sasidharan wrote: Kris Kennaway wrote: On Sat, Aug 11, 2007 at 11:37:22AM +0200, Tobias Roth wrote: Kris Kennaway wrote: On Sat, Aug 11, 2007 at 12:03:33PM +0400, Rakhesh Sasidharan wrote: Hi, Just a question that struck me today. Before there were the portupgrade and other tools for upgrading installed applications to their newer versions, how did things work out? Did one upgrade applications through a series of make deinstall reinstall commands (I wonder if these commands take care of dependencies too) or was there any other way? Basically that, plus a lot of other manual steps that were always somewhat hard to get exactly right but which more or less worked back in the days when Gnomes lived in your garden and Java was a place on the map. It was a simpler, more innocent age. There was pkg_version -c that printed a sequence of cd /usr/ports/foo/bar make make deinstall make install make clean or something like that. Whatever broke was fixed manually afterwards :-) Oh, and there was no UPDATING in /usr/ports/ as well I think. Old-timers will tell you that pkg_version is a new-fangled invention, and back in the day they had to slave for hours over a hot keyboard to run all those make commands by hand :) I see. That bad eh ... no pkg_version even! I guess then the only way was to track cvs updates to see if anything of interest has been updated recently and then upgrade it manually. :-/ Pretty much. It more or less worked when the ports collection was small, applications were mostly self-contained (few dependencies), and updates were infrequent. Obviously that approach did not scale. I'm curious now -- how does portupgrade (that's the tool I know/ use so I'll use that as an example) do its upgrading? I have seen that in case of an upgrade in builds the newer version, uninstalls the previous one (even though it might be required by other apps), and then installs the newer version. How does it do that -- by some magic of its own or does it use the usual ports commands etc? At the core it uses the usual port commands, but there is a lot of higher level logic around that to do things like planning and ordering of upgrades and error recovery. Any place where I can get more info on these 3rd party tools? Not too techie, but a bird's eye view of things ... Apart from the code itself I am not sure about this. Kris pgp7EcE6tyGyE.pgp Description: PGP signature
Re: How did upgrading applications happen before portupgrade etc?
On Sat, Aug 11, 2007 at 12:03:33PM +0400, Rakhesh Sasidharan wrote: Did one upgrade applications through a series of make deinstall reinstall commands (I wonder if these commands take care of dependencies too) or was there any other way? I used pkg_version -c, which has now been removed. It spit out a shell script that did the relevant upgrade commands, but wasn't perfect. OpenBSD still has this option, and it's a perl script, so you can probably still use that one if you're allergic to portupgrade/portmaster. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
Kris Kennaway wrote: I'm curious now -- how does portupgrade (that's the tool I know/ use so I'll use that as an example) do its upgrading? I have seen that in case of an upgrade in builds the newer version, uninstalls the previous one (even though it might be required by other apps), and then installs the newer version. How does it do that -- by some magic of its own or does it use the usual ports commands etc? At the core it uses the usual port commands, but there is a lot of higher level logic around that to do things like planning and ordering of upgrades and error recovery. I see. And I suppose that's how it avoids having to uninstall all the apps and then re-install them. Neat! It really came to me as a shock that once upon a time (and even now, those who prefer the old fashioned way) you had to actually uninstall all the apps and then re-install. I began using FreeBSD with 6.2 and so had taken portupgrade to be for granted. Then I happened to read a blog post by a NetBSD developer on how he keeps updating applications through pkgsrc, and from there I went to the NetBSD Wiki and their page on updating applications, and both these articles talk about uninstalling all the apps and then installing newer versions etc ... which is when it struck me that FreeBSD doesn't seem to have a cumbersome process such as that. Hats off to portupgrade and other such tools! :-) Regards, Rakhesh ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
On Sat, Aug 11, 2007 at 10:51:41AM -0700, David Ericson Thiel wrote: On Sat, Aug 11, 2007 at 12:03:33PM +0400, Rakhesh Sasidharan wrote: Did one upgrade applications through a series of make deinstall reinstall commands (I wonder if these commands take care of dependencies too) or was there any other way? I used pkg_version -c, which has now been removed. It spit out a shell script that did the relevant upgrade commands, but wasn't perfect. OpenBSD still has this option, and it's a perl script, so you can probably still use that one if you're allergic to portupgrade/portmaster. It was removed because it was dangerous. You could still use it if you choose, but as we're discussing it's an antiquated and obsoleted not-really-upgrade method. Kris pgpuwLCzmHJuj.pgp Description: PGP signature
Re: How did upgrading applications happen before portupgrade etc?
On Sat, Aug 11, 2007 at 03:02:53PM +0400, Rakhesh Sasidharan wrote: 5. pkg_delete port I see. In step 5, pkg_delete port wont work if port is required by others right? So you delete those apps too? Could be a lot of stuff to uninstall, right? Absolutely correct. That might seem like a nightmare to most people, but to me it's not. I suppose this is also why I avoid ports that have too many dependencies (mail/p5-Mail-SpamAssassin, for example, is starting to make me consider using dspam instead). Note that we keep all of our configuration files in a directory called /conf/ME and in /usr/local simply use symlinks. (Matt Dillon might be grinning over /conf/ME, since it's a Best Internet-ism :) ). I don't trust that all ports will play nice with existing configuration files in /usr/local, and I have seen much evidence of this in the past (nuking files without your knowledge, mainly. Oh crap! It nuked our entire configuration for the apache/mail/whatever server!!!). That's a smashing idea! Thanks for mentioning that here. :-) What does the ME stand for in /conf/ME btw? ME stands for me, e.g. myself. I guess it'd help if I explained how it worked: The /conf directory on all machines contained the configuration files for itself as well as every other machine on the network. So for shell01.whatever.com you'd find this in /conf: /conf/ME -- /conf/shell01.whatever.com /conf/mail01.whatever.com/ /conf/shell01.whatever.com/ /conf/shell02.whatever.com/ /conf/sql01.whatever.com/ /conf/test.whatever.com/ Each directory contained configuration files specific to that system. During system install, either done automatically or manually (I forget), the /conf directory on a machine is populated, and the ME symlink is made. Then the script would populate /usr/local with symlinks, and I believe also did a mkdir -p on directories. For example, if /conf/ME contained: etc/Muttrc etc/apache22/httpd.conf etc/mail/spamassassin/local.cf etc/pine.conf etc/sudoers You'd find this in /usr/local: etc/Muttrc -- /conf/ME/etc/Muttrc etc/apache22/httpd.conf -- /conf/ME/etc/apache22/httpd.conf etc/mail/spamassassin/local.cf -- /conf/ME/etc/mail/spamassassin/local.cf etc/pine.conf -- /conf/ME/etc/pine.conf etc/sudoers -- /conf/ME/etc/sudoers I believe the reason each machine had a copy of all the other machines' configs was in case one got munged on the repository server (or if you needed to roll back and didn't make a backup of your changes, oops), and/or if you wanted to change the role of a machine on the fly. All of the packages/ports installed on all the machines were 100% identical; that is, every machine, regardless of role, had the same packages/ports installed. Thanks for sharing your traditional steps with here btw. No problem. :-) You asked a good question, so hopefully you'll get some good answers back. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
On Sat, Aug 11, 2007 at 05:42:30AM -0500, Matthew D. Fuller wrote: portupgrade manages all the regular /var/db/pkg/*/* files just like any other tool. It's got its db files, sure, but they're caches, not alternate masters. I've never seen a sync mismatch (or rather, I see them all the time, when portupgrade sees that the source is newer than its cache and updates the cache). I'm aware it manages all of the regular /var/db/pkg files just like the pkg_* tools do. It has to for it to play nice. But it also maintains a separate database for (I think) versioning and dependency matching. I've been told the reason it does this is due to shortcomings in /var/db/pkg or the pkg_* tools. Also, if you use portupgrade, you pretty much HAVE to use it exclusively for everything, e.g. you can't go mix-and-matching use of portupgrade and the base pkg_* tools without there being some inconsistency induced and thus breaking portupgrade. Am I correct? I blow 'em away every time I upgrade ruby or ruby-bdb or bdb just out of GP to head off potential troubles. With the growing number of installed ports, rebuilding the pkgdb.db files takes a long time, but it's what, a minute? Minute and a half? I think you're referring to things like this: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=367904+0+archive/2007/freebsd-ports/20070204.freebsd-ports Where the only solution is to rm the pkgdb.db file and then run pkgdb -u, correct? There are certainly reasons to dislike portupgrade (like that it's slow. Godawful long. Where's-War-And-Peace-I-need-something-to-read slow. It's almost as slow as yum is on a machine twice as fast), but I don't understand this one. The db's dont go wonky with any notable regularity IME, and when they do you just rm 'em and move on. What's the big deal? If people use portupgrade, awesome. If it makes their lives easier, great. I won't get religious about it if I'm helping someone with their box and they say Oh, btw, I use portupgrade. My response will be Alright cool. But if someone asks me Why don't you use portupgrade I'm going to tell them why. I see claims like the above (you just nuke the db and move on. What's the problem?) and I think What the f***?. Removing a file and ignoring the problem because the problem then goes away (until the next time it happens...) makes me think of a Windows-like solution. Reboot the box and the problem goes away. IIS /REBOOTONERROR. No thanks. When something breaks on my boxes, I want to know exactly why, and if it's not due to my own fault, I want to either fix it myself + supply a patch (if I'm capable of doing so), or report it and have it fixed. The fact that portupgrade has existed since early 2001 and still continues to have problems like the aforementioned is uncomfortable (in my opinion). It's major enough that it doesn't instill confidence. If it's a minor enough issue to you, then great, no harm done. But it's not a minor issue to me. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
On Sat, 11 Aug 2007 13:33:22 -0700 Jeremy Chadwick [EMAIL PROTECTED] wrote: On Sat, Aug 11, 2007 at 03:02:53PM +0400, Rakhesh Sasidharan wrote: 5. pkg_delete port I see. In step 5, pkg_delete port wont work if port is required by others right? So you delete those apps too? Could be a lot of stuff to uninstall, right? Absolutely correct. That might seem like a nightmare to most people, but to me it's not. It's not correct, pkg_delete -f can force the deletion. I would manually upgrade a port like this: cd /usr/ports/misc/foo make ; do the build pkg_info -qO misc/foo ; get old package name pkg_create -b old-package-name ; backup existing package /usr/local/etc/rc.d/foo stop ; stop the daemon if needed pkg_delete -f old-package-name ; force removal make install /usr/local/etc/rc.d/foo start ; start the daemon if needed make clean rm back-up package ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
Jeremy Chadwick wrote: On Sat, Aug 11, 2007 at 03:02:53PM +0400, Rakhesh Sasidharan wrote: 5. pkg_delete port I see. In step 5, pkg_delete port wont work if port is required by others right? So you delete those apps too? Could be a lot of stuff to uninstall, right? Absolutely correct. That might seem like a nightmare to most people, but to me it's not. I suppose this is also why I avoid ports that have too many dependencies (mail/p5-Mail-SpamAssassin, for example, is starting to make me consider using dspam instead). It did to me. But yeah, if you limit yourself to ports without too many dependencies, then its fine. And also, if you are doing this only once in a while, it limits downtime too. Makes sense. Note that we keep all of our configuration files in a directory called /conf/ME and in /usr/local simply use symlinks. (Matt Dillon might be grinning over /conf/ME, since it's a Best Internet-ism :) ). I don't trust that all ports will play nice with existing configuration files in /usr/local, and I have seen much evidence of this in the past (nuking files without your knowledge, mainly. Oh crap! It nuked our entire configuration for the apache/mail/whatever server!!!). That's a smashing idea! Thanks for mentioning that here. :-) What does the ME stand for in /conf/ME btw? ME stands for me, e.g. myself. I guess it'd help if I explained how it worked: The /conf directory on all machines contained the configuration files for itself as well as every other machine on the network. So for shell01.whatever.com you'd find this in /conf: /conf/ME -- /conf/shell01.whatever.com /conf/mail01.whatever.com/ /conf/shell01.whatever.com/ /conf/shell02.whatever.com/ /conf/sql01.whatever.com/ /conf/test.whatever.com/ Each directory contained configuration files specific to that system. Wicked! I just take backups of the /etc and /usr/local/etc of each machine onto every other machine but the method above is even better. Esp in light of what you mention later on ... During system install, either done automatically or manually (I forget), the /conf directory on a machine is populated, and the ME symlink is made. Then the script would populate /usr/local with symlinks, and I believe also did a mkdir -p on directories. For example, if /conf/ME contained: etc/Muttrc etc/apache22/httpd.conf etc/mail/spamassassin/local.cf etc/pine.conf etc/sudoers You'd find this in /usr/local: etc/Muttrc -- /conf/ME/etc/Muttrc etc/apache22/httpd.conf -- /conf/ME/etc/apache22/httpd.conf etc/mail/spamassassin/local.cf -- /conf/ME/etc/mail/spamassassin/local.cf etc/pine.conf -- /conf/ME/etc/pine.conf etc/sudoers -- /conf/ME/etc/sudoers Dodo question maybe: but instead of symlinking specific files, why not just symlink /usr/local/etc to something like /conf/ME/etc? The way you described it, would you also have to keep track of new config files, move them over to /conf/ME/etc, and then symlink from /usr/local/etc? If you symlink the whole directory, you save that hassle right? Plus, later on, when you want the machine to switch roles, its just a matter of changing the directory symlink rather than of each file ... I believe the reason each machine had a copy of all the other machines' configs was in case one got munged on the repository server (or if you needed to roll back and didn't make a backup of your changes, oops), and/or if you wanted to change the role of a machine on the fly. All of the packages/ports installed on all the machines were 100% identical; that is, every machine, regardless of role, had the same packages/ports installed. A very interesting tip btw. So you essentially have all the machines ready to switch roles at the change of a symlink. Neat! This idea was by Matt Dillon eh? :-) You have any scripts etc to keep this running? Would be nice if you could drop a copy so I can poke around ... Thanks, Rakhesh ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How did upgrading applications happen before portupgrade etc?
On Sat, 11 Aug 2007, Rakhesh Sasidharan wrote: Afaik portmaster's beauty is that it doesn't have any dependencies like portuprade does. Doesn't require ruby etc, so upgrades to them don't cause problems like they could with portupgrade. (Again, haven't used portmaster, so I don't know; this is my understanding). Yes, that's correct. It's written in /bin/sh, and the script is actually in the files directory of the port. That's about as non-depedent as you can get. :) Seriously though, I think it's worth mentioning again that portmaster is not in competition with portupgrade. The goals that they are designed for have a lot of overlap, but there are things that you can do with portupgrade that you can't do with portmaster (especially managing upgrades using packages). That said, if you're upgrading or installing ports, I think portmaster does a pretty good job, and it handles the exact scenario that started this thread, How do I update a port (or ports) from the middle of a dependency chain without breaking stuff? For anyone interested, you can install the port and read the man page without changing or committing to anything, FYI. hope this helps, Doug -- This .signature sanitized for your protection ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
