Re: package upgrade strategy
Hi All, Thanks again for all the answers in the topic! I read all of them but I cannot answer each in separate emails so I just picked the latest to continue the thread. As earlier mentioned I agree with the advantages of the pkgsrc based solutions and in the meantime I also found a nice summary of the different approaches at https://wiki.netbsd.org/pkgsrc/how_to_upgrade_packages/ (better later than never). However, I pursued the issue further to get a workaround (till I have time to rebuild things via pkgsrc) with the binary packages and I think I found something based on which I could build something. I only tried it manually and it seems to work but of course in case of more pkgs a script would be better to manage all this: -in case a package needs to be updated to a newer version, copy its shared libs to a specific directory (say soold) -record the old pkg name (together with the one that required updating it) in an sqlite db and the files copied to the soold directory -add soold directory to LD_LIBRARY_PATH if it's not yet added The solution would be better if it was somehow possible to revert the changes i.e. if I don't like the newly installed stuff, i.e. just downgrade the packages to their earlier versions which could be figured out from the db file that keeps track of the removed pkgs. However, afaik binary pkg downgrade is not possible either pkgin or by the pkg_* tools. I guess this is against quite a many things and how they're meant to be done in NetBSD but as a workaround this may come handy. What do you think? Can anyone forsee already a "design flaw" or whatever that'd prevent this from being feasible? Thanks & regards, r0ller Eredeti levél Feladó: Greg Troxel < g...@lexort.com (Link -> mailto:g...@lexort.com) > Dátum: 2017 szeptember 30 18:09:06 Tárgy: Re: package upgrade strategy Címzett: r0ller < r0l...@freemail.hu (Link -> mailto:r0l...@freemail.hu) > pkgsrc is designed, more or less, to have packages built consistently from the same source tree. You can often get away with having some packages be old. Other than the old packages not depending on or being dependencies of newer ones, you're on thin ice. If a package is not available in a newer branch, that could be because it was removed from pkgsrc, or because it didn't build. You can check out the sources and have a look. You can also look at the mail archives for pkgsrc-bulk and see what happened, and find the build logs. It may be that midori builds on NetBSD 8 (not yet out) but not 7, or 7 but not 6. This can be about the versions of X things in the base system, or it can be about compilers. So looking at the build logs for various systems can be useful. pkgsrc does not require you to choose between building from source and binary packages. But, you should use a source tree consistent with the binary packages you are using. Right now I think the latest binary packages for NetBSD are from 2017Q2. 2017Q3 exists in source form (and I'm behind on drafting an announcement), but NetBSD's package building machines are down due to data center work (we are grateful guests). I expect binary packages within a few weeks. Also, you can build your own binary packages, and use them with pkgin. You just have to make a summary file, which is easy. pkg_info -X *gz | bzip2 > pkg_summary.bz2 So if you have a fast machine, you can build, and install them on the slower machine (just rsync them and add a repositories.conf line). Packages without active upstreams are harder to maintain and can tend to run into problems. Midori's last release appears to be from August of 2015. That's not really really old, but it's long enough to suggest a possible issue. If you want to run an older build of midori, it's going to expect dependencies that are ABI-compatible, and this rapidly gets difficult. Overall, I think your best (but not easiest) path to success will be to use 2017Q3 sources, update everything you can via the forthcoming binary packages, and then to build midori from source, perhaps fixing the build. We would then want to commit the fixes, and for anything that belongs upstream to be comitted there and then have a new release. So far it looks like the patches in pkgsrc for midori are very minor and not obviously appropriate for upstream.
Re: package upgrade strategy
pkgsrc is designed, more or less, to have packages built consistently from the same source tree. You can often get away with having some packages be old. Other than the old packages not depending on or being dependencies of newer ones, you're on thin ice. If a package is not available in a newer branch, that could be because it was removed from pkgsrc, or because it didn't build. You can check out the sources and have a look. You can also look at the mail archives for pkgsrc-bulk and see what happened, and find the build logs. It may be that midori builds on NetBSD 8 (not yet out) but not 7, or 7 but not 6. This can be about the versions of X things in the base system, or it can be about compilers. So looking at the build logs for various systems can be useful. pkgsrc does not require you to choose between building from source and binary packages. But, you should use a source tree consistent with the binary packages you are using. Right now I think the latest binary packages for NetBSD are from 2017Q2. 2017Q3 exists in source form (and I'm behind on drafting an announcement), but NetBSD's package building machines are down due to data center work (we are grateful guests). I expect binary packages within a few weeks. Also, you can build your own binary packages, and use them with pkgin. You just have to make a summary file, which is easy. pkg_info -X *gz | bzip2 > pkg_summary.bz2 So if you have a fast machine, you can build, and install them on the slower machine (just rsync them and add a repositories.conf line). Packages without active upstreams are harder to maintain and can tend to run into problems. Midori's last release appears to be from August of 2015. That's not really really old, but it's long enough to suggest a possible issue. If you want to run an older build of midori, it's going to expect dependencies that are ABI-compatible, and this rapidly gets difficult. Overall, I think your best (but not easiest) path to success will be to use 2017Q3 sources, update everything you can via the forthcoming binary packages, and then to build midori from source, perhaps fixing the build. We would then want to commit the fixes, and for anything that belongs upstream to be comitted there and then have a new release. So far it looks like the patches in pkgsrc for midori are very minor and not obviously appropriate for upstream. signature.asc Description: PGP signature
Re: package upgrade strategy
Date:Thu, 28 Sep 2017 22:20:50 +0200 (CEST) From:r0ller Message-ID: | but I hoped that there's a way to avoid compiling relatively big programs | (like seamonkey) in a case when I just want to check something out [...] There is - you can keep using the binary packages, for everything that still exists in pkgsrc (and for which binary packages are available). You just need to use the sources for midori (or any other package that has been deleted) because there are no new binary packages being produced for that. For everything else you just do binary package upgrades, then when everything is in place (which will have resulted in midori having been deleted) you just compile your copy of that (using the old pkgsrc directory for it) from sources. As long as it continues to build, that should be easy enough. Of course, over time, that gets less and less likely, unless you are upgrading it yourself. If you want to protect it so you have a binary that will work like it does now, regardless of anything else that happens, you should be able to make that work, by setting up an entirely separate /usr/pkg tree (ie: storing it someplace else) - get a version of pkgsrc that includes midori (an older one than current) change the PKG_DBDIR and LOCALBASE / PREFIX, and build just midori from source with those settings (which will also build all of its dependencies, since in that tree, nothing will exist yet). Once that's done, simply leave that tree alone forever, and never touch it again. It will remain completely independent of anything that is normally done with pkgsrc, not influencing it at all. In the regular /usr/pkg you just keep doing binary pkg_add pkg_delete ... as desired to install and delete packages you want to test. kre ps: "kids" is perfectly OK to use at least for this native English speaker.
Re: package upgrade strategy
Hi All, Thanks for all your answers (Alistair, Brad, Jeff, Jeremy)! They all seem to point in one direction i.e. to use pkgsrc with which I don't have any problem. I'm kind of used to it but I hoped that there's a way to avoid compiling relatively big programs (like seamonkey) in a case when I just want to check something out and get rid of it if it does not fit the bill. The scenario is something like that I have a 10 years old desktop for our kids (some say this word is negative for native English speakers, I'm not one of them so please, forgive) to play online flash games, watch youtube videos, etc. with 3gb ram, intel core2 conroe cpu with 1 core+igp and a conroe asrock board which all serves pretty well at their age for this purpose -at least with NetBSD;) Midori did fit the bill for everything they wanted till they bumped into codecombat which doesn't work. So I wanted to quickly check out another browser and this is exactly when I don't have time to compile anything:) As mentioned, by checking out the earlier 7.0 Q1 repo helped with seamonkey and codecombat runs fine in that but in such cases compiling each browser till I find the right one is not an option. I mean, before seamonkey I also checked out epiphany and firefox, which have the same bug as midori in that online game. Finding out that seamonkey fits the bill took roughly 15 mins, but compiling three browsers would have taken a lot more than that. Don't get me wrong, in general I agree with all your suggestions, I just hoped that there's still a way to keep different versions of pkgs in parallel. Best regards, r0ller Eredeti levél Feladó: Alistair Crooks < a...@pkgsrc.org (Link -> mailto:a...@pkgsrc.org) > Dátum: 2017 szeptember 28 18:20:31 Tárgy: Re: package upgrade strategy Címzett: Jeff_W < j...@sdf.org (Link -> mailto:j...@sdf.org) > On 28 September 2017 at 08:40, Jeff_W wrote: > % cd /pkgsrc/foo/pkg-A > % sudo make clean && sudo make replace > > If the above breaks pkg-C: > > % cd /pkgsrc/foo/pkg-C > % sudo make clean && sudo make clean-depends && sudo make update Orthogonal to this discussion - pkgsrc was modified to use just-in-time su in the early 2000s, and avoids interesting issues like fetching sources as root. You are definitely encouraged to use it. If you'd prefer to use sudo, rather than su, put this in your etc/mk.conf: .if exists(${LOCALBASE}/bin/sudo) SU_CMD= ${LOCALBASE}/bin/sudo /bin/sh -c .endif Best, Alistair
Re: package upgrade strategy
I suspect this is more fallout from people mis-updating libraries. try manually deleting and adding midori again. I'm hoping it rebuilt the package even though there's no indication to the binary package bits that it changed.
Re: package upgrade strategy
r0ller writes: > Hi Brad, > > I'd do that if it was possible (having the newest versions of all my > packages) but as said, midori is not available any more in the 7.1 package > repo so it cannot be upgraded (not even the same version of midori is > available with just an updated list of dependencies) and if I upgrade icu (to > version 59), midori starts complaining about not finding the earlier version > of it (version 58). What if I just used pkg_add for icu-59? Or if I mark > icu-58 non-autoremovable by pkgin, would the install of icu-59 leave icu-58 > untouched? > > Best regards, > r0ller > Ya, you are in a bad situation that will likely never allow you to update using anything resembling "normal" using prebuilt binary packages. I.e. no more pkgin anymore from a remote repo. You are facing the requirement to build packages yourself from source and the need to create a consistant "bundle" of packages for a system. You may be able to resolve the issue by creating a local midori package and build from that, for example... I have never used the prebuilt binary packages, and have always built and rebuilt from source due to various complicated reasons for more than 15 years. It has allowed me to do things like include a local version of a package that isn't "out yet" in pkgsrc and has allowed me to use older packages that have been removed from pkgsrc [or updated to a version I didn't want] as long as they would be ok with the dependent packages in pkgsrc. I could describe what I am doing if you are interested and want to do this sort of thing. It honestly isn't hard and mostly just requires a bit of organization and some disk space. -- Brad Spencer - b...@anduin.eldar.org - KC8VKS http://anduin.eldar.org - & - http://anduin.ipv6.eldar.org [IPv6 only]
Re: package upgrade strategy
On Thu, 28 Sep 2017, r0ller wrote: > By the way, what kind of difference is indicated by the number in the > 'nb' suffix? Means the original code (upstream source) was not changed. The nb means we may be building or installing it differently (like due to a new patch, new build option, or a dependency change, for example). > Another question would be if it's possible to keep different > versions of a package installed? I know in case of shared libs it may > be tricky because of the symlinks but the runtime linker is not > looking for the symlink I hope but the versioned soname, right? Any > hints are welcome! Well if you build your own from pkgsrc, you can use a different PKG_DBDIR and LOCALBASE. (Maybe bootstrap it with different settings.) But then you have an additional install to manage and lots of resources potentially wasted. (Sorry you had a package disappear.)
Re: package upgrade strategy
Alistair Crooks wrote: > On 28 September 2017 at 08:40, Jeff_W wrote: > > % cd /pkgsrc/foo/pkg-A > > % sudo make clean && sudo make replace > > > > If the above breaks pkg-C: > > > > % cd /pkgsrc/foo/pkg-C > > % sudo make clean && sudo make clean-depends && sudo make update > > Orthogonal to this discussion - pkgsrc was modified to use > just-in-time su in the early 2000s, and avoids interesting issues like > fetching sources as root. You are definitely encouraged to use it. > > If you'd prefer to use sudo, rather than su, put this in your etc/mk.conf: > > .if exists(${LOCALBASE}/bin/sudo) > SU_CMD= ${LOCALBASE}/bin/sudo /bin/sh -c > .endif Nice; wasn't aware of that - thanks Alistair
Re: package upgrade strategy
On 28 September 2017 at 08:40, Jeff_W wrote: > % cd /pkgsrc/foo/pkg-A > % sudo make clean && sudo make replace > > If the above breaks pkg-C: > > % cd /pkgsrc/foo/pkg-C > % sudo make clean && sudo make clean-depends && sudo make update Orthogonal to this discussion - pkgsrc was modified to use just-in-time su in the early 2000s, and avoids interesting issues like fetching sources as root. You are definitely encouraged to use it. If you'd prefer to use sudo, rather than su, put this in your etc/mk.conf: .if exists(${LOCALBASE}/bin/sudo) SU_CMD= ${LOCALBASE}/bin/sudo /bin/sh -c .endif Best, Alistair
Re: package upgrade strategy
> I hope someone can give an answer to this question from a newbie: how can > I handle situations when an already installed package (say A) needs to be > upgraded due to a newly installed package (say B) as it's being a > dependency for it but another already installed package (say C) still > needs the older version of package A? Upgrading package C may be thought > of as help but it disappeared from the 7.1 package repo and as far as I > could see it was last present in 7.0_2017Q1. % cd /pkgsrc/foo/pkg-A % sudo make clean && sudo make replace If the above breaks pkg-C: % cd /pkgsrc/foo/pkg-C % sudo make clean && sudo make clean-depends && sudo make update HTH, jgw
Re: package upgrade strategy
Hi Brad, I'd do that if it was possible (having the newest versions of all my packages) but as said, midori is not available any more in the 7.1 package repo so it cannot be upgraded (not even the same version of midori is available with just an updated list of dependencies) and if I upgrade icu (to version 59), midori starts complaining about not finding the earlier version of it (version 58). What if I just used pkg_add for icu-59? Or if I mark icu-58 non-autoremovable by pkgin, would the install of icu-59 leave icu-58 untouched? Best regards, r0ller Eredeti levél Feladó: Brad Spencer < b...@anduin.eldar.org (Link -> mailto:b...@anduin.eldar.org) > Dátum: 2017 szeptember 28 14:13:53 Tárgy: Re: package upgrade strategy Címzett: r0ller < r0l...@freemail.hu (Link -> mailto:r0l...@freemail.hu) > r0ller writes: > Hi All, > > I hope someone can give an answer to this question from a newbie: how can I > handle situations when an already installed package (say A) needs to be > upgraded due to a newly installed package (say B) as it's being a dependency > for it but another already installed package (say C) still needs the older > version of package A? Upgrading package C may be thought of as help but it > disappeared from the 7.1 package repo and as far as I could see it was last > present in 7.0_2017Q1. > > If anyone is interested in the details, package A is icu, package B is > seamonkey and package C is midori. What I did for now was that I changed > /usr/pkg/etc/pkgin/repositories.conf so that it points to 7.0_2017Q1 and > installed seamonkey from that repo as the versions are not that different > (seamonkey-2.46nb7 in 7.0_2017Q1 and seamonkey-2.46nb8 in 7.1) and it left > icu untouched. By the way, what kind of difference is indicated by the number > in the 'nb' suffix? Another question would be if it's possible to > keep different versions of a package installed? I know in case of shared libs > it may be tricky because of the symlinks but the runtime linker is not > looking for the symlink I hope but the versioned soname, right? Any hints are > welcome! > > Best regards, > r0ller Hello... I have been running pkgsrc for a very long time and you won't like my answer Basically, you don't try to do this with piecemeal updates. It honestly will be simpler to delete all of the packages, saving configs first, and reinstall the new versions. This may sound quite extreme, but if you have the situation you describe there will be too many other interdependencies to make anything else reasonable and sane, especially in the long run. There are techniques to make this much less harsh then it would be otherwise, and I can tell you about them if you want, but I really suspect you will find it much simpler to do as I suggested. -- Brad Spencer - b...@anduin.eldar.org - KC8VKS http://anduin.eldar.org - & - http://anduin.ipv6.eldar.org [IPv6 only]
package upgrade strategy
Hi All, I hope someone can give an answer to this question from a newbie: how can I handle situations when an already installed package (say A) needs to be upgraded due to a newly installed package (say B) as it's being a dependency for it but another already installed package (say C) still needs the older version of package A? Upgrading package C may be thought of as help but it disappeared from the 7.1 package repo and as far as I could see it was last present in 7.0_2017Q1. If anyone is interested in the details, package A is icu, package B is seamonkey and package C is midori. What I did for now was that I changed /usr/pkg/etc/pkgin/repositories.conf so that it points to 7.0_2017Q1 and installed seamonkey from that repo as the versions are not that different (seamonkey-2.46nb7 in 7.0_2017Q1 and seamonkey-2.46nb8 in 7.1) and it left icu untouched. By the way, what kind of difference is indicated by the number in the 'nb' suffix? Another question would be if it's possible to keep different versions of a package installed? I know in case of shared libs it may be tricky because of the symlinks but the runtime linker is not looking for the symlink I hope but the versioned soname, right? Any hints are welcome! Best regards, r0ller