Andrej, > -----Original Message----- > From: Valek, Andrej <andrej.va...@siemens.com> > Sent: Saturday, January 22, 2022 2:26 AM > To: openembedded-core@lists.openembedded.org; Bryan Evenson > <beven...@melinkcorp.com> > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary busybox > links during install > > Hello again, > > Maybe a general question. Is it working in current master? I do not want to > brake dunfell, just applying something, which will create a lot of divergence. >
I wasn't able to build an image based on the latest master as of Monday. From the error messages I think it had something to do with the addition of setuptools3-base that wasn't complete yet. I was able to build and run an image based on the latest honister branch. The master branch has four commits for busybox that honsiter does not have. In those four commits, I see no changes to any of the installation steps. Based on this information, I'd say its safe to say that testing on the latest honister branch should confirm whether the problem is still present or not. To test, I first built my image and directly programmed it on my hardware. My image is based upon core-image-minimal, uses sysvinit for an init system and opkg for a package manager. I have also added bash, cronie, dhcpcd and util-linux to my image; I've added others but these seem to produce the most obvious errors. After programming the image on my hardware, I added PR="r1" to the gcc recipe. This forced all the packages to get rebuilt and increment the version number (if anyone knows a simpler way to accomplish this, let me know). I then created a package feed with this update and attempted an upgrade. To upgrade, I did: opkg update opkg --download-only upgrade opkg upgrade > full_upgrade.txt 2>&1 & I download all the packages to be upgraded first. I then sent the upgrade process to the background so I could check other things during the upgrade process. During the upgrade, I would occasionally execute "ls -l" to verify that the file full_upgrade.txt existed and was still growing. About a minute into the upgrade process when I executed "ls -l" I got the error message: ls: not found From that point on until upgrade was complete I had to execute "busybox ls -l" to see the current directory file list. After upgrade was complete, I checked full_upgrade.txt for error messages. I saw the following: /usr/bin/update-alternatives: line 1: sed: not found (occurred 102 times) /usr/sbin/update-rc.d: line 54: rm: not found (occurred 36 times) Stopping crond: /etc/init.d/crond: line 37: start-stop-daemon: not found /tmp/opkg-yBH1Ta/cronie-zHpym4/preinst: line 1: tr: not found /etc/init.d/udev: line 74: start-stop-daemon: not found /tmp/opkg-yBH1Ta/dhcpcd-c86ca9/preinst: line 1: tr: not found In this case, a lot of alternatives and init scripts may not be setup correctly because either the old ones were not removed or the new ones were not installed. Prior to my patch below, I saw similar errors when upgrading my image from a build based off the morty branch to one based off dunfell (and my system was not operational after the upgrade). After my patch, I performed the same upgrade with no errors and my system was fully operational. I'm up for any thoughts on a different approach that accomplishes the same goal. Otherwise, I'm going to work with what I have and update my changes so they patch cleanly on master. Thanks, Bryan > Cheers, > Andrej > > On Fri, 2022-01-21 at 15:02 +0000, Bryan Evenson wrote: > > Andrej, > > > > Thanks for the response. This is an attempt to fix a problem I am > > having with automated firmware upgrades for my system. I am using > > opkg for a package manager; not sure if the same problem exists with > > other package managers. I run into problems whenever busybox is one > > of the packages that needs to get updated. I enact my distribution > > firmware upgrade by calling "opkg --download-only upgrade; opkg > > upgrade". What I see happen is: > > > > 1. In the busybox pkg_prerm stage sets up some soft links for some > > common applets in a temporary directory and exports a path to that > > directory. It might also setup a temporary alternative to /bin/sh if > > it is the last shell. > > 2. After the remove stage, the busybox binary is gone. The softlinks > > created in the prerm stage are useless since they point to binary that > > no longer exists. > > 3. opkg continues with upgrade on other packages which may depend on a > > command provided by busybox in a prerm, postrm, preinst or postinst > > script. These upgrades then fail since the commands are no longer > > available. > > 4. The busybox upgrade completes, which may or may not complete > > successfully. For what I am attempting, I am upgrading my system from > > the morty branch to dunfell. I have util-linux on my system which > > shares some alternatives with busybox. The util-linux upgrade fails > > because it needs some busybox applets during its upgrade process. > > Then the busybox upgrade fails because the final update- alternatives > > doesn’t work; some files still exist that util-linux couldn't remove > > during its upgrade that clash with busybox's alternatives. > > > > After trying several ways to get my upgrade to work, I landed on the > > approach below. I'm creating a temporary directory and copying the > > busybox binary and the busybox.links files to that directory. I then > > install an alternative for every applet for busybox listed in all of > > its busybox.links files that points to the temporary copy of the > > busybox binary. This means that any package that uses a busybox > > applet during its install process should still work. Then during the > > postinst step I am removing all the temporary alternative links. I > > use the temporary busybox.links files for removing the alternative > > links in case the upgraded busybox is now configured with a different > > set of applets. > > > > This is a heavy handed approach, and it does extend the upgrade > > process for me by a few minutes since it runs through update- > > alternatives for busybox two more times. But, the approach works for > > me and I think would be more resilient than past approaches. I tried > > to mimic the existing code in my additions. If a more widescale > > rewrite makes sense than that works for me also. > > > > Thanks, > > Bryan > > > > > > > -----Original Message----- > > > From: Valek, Andrej <andrej.va...@siemens.com> > > > Sent: Friday, January 21, 2022 9:01 AM > > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > > <beven...@melinkcorp.com> > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > busybox links during install > > > > > > Hi Bryan, > > > > > > Sorry, maybe I didn't fully understand the use-case. Are you trying > > > to upgrade the busybox on demand? If yes, that is not a good idea. > > > > > > I'm little bit scary about doing "export PATH=$busybox_rmdir:$PATH" > > > and > > > creating a custom locks is not a good at all. > > > > > > Cheers, > > > Andrej > > > > > > On Fri, 2022-01-21 at 13:29 +0000, Bryan Evenson wrote: > > > > All, > > > > > > > > Ping on this RFC. It works for me, but I have a feeling there is > > > > a better way to do this. It still seems a little messy and could > > > > probably be simplified for the same effect. > > > > > > > > Thanks, > > > > Bryan > > > > > > > > > -----Original Message----- > > > > > From: Bryan Evenson > > > > > Sent: Thursday, December 23, 2021 9:50 AM > > > > > To: openembedded-core@lists.openembedded.org > > > > > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > busybox links during install > > > > > > > > > > Busybox upgrades sometimes fail, especially if there is a major > > > > > distribution upgrade and all packages need to be updated. > > > > > Success > > > > > is highly dependent on the package upgrade order. > > > > > > > > > > Commit [1] attempts to ensure a shell is still present by adding > > > > > an alternative to /bin/sh if busybox is the only shell. > > > > > However, if busybox is not the only shell present and the other > > > > > shells are upgrading, it may then be possible that all shells > > > > > will be removed during the upgrade process. > > > > > > > > > > Commit [2] creates temporary symbolic links for all the busybox > > > > > links during busybox's postinst step. However, this is too late > > > > > in the process as some packages attempt to use 'rm' and 'sed' > > > > > after > > > > > update-alternatives removes the old links and prior to when > > > > > busybox's postinst step runs. > > > > > > > > > > This fix is similar to [2] but runs during the preinst step. For > > > > > opkg, this is the first step that is guaranteed to run from the > > > > > new package (prerm is run from the old package) and will > > > > > therefore be a backwards-compatible fix for upgrading older > > > > > systems. > > > > > > > > > > Copies the existing busybox binary and the busybox.links files > > > > > to a temporary directory and then creates alternative links for > > > > > all installed busybox commands. The temporary links and > > > > > directory are cleaned up during the postinst step. > > > > > > > > > > RFC: This works for me, but there may be room for improvement. I > > > > > don't know if the current pkg_prerm steps are necessary anymore. > > > > > However, in > > > > > my testing I did need the links for update-alternatives to work > > > > > in the preinst step. I am also not certain if the > > > > > populate_packages_updatealternatives_append > > > > > step is necessary anymore. I have also only tested this fix on > > > > > dunfell, as I don't have a working image based on master yet. > > > > > It > > > > > may be more appropriate for this to go to master and then be > > > > > backported to dunfell, but I would need assistance in testing. > > > > > > > > > > [1] https://git.openembedded.org/openembedded- > > > > > core/commit/meta/recipes- > > > > > > > > > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23 > > > > > a > > > > > [2] https://git.openembedded.org/openembedded- > > > > > core/commit/meta/recipes- > > > > > > > > > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > > > > > d2 > > > > > > > > > > Signed-off-by: Bryan Evenson <beven...@melinkcorp.com> > > > > > --- > > > > > meta/recipes-core/busybox/busybox.inc | 57 > > > > > ++++++++++++++++++++++++++- > > > > > 1 file changed, 55 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/meta/recipes-core/busybox/busybox.inc > > > > > b/meta/recipes- > > > > > core/busybox/busybox.inc index e0522be729..c85402411b 100644 > > > > > --- a/meta/recipes-core/busybox/busybox.inc > > > > > +++ b/meta/recipes-core/busybox/busybox.inc > > > > > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > > > > > > > > > pkg_postinst_${PN}_append () { > > > > > - # If busybox exists in the remove directory it is > > > > > because it was the only shell left. > > > > > if [ "x$D" = "x" ] ; then > > > > > + # If busybox exists in the remove directory it is > > > > > because it was the only > > > > > shell left. > > > > > if [ "x$BUSYBOX" != "x" ] ; then > > > > > update-alternatives --remove sh $BUSYBOX > > > > > - rm -f $BUSYBOX > > > > > fi > > > > > + # Remove the temporary alternatives > > > > > + for busybox_preinstdir in /tmp/busyboxpreinst-*; do > > > > > + if [ "$busybox_preinstdir" != > > > > > '/tmp/busyboxpreinst- > > > > > *' ] ; then > > > > > + BUSYBOX_PREINST_DIR="$busybox_preinstdir" > > > > > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > > > > > + if [ -e $BUSYBOX ] ; then > > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > > + if [ -e > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > > > > > + while read link; do > > > > > + update-alternatives --remove > > > > > $($BUSYBOX basename > > > > > $link) $BUSYBOX > > > > > + done < > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix > > > > > + fi > > > > > + done > > > > > + fi > > > > > + rm -rf $BUSYBOX_PREINST_DIR > > > > > + fi > > > > > + done > > > > > fi > > > > > } > > > > > > > > > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > > > > > fi > > > > > } > > > > > > > > > > +pkg_preinst_${PN} () { > > > > > + # Create a temporary copy the busybox binary and the > > > > > links > > > > > files. Then, > > > > > + # install an alternative link for all the links. > > > > > Other > > > > > packages use these > > > > > + # commands during their upgrade process. This ensures > > > > > the > > > > > links are > > > > > available > > > > > + # to all the other packages. We do this in the > > > > > preinst > > > > > step because it is > > > > > + # the first step guaranteed to be used from the new > > > > > package. The > > > > > prerm is > > > > > + # used from the old package. Placing this here > > > > > ensures it > > > > > runs on > > > > > upgrade even > > > > > + # on older systems. > > > > > + > > > > > + if [ "x$D" = "x" ] ; then > > > > > + # update-alternatives may need the links from > > > > > commands > > > > > added in > > > > > the prerm step > > > > > + # to operate. Make sure we can get to that path. > > > > > + for busybox_rmdir in /tmp/busyboxrm-*; do > > > > > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' ] ; > > > > > then > > > > > + export PATH=$busybox_rmdir:$PATH > > > > > + fi > > > > > + done > > > > > + > > > > > + # Create a temporary directory for the busybox > > > > > binary > > > > > and the link lists > > > > > + BUSYBOX=${base_bindir}/busybox > > > > > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d > > > > > /tmp/busyboxpreinst- > > > > > XXXXXX` > > > > > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > > > > > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > > > > > + > > > > > + # Go through all the links and install an > > > > > alternative > > > > > that points to the > > > > > temporary > > > > > + # busybox binary. > > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > > + if [ -e ${sysconfdir}/busybox.links$suffix ] ; > > > > > then > > > > > + $BUSYBOX cp > > > > > ${sysconfdir}/busybox.links$suffix > > > > > $BUSYBOX_TMP_DIR > > > > > + while read link; do > > > > > + update-alternatives --install $link > > > > > $($BUSYBOX basename $link) > > > > > $BUSYBOX_TMP_LOC 1 > > > > > + done < > > > > > $BUSYBOX_TMP_DIR/busybox.links$suffix > > > > > + fi > > > > > + done > > > > > + fi > > > > > +} > > > > > + > > > > > pkg_postrm_${PN} () { > > > > > # Add path to remove dir in case we removed our only > > > > > grep > > > > > if [ "x$D" = "x" ] ; then > > > > > -- > > > > > 2.17.1 > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#161041): https://lists.openembedded.org/g/openembedded-core/message/161041 Mute This Topic: https://lists.openembedded.org/mt/87919018/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
