[Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
Hi all, after fixing my problems with the errata import, I found another problem with Debian...especially with Debian "stretch" and SW 2.7. At least, I did not yet see a major (or even minor) error with Debian jessie on SW 2.7 I can successfully deploy a Debian stretch system via SW 2.7, register it and install packages. The "installed" packages on the client is reported to SW. No problem so far. But, even if SW reports, that there are no newer packages in SW for the client, and I have done "apt-get upgrade" several times, the "client" does not recognize, that a particular package is already installed. I can install (for example "adduser") with "apt-get install adduser" over and over again. It does not tell.."that package is already installed with the newest version" The client also reports, that it installes the "same" version above "the already" installed version of this package. So, for example, my freshly installed debian system has 372 packages. After installation, "apt-get upgrade", it "always" upgrades about 260 packages again and again and again. Also after applying https://github.com/spacewalkproject/spacewalk/wiki/DebianUbuntuSupportIn27 on the client. It does not change. What is changed then, if that the "installed" packages is correctly "reflected" in the SW WebUI for that system. I *think*, that the problem is located on the client, especially in the "improvement of apt" in Debian 9 (see https://www.debian.org/releases/stable/amd64/release-notes/ch-whats-new.en.html#apt-improvements and https://wiki.debian.org/DebianRepository/Format ) ... If the following fields exist in the control file of a .deb file they also must exist in the record about the package in the Packages file and the value must match /exactly/ or a client might recognize a metadata mismatch and redownloads/reinstalls a package: * Depends et al * Installed-Size * Multi-Arch ... I know that the "Multi-Arch" header is not written to the packages list (see https://bugzilla.redhat.com/show_bug.cgi?id=1243387) and the workaround mentioned there (that I use for debian jessie) does not work here. The "Multi-Arch" header often has different values. I just want to know, if anybody can confirm this behavior. Next time I'm in the office, I'll try to modify the packages list to add the "Multi-Arch" headers (and their) values to the packages list and see, if this is really the problem. Kind regards, Robert ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
[Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
On Fri, 13 Oct 2017 19:17:12, Robert Paschedag wrote: > >Hi all, > >after fixing my problems with the errata import, I found another problem with >Debian...especially with > >Debian "stretch" and SW 2.7. At least, I did not yet see a major (or even >minor) error with Debian jessie on SW 2.7 > >I can successfully deploy a Debian stretch system via SW 2.7, register >it and install packages. > >The "installed" packages on the client is reported to SW. No problem so far. > >But, even if SW reports, that there are no newer packages in SW for the >client, and I have done > >"apt-get upgrade" several times, the "client" does not recognize, that a >particular package > Hi Robert, I can confirm that. I've been experiencing the same thing with Ubuntu 14 clients last month and Ubuntu 16 clients this month. The python packages had updated and attempts to update kept on cycling through with the same packages identified as needing updates, until I went through and manually copied the Multi-Arch: allowed settings for the packages that had it (looking at the Packages on the Ubuntu repos). I was thinking of doing what you propose but I've been to busy to dig into the code to try to add the Multi-Arch field to the copied data. I look forward to your changes. Paul-Andre Panon Senior systems administrator ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
ok. Couldn't sit still. Tested the "Multi-Arch" header on stretch, andthat seems to work Using the Packages.gz file created by spacewalk, I can install "alex" over and over again root@stretch:/tmp# dpkg -l alex Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==---= ii alex 3.1.7-4 amd64lexical analyser generator for Ha root@stretch:/tmp# apt-get install alex Reading package lists... Done Building dependency tree Reading state information... Done The following packages will be upgraded: alex 1 upgraded, 0 newly installed, 0 to remove and 358 not upgraded. Need to get 0 B/605 kB of archives. After this operation, 0 B of additional disk space will be used. WARNING: The following packages cannot be authenticated! alex Authentication warning overridden. (Reading database ... 43532 files and directories currently installed.) Preparing to unpack .../alex_3.1.7-4_amd64.deb ... Unpacking alex (3.1.7-4) over (3.1.7-4) ... Setting up alex (3.1.7-4) ... Processing triggers for man-db (2.7.6.1-2) ... Apt-Spacewalk: Updating package profile root@stretch:/tmp# apt-get install alex Reading package lists... Done Building dependency tree Reading state information... Done The following packages will be upgraded: alex 1 upgraded, 0 newly installed, 0 to remove and 358 not upgraded. Need to get 0 B/605 kB of archives. After this operation, 0 B of additional disk space will be used. WARNING: The following packages cannot be authenticated! alex Authentication warning overridden. (Reading database ... 43532 files and directories currently installed.) Preparing to unpack .../alex_3.1.7-4_amd64.deb ... Unpacking alex (3.1.7-4) over (3.1.7-4) ... Setting up alex (3.1.7-4) ... Processing triggers for man-db (2.7.6.1-2) ... Apt-Spacewalk: Updating package profile root@stretch:/tmp# Now changing /var/lib/apt/lists/ and add "Multi-Arch: foreign" as included in "control" of "alex" package, results in root@stretch:/tmp# apt-get install alex Reading package lists... Done Building dependency tree Reading state information... Done alex is already the newest version (3.1.7-4). 0 upgraded, 0 newly installed, 0 to remove and 358 not upgraded. root@stretch:/tmp# If I remove the "Multi-Arch" header again, the old behaviour just returns. So I think, including the "Multi-Arch" header for the package into the database is a must have to support "stretch". But maybe I just have a strange error and someone can give me a hint. I still have to test that with *all* packages. Regards, Robert On 10/13/17 21:17, Robert Paschedag wrote: Hi all, after fixing my problems with the errata import, I found another problem with Debian...especially with Debian "stretch" and SW 2.7. At least, I did not yet see a major (or even minor) error with Debian jessie on SW 2.7 I can successfully deploy a Debian stretch system via SW 2.7, register it and install packages. The "installed" packages on the client is reported to SW. No problem so far. But, even if SW reports, that there are no newer packages in SW for the client, and I have done "apt-get upgrade" several times, the "client" does not recognize, that a particular package is already installed. I can install (for example "adduser") with "apt-get install adduser" over and over again. It does not tell.."that package is already installed with the newest version" The client also reports, that it installes the "same" version above "the already" installed version of this package. So, for example, my freshly installed debian system has 372 packages. After installation, "apt-get upgrade", it "always" upgrades about 260 packages again and again and again. Also after applying https://github.com/spacewalkproject/spacewalk/wiki/DebianUbuntuSupportIn27 on the client. It does not change. What is changed then, if that the "installed" packages is correctly "reflected" in the SW WebUI for that system. I *think*, that the problem is located on the client, especially in the "improvement of apt" in Debian 9 (see https://www.debian.org/releases/stable/amd64/release-notes/ch-whats-new.en.html#apt-improvements and https://wiki.debian.org/DebianRepository/Format ) ... If the following fields exist in the control file of a .deb file they also must exist in the record about the package in the Packages file and the value must match /exactly/ or a client might recognize a metadata mismatch and redownloads/reinstalls a package: * Depends et al * Installed-Size * Multi-Arch ... I know that the "Multi-Arch" header is not written to the packages list (see https://bugzilla.redhat.com/show_bug.cgi?id=1243387) and the workaround mentioned there (that I use for debian jessie) d
Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
Am 14. Oktober 2017 01:38:59 MESZ schrieb Paul-Andre Panon : >On Fri, 13 Oct 2017 19:17:12, Robert Paschedag > wrote: >> >>Hi all, >> >>after fixing my problems with the errata import, I found another >problem with Debian...especially with >> >>Debian "stretch" and SW 2.7. At least, I did not yet see a major (or >even minor) error with Debian jessie on SW 2.7 >> >>I can successfully deploy a Debian stretch system via SW 2.7, register > >>it and install packages. >> >>The "installed" packages on the client is reported to SW. No problem >so far. >> >>But, even if SW reports, that there are no newer packages in SW for >the >>client, and I have done >> >>"apt-get upgrade" several times, the "client" does not recognize, that >a >>particular package >> > >Hi Robert, > >I can confirm that. I've been experiencing the same thing with Ubuntu >14 clients last month and Ubuntu 16 clients this month. The python >packages had updated and attempts to update kept on cycling through >with the same packages identified as needing updates, until I went >through and manually copied the Multi-Arch: allowed settings for the >packages that had it (looking at the Packages on the Ubuntu repos). I >was thinking of doing what you propose but I've been to busy to dig >into the code to try to add the Multi-Arch field to the copied data. I >look forward to your changes. > >Paul-Andre Panon >Senior systems administrator > > >___ >Spacewalk-list mailing list >Spacewalk-list@redhat.com >https://www.redhat.com/mailman/listinfo/spacewalk-list Hi Paul-Andre, the big problem is... "allowed" is not the only value for Multi-Arch. In Debian stretch "main" repo, there are 8979 Packages with "Multi-Arch" header. No way to do that by hand. Another thing. I hope you can give me a hint. Apt now tries (or even just generates) "binary-all" package files for all "amd64" repos, the client has subscribed. How can I stop that? All "architecture" settings are set to "amd64". Don't know, where this *feature* came from but it looks like breaking my "workaround" I'm trying to implement. Thanks Robert ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
On 10/16/17 21:25, Paul-Andre Panon wrote: On Saturday, October 14, 2017 12:02 AM, Robert Paschedag [mailto:robert.pasche...@web.de] wrote: Am 14. Oktober 2017 01:38:59 MESZ schrieb Paul-Andre Panon : On Fri, 13 Oct 2017 19:17:12, Robert Paschedag Hi Robert, I can confirm that. I've been experiencing the same thing with Ubuntu 14 clients last month and Ubuntu 16 clients this month. The python packages had updated and attempts to update kept on cycling through with the same packages identified as needing updates, until I went through and manually copied the Multi-Arch: allowed settings for the packages that had it (looking at the Packages on the Ubuntu repos). I was thinking of doing what you propose but I've been to busy to dig into the code to try to add the Multi-Arch field to the copied data. I look forward to your changes. Paul-Andre Panon Senior systems administrator Hi Paul-Andre, the big problem is... "allowed" is not the only value for Multi-Arch. In Debian stretch "main" repo, there are 8979 Packages with "Multi-Arch" header. No way to do that by hand. Agreed, but so far I have only had problems with packages where the missing Multi-Arch value is "allowed". Maybe I've just been lucky though. The annoying thing is that, even with the ~couple dozen cases of "Multi-Arch: allowed" in the Ubuntu main and universe repos (mostly from python and the gcc toolchain), every sync wipes out your last set of manual fixes. When I ran into the problem, I found the clue to a workaround in matus' Jan 30th 2015 comment at http://www.devops-blog.net/spacewalk/registering-ubuntu-and-debian-servers-with-spacewalk, so this isn't a new issue. I'm not happy with his kludgy "fix" though since it also winds up adding that property to many packages where it doesn't apply. Yeah. I'm also using that workaround from the bugzilla report about the missing Multi-Arch headers and in jessie, this really works quite good. Now, with "stretch", this is just a nightmare. I was so happy, that now in SW2.7, the debian version string parsing has been improved and now debian has "improved" "apt". But that improvement somehow throws a big stick between your legs while you're running at max speed, causing you to crash and break with your face on the ground :-P Another thing. I hope you can give me a hint. Apt now tries (or even just generates) "binary-all" package files for all "amd64" repos, the client has subscribed. How can I stop that? All "architecture" settings are set to "amd64". Don't know, where this *feature* came from but it looks like breaking my "workaround" I'm trying to implement. I'm not sure, we've only been using the 64-bit amd64 images, not the 32-bit i386 images. When I look at the Packages file on the binary-amd64 Ubuntu directory from the closest mirror, there doesn't seem to be a package with something other than Architecture: amd64 . Since we've been all 64-bit it hasn't been an issue for us, and maybe that's why I'm not having any issues with other possible values of Multi-Arch as well. Thanks Robert Paul-Andre Panon Senior systems administrator Office: 604.629.5182 ext 2341 Mobile: 604.679.1617 I found the "error". It is really because of the "missing" "Release" file. If this is found, and there is an "Architectures:" header, the new "apt" will download the Packages for this architectures. If the "Release" is missing, "apt" will download the Packages for the clients configured "architectures" + "all". This is a new behaviour. But I think, that this is not my main error. Because I'm still using Steve Meier's "debian sync" tool to download and import the debian packages, I modified that to create a "temporary" file with the "package" name and possible "multi-arch" header, which I can insert, after SW has created his own "Packages" file. But this does not yet work 100%, because there are sometimes more than one package with exact same name. I have to improve that and keep trying. Really the best thing would be, if the "Multi-Arch" header would be added to the database. If I'm right, this header is already "present", while the .deb files gets parsed but it just gets ignored. And I don't know, how much work and modification is needed to fully implement this header to every .deb package. Would be cool, if the main developers of SW could look into that issue and I'll really try to support them with as much information as possible to get this fixed. As final information. This is not an error in SW. This is "an improvement" in Debian 9, which most of us (and I think most of SW developers) just did not get but its a major problem to "manage" debian 9 with spacewalk. Robert ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
On Monday, October 16, 2017 2:08 PM, Robert Paschedag [mailto:robert.pasche...@web.de] wrote: >On 10/16/17 21:25, Paul-Andre Panon wrote: >> On Saturday, October 14, 2017 12:02 AM, Robert Paschedag >> [mailto:robert.pasche...@web.de] wrote: >>> Am 14. Oktober 2017 01:38:59 MESZ schrieb Paul-Andre Panon >>> : >>> Hi Paul-Andre, >>> >>> the big problem is... "allowed" is not the only value for Multi-Arch. In >>> Debian stretch "main" repo, there are 8979 Packages with "Multi-Arch" >>> header. No way to do that by hand. >> Agreed, but so far I have only had problems with packages where the missing >> Multi-Arch value is "allowed". Maybe I've just been lucky though. The >> annoying thing is that, even with the ~couple dozen cases of "Multi-Arch: >> allowed" in the Ubuntu main and universe repos (mostly from python and the >> gcc toolchain), every sync wipes out your last set of manual fixes. When I >> ran into the problem, I found the clue to a workaround in matus' Jan 30th >> 2015 comment at >> http://www.devops-blog.net/spacewalk/registering-ubuntu-and-debian-servers-with-spacewalk, >> so this isn't a new issue. I'm not happy with his kludgy "fix" though since >> it also winds up adding that property to many packages where it doesn't >> apply. >Yeah. I'm also using that workaround from the bugzilla report about the >missing Multi-Arch headers and in jessie, this really works quite good. >Now, with "stretch", this is just a nightmare. I was so happy, that now in >SW2.7, the debian version string parsing has been improved and now debian has >"improved" "apt". But that improvement somehow throws a big stick between your >legs while you're running at max speed, causing you to crash and break with >your face on the ground :-P > >>> Thanks >>> Robert >> Paul-Andre Panon >> Senior systems administrator >> >> Office: 604.629.5182 ext 2341 Mobile: 604.679.1617 > >I found the "error". It is really because of the "missing" "Release" >file. If this is found, and there is an "Architectures:" header, the new "apt" >will download the Packages for this architecture. If the "Release" is missing, >"apt" will download the Packages for the clients configured "architectures" + >"all". > >This is a new behaviour. Ah, that explains why I haven't been seeing the same problems then then. I've been building Release files using another of Philip's blog postings (and customized it to build InRelease files as well). http://www.devops-blog.net/spacewalk/gpg-signing-apt-repository-in-spacewalk > >But I think, that this is not my main error. Because I'm still using Steve >Meier's "debian sync" tool to download and import the debian packages, I >modified that to create a "temporary" file with the "package" name and >possible "multi-arch" header, which I can insert, after SW has created his own >"Packages" file. > >But this does not yet work 100%, because there are sometimes more than one >package with exact same name. I have to improve that and keep trying. > >Really the best thing would be, if the "Multi-Arch" header would be added to >the database. If I'm right, this header is already "present", while the .deb >files gets parsed but it just gets ignored. And I don't know, how much work >and modification is needed to fully implement this header to every .deb >package. It appears to be in the .deb package so if that's what is being used as a metadata source (and not the Package file from the mirror site) then it would still be valid. ~$ dpkg-deb -I /var/cache/apt/archives/python2.7-minimal_2.7.12-1ubuntu0~16.04.1_amd64.deb new debian package, version 2.0. size 1295108 bytes: control archive=2355 bytes. 826 bytes,21 lines control 338 bytes, 5 lines md5sums 2365 bytes,78 lines * postinst #!/bin/sh 416 bytes,21 lines * postrm #!/bin/sh 900 bytes,39 lines * preinst #!/bin/sh 813 bytes,36 lines * prerm#!/bin/sh Package: python2.7-minimal Source: python2.7 Version: 2.7.12-1ubuntu0~16.04.1 Architecture: amd64 Maintainer: Ubuntu Core Developers Installed-Size: 3584 Pre-Depends: libc6 (>= 2.15) Depends: libpython2.7-minimal (= 2.7.12-1ubuntu0~16.04.1), zlib1g (>= 1:1.2.0) Recommends: python2.7 Suggests: binfmt-support Conflicts: binfmt-support (<< 1.1.2) Replaces: python2.7 (<< 2.7.8-7~) Section: python Priority: standard Multi-Arch: allowed Description: Minimal subset of the Python language (version 2.7) This package contains the interpreter and some essential modules. It can be used in the boot process for some basic tasks. See /usr/share/doc/python2.7-minimal/README.Debian for a list of the modules contained in this package. Original-Maintainer: Matthias Klose I agree that adding it to the database was what I thought would be needed to do it right as well. However it's an optional field that isn't set on all packages. I looked at the database
Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
-Ursprüngliche Nachricht- Von: spacewalk-list-boun...@redhat.com [mailto:spacewalk-list-boun...@redhat.com] Im Auftrag von Paul-Andre Panon Gesendet: Dienstag, 17. Oktober 2017 03:53 An: Robert Paschedag ; spacewalk-list@redhat.com Betreff: Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again On Monday, October 16, 2017 2:08 PM, Robert Paschedag [mailto:robert.pasche...@web.de] wrote: >On 10/16/17 21:25, Paul-Andre Panon wrote: >> On Saturday, October 14, 2017 12:02 AM, Robert Paschedag >> [mailto:robert.pasche...@web.de] wrote: >>> Am 14. Oktober 2017 01:38:59 MESZ schrieb Paul-Andre Panon >>> : >>> Hi Paul-Andre, >>> >>> the big problem is... "allowed" is not the only value for Multi-Arch. In >>> Debian stretch "main" repo, there are 8979 Packages with "Multi-Arch" >>> header. No way to do that by hand. >> Agreed, but so far I have only had problems with packages where the missing >> Multi-Arch value is "allowed". Maybe I've just been lucky though. The >> annoying thing is that, even with the ~couple dozen cases of "Multi-Arch: >> allowed" in the Ubuntu main and universe repos (mostly from python and the >> gcc toolchain), every sync wipes out your last set of manual fixes. When I >> ran into the problem, I found the clue to a workaround in matus' Jan 30th >> 2015 comment at >> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.devops-blog.net%2Fspacewalk%2Fregistering-ubuntu-and-debian-servers-with-spacewalk&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C9ed97f18f9e14fe781fa08d51501f142%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438020499979024&sdata=%2Bjf%2FUgIwxmI40%2FHPcdHSgqij6J4h98inzkmH35xLSuU%3D&reserved=0, >> so this isn't a new issue. I'm not happy with his kludgy "fix" though since >> it also winds up adding that property to many packages where it doesn't >> apply. >Yeah. I'm also using that workaround from the bugzilla report about the >missing Multi-Arch headers and in jessie, this really works quite good. >Now, with "stretch", this is just a nightmare. I was so happy, that now >in SW2.7, the debian version string parsing has been improved and now >debian has "improved" "apt". But that improvement somehow throws a big >stick between your legs while you're running at max speed, causing you >to crash and break with your face on the ground :-P > >>> Thanks >>> Robert >> Paul-Andre Panon >> Senior systems administrator >> >> Office: 604.629.5182 ext 2341 Mobile: 604.679.1617 > >I found the "error". It is really because of the "missing" "Release" >file. If this is found, and there is an "Architectures:" header, the new "apt" >will download the Packages for this architecture. If the "Release" is missing, >"apt" will download the Packages for the clients configured "architectures" + >"all". > >This is a new behaviour. Ah, that explains why I haven't been seeing the same problems then then. I've been building Release files using another of Philip's blog postings (and customized it to build InRelease files as well). https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.devops-blog.net%2Fspacewalk%2Fgpg-signing-apt-repository-in-spacewalk&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C9ed97f18f9e14fe781fa08d51501f142%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438020499979024&sdata=POm2n8ppgRctqjdYyw5kzWLrcz4CDIYHarcla2Mhmjg%3D&reserved=0 > >But I think, that this is not my main error. Because I'm still using Steve >Meier's "debian sync" tool to download and import the debian packages, I >modified that to create a "temporary" file with the "package" name and >possible "multi-arch" header, which I can insert, after SW has created his own >"Packages" file. > >But this does not yet work 100%, because there are sometimes more than one >package with exact same name. I have to improve that and keep trying. > >Really the best thing would be, if the "Multi-Arch" header would be added to >the database. If I'm right, this header is already "present", while the .deb >files gets parsed but it just gets ignored. And I don't know, how much work >and modification is needed to fully implement this header to every .deb >package. It appears to be in the .deb package so if that's what is being used as a metadata source (and not the Package file from the mir
Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
Ok. Looks like I got it working now. After creating a very, very ugly script to add the missing headers to the Packages file, I discovered the deb822 module, already installed on the spacewalk-server (python-debian), which can easily parse a Packages file. With the information extracted with Steven Meiers "spacewalk-debian-sync" script (while syncing the Debian repo), modifying Packages was pretty easy. See a modified version of Steve Meiers script and the new script to add the missing "Multi-Arch" header here https://github.com/rpasche/spacewalk-debian-sync/tree/devel At least, this works pretty good on my test system, and after "apt-get upgrade", another "apt-get upgrade" shows "no packages to upgrade" 😊 root@stretch:~# apt-get upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. root@stretch:~# Robert -Ursprüngliche Nachricht- Von: spacewalk-list-boun...@redhat.com [mailto:spacewalk-list-boun...@redhat.com] Im Auftrag von Paschedag, Robert Gesendet: Dienstag, 17. Oktober 2017 10:02 An: spacewalk-list@redhat.com; Robert Paschedag Betreff: Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again -Ursprüngliche Nachricht- Von: spacewalk-list-boun...@redhat.com [mailto:spacewalk-list-boun...@redhat.com] Im Auftrag von Paul-Andre Panon Gesendet: Dienstag, 17. Oktober 2017 03:53 An: Robert Paschedag ; spacewalk-list@redhat.com Betreff: Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again On Monday, October 16, 2017 2:08 PM, Robert Paschedag [mailto:robert.pasche...@web.de] wrote: >On 10/16/17 21:25, Paul-Andre Panon wrote: >> On Saturday, October 14, 2017 12:02 AM, Robert Paschedag >> [mailto:robert.pasche...@web.de] wrote: >>> Am 14. Oktober 2017 01:38:59 MESZ schrieb Paul-Andre Panon >>> : >>> Hi Paul-Andre, >>> >>> the big problem is... "allowed" is not the only value for Multi-Arch. In >>> Debian stretch "main" repo, there are 8979 Packages with "Multi-Arch" >>> header. No way to do that by hand. >> Agreed, but so far I have only had problems with packages where the missing >> Multi-Arch value is "allowed". Maybe I've just been lucky though. The >> annoying thing is that, even with the ~couple dozen cases of "Multi-Arch: >> allowed" in the Ubuntu main and universe repos (mostly from python and the >> gcc toolchain), every sync wipes out your last set of manual fixes. When I >> ran into the problem, I found the clue to a workaround in matus' Jan 30th >> 2015 comment at >> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.devops-blog.net%2Fspacewalk%2Fregistering-ubuntu-and-debian-servers-with-spacewalk&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C9ed97f18f9e14fe781fa08d51501f142%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438020499979024&sdata=%2Bjf%2FUgIwxmI40%2FHPcdHSgqij6J4h98inzkmH35xLSuU%3D&reserved=0, >> so this isn't a new issue. I'm not happy with his kludgy "fix" though since >> it also winds up adding that property to many packages where it doesn't >> apply. >Yeah. I'm also using that workaround from the bugzilla report about the >missing Multi-Arch headers and in jessie, this really works quite good. >Now, with "stretch", this is just a nightmare. I was so happy, that now >in SW2.7, the debian version string parsing has been improved and now >debian has "improved" "apt". But that improvement somehow throws a big >stick between your legs while you're running at max speed, causing you >to crash and break with your face on the ground :-P > >>> Thanks >>> Robert >> Paul-Andre Panon >> Senior systems administrator >> >> Office: 604.629.5182 ext 2341 Mobile: 604.679.1617 > >I found the "error". It is really because of the "missing" "Release" >file. If this is found, and there is an "Architectures:" header, the new "apt" >will download the Packages for this architecture. If the "Release" is missing, >"apt" will download the Packages for the clients configured "architectures" + >"all". > >This is a new behaviour. Ah, that explains why I haven't been seeing the same problems then then. I've been building Release files using another of Philip's blog postings (and customized it to build InRelease files as well). https://emea01.safelinks.protection.outlook.com
Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
On Tue, 17 Oct 2017 5:36:50 , Robert Paschedag wrote: > >Ok. > >Looks like I got it working now. > >After creating a very, very ugly script to add the missing headers to the >Packages file, I discovered the deb822 module, already installed on the >spacewalk-server (python-debian), which can easily parse a Packages file. > >With the information extracted with Steven Meiers "spacewalk-debian-sync" >script (while syncing the Debian repo), modifying Packages was pretty easy. > >See a modified version of Steve Meiers script and the new script to add the >missing "Multi-Arch" header here >https://github.com/rpasche/spacewalk-debian-sync/tree/devel > >At least, this works pretty good on my test system, and after "apt-get >upgrade", another "apt-get upgrade" shows "no packages to upgrade" ? > >Robert I think going back to using a modified version of Steve Meier's script would be a step back that I would prefer to avoid. I would prefer to update the spacewalk-repo-sync command. In addition to the rhn_deb.py file you indicated, there also appears to be a repo-specific plugin, deb_src.py, that appears to be pulling down the Packages file from the repo and determining what packages (and versions) and corresponding URLs are available, https://github.com/spacewalkproject/spacewalk/blob/45553919f852c4dbc5fcfe1c24499b2f9e9f544b/backend/satellite_tools/repo_plugins/deb_src.py One interesting thing is that that file has an empty get_updates() method which may be for Errata. If you were going to add a hook to incorporate Ubuntu errata (from https://lists.ubuntu.com/archives/ubuntu-security-announce/) and/or Debian errata, that may be where it would have to go but you would have to prepare a data/class structure similar to what the yum_src plugin uses (from yum.update_md import UpdateMetadata, UpdateNoticeException, UpdateNotice). You would want new classes to parse the external Errata sources (sub-classed for the distro, but you would need a new repo metadata field to determine the distro) and then would cross-correlate the results for what applies to that repo. Seems messy and an externally scheduled process may be somewhat cleaner. Anyways, I think you're correct that rhn_deb is used to parse the deb packages. Instead of subclassing per repo, rhn_pkg fronts rhn_deb and the other package header parsers. What populates the db tables (to add a multi-arch table row)? The sync() method in spacewalk/backend/satellite_tools/reposync.py probably invokes spacewalk/backend/satellite_tools/repo_plugins/__init__.py via the repo plugins (and load_checksum_from_header() calls the deb header parser), and the plugin __init__.py also has an upload_package() method which calls *push_package()* in spacewalk/backend/server/rhnPackageUpload.py to populate the db with the parsed header values. What I haven't yet figured out is a) what generates the Spacewalk Packages file from the DB tables or header data structures. b) The Package class in importLib.py and packageImport.py in https://github.com/spacewalkproject/spacewalk/tree/45553919f852c4dbc5fcfe1c24499b2f9e9f544b/backend/server/importlib may be involved but I'm still figuring them out. I've seen one reference to rhn_pkg in packageImport.py, but it's in _import_signatures so they might not need to change. So far, it looks like you would need to a) add the table with the DDL I gave earlier b) change rhn_deb to include parsing the Multi-Arch header value c) change push_package() in rhnPackageUpload.py to populate the new table d) change get_info_for_package in spacewalk/backend/server/rhnPackage.py to download the multi-arch value from the new table. e) change ? to add Multi-Arch to the Package file creation f) it's non-profit Cheers, Paul-Andre Panon Senior systems administrator ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
Am 18. Oktober 2017 05:38:11 MESZ schrieb Paul-Andre Panon : >On Tue, 17 Oct 2017 5:36:50 , Robert Paschedag > wrote: >> >>Ok. >> >>Looks like I got it working now. >> >>After creating a very, very ugly script to add the missing headers to >the Packages file, I discovered the deb822 module, already installed on >the spacewalk-server (python-debian), which can easily parse a Packages >file. >> >>With the information extracted with Steven Meiers >"spacewalk-debian-sync" script (while syncing the Debian repo), >modifying Packages was pretty easy. >> >>See a modified version of Steve Meiers script and the new script to >add the missing "Multi-Arch" header here >https://github.com/rpasche/spacewalk-debian-sync/tree/devel >> >>At least, this works pretty good on my test system, and after "apt-get >upgrade", another "apt-get upgrade" shows "no packages to upgrade" ? >> >>Robert > >I think going back to using a modified version of Steve Meier's script >would be a step back that I would prefer to avoid. I would prefer to >update the spacewalk-repo-sync command. > >In addition to the rhn_deb.py file you indicated, there also appears to >be a repo-specific plugin, deb_src.py, that appears to be pulling down >the Packages file from the repo and determining what packages (and >versions) and corresponding URLs are available, >https://github.com/spacewalkproject/spacewalk/blob/45553919f852c4dbc5fcfe1c24499b2f9e9f544b/backend/satellite_tools/repo_plugins/deb_src.py >One interesting thing is that that file has an empty get_updates() >method which may be for Errata. If you were going to add a hook to >incorporate Ubuntu errata (from >https://lists.ubuntu.com/archives/ubuntu-security-announce/) and/or >Debian errata, that may be where it would have to go but you would have >to prepare a data/class structure similar to what the yum_src plugin >uses (from yum.update_md import UpdateMetadata, UpdateNoticeException, >UpdateNotice). You would want new classes to parse the external Errata >sources (sub-classed for the distro, but you would need a new repo >metadata field to determine the distro) and then would cross-correlate >the results for what applies to that repo. Seems messy and an >externally scheduled process may be somewhat cleaner. > >Anyways, I think you're correct that rhn_deb is used to parse the deb >packages. Instead of subclassing per repo, rhn_pkg fronts rhn_deb and >the other package header parsers. >What populates the db tables (to add a multi-arch table row)? The >sync() method in spacewalk/backend/satellite_tools/reposync.py >probably invokes >spacewalk/backend/satellite_tools/repo_plugins/__init__.py via the repo >plugins (and load_checksum_from_header() calls the deb header parser), >and the plugin __init__.py also has an upload_package() method which >calls *push_package()* in spacewalk/backend/server/rhnPackageUpload.py >to populate the db with the parsed header values. > >What I haven't yet figured out is >a) what generates the Spacewalk Packages file from the DB tables or >header data structures. > >b) The Package class in importLib.py and packageImport.py in >https://github.com/spacewalkproject/spacewalk/tree/45553919f852c4dbc5fcfe1c24499b2f9e9f544b/backend/server/importlib >may be involved but I'm still figuring them out. I've seen one >reference to rhn_pkg in packageImport.py, but it's in >_import_signatures so they might not need to change. > >So far, it looks like you would need to >a) add the table with the DDL I gave earlier >b) change rhn_deb to include parsing the Multi-Arch header value >c) change push_package() in rhnPackageUpload.py to populate the new >table >d) change get_info_for_package in >spacewalk/backend/server/rhnPackage.py to download the multi-arch value >from the new table. >e) change ? to add Multi-Arch to the Package file creation >f) it's non-profit > >Cheers, > >Paul-Andre Panon >Senior systems administrator Your right, but I came from a SW2.4, which was unable to sync Debian repos. So this is currently no step back (for me) and now with that workaround, I can tell the customer that "I" support Debian 9 now in the company, as all problems with that have been solved. Many people already asked for Debian 9 and that update problem was a showstopper. And I agree That should be fixed and integrated in SW.. Robert ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
-Ursprüngliche Nachricht- Von: spacewalk-list-boun...@redhat.com [mailto:spacewalk-list-boun...@redhat.com] Im Auftrag von Paul-Andre Panon Gesendet: Mittwoch, 18. Oktober 2017 05:38 An: spacewalk-list@redhat.com; Robert Paschedag Betreff: Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again On Tue, 17 Oct 2017 5:36:50 , Robert Paschedag wrote: > >Ok. > >Looks like I got it working now. > >After creating a very, very ugly script to add the missing headers to the >Packages file, I discovered the deb822 module, already installed on the >spacewalk-server (python-debian), which can easily parse a Packages file. > >With the information extracted with Steven Meiers "spacewalk-debian-sync" >script (while syncing the Debian repo), modifying Packages was pretty easy. > >See a modified version of Steve Meiers script and the new script to add the >missing "Multi-Arch" header here >https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frpasche%2Fspacewalk-debian-sync%2Ftree%2Fdevel&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C97788f18a5b342344d0c08d515da0173%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438948434825640&sdata=Ii%2BQ3AZE%2FpM0jxefRMS4p6s%2Bn7CFkUrBloramIPkjRw%3D&reserved=0 > >At least, this works pretty good on my test system, and after "apt-get >upgrade", another "apt-get upgrade" shows "no packages to upgrade" ? > >Robert I think going back to using a modified version of Steve Meier's script would be a step back that I would prefer to avoid. I would prefer to update the spacewalk-repo-sync command. In addition to the rhn_deb.py file you indicated, there also appears to be a repo-specific plugin, deb_src.py, that appears to be pulling down the Packages file from the repo and determining what packages (and versions) and corresponding URLs are available, https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fspacewalkproject%2Fspacewalk%2Fblob%2F45553919f852c4dbc5fcfe1c24499b2f9e9f544b%2Fbackend%2Fsatellite_tools%2Frepo_plugins%2Fdeb_src.py&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C97788f18a5b342344d0c08d515da0173%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438948434825640&sdata=UR26BszhI8uyGO3zYrLV4bVKgT4%2FmlV7r5LoPQI5seM%3D&reserved=0 One interesting thing is that that file has an empty get_updates() method which may be for Errata. If you were going to add a hook to incorporate Ubuntu errata (from https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.ubuntu.com%2Farchives%2Fubuntu-security-announce%2F&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C97788f18a5b342344d0c08d515da0173%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438948434825640&sdata=K5vt9T%2FhobkSX1GeFyVJUQGLymRTz7tOcM9ei4HB%2FNg%3D&reserved=0) and/or Debian errata, that may be where it would have to go but you would have to prepare a data/class structure similar to what the yum_src plugin uses (from yum.update_md import UpdateMetadata, UpdateNoticeException, UpdateNotice). You would want new classes to parse the external Errata sources (sub-classed for the distro, but you would need a new repo metadata field to determine the distro) and then would cross-correlate the results for what applies to that repo. Seems messy and an externally scheduled process may be somewhat cleaner. Anyways, I think you're correct that rhn_deb is used to parse the deb packages. Instead of subclassing per repo, rhn_pkg fronts rhn_deb and the other package header parsers. What populates the db tables (to add a multi-arch table row)? The sync() method in spacewalk/backend/satellite_tools/reposync.py probably invokes spacewalk/backend/satellite_tools/repo_plugins/__init__.py via the repo plugins (and load_checksum_from_header() calls the deb header parser), and the plugin __init__.py also has an upload_package() method which calls *push_package()* in spacewalk/backend/server/rhnPackageUpload.py to populate the db with the parsed header values. What I haven't yet figured out is a) what generates the Spacewalk Packages file from the DB tables or header data structures. It's https://github.com/spacewalkproject/spacewalk/blob/master/java/code/src/com/redhat/rhn/taskomatic/task/repomd/DebPackageWriter.java and https://github.com/spacewalkproject/spacewalk/blob/master/java/code/src/com/redhat/rhn/taskomatic/task/repomd/DebRepositoryWriter.java This might also be a good place to put creation of "Release" files in future? Packages use the PackageDto class https://github.com/spacewalkproject/spacewalk/blob/master/java/code/src/com/redhat/rhn/frontend/dto/PackageDto.java which should then include some sort of private String multiArch; with get/set methods b) The Package class in importLib.py and pac
