Thanks Stephen. That is exactly what I had done on my end. Sorry I couldn't send the patch directly, but you made the identical changes.
Brian On 02/13/2015 01:10 PM, Stephen Herr wrote: > Thanks guys, hopefully this looks reasonable: > https://github.com/spacewalkproject/spacewalk/commit/76883065ff72449726adc82d081ae681f2a5bb59 > > > RHEL 6 does not have pyliblzma installed by default but it's available > in EPEL so I had to add a requirement to the spec file. RHEL 5 doesn't > have it available but it should be fine since we shouldn't be > releasing Spacewalk 2.3 for RHEL 5 anyway. > > -Stephen > > On 02/03/2015 01:04 PM, Michael Guidero wrote: >> I have Spacewalk 2.2 and this does occur with it as well. I have the >> issue with the Scientific Linux repository. >> >> http://ftp.scientificlinux.org/linux/scientific/7x/x86_64/os/repodata/ >> >> Spacewalk delivers a compressed comps.xml file to my client systems with >> no .xz extension. >> >> [root@swtest01 sl-7-x86_64-7x]# pwd >> /var/cache/yum/x86_64/7/sl-7-x86_64-7x >> [root@swtest01 sl-7-x86_64-7x]# file comps.xml >> comps.xml: XZ compressed data >> >> Yum complains: >> >> [root@swtest01 sl-7-x86_64-7x]# yum group list >> Loaded plugins: rhnplugin, versionlock >> This system is receiving updates from RHN Classic or Red Hat Satellite. >> There is no installed groups file. >> Maybe run: yum groups mark convert (see man yum) >> Failed to add groups file for repository: sl-7-x86_64-7x - comps file is >> empty/damaged >> Available environment groups: >> ... >> >> >> >> >> >> On Fri, Jan 30, 2015 at 8:21 AM, Brian Buesker >> <[email protected] <mailto:[email protected]>> wrote: >> >> Since I don't have Spacewalk 2.2 or one of the nightly versions >> installed here, I can't say for sure that the issue does not >> exist in >> them. Note though that I was able to sync the repository >> successfully >> even before I made the change. The problem is that spacewalk is >> still >> storing the compressed comps.xml but serves it to yum clients as >> if it >> was not compressed. >> >> Can you confirm that the comps.xml as stored in the >> satellite/rhn/comps >> directory for the channel is the uncompressed XML file? I found >> that it >> would always be the LZMA-compressed version and be named >> >> 503597f15d6c693a96558f46fbd8ebb5853b92ef528de2a7d55a4dd021162873-comps-f21.xml.xz. >> However, the repomd.xml file indicates that the comps file is the >> uncompressed form. As best I can tell, spacewalk is serving the >> compressed comps.xml as if it were uncompressed (as far as yum is >> concerned). This causes yum to expect it to be an uncompressed XML >> document and it thus generates an error indicating it is >> malformed. You >> would only notice this if you actually subscribed a system to the >> channel and tried to do some sort of yum operation. From the >> perspective >> of Spacewalk itself, everything seems to work fine. >> >> The fix I found was to make sure spacewalk stores the >> uncompressed one, >> since that is what it seems to indicate is present in repomd.xml. >> This >> appears to be consistent with previous distros where the >> comps.xml is >> gzip compressed but is stored/served by spacewalk in uncompressed >> form. >> >> Brian >> >> On 01/30/2015 05:20 AM, Tomas Lestach wrote: >> > I was able to sync fc21 repo on Spacewalk nightly together with >> the comps file. >> > Feel free to check [1] for more information. >> > >> > Even if Brian installed pyliblzma, it's possible there were more >> changes done >> > since Spacewalk 2.1 that affect the issue. >> > >> > Regards, >> > -- >> > Tomas Lestach >> > Red Hat Satellite Engineering, Red Hat >> > >> > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1144008 >> > >> > >> > ----- Original Message ----- >> >> From: "Stephen Herr" <[email protected] >> <mailto:[email protected]>> >> >> To: [email protected] <mailto:[email protected]> >> >> Sent: Thursday, January 29, 2015 4:07:08 PM >> >> Subject: Re: [Spacewalk-list] Support for LZMA-compressed >> comps.xml >> >> >> >> Here you go Tomas: >> >> >> >> >> >> https://dl.fedoraproject.org/pub/fedora/linux/releases/21/Everything/x86_64/os/repodata/ >> >> >> >> Take a look at *-comps-f21.xml.xz >> >> >> >> -Stephen >> >> >> >> On 01/29/2015 06:23 AM, Tomas Lestach wrote: >> >>> Would you share a public repo with lzma compressed comps? >> >>> >> >>> Regards, >> >>> -- >> >>> Tomas Lestach >> >>> Red Hat Satellite Engineering, Red Hat >> >>> >> >>> >> >>> ----- Original Message ----- >> >>>> From: "Brian Buesker" <[email protected] >> <mailto:[email protected]>> >> >>>> To: [email protected] >> <mailto:[email protected]> >> >>>> Sent: Wednesday, January 28, 2015 9:53:48 PM >> >>>> Subject: [Spacewalk-list] Support for LZMA-compressed >> comps.xml >> >>>> >> >>>> >> >>>> I've started trying to bring online a Fedora 21 >> distro/kickstart >> >>>> profile against our Spacewalk 2.1 server. Unlike others who >> have >> >>>> had >> >>>> trouble getting the fedora installer generation to work, it >> mostly >> >>>> seems to be working for me. I'm not sure if that is because >> I am >> >>>> using 2.1 instead of 2.2 or if there is some other >> explanation. >> >>>> >> >>>> However, one thing I ran into is that when I created a >> Fedora 21 >> >>>> Everything channel, Spacewalk ended up not handling the >> comps.xml >> >>>> properly. In the upstream yum repo, the comps.xml is >> actually LZMA >> >>>> compressed. It does not look like Spacewalk decompresses >> this when >> >>>> staging the comps in the local file system. Thus, when a >> client >> >>>> fetches the repo metadata, it gets a file named comps.xml >> that is >> >>>> actually compressed and thus it complains about the XML not >> being >> >>>> well formed. >> >>>> >> >>>> I looked at the Spacewalk git repo and I see that an RPM >> >>>> dependency >> >>>> was added for pyliblzma. I already have that installed on my >> >>>> system, >> >>>> but despite that, Spacewalk still does not handle the >> compressed >> >>>> comps properly. I dug further into this and found that two >> python >> >>>> modules need to be updated, namely reposync.py and >> fileutils.py. >> >>>> >> >>>> I would provide a patch with the fixes, although I have to go >> >>>> through >> >>>> a somewhat lengthy procedure before I can get permission >> from my >> >>>> employer to release the patch. Therefore, I'll describe the >> >>>> necessary changes in case others want to make the same ones >> (or >> >>>> even >> >>>> better, get this updated in git). >> >>>> >> >>>> >> >>>> 1. In reposync.py, the list of suffixes in >> import_groups() >> >>>> needs >> >>>> to include '.xz'. >> >>>> 2. In fileutils.py, decompress_open() needs to be >> updated to >> >>>> use >> >>>> the lzma.LZMAFile() function on files ending in '.xz' >> (and of >> >>>> course an import is needed as well). >> >>>> >> >>>> >> >>>> Let me know if you have any questions. >> >>>> >> >>>> >> >>>> Thanks, >> >>>> Brian Buesker >> >>>> >> >>>> _______________________________________________ >> >>>> Spacewalk-list mailing list >> >>>> [email protected] <mailto:[email protected]> >> >>>> https://www.redhat.com/mailman/listinfo/spacewalk-list >> >>> _______________________________________________ >> >>> Spacewalk-list mailing list >> >>> [email protected] <mailto:[email protected]> >> >>> https://www.redhat.com/mailman/listinfo/spacewalk-list >> >>> >> >> _______________________________________________ >> >> Spacewalk-list mailing list >> >> [email protected] <mailto:[email protected]> >> >> https://www.redhat.com/mailman/listinfo/spacewalk-list >> >> >> > _______________________________________________ >> > Spacewalk-list mailing list >> > [email protected] <mailto:[email protected]> >> > https://www.redhat.com/mailman/listinfo/spacewalk-list >> > >> >> >> >> _______________________________________________ >> Spacewalk-list mailing list >> [email protected] <mailto:[email protected]> >> https://www.redhat.com/mailman/listinfo/spacewalk-list >> >> >> >> >> -- >> >> Michael Guidero >> Sococo IT >> 650-265-7013 Ext 1000 >> >> >> >> _______________________________________________ >> Spacewalk-list mailing list >> [email protected] >> https://www.redhat.com/mailman/listinfo/spacewalk-list >> > > _______________________________________________ > Spacewalk-list mailing list > [email protected] > https://www.redhat.com/mailman/listinfo/spacewalk-list > > _______________________________________________ Spacewalk-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/spacewalk-list
