Hi Christina Just replacing sha256 with sha does not work. (including deletion of .py[o,c]) Then the generated hash in the pulp metadata is wrong. And the hash in the repodata/primary.xml.gz is still sha256!
pulp-admin rpm repo content rpm --repo-id centos.5.x86_64.puzzle | grep Data-Random -C 20 Checksum: d44cb4afbbdd2cca8cac107a19ff878847ec03e0 Checksumtype: sha should be # sha1sum perl-Data-Random-0.05-1.el5.noarch.rpm 4ad4b6eb87df53002f2c25223495a1503458420f perl-Data-Random-0.05-1.el5.noarch.rpm After # pulp-admin rpm repo publish run --repo-id centos.5.x86_64.puzzle the repodata/...-primary.xml.gzl file still uses the sha256sum: <name>perl-Data-Random</name> <arch>noarch</arch> <version epoch="0" ver="0.05" rel="1.el5"/> <checksum type="sha256" pkgid="YES">c45114bb7c1b2331e1bc352a691d3eaac1a14f3a6a5011eeae73c1eb8c8f86a2</checksum> # sha256sum perl-Data-Random-0.05-1.el5.noarch.rpm c45114bb7c1b2331e1bc352a691d3eaac1a14f3a6a5011eeae73c1eb8c8f86a2 perl-Data-Random-0.05-1.el5.noarch.rpm Does it work differently on your system? # rpm -qi pulp-admin-client Version : 2.3.1 Regards, Philipp Gassmann Am 17.06.2014 17:43, schrieb Christina Plummer: > Hi Philipp, > > I believe there were 3 lines that needed sha256 to be replaced by sha in > package.py per my diff: > > 70: checksum = _calculate_checksum('sha', bundle.filename) > 78: 'checksumtype' : 'sha', > 143: unit_key['checksumtype'] = 'sha' # hardcoded to this in v1 so > leaving this way for now > > Also, did you make sure to set aside the old .pyc and .pyo files and > restart httpd before performing the upload? > This will, of course, force all your uploaded packages to use 'sha' - so > you would also need to change the checksum-type on your EL6 repos from > sha256 to sha and republish. > Christina > On Tue, Jun 17, 2014 at 11:33 AM, Philipp Gassmann <gassm...@puzzle.ch > <mailto:gassm...@puzzle.ch>> wrote: > > Hi, > > I tried no to modify the > > /usr/lib/python2.6/site-packages/pulp_rpm/extension/admin/upload/package.py > as suggested by Christina Plummer. > (on the separate build-server, not on the pulp server) > just replacing sha256 with sha didn't work. > > _checksum_calculate_checksum(type,file) needs 'sha1' as type to generate > a correct sha1 hash, but the checksum type for the pulp-metadata needs > to be 'sha'. > > Now I managed to get the pulp-metadata right when i check on the command > line: > # pulp-admin rpm repo content rpm --repo-id centos.5.x86_64.puzzle | > grep Data-Random -C 20 > > Arch: noarch > Buildhost: buildserver.not.puzzle.ch > <http://buildserver.not.puzzle.ch/> > Checksum: 4ad4b6eb87df53002f2c25223495a1503458420f > Checksumtype: sha > Description: A module used to generate random data. Useful mostly > for test > programs. > Epoch: 0 > Filename: perl-Data-Random-0.05-1.el5.noarch.rpm > License: Artistic/GPL > Name: perl-Data-Random > Provides: perl(Data::Random) = 0.05-0, > perl(Data::Random::WordList) = > 0.05-0, perl-Data-Random = 0.05-1.el5-0 > Release: 1.el5 > Requires: perl, perl(Carp), perl(Date::Calc), perl(Exporter), > perl(GD) > Vendor: Dag Apt Repository, http://dag.wieers.com/apt/ > Version: 0.05 > > BUT then I run the repo publish command to generate the xml metadata > # pulp-admin rpm repo publish run --repo-id centos.5.x86_64.puzzle > +----------------------------------------------------------------------+ > Publishing Repository [centos.5.x86_64.puzzle] > +----------------------------------------------------------------------+ > > This command may be exited by pressing ctrl+c without affecting the > actual > operation on the server. > > Publishing distributions... > [==================================================] 100% > Distributions: 0/0 items > ... completed > > Publishing packages... > [==================================================] 100% > Packages: 157/157 items > ... completed > > Generating metadata > [\] > ... completed > > Publishing repository over HTTPS > [-] > ... completed > > Publishing repository over HTTP > [\] > ... completed > > AND it still generates the WRONG Hash to the repodata/primary.xml.gz: > > <package type="rpm"> > <name>perl-Data-Random</name> > <arch>noarch</arch><version epoch="0" ver="0.05" rel="1.el5"/> > <checksum type="sha256" > > pkgid="YES">c45114bb7c1b2331e1bc352a691d3eaac1a14f3a6a5011eeae73c1eb8c8f86a2</checksum> > ... > > It looks completely broken, inconsistent on various levels > (architecture/implementation/process). > - checksum calculated per upload > - checksum type per file in a repository > - checksum type as repository option (OK) but without effect. > - inconsistent checksum (type) between metadata in database and in xml > repodata > - type sha means hashlib.new('sha1') > - Manual Package upload should be a core functionality of pulp and it is > so broken > - Bug is known since September last year > - Bug is categorized to "user-experience" > - Pulp is supposed to be a core part of Satellite 6 > > Sorry for the rant, but I expect better quality from Red Hat. > We want to use pulp as yum repository mirror and Server for RHEL/CentOS > 5 & 6, stumbling over such a blocker is very disappointing. > > Regards, > Philipp Gassmann > > Am 17.06.2014 15:36, schrieb Philipp Gassmann: > > Hi Barnaby, > > > > This option (--checksum-type) is not present in the current stable > > version of pulp (2.3.1) as Christina Plummer already mentioned in > March. > > > > And as mentioned below, it is still hardcoded in the current > master branch: > > # Checksum > > unit_key['checksumtype'] = 'sha256' # hardcoded to this in v1 so > > leaving this way for now > > > > > > https://github.com/pulp/pulp_rpm/blame/master/extensions_admin/pulp_rpm/extensions/admin/upload/package.py#L178 > > > > > > from the details of repo list --detail (see below) it looks like > > "Checksum Type" is a property of the distributor of the > repository. That > > makes sense to me, as the whole repository is to be consumed by a EL 5 > > client. > > BTW. "Auto Publish: True" does not seem to work when manually > uploading > > a package (it did in pulp v1). > > > > Regards, > > Philipp > > > > Am 17.06.2014 15:02, schrieb Barnaby Court: > >> Hi, Pulp does support EL5 platforms. The trick is that you have > to specify the checksum during the upload of RPM content units. > Pulp treats files with a different checksum as completely different > entities. Use the "pulp-admin rpm repo uploads rpm --checksum-type > sha ..." for uploading the package for RHEL5. This allows you to > override the default checksum type of sha256 whenever you upload an > RPM. Upload the same rpm twice, once with --checksum-type sha for > your el5 repo and once without the option specified or specified as > sha256 for your EL6 repo. Regards, > >> > >> Barnaby > >> > >> ----- Original Message ----- > >> From: "Philipp Gassmann" <gassm...@puzzle.ch > <mailto:gassm...@puzzle.ch>> > >> To: "Martin Collins" <martin.coll...@framestore.com > <mailto:martin.coll...@framestore.com>>, "Christina Plummer" > <cplum...@gmail.com <mailto:cplum...@gmail.com>>, "Barnaby Court" > <bco...@redhat.com <mailto:bco...@redhat.com>> > >> Cc: pulp-list@redhat.com <mailto:pulp-list@redhat.com> > >> Sent: Tuesday, June 17, 2014 4:51:57 AM > >> Subject: Re: [Pulp-list] Repo configured to use SHA checksums, > but uploaded packages still get SHA256 > >> > >> Thanks for the replys! > >> Unfortuately those workarounds are not really usable if you need to > >> upload packages to existing repos for EL5 and EL6. > >> I migrated the EL5 repos from pulp_v1 by initially syncing from those > >> repos. Now I need to add additional packages to those repositories. > >> > >> I'm quite disappointed to find out that Pulp 2 does support > repositories > >> for EL5, even though RHEL 5 is listed as supported Platform. > >> > >> http://www.pulpproject.org/ > >> Supported Platforms: > >> Fedora 18, Fedora 19, Red Hat Enterprise Linux 5, Red Hat > Enterprise Linux 6 > >> > >> The sha256 is still hardcoded: > >> > > https://github.com/pulp/pulp_rpm/blob/master/extensions_admin/pulp_rpm/extensions/admin/upload/package.py > >> > >> The checksumtype is a repository option so the checksum should not be > >> different for different packages of the same repository. > >> Pulp 1 had the command to generate the metadata. > >> Pulp 2 has the publish command which has a stage "generating > metadata", > >> there the checksum should be created. > >> Am I wrong on that? > >> Why does pulp 2.4 add an option "--checksum-type" to the upload > command? > >> Shouln't it use the repository checksum-type option? > >> What is the checksum-type option of the repository for? > >> > >> pulp-admin rpm repo update --repo-id centos.5.x86_64.custom > >> --checksum-type=sha1 > >> > >> Id: centos.5.i386.updates > >> Display Name: centos.5.i386.updates > >> Description: Last checked by Puppet on Tue Jun 17 > 06:43:30 CEST 2014 > >> Content Unit Counts: > >> Rpm: 408 > >> Notes: > >> Importers: > >> Config: > >> Feed: http://mirror.switch.ch/ftp/mirror/centos/5/updates/i386/ > >> Id: yum_importer > >> Importer Type Id: yum_importer > >> Last Sync: 2014-05-21T11:54:49+02:00 > >> Repo Id: centos.5.i386.updates > >> Scheduled Syncs: > >> Distributors: > >> Auto Publish: True > >> Config: > >> Checksum Type: sha1 > >> Http: True > >> Https: True > >> Relative URL: centos.5.i386.updates > >> Distributor Type Id: yum_distributor > >> Id: yum_distributor > >> Last Publish: 2014-05-21T11:55:24+02:00 > >> Repo Id: centos.5.i386.updates > >> Scheduled Publishes: > >> > >> > >> The Bug Report is here: > >> https://bugzilla.redhat.com/show_bug.cgi?id=1004580 > >> > >> > >> Am 16.06.2014 23:37, schrieb Martin Collins: > >>> My workaround (as I have a large number of rhel6 and only a few for > >>> rhel5) was to just sync from a file:/// feed URL for rhel5 then > just put > >>> the rpms in that dir, createrepo -s sha, then sync and continue > normally > >>> from there > >>> > >>> Cheers > >>> Martin > >>> > >>> On 16 Jun 2014 22:33, "Christina Plummer" <cplum...@gmail.com > <mailto:cplum...@gmail.com> > >>> <mailto:cplum...@gmail.com <mailto:cplum...@gmail.com>>> wrote: > >>> > >>> Hi Philipp, > >>> > >>> My workaround was to modify the upload/package.py and explicitly > >>> force it to use "sha" instead of "sha256". On my system, > the file > >>> was > >>> > > /usr/lib/python2.6/site-packages/pulp_rpm/extension/admin/upload/package.py > >>> . See attached diff. > >>> > >>> Be sure to set aside the package.pyc and package.pyo files > as well, > >>> and restart httpd. Then delete and re-upload the package to > your repos. > >>> > >>> Belated public thanks to Ryan Bowlby for the pointer on this > one. > >>> > >>> Christina > >>> > >>> > >>> On Mon, Jun 16, 2014 at 10:19 AM, Philipp Gassmann > >>> <gassm...@puzzle.ch <mailto:gassm...@puzzle.ch> > <mailto:gassm...@puzzle.ch <mailto:gassm...@puzzle.ch>>> wrote: > >>> > >>> Hello Christina > >>> > >>> I just stumbled over the same issue. I uploaded the same > noarch > >>> package > >>> to multiple repositories. now the installation of the > package on > >>> CentOS > >>> 5 fails because of the wrong checksum. > >>> > >>> Did you find a solution or workaround? > >>> I can install the package if I download it manually and > install > >>> it with > >>> yum --nogpgcheck. > >>> > >>> Regards, > >>> Philipp > >>> > >>> > >>> Am 11.03.2014 14:26, schrieb Christina Plummer: > >>> > Hi Barnaby, > >>> > > >>> > Thanks for your note. Unfortunately, --checksum-type > doesn't > >>> seem to be > >>> > an option for the "pulp-admin rpm repo uploads rpm" > command: > >>> > > >>> > CWC\cplummer1@cbigdc-rhel6test ~ $ pulp-admin rpm repo > uploads rpm > >>> > --repo-id=q1-ol5-x86_64 --file > >>> oracle-validated-1.0.0-34.el5.x86_64.rpm > >>> > --checksum-type sha > >>> > Command: rpm > >>> > Description: uploads one or more RPMs into a repository > >>> > > >>> > Available Arguments: > >>> > > >>> > --repo-id - (required) unique identifier; only > >>> alphanumeric, ., > >>> > -, and _ > >>> > allowed > >>> > --file, -f - full path to a file to upload; may be > >>> specified multiple > >>> > times for multiple files > >>> > --dir, -d - full path to a directory > containing files > >>> to upload; > >>> > may be > >>> > specified multiple times for multiple > >>> directories > >>> > -v - display extra information about > the upload > >>> process > >>> > --skip-existing - if specified, RPMs that already > exist on > >>> the server > >>> > will not > >>> > be uploaded > >>> > The following options were specified but do not exist > on the > >>> command: > >>> > --checksum-type > >>> > > >>> > I am using the RPMs for 2.3.1-1. Shouldn't it default > to the > >>> > checksum-type for the repository, if specified? > >>> > > >>> > Thanks, > >>> > Christina > >>> > > >>> > > >>> > On Mon, Mar 10, 2014 at 2:31 PM, Barnaby Court > >>> <bco...@redhat.com <mailto:bco...@redhat.com> > <mailto:bco...@redhat.com <mailto:bco...@redhat.com>> > >>> > <mailto:bco...@redhat.com <mailto:bco...@redhat.com> > <mailto:bco...@redhat.com <mailto:bco...@redhat.com>>>> wrote: > >>> > > >>> > Christina, > >>> > > >>> > The checksum on the RPM is set at upload time. Was > >>> "--checksum-type > >>> > SHA1" set on the "pulp-admin rpm repo uploads rpm" > >>> command? If no > >>> > checksum type is specified it defaults to SHA256. > >>> > > >>> > Regards, > >>> > Barnaby > >>> > > >>> > ----- Original Message ----- > >>> > From: "Christina Plummer" <cplum...@gmail.com > <mailto:cplum...@gmail.com> > >>> <mailto:cplum...@gmail.com <mailto:cplum...@gmail.com>> > >>> > <mailto:cplum...@gmail.com > <mailto:cplum...@gmail.com> <mailto:cplum...@gmail.com > <mailto:cplum...@gmail.com>>>> > >>> > To: pulp-list@redhat.com > <mailto:pulp-list@redhat.com> <mailto:pulp-list@redhat.com > <mailto:pulp-list@redhat.com>> > >>> <mailto:pulp-list@redhat.com > <mailto:pulp-list@redhat.com> <mailto:pulp-list@redhat.com > <mailto:pulp-list@redhat.com>>> > >>> > Sent: Monday, March 10, 2014 1:42:14 PM > >>> > Subject: [Pulp-list] Repo configured to use SHA > checksums, > >>> but > >>> > uploaded packages still get SHA256 > >>> > > >>> > On Pulp 2.1.3, I had run into issues with sha256 > checksums > >>> being > >>> > created for some of the repo metadata files, even > though > >>> the repo > >>> > was configured to use sha instead. > >>> > > >>> > On Pulp 2.3.1, this seems to have been fixed... > however, I > >>> am now > >>> > seeing that packages manually uploaded into a repo are > >>> still getting > >>> > a sha256 checksum in the repodata, which means > that RHEL5/OL5 > >>> > clients can't handle them. Packages that were > synced, or > >>> copied from > >>> > another repository, have the correct checksum type. > >>> > > >>> > OS: RHEL 6.5 x86_64 > >>> > Pulp: 2.3.1-1 (RPMs from Pulp repo) > >>> > Steps to reproduce: > >>> > 1. Created "live" repo fed from Oracle public yum > >>> repositories, > >>> > checksum-type=sha, retain-old-count=1 > >>> > 2. Created "q1" repo, checksum-type=sha > >>> > 3. Copied rpms from "live" to "q1 > >>> > 4. Deleted "known-broken" versions of > >>> oracle-validated-1.1.0 package > >>> > 5. Uploaded "known-working" version of > >>> oracle-validated-1.0.0 package > >>> > 6. Published repo. Every package except for > >>> oracle-validated-1.0.0 > >>> > has a 'sha' checksum; oracle-validated-1.0.0 alone > has a > >>> 'sha256' > >>> > checksum. > >>> > > >>> > Is this a bug? > >>> > > >>> > Thanks, > >>> > Christina > >>> > > >>> > _______________________________________________ > >>> > Pulp-list mailing list > >>> > Pulp-list@redhat.com <mailto:Pulp-list@redhat.com> > <mailto:Pulp-list@redhat.com <mailto:Pulp-list@redhat.com>> > >>> <mailto:Pulp-list@redhat.com > <mailto:Pulp-list@redhat.com> <mailto:Pulp-list@redhat.com > <mailto:Pulp-list@redhat.com>>> > >>> > https://www.redhat.com/mailman/listinfo/pulp-list > >>> > > >>> > > >>> > > >>> > > >>> > _______________________________________________ > >>> > Pulp-list mailing list > >>> > Pulp-list@redhat.com <mailto:Pulp-list@redhat.com> > <mailto:Pulp-list@redhat.com <mailto:Pulp-list@redhat.com>> > >>> > https://www.redhat.com/mailman/listinfo/pulp-list > >>> > > >>> > >>> > >>> -- > >>> Philipp Gassmann > >>> > >>> Puzzle ITC GmbH > >>> http://www.puzzle.ch <http://www.puzzle.ch/> > >>> > >>> Telefon +41 31 370 22 00 > <tel:%2B41%2031%20370%2022%2000> <tel:%2B41%2031%20370%2022%2000> > >>> Werfen Sie einen Blick in unseren Blog: > >>> http://www.puzzle.ch/blog > >>> > >>> > >>> > >>> _______________________________________________ > >>> Pulp-list mailing list > >>> Pulp-list@redhat.com <mailto:Pulp-list@redhat.com> > <mailto:Pulp-list@redhat.com <mailto:Pulp-list@redhat.com>> > >>> https://www.redhat.com/mailman/listinfo/pulp-list > >>> > >> > >> > > > > > > > -- > Philipp Gassmann > > Puzzle ITC GmbH > http://www.puzzle.ch <http://www.puzzle.ch/> > > Telefon +41 31 370 22 00 <tel:%2B41%2031%20370%2022%2000> > Direkt +41 31 370 22 02 <tel:%2B41%2031%20370%2022%2002> > Fax +41 31 370 22 01 <tel:%2B41%2031%20370%2022%2001> > > Werfen Sie einen Blick in unseren Blog: > http://www.puzzle.ch/blog > > -- Philipp Gassmann Puzzle ITC GmbH http://www.puzzle.ch Telefon +41 31 370 22 00 Direkt +41 31 370 22 02 Fax +41 31 370 22 01 Werfen Sie einen Blick in unseren Blog: http://www.puzzle.ch/blog
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list