Re: [Pulp-list] RHEL 8 rpm repo sync errors

Venkataramana Bora Thu, 19 Mar 2020 13:57:25 -0700

Hi Dennis , Thank you very much for all these details . I will try to create repo as shown on that url " Sync a Protected Repo".

Yes , our other repositories are able to sync Red Hat content . To compare their configuration I did not know exactly where to look for that .

Sincerely,
Ramana Bora

----- Original message -----
From: Dennis Kliban <dkli...@redhat.com>
To: Venkataramana Bora <venkb...@in.ibm.com>
Cc: pulp-list <pulp-list@redhat.com>
Subject: [EXTERNAL] Re: [Pulp-list] RHEL 8 rpm repo sync errors
Date: Thu, Mar 19, 2020 1:24 AM

I was referring to the certificate and key configured on the importer associated with the repository. Our docs explain how to configure a new repository to sync from the Red Hat servers[0].

If your other repositories are able to sync Red Hat content, I would compare the configuration of those repositories with the broken one.

[0] https://docs.pulpproject.org/en/2.21/plugins/pulp_rpm/user-guide/recipes.html#sync-a-protected-repo

On Wed, Mar 18, 2020 at 3:43 PM Venkataramana Bora <venkb...@in.ibm.com> wrote:

Hi Dennis , Thanks a lot for your reply.

Sorry, for this question . I'm not well versed with Puppet configuration , some one else did the config part here, no longer with company.

You said , we should have the correct client certificate and client key configured.

Is that SSLCertificateFile and SSLCertificateKeyFile in /etc/httpd/conf.d/ssl.conf ?

Can you please let me know patch to check?

Sincerely,
Ramana Bora

----- Original message -----
From: Dennis Kliban <dkli...@redhat.com>
To: Venkataramana Bora <venkb...@in.ibm.com>
Cc: pulp-list <pulp-list@redhat.com>
Subject: [EXTERNAL] Re: [Pulp-list] RHEL 8 rpm repo sync errors
Date: Wed, Mar 18, 2020 7:25 PM

This looks like a problem with the client certificate. The very first request for a file that pulp tries to download is receiving a 403 response from the CDN. Please make sure you have the correct client certificate and client key configured.

On Tue, Mar 17, 2020 at 5:53 PM Venkataramana Bora <venkb...@in.ibm.com> wrote:

Hi Teamn ,
Getting errors as shown here when we are trying to sync RHEL 8 repos (Base OS and Appstream) creatd on Pulp master. The same Pulp master we already using for RHEL 6 and 7 repos with out any issues. Strugling with RHEL 8 repos creation,
1.Could you please let us know whethere RHEL 8 rpm repos sync works on Pulp master version 2.16 or not ?
2.feed_ca_cert "redhat-uep.pem" and feed_cert "rhel-identity.pem" are there . We have no issues with RHEL 6 and 7 repo syncs, those created some years ago and working well.
Is there any thing specifically need to do for RHEL 8 repos in terms of ca cert/feed cert ? We see "failed with code 403 Forbiddenin" in var log messages.

3.pulp-admin tasks details Traceback shows "/usr/lib/python2.7/..."

For this added these 3 lines in /etc/pulp/server/plugins.conf.d/yum_importer.json as recommended here https://pulp.plan.io/issues/6327

{
   "validate":true
}

After adding that json , I see that RHEL 8 repodata xml.gz in rhel8/x86_64/baseos/os/repodata/ but not "packages" folder created yet , checked that thru pulp web access .
Please help resolving this issue .

Feed url: https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/baseos/os
Feed url: https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/appstream/os
Repoid for these urls are :
rhel-8-server-baseos-x86_64
rhel-8-server-appstream-x86_64

Note: Tried with "8" in place "$releasever" in feed but same error . Our Pulp master ver.2.16 is on CentOS 7.7.

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Tasks performed:

[root@swy01opplppr01 ~]# pulp-admin rpm repo sync run --force-full --repo-id rhel-8-server-baseos-x86_64
+----------------------------------------------------------------------+
         Synchronizing Repository [rhel-8-server-baseos-x86_64]
+----------------------------------------------------------------------+

The following sync configuration options will be used:

Force Full: True

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[\]
... completed

Downloading repository content...
[-]
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items

Task Failed

Error retrieving metadata: Forbidden

-------------------------------------------------------------------------------------------------------------------------
From /var/log/messages:

Mar 17 17:11:23 swy01opplppr01 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[c5b01662-a20a-4c44-b114-b1ba595f05fc]
Mar 17 17:11:23 swy01opplppr01 pulp: celery.app.trace:INFO: [55ba0954] Task pulp.server.async.tasks._queue_reserved_task[55ba0954-1a18-410a-9507-f61d74c4776f] succeeded in 0.115055091001s: None
Mar 17 17:11:23 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/.
Mar 17 17:11:23 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Mar 17 17:11:23 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml failed with code 403: Forbidden
Mar 17 17:11:24 swy01opplppr01 pulp: urllib3.connectionpool:INFO: [fbbc36d3] Starting new HTTPS connection (1): cdn.redhat.com
Mar 17 17:11:24 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: [fbbc36d3] Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os failed with code 403: Forbidden
Mar 17 17:11:24 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/.
Mar 17 17:11:24 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Mar 17 17:11:24 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml failed with code 403: Forbidden
Mar 17 17:11:25 swy01opplppr01 pulp: urllib3.connectionpool:INFO: [fbbc36d3] Starting new HTTPS connection (1): cdn.redhat.com
Mar 17 17:11:25 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: [fbbc36d3] Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os failed with code 403: Forbidden
Mar 17 17:11:25 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/.
Mar 17 17:11:25 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Mar 17 17:11:25 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml failed with code 403: Forbidden
Mar 17 17:11:26 swy01opplppr01 pulp: urllib3.connectionpool:INFO: [fbbc36d3] Starting new HTTPS connection (1): cdn.redhat.com
Mar 17 17:11:26 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: [fbbc36d3] Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os failed with code 403: Forbidden
Mar 17 17:11:26 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/.
Mar 17 17:11:26 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Mar 17 17:11:27 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml failed with code 403: Forbidden
Mar 17 17:11:27 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading additional units.
Mar 17 17:11:27 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Mar 17 17:11:28 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/.treeinfo failed with code 404: Not Found
Mar 17 17:11:28 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Mar 17 17:11:29 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/treeinfo failed with code 404: Not Found
Mar 17 17:11:30 swy01opplppr01 pulp: pulp.server.async.tasks:INFO: [fbbc36d3] Task failed : [fbbc36d3-fddd-4f54-9be6-770acecf374f] : Error retrieving metadata: Forbidden
Mar 17 17:11:30 swy01opplppr01 pulp: celery.app.trace:INFO: [fbbc36d3] Task pulp.server.managers.repo.sync.sync[fbbc36d3-fddd-4f54-9be6-770acecf374f] raised expected: PulpCodedException()
Mar 17 17:11:30 swy01opplppr01 pulp: celery.app.trace:INFO: [c5b01662] Task pulp.server.async.tasks._release_resource[c5b01662-a20a-4c44-b114-b1ba595f05fc] succeeded in 0.00344043700079s: None

-----------------------------------------------------------------------------------------------------

[root@swy01opplppr01 ~]# pulp-admin tasks details --task-id fbbc36d3-fddd-4f54-9be6-770acecf374f
+----------------------------------------------------------------------+
                              Task Details
+----------------------------------------------------------------------+

Operations:       sync
Resources:        rhel-8-server-baseos-x86_64 (repository)
State:            Failed
Start Time:       2020-03-17T21:11:23Z
Finish Time:      2020-03-17T21:11:30Z
Result:           N/A
Task Id:          fbbc36d3-fddd-4f54-9be6-770acecf374f
Worker Name:      reserved_resource_worke...@swy01opplppr01.ops.hosting.knxa
Progress Report:
Yum Importer:
    Comps:
      State: FINISHED
    Content:
      Details:
        Drpm Done: 0
        Drpm Total: 0
        Rpm Done:   0
        Rpm Total: 0
      Error Details:
      Items Left:    0
      Items Total:   0
      Size Left:     0
      Size Total:    0
      State:         FINISHED
    Distribution:
      Error Details:
      Items Left:    0
      Items Total:   0
      State:         FINISHED
    Errata:
      State: FINISHED
    Metadata:
      State: FINISHED
    Purge Duplicates:
      State: FINISHED
Traceback:        Traceback (most recent call last):   File
                  "/usr/lib/python2.7/site-packages/celery/app/trace.py", line
                  382, in trace_task     R = retval = fun(*args, **kwargs)
                  File
                  "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py",
                  line 529, in __call__     return super(Task,
                  self).__call__(*args, **kwargs)   File
                  "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py",
                  line 107, in __call__     return super(PulpTask,
                  self).__call__(*args, **kwargs)   File
                  "/usr/lib/python2.7/site-packages/celery/app/trace.py", line
                  641, in __protected_call__     return self.run(*args,
                  **kwargs)   File
                  "/usr/lib/python2.7/site-packages/pulp/server/controllers/repo
                  sitory.py", line 769, in sync     sync_report =
                  sync_repo(transfer_repo, conduit, call_config)   File
                  "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py",
                  line 737, in wrap_f     return f(*args, **kwargs)   File
                  "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/y
                  um/importer.py", line 82, in sync_repo     report =
                  self._current_sync.run()   File
                  "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/y
                  um/sync.py", line 335, in run
                  reason=self.repomd_not_found_reason) PulpCodedException: Error
                  retrieving metadata: Forbidden

-------------------------------------------------------------------------------

Sincerely,
Ramana Bora

_______________________________________________
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

_______________________________________________
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] RHEL 8 rpm repo sync errors

Reply via email to