It looks like the last error caused failed sync process. I reset the db on a
dev host and try to sync the same upstream repo and got
""An error occurred (QuotaExceeded) when calling the PutObject operation:
Unknown"
Any idea how to fix this?
"traceback": " File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rq/worker.py\", line
886, in perform_job\n rv = job.perform()\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rq/job.py\", line
664, in perform\n self._result = self._execute()\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rq/job.py\", line
670, in _execute\n return self.func(*self.args, **self.kwargs)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulp_rpm/app/tasks/synchronizing.py\",
line 266, in synchronize\n dv.create()\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/plugin/stages/declarative_version.py\",
line 148, in create\n loop.run_until_complete(pipeline)\n File
\"/opt/python/3.7.3/lib64/python3.7/asyncio/base_events.py\", line 584, in
run_until_complete\n return future.result()\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/plugin/stages/api.py\",
line 225, in create_pipeline\n await asyncio.gather(*futures)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/plugin/stages/api.py\",
line 43, in __call__\n await self.run()\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/plugin/stages/artifact_stages.py\",
line 219, in run\n d_artifact.artifact for d_artifact in da_to_save\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/app/models/content.py\",
line 87, in bulk_get_or_create\n return super().bulk_create(objs,
batch_size=batch_size)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/manager.py\",
line 82, in manager_method\n return getattr(self.get_queryset(),
name)(*args, **kwargs)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/query.py\",
line 468, in bulk_create\n self._batched_insert(objs_with_pk, fields,
batch_size, ignore_conflicts=ignore_conflicts)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/query.py\",
line 1204, in _batched_insert\n ignore_conflicts=ignore_conflicts,\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/query.py\",
line 1186, in _insert\n return
query.get_compiler(using=using).execute_sql(return_id)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/sql/compiler.py\",
line 1376, in execute_sql\n for sql, params in self.as_sql():\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django_readonly_field/compiler.py\",
line 31, in as_sql\n return super(ReadonlySQLCompilerMixin,
self).as_sql()\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/sql/compiler.py\",
line 1320, in as_sql\n for obj in self.query.objs\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/sql/compiler.py\",
line 1320, in <listcomp>\n for obj in self.query.objs\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/sql/compiler.py\",
line 1319, in <listcomp>\n [self.prepare_value(field,
self.pre_save_val(field, obj)) for field in fields]\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/sql/compiler.py\",
line 1270, in pre_save_val\n return field.pre_save(obj, add=True)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/app/models/fields.py\",
line 68, in pre_save\n return super().pre_save(model_instance, add)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/fields/files.py\",
line 288, in pre_save\n file.save(file.name, file.file, save=False)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/fields/files.py\",
line 87, in save\n self.name = self.storage.save(name, content,
max_length=self.field.max_length)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/files/storage.py\",
line 52, in save\n return self._save(name, content)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/storages/backends/s3boto3.py\",
line 447, in _save\n obj.upload_fileobj(content, ExtraArgs=params)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/boto3/s3/inject.py\",
line 621, in object_upload_fileobj\n ExtraArgs=ExtraArgs,
Callback=Callback, Config=Config)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/boto3/s3/inject.py\",
line 539, in upload_fileobj\n return future.result()\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/s3transfer/futures.py\",
line 106, in result\n return self._coordinator.result()\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/s3transfer/futures.py\",
line 265, in result\n raise self._exception\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/s3transfer/tasks.py\",
line 126, in __call__\n return self._execute_main(kwargs)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/s3transfer/tasks.py\",
line 150, in _execute_main\n return_value = self._main(**kwargs)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/s3transfer/upload.py\",
line 692, in _main\n client.put_object(Bucket=bucket, Key=key, Body=body,
**extra_args)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/botocore/client.py\",
line 357, in _api_call\n return self._make_api_call(operation_name,
kwargs)\n File
\"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/botocore/client.py\",
line 676, in _make_api_call\n raise error_class(parsed_response,
operation_name)\n"
From: Bin Li (BLOOMBERG/ 120 PARK) At: 03/09/21 11:34:57To:
[email protected]
Cc: [email protected], [email protected]
Subject: Re: [Pulp-list] pulp 3.7.3 sync with checksum error
Got a new errors "Package matching query does not exist.". Is this also related
to upstream repo? Can we have more details when this happens?
From: [email protected] At: 03/05/21 15:06:40To: Bin Li (BLOOMBERG/ 120
PARK )
Cc: [email protected], [email protected]
Subject: Re: [Pulp-list] pulp 3.7.3 sync with checksum error
Great, thanks for the update.
David
On Fri, Mar 5, 2021 at 2:47 PM Bin Li (BLOOMBERG/ 120 PARK)
<[email protected]> wrote:
Thanks Dave. Got the filename which has the inconsistent checksum after
patching. We will ask upstream remote to update the repodata.
From: [email protected] At: 03/05/21 12:42:56To: [email protected]
Cc: Bin Li (BLOOMBERG/ 120 PARK ) , [email protected]
Subject: Re: [Pulp-list] pulp 3.7.3 sync with checksum error
Regarding the error message, I've observed the problem myself. and have filed
an issue:
https://pulp.plan.io/issues/8357
In the meantime, if you can patch the code, this should tell you give you more
info:
https://gist.github.com/daviddavis/2e6ab1872d97230d144a6cd1f9d05e31
David
On Fri, Mar 5, 2021 at 12:35 PM Brian Bouterse <[email protected]> wrote:
Did this happen inside a task? Did you see a traceback for it also?
On Fri, Mar 5, 2021 at 12:00 PM Bin Li (BLOOMBERG/ 120 PARK)
<[email protected]> wrote:
The sync process gave an error "A file failed validation due to checksum". Is
this error caused by remote repo? Is there a way to find out which file cause
the issue?_______________________________________________
Pulp-list mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/pulp-list
_______________________________________________
Pulp-list mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/pulp-list
_______________________________________________
Pulp-list mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/pulp-list
