Re: [Pulp-list] Can't build PEX out of pulp-cli because of missing dependency

[Konstantin M. Khankin]

Hello Matthias!

This workaround is sufficient for me. However, I feel like the dependency
should be added, yes. I don't have (and wouldn't want to have) GitHub
account, so I can't file a PR or an issue, sorry. Not going to ask you to
do this as I'm fine with a workaround :)

Thanks!

ср, 15 дек. 2021 г. в 13:01, Matthias Dellweg :

> Hello!
> Thank you for verifying this. Is this workaround sufficient for you?
> Should we still add a dependency on setuptools (It kind of felt
> implicit, because you cannot use setup.py without it anyway.)? If so,
> can you file a PR for it?
>
>   Matthias
>
> On Wed, Dec 15, 2021 at 11:56 AM Konstantin M. Khankin
>  wrote:
> >
> > Validated PEX build fails because of missing dependency with this
> workaround:
> >
> > $ echo setuptools > pulp_cli.req
> > $ pex -r pulp_cli.req -v -c pulp -o pulp.pex 'pulp-cli[pygments]'
> > $ deactivate
> > $ ./pulp.pex --version
> > pulp3 command line interface, version 0.12.0
> >
> > ср, 15 дек. 2021 г. в 12:43, Konstantin M. Khankin <
> khankin.konstan...@gmail.com>:
> >>
> >> Hello!
> >>
> >> I wanted to make pulp-cli portable and build PEX archive out of it but
> the latest pex (2.1.56) fails to do so:
> >>
> >> ```
> >> $ pex -v -c pulp 'pulp-cli[pygments]'
> >> ...
> >> Running PEX file at /tmp/tmpdy35exye with args []
> >> pex: PEX.run invoking /home/hc/pex/bin/python3 -s -E /tmp/tmpdy35exye
> >> pex: Re-executing: cmdline='/usr/bin/python3.10 /tmp/tmpdy35exye',
> sys.executable='/home/hc/pex/bin/python3', PEX_PYTHON=None,
> PEX_PYTHON_PATH=None, interpreter_constraints=[]
> >> pex: Found site-library: /usr/local/lib/python3.10/site-packages
> >> pex: Found site-library: /usr/lib/python3.10/site-packages
> >> pex: Found site-library: /usr/local/lib64/python3.10/site-packages
> >> pex: Found site-library: /usr/lib64/python3.10/site-packages
> >> pex: Tainted path element: /usr/lib64/python3.10/site-packages
> >> pex: Tainted path element: /usr/lib/python3.10/site-packages
> >> pex: Scrubbing from user site:
> /home/hc/.local/lib/python3.10/site-packages
> >> pex: Scrubbing from site-packages: /usr/lib64/python3.10/site-packages
> >> pex: Scrubbing from site-packages: /usr/lib/python3.10/site-packages
> >> pex: New sys.path: ['/tmp/tmpdy35exye/.bootstrap', '/tmp/tmpdy35exye',
> '/usr/lib64/python310.zip', '/usr/lib64/python3.10',
> '/usr/lib64/python3.10/lib-dynload']
> >> pex: Activating PEX virtual environment from /tmp/tmpdy35exye: 54.7ms
> >> pex: Bootstrap complete, performing final sys.path modifications...
> >> pex: PYTHONPATH contains:
> >> pex: /tmp/tmpdy35exye
> >> pex:   * /usr/lib64/python310.zip
> >> pex: /usr/lib64/python3.10
> >> pex: /usr/lib64/python3.10/lib-dynload
> >> pex:
>  
> /home/hc/.pex/installed_wheels/7679cb4a5378680b93c1f33b463094a55cf8c547/pulp_cli-0.12.0-py3-none-any.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/23fff3a23c744ec73d03f9639bec17d3745b762c/click-8.0.3-py3-none-any.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/bc1215388f22b27f973bc1bc3d74a96373404ac9/packaging-21.3-py3-none-any.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/33b6478f39bfd332868d887340fd44164373dfb4/pyparsing-3.0.6-py3-none-any.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/5aa8517e7212729e4c0b95fab8b765c01e48f97c/PyYAML-5.4.1-cp310-cp310-linux_x86_64.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/cd6d3dfd86e52a31a3664cff69214a5dd94161e7/schema-0.7.4-py2.py3-none-any.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/c29688d87604522b7f9aa8a178a6a21b6a9fe775/contextlib2-21.6.0-py2.py3-none-any.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/d20e2df000dd43249c3a9eed041f08a812a93423/requests-2.26.0-py2.py3-none-any.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/601e0a29cd7d663eb8ea40c3ba7a52200e2544c0/urllib3-1.26.7-py2.py3-none-any.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/883fca1b3199ac78822a54c4af0a01c1f7a7ea74/certifi-2021.10.8-py2.py3-none-any.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/7e4deb9e3d26e447cfdd8c00b13805d8d2a65fca/charset_normalizer-2.0.9-py3-none-any.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/5485a373e7aae71ef1d2c02b8b09c330b75c5248/idna-3.3-py3-none-any.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/a55ae166e643e6c7a28c16fe005efc32ee98ee76/toml-0.10.2-py2.py3-none-any.whl
> >> pex:
>  
> /home/hc/.pex/installed_wheels/8dd8170137ab3f4dc94d5006227635731b451dd1/Pygmen

[Pulp-list] Can't build PEX out of pulp-cli because of missing dependency

[Konstantin M. Khankin]

Hello!

I wanted to make pulp-cli portable and build PEX archive out of it but the
latest pex (2.1.56) fails to do so:

```
$ pex -v -c pulp 'pulp-cli[pygments]'
...
Running PEX file at /tmp/tmpdy35exye with args []
pex: PEX.run invoking /home/hc/pex/bin/python3 -s -E /tmp/tmpdy35exye
pex: Re-executing: cmdline='/usr/bin/python3.10 /tmp/tmpdy35exye',
sys.executable='/home/hc/pex/bin/python3', PEX_PYTHON=None,
PEX_PYTHON_PATH=None, interpreter_constraints=[]
pex: Found site-library: /usr/local/lib/python3.10/site-packages
pex: Found site-library: /usr/lib/python3.10/site-packages
pex: Found site-library: /usr/local/lib64/python3.10/site-packages
pex: Found site-library: /usr/lib64/python3.10/site-packages
pex: Tainted path element: /usr/lib64/python3.10/site-packages
pex: Tainted path element: /usr/lib/python3.10/site-packages
pex: Scrubbing from user site: /home/hc/.local/lib/python3.10/site-packages
pex: Scrubbing from site-packages: /usr/lib64/python3.10/site-packages
pex: Scrubbing from site-packages: /usr/lib/python3.10/site-packages
pex: New sys.path: ['/tmp/tmpdy35exye/.bootstrap', '/tmp/tmpdy35exye',
'/usr/lib64/python310.zip', '/usr/lib64/python3.10',
'/usr/lib64/python3.10/lib-dynload']
pex: Activating PEX virtual environment from /tmp/tmpdy35exye: 54.7ms
pex: Bootstrap complete, performing final sys.path modifications...
pex: PYTHONPATH contains:
pex: /tmp/tmpdy35exye
pex:   * /usr/lib64/python310.zip
pex: /usr/lib64/python3.10
pex: /usr/lib64/python3.10/lib-dynload
pex:
/home/hc/.pex/installed_wheels/7679cb4a5378680b93c1f33b463094a55cf8c547/pulp_cli-0.12.0-py3-none-any.whl
pex:
/home/hc/.pex/installed_wheels/23fff3a23c744ec73d03f9639bec17d3745b762c/click-8.0.3-py3-none-any.whl
pex:
/home/hc/.pex/installed_wheels/bc1215388f22b27f973bc1bc3d74a96373404ac9/packaging-21.3-py3-none-any.whl
pex:
/home/hc/.pex/installed_wheels/33b6478f39bfd332868d887340fd44164373dfb4/pyparsing-3.0.6-py3-none-any.whl
pex:
/home/hc/.pex/installed_wheels/5aa8517e7212729e4c0b95fab8b765c01e48f97c/PyYAML-5.4.1-cp310-cp310-linux_x86_64.whl
pex:
/home/hc/.pex/installed_wheels/cd6d3dfd86e52a31a3664cff69214a5dd94161e7/schema-0.7.4-py2.py3-none-any.whl
pex:
/home/hc/.pex/installed_wheels/c29688d87604522b7f9aa8a178a6a21b6a9fe775/contextlib2-21.6.0-py2.py3-none-any.whl
pex:
/home/hc/.pex/installed_wheels/d20e2df000dd43249c3a9eed041f08a812a93423/requests-2.26.0-py2.py3-none-any.whl
pex:
/home/hc/.pex/installed_wheels/601e0a29cd7d663eb8ea40c3ba7a52200e2544c0/urllib3-1.26.7-py2.py3-none-any.whl
pex:
/home/hc/.pex/installed_wheels/883fca1b3199ac78822a54c4af0a01c1f7a7ea74/certifi-2021.10.8-py2.py3-none-any.whl
pex:
/home/hc/.pex/installed_wheels/7e4deb9e3d26e447cfdd8c00b13805d8d2a65fca/charset_normalizer-2.0.9-py3-none-any.whl
pex:
/home/hc/.pex/installed_wheels/5485a373e7aae71ef1d2c02b8b09c330b75c5248/idna-3.3-py3-none-any.whl
pex:
/home/hc/.pex/installed_wheels/a55ae166e643e6c7a28c16fe005efc32ee98ee76/toml-0.10.2-py2.py3-none-any.whl
pex:
/home/hc/.pex/installed_wheels/8dd8170137ab3f4dc94d5006227635731b451dd1/Pygments-2.10.0-py3-none-any.whl
pex: /tmp/tmpdy35exye/.bootstrap
pex:   * - paths that do not exist or will be imported via zipimport
Traceback (most recent call last):
  File "/tmp/tmpdy35exye/.bootstrap/pex/pex.py", line 476, in execute
exit_value = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/tmp/tmpdy35exye/.bootstrap/pex/pex.py", line 401, in _wrap_coverage
return runner(*args)
  File "/tmp/tmpdy35exye/.bootstrap/pex/pex.py", line 432, in
_wrap_profiling
return runner(*args)
  File "/tmp/tmpdy35exye/.bootstrap/pex/pex.py", line 532, in _execute
return self.execute_entry(self._pex_info.entry_point)
  File "/tmp/tmpdy35exye/.bootstrap/pex/pex.py", line 668, in execute_entry
return self.execute_pkg_resources(entry_point)
  File "/tmp/tmpdy35exye/.bootstrap/pex/pex.py", line 699, in
execute_pkg_resources
runner = entry.resolve()
  File
"/tmp/tmpdy35exye/.bootstrap/pex/vendor/_vendored/setuptools/pkg_resources/__init__.py",
line 2481, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File
"/home/hc/.pex/installed_wheels/7679cb4a5378680b93c1f33b463094a55cf8c547/pulp_cli-0.12.0-py3-none-any.whl/pulpcore/cli/common/__init__.py",
line 7, in 
import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'
```

As I understand, this happens because setup.py does not

explicitly declare a dependency on setuptools (which contains pkg_resources
unlike the packaging lib). If that's correct, could you add the dependency
please? If not, do you know what can be the issue here?

Also, it seems to me pkg_resources is only used to load CLI plugins here
.
However, the document which this code mentions does not include any

Re: [Pulp-list] Custom list of CA certificates for pulp-cli

[Konstantin M. Khankin]

Yes, it works. Thank you!

пт, 26 нояб. 2021 г. в 20:59, Matthias Dellweg :

> Can you please verify that this change solves your problem:
> https://github.com/pulp/pulp-cli/pull/434
>
> Thank you!
>
> On Fri, Nov 26, 2021 at 6:09 PM Konstantin M. Khankin
>  wrote:
> >
> > Hello!
> >
> > Today I got unpleasantly surprised about the inability to change the
> list of CA certificates pulp-cli uses to verify SSL connections. Pulp-cli
> uses the requests library, so I expected to be able to just specify
> REQUESTS_CA_BUNDLE envvar and be happy. However, this did not work. So I
> added some print_tb() here and there and saw this:
> >
> > ...
> >   File
> "/usr/local/lib/python3.6/site-packages/pulpcore/cli/python/content.py",
> line 88, in upload
> > artifact_href = PulpArtifactContext(pulp_ctx).upload(file,
> chunk_size)
> >   File
> "/usr/local/lib/python3.6/site-packages/pulpcore/cli/core/context.py", line
> 43, in upload
> > result = self.list(limit=1, offset=0, parameters={"sha256":
> sha256_digest})
> >   File
> "/usr/local/lib/python3.6/site-packages/pulpcore/cli/common/context.py",
> line 436, in list
> > result: Dict[str, Any] = self.pulp_ctx.call(self.LIST_ID,
> parameters=payload)
> >   File
> "/usr/local/lib/python3.6/site-packages/pulpcore/cli/common/context.py",
> line 134, in call
> > result = self.api.call(operation_id, *args, **kwargs)
> >   File
> "/usr/local/lib/python3.6/site-packages/pulpcore/cli/common/openapi.py",
> line 263, in call
> > response: requests.Response = self._session.send(request)
> >   File "/usr/local/lib/python3.6/site-packages/requests/sessions.py",
> line 657, in send
> > r = adapter.send(request, **kwargs)
> >   File "/usr/local/lib/python3.6/site-packages/requests/adapters.py",
> line 420, in send
> > self.cert_verify(conn, request.url, verify, cert)
> > ...
> >
> > pulpcore/cli/common/openapi.py calls requests.Session.send(). This
> works, but it means merge_environment_settings is not called, because it is
> in request() (which later calls send()). This means envvar is not applied.
> >
> > I tried to abuse Python's dynamic typing and change the value of the
> verify_ssl flag to what I'd put into the envvar, but click is smart enough
> to catch this :(
> >
> > I found the FR to support custom CA certificates. It may be that this FR
> is as easy as changing send() to request(), allowing users to use an
> already existing mechanism from the requests library.
> >
> > Is there something which I missed?
> >
> > Thanks!
> >
> > --
> > Konstantin Khankin
> > ___
> > Pulp-list mailing list
> > Pulp-list@redhat.com
> > https://listman.redhat.com/mailman/listinfo/pulp-list
>
>

-- 
Ханкин Константин
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Custom list of CA certificates for pulp-cli

[Konstantin M. Khankin]

Hello!

Today I got unpleasantly surprised about the inability to change the list
of CA certificates pulp-cli uses to verify SSL connections. Pulp-cli uses
the requests library, so I expected to be able to just
specify REQUESTS_CA_BUNDLE envvar and be happy. However, this did not work.
So I added some print_tb() here and there and saw this:

...
  File
"/usr/local/lib/python3.6/site-packages/pulpcore/cli/python/content.py",
line 88, in upload
artifact_href = PulpArtifactContext(pulp_ctx).upload(file, chunk_size)
  File
"/usr/local/lib/python3.6/site-packages/pulpcore/cli/core/context.py", line
43, in upload
result = self.list(limit=1, offset=0, parameters={"sha256":
sha256_digest})
  File
"/usr/local/lib/python3.6/site-packages/pulpcore/cli/common/context.py",
line 436, in list
result: Dict[str, Any] = self.pulp_ctx.call(self.LIST_ID,
parameters=payload)
  File
"/usr/local/lib/python3.6/site-packages/pulpcore/cli/common/context.py",
line 134, in call
result = self.api.call(operation_id, *args, **kwargs)
  File
"/usr/local/lib/python3.6/site-packages/pulpcore/cli/common/openapi.py",
line 263, in call
response: requests.Response = self._session.send(request)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line
657, in send
r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line
420, in send
self.cert_verify(conn, request.url, verify, cert)
...

pulpcore/cli/common/openapi.py

calls requests.Session.send
().
This works, but it means merge_environment_settings

is
not called, because it is in request() (which later calls send()). This
means envvar is not applied.

I tried to abuse Python's dynamic typing and change the value of the
verify_ssl flag to what I'd put into the envvar, but click is smart enough
to catch this :(

I found the FR  to support
custom CA certificates. It may be that this FR is as easy as changing
send() to request(), allowing users to use an already existing mechanism
from the requests library.

Is there something which I missed?

Thanks!

-- 
Konstantin Khankin
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Pulp2 2.21.4-2 [RELEASED]

[Konstantin M. Khankin]

Looks like there is one more place where client relies on libraries
available only in the server package:

  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
handler "bind", import failed
  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
Traceback (most recent call last):
  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
File "/usr/lib/python2.7/site-packages/pulp/agent/lib/container.py", line
284, in __load
  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
mod = self.__import_module(path)
  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
File "/usr/lib/python2.7/site-packages/pulp/agent/lib/container.py", line
313, in __import_module
  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
mod = __import__(path[0], globals(), locals(), [path[-1]])
  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
File "/usr/lib/python2.7/site-packages/pulp_rpm/handlers/bind.py", line 9,
in 
  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
from pulp_rpm.handlers import repolib
  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
File "/usr/lib/python2.7/site-packages/pulp_rpm/handlers/repolib.py", line
16, in 
  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
from pulp_rpm.handlers.repo_file import Repo, RepoFile, MirrorListFile,
RepoKeyFiles, CertFiles
  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
File "/usr/lib/python2.7/site-packages/pulp_rpm/handlers/repo_file.py",
line 6, in 
  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
from pulp.plugins.util import misc
  goferd: [ERROR][worker-0] pulp.agent.lib.container:290 -
ImportError: No module named plugins.util

вт, 10 нояб. 2020 г. в 17:52, Grant Gainey :

> 2.21.4-2 has been released. The only change is the fix for 7803
> 
>
> G
>
> On Mon, Nov 9, 2020 at 10:26 AM Grant Gainey  wrote:
>
>> Hey folks,
>>
>> Just a heads-up that there will be a 2.21.4-2 turn of the build crank
>> this week in order to address issue 7803
>>  . I will send out a notification once
>> the fixed RPMs are available.
>>
>> G
>> --
>> Grant Gainey
>> Principal Software Engineer, Red Hat System Management Engineering
>>
>
>
> --
> Grant Gainey
> Principal Software Engineer, Red Hat System Management Engineering
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list



-- 
Ханкин Константин
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] "Plugin "pulp-profile-update" can't be imported" at 2.21.4

[Konstantin M. Khankin]

Hi!

This report is very similar to the one in https://pulp.plan.io/issues/5126
or mail thread
https://www.redhat.com/archives/pulp-list/2019-July/msg00021.html, the only
difference is I get this error with the latest Pulp update 2.21.4:

# rpm -qa | grep pulp
python-pulp-client-lib-2.21.4-1.el7.noarch
python-pulp-common-2.21.4-1.el7.noarch
pulp-rpm-handlers-2.21.4-1.el7.noarch
python-pulp-rpm-common-2.21.4-1.el7.noarch
python-pulp-agent-lib-2.21.4-1.el7.noarch
pulp-agent-2.21.4-1.el7.noarch
pulp-puppet-consumer-extensions-2.21.4-1.el7.noarch
pulp-consumer-client-2.21.4-1.el7.noarch
pulp-rpm-yumplugins-2.21.4-1.el7.noarch
python-pulp-bindings-2.21.4-1.el7.noarch
pulp-rpm-consumer-extensions-2.21.4-1.el7.noarch
python-pulp-puppet-common-2.21.4-1.el7.noarch
pulp-puppet-handlers-2.21.4-1.el7.noarch

>>> import os
>>> d, m = os.path.split('/usr/lib/yum-plugins/pulp-profile-update.py')
>>> m = m.split('.py')[0]
>>> m
'pulp-profile-update'
>>> d
'/usr/lib/yum-plugins'
>>> import imp
>>> imp.find_module(m, [d])
(, '/usr/lib/yum-plugins/pulp-profile-update.py', ('.py',
'U', 1))
>>> f, p, d = imp.find_module(m, [d])
>>> imp.load_module(m, f, p, d)
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/yum-plugins/pulp-profile-update.py", line 7, in 
from pulp.common.bundle import Bundle as BundleImpl
  File "/usr/lib/python2.7/site-packages/pulp/common/bundle.py", line 4, in

from pulp.plugins.util import misc
ImportError: No module named plugins.util

# ls /usr/lib/python2.7/site-packages/pulp/
agent  bindings  client  common  __init__.py  __init__.pyc  __init__.pyo

===
At host where I don't have this error, ls gives:
# ls /usr/lib/python2.7/site-packages/pulp
agent  bindings  client  common  __init__.py  __init__.pyc  __init__.pyo
 oid_validation  plugins  repoauth  server

# ls /usr/lib/python2.7/site-packages/pulp/plugins/
cataloger.py   conduitsconfig.pyo   distributor.pyo  importer.pyc
 __init__.pyc  migration  model.pyo profiler.pyo  util
cataloger.pyc  config.py   distributor.py   file importer.pyo
 __init__.pyo  model.py   profiler.py   rsync
cataloger.pyo  config.pyc  distributor.pyc  importer.py  __init__.py
loadermodel.pyc  profiler.pyc  types

# rpm -qf /usr/lib/python2.7/site-packages/pulp/plugins/util/misc.py
pulp-server-2.21.4-1.el7.noarch

===
Obviously if plugins/util/misc is included into the pulp-server package, it
can only be found on a server. So same as in
https://pulp.plan.io/issues/5126, 2.21.4 introduced server dependency on a
client.

Could you move the plugins package into the pulp-common package please?

Thanks!

-- 
Konstantin Khankin
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Can't get CentOS 7 Updates repo published

[Konstantin M. Khankin]

Thanks for the info Dennis!

I tried 'rpm repo sync sun --force-full' but it didn't help to clear the
corrupted entry. How can I drop this on the server side? Should I just
directly edit an entry in mongodb?

пт, 2 окт. 2020 г. в 14:39, Dennis Kliban :

> Filelists are stored in the database in a compressed form[0]. During
> publishing of metadata, each fillist is decompressed before being written
> to a file[1]. It's hard to speculate when data corruption occurred. It's
> possible that the data was corrupted in the remote repository the first
> time that this RPM was downloaded. That metadata could have been fixed
> since then, but Pulp thinks it already has the information and is not
> trying to re-download.
>
> [0]
> https://github.com/pulp/pulp_rpm/blob/2-master/plugins/pulp_rpm/plugins/db/models.py#L1051
> [1]
> https://github.com/pulp/pulp_rpm/blob/2-master/plugins/pulp_rpm/plugins/db/models.py#L1064
>
> On Fri, Oct 2, 2020 at 3:27 AM Konstantin M. Khankin <
> khankin.konstan...@gmail.com> wrote:
>
>> Sorry for reviving this thread but I think I found something.
>>
>> Today I tried to install CentOS 7 and got an error:
>> "TypeError: Parsing filelists.xml error: expected '>'"
>>
>> So I did this:
>>
>> >>> import xml.etree.ElementTree as ET
>> >>> tree =
>> ET.parse('/mnt/sysimage/var/tmp/yum.cache/centos_updates/gen/filelists.xml')
>> ...
>>   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in
>> _raiseerror
>> raise err
>> xml.etree.ElementTree.ParseError: not well-formed (invalid token): line
>> 1884123, column 136
>>
>> I opened filelists.xml and found that line 1884123 is indeed corrupt and
>> looks like this:
>>
>> /usr/share/javadoc/java-1.8.0-openjdk-1.8.0.111-1.b15.el7_2-debug/api/javax/sql/rowset/serial/compact3-package-frame.html>   !,^@ 0
>> 480!share/javadoc/java-1.8.0-openjdk-1.8.0.111-1.b15.el7_2-debug/api/javax/sql/rowset/serial/compact3-package-summary.html
>>
>> I verified another consumer has exactly the same issue with this file.
>>
>> Do I understand correctly Pulp generates filelists.xml? If so, can it be
>> because of a bug in Pulp or should I look for a silent data corruption
>> issue on my pulp server?
>>
>> Thanks!
>>
>>
>> вт, 1 сент. 2020 г. в 17:14, Dennis Kliban :
>>
>>> The repoview feature requires the sqlite db to be generated. This
>>> feature generates HTML pages for browsing the repository in a web browser.
>>> I believe that users are still able to browse the repository without it,
>>> but in that case the HTML listing of directories is generated by the web
>>> server each time a user requests it.
>>>
>>> On Tue, Sep 1, 2020 at 4:53 AM Konstantin M. Khankin <
>>> khankin.konstan...@gmail.com> wrote:
>>>
>>>> Thank you, that obviously helped. Still would be useful to know the
>>>> root cause.
>>>>
>>>> OTOH, if sqlite creation functionality is not that critical - why would
>>>> we ever enable something which only consumes time during sync for no
>>>> benefit?
>>>>
>>>> вт, 1 сент. 2020 г. в 00:26, Dennis Kliban :
>>>>
>>>>> Looks like everything is up to date. I have no idea what the root
>>>>> cause is, but according to this comment[0], you can work around the 
>>>>> problem
>>>>> by disabling the generation of sqlite db. I am not sure what the exact
>>>>> effect of this will be for the clients that are consuming this content, 
>>>>> but
>>>>> the repository will be usable.
>>>>>
>>>>> [0] https://pulp.plan.io/issues/2019#note-2
>>>>>
>>>>> On Mon, Aug 31, 2020 at 4:10 PM Konstantin M. Khankin <
>>>>> khankin.konstan...@gmail.com> wrote:
>>>>>
>>>>>> # rpm -qa | grep pulp | sort
>>>>>> pulp-admin-client-2.21.3-1.el7.noarch
>>>>>> pulp-agent-2.21.3-1.el7.noarch
>>>>>> pulp-consumer-client-2.21.3-1.el7.noarch
>>>>>> pulp-deb-admin-extensions-1.10.1-1.el7.noarch
>>>>>> pulp-deb-plugins-1.10.1-1.el7.noarch
>>>>>> pulp-docker-admin-extensions-3.2.6-1.el7.noarch
>>>>>> pulp-docker-plugins-3.2.6-1.el7.noarch
>>>>>> pulp-puppet-consumer-extensions-2.21.3-1.el7.noarch
>>>>>> pulp-puppet-handlers-2.21.3-1.el7.noarch
>>>>>> pulp-python-admin-extensions-2.0.4-1.el7.noarc

Re: [Pulp-list] Can't get CentOS 7 Updates repo published

[Konstantin M. Khankin]

Sorry for reviving this thread but I think I found something.

Today I tried to install CentOS 7 and got an error:
"TypeError: Parsing filelists.xml error: expected '>'"

So I did this:

>>> import xml.etree.ElementTree as ET
>>> tree =
ET.parse('/mnt/sysimage/var/tmp/yum.cache/centos_updates/gen/filelists.xml')
...
  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in
_raiseerror
raise err
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line
1884123, column 136

I opened filelists.xml and found that line 1884123 is indeed corrupt and
looks like this:

/usr/share/javadoc/java-1.8.0-openjdk-1.8.0.111-1.b15.el7_2-debug/api/javax/sql/rowset/serial/compact3-package-frame.html

I verified another consumer has exactly the same issue with this file.

Do I understand correctly Pulp generates filelists.xml? If so, can it be
because of a bug in Pulp or should I look for a silent data corruption
issue on my pulp server?

Thanks!


вт, 1 сент. 2020 г. в 17:14, Dennis Kliban :

> The repoview feature requires the sqlite db to be generated. This  feature
> generates HTML pages for browsing the repository in a web browser. I
> believe that users are still able to browse the repository without it, but
> in that case the HTML listing of directories is generated by the web server
> each time a user requests it.
>
> On Tue, Sep 1, 2020 at 4:53 AM Konstantin M. Khankin <
> khankin.konstan...@gmail.com> wrote:
>
>> Thank you, that obviously helped. Still would be useful to know the root
>> cause.
>>
>> OTOH, if sqlite creation functionality is not that critical - why would
>> we ever enable something which only consumes time during sync for no
>> benefit?
>>
>> вт, 1 сент. 2020 г. в 00:26, Dennis Kliban :
>>
>>> Looks like everything is up to date. I have no idea what the root cause
>>> is, but according to this comment[0], you can work around the problem by
>>> disabling the generation of sqlite db. I am not sure what the exact effect
>>> of this will be for the clients that are consuming this content, but the
>>> repository will be usable.
>>>
>>> [0] https://pulp.plan.io/issues/2019#note-2
>>>
>>> On Mon, Aug 31, 2020 at 4:10 PM Konstantin M. Khankin <
>>> khankin.konstan...@gmail.com> wrote:
>>>
>>>> # rpm -qa | grep pulp | sort
>>>> pulp-admin-client-2.21.3-1.el7.noarch
>>>> pulp-agent-2.21.3-1.el7.noarch
>>>> pulp-consumer-client-2.21.3-1.el7.noarch
>>>> pulp-deb-admin-extensions-1.10.1-1.el7.noarch
>>>> pulp-deb-plugins-1.10.1-1.el7.noarch
>>>> pulp-docker-admin-extensions-3.2.6-1.el7.noarch
>>>> pulp-docker-plugins-3.2.6-1.el7.noarch
>>>> pulp-puppet-consumer-extensions-2.21.3-1.el7.noarch
>>>> pulp-puppet-handlers-2.21.3-1.el7.noarch
>>>> pulp-python-admin-extensions-2.0.4-1.el7.noarch
>>>> pulp-python-plugins-2.0.4-1.el7.noarch
>>>> pulp-rpm-admin-extensions-2.21.3-1.el7.noarch
>>>> pulp-rpm-consumer-extensions-2.21.3-1.el7.noarch
>>>> pulp-rpm-handlers-2.21.3-1.el7.noarch
>>>> pulp-rpm-plugins-2.21.3-1.el7.noarch
>>>> pulp-rpm-yumplugins-2.21.3-1.el7.noarch
>>>> pulp-selinux-2.21.3-1.el7.noarch
>>>> pulp-server-2.21.3-1.el7.noarch
>>>> python-pulp-agent-lib-2.21.3-1.el7.noarch
>>>> python-pulp-bindings-2.21.3-1.el7.noarch
>>>> python-pulp-client-lib-2.21.3-1.el7.noarch
>>>> python-pulp-common-2.21.3-1.el7.noarch
>>>> python-pulp-deb-common-1.10.1-1.el7.noarch
>>>> python-pulp-docker-common-3.2.6-1.el7.noarch
>>>> python-pulp-oid_validation-2.21.3-1.el7.noarch
>>>> python-pulp-puppet-common-2.21.3-1.el7.noarch
>>>> python-pulp-python-common-2.0.4-1.el7.noarch
>>>> python-pulp-repoauth-2.21.3-1.el7.noarch
>>>> python-pulp-rpm-common-2.21.3-1.el7.noarch
>>>>
>>>> # rpm -qa | grep createrepo
>>>> createrepo-0.9.9-28.el7.noarch
>>>> createrepo_c-0.10.0-20.el7.x86_64
>>>> createrepo_c-libs-0.10.0-20.el7.x86_64
>>>>
>>>> # cat /etc/redhat-release
>>>> CentOS Linux release 7.8.2003 (Core)
>>>>
>>>> пн, 31 авг. 2020 г. в 22:48, Dennis Kliban :
>>>>
>>>>> This looks exactly like the issue that was reported here[0].
>>>>>
>>>>> What version of pulp are you using? What version of createrepo_c is
>>>>> installed?
>>>>>
>>>>> [0] https://pulp.plan.io/issues/2019
>>>>>
>

Re: [Pulp-list] Can't get CentOS 7 Updates repo published

[Konstantin M. Khankin]

Thank you, that obviously helped. Still would be useful to know the root
cause.

OTOH, if sqlite creation functionality is not that critical - why would we
ever enable something which only consumes time during sync for no benefit?

вт, 1 сент. 2020 г. в 00:26, Dennis Kliban :

> Looks like everything is up to date. I have no idea what the root cause
> is, but according to this comment[0], you can work around the problem by
> disabling the generation of sqlite db. I am not sure what the exact effect
> of this will be for the clients that are consuming this content, but the
> repository will be usable.
>
> [0] https://pulp.plan.io/issues/2019#note-2
>
> On Mon, Aug 31, 2020 at 4:10 PM Konstantin M. Khankin <
> khankin.konstan...@gmail.com> wrote:
>
>> # rpm -qa | grep pulp | sort
>> pulp-admin-client-2.21.3-1.el7.noarch
>> pulp-agent-2.21.3-1.el7.noarch
>> pulp-consumer-client-2.21.3-1.el7.noarch
>> pulp-deb-admin-extensions-1.10.1-1.el7.noarch
>> pulp-deb-plugins-1.10.1-1.el7.noarch
>> pulp-docker-admin-extensions-3.2.6-1.el7.noarch
>> pulp-docker-plugins-3.2.6-1.el7.noarch
>> pulp-puppet-consumer-extensions-2.21.3-1.el7.noarch
>> pulp-puppet-handlers-2.21.3-1.el7.noarch
>> pulp-python-admin-extensions-2.0.4-1.el7.noarch
>> pulp-python-plugins-2.0.4-1.el7.noarch
>> pulp-rpm-admin-extensions-2.21.3-1.el7.noarch
>> pulp-rpm-consumer-extensions-2.21.3-1.el7.noarch
>> pulp-rpm-handlers-2.21.3-1.el7.noarch
>> pulp-rpm-plugins-2.21.3-1.el7.noarch
>> pulp-rpm-yumplugins-2.21.3-1.el7.noarch
>> pulp-selinux-2.21.3-1.el7.noarch
>> pulp-server-2.21.3-1.el7.noarch
>> python-pulp-agent-lib-2.21.3-1.el7.noarch
>> python-pulp-bindings-2.21.3-1.el7.noarch
>> python-pulp-client-lib-2.21.3-1.el7.noarch
>> python-pulp-common-2.21.3-1.el7.noarch
>> python-pulp-deb-common-1.10.1-1.el7.noarch
>> python-pulp-docker-common-3.2.6-1.el7.noarch
>> python-pulp-oid_validation-2.21.3-1.el7.noarch
>> python-pulp-puppet-common-2.21.3-1.el7.noarch
>> python-pulp-python-common-2.0.4-1.el7.noarch
>> python-pulp-repoauth-2.21.3-1.el7.noarch
>> python-pulp-rpm-common-2.21.3-1.el7.noarch
>>
>> # rpm -qa | grep createrepo
>> createrepo-0.9.9-28.el7.noarch
>> createrepo_c-0.10.0-20.el7.x86_64
>> createrepo_c-libs-0.10.0-20.el7.x86_64
>>
>> # cat /etc/redhat-release
>> CentOS Linux release 7.8.2003 (Core)
>>
>> пн, 31 авг. 2020 г. в 22:48, Dennis Kliban :
>>
>>> This looks exactly like the issue that was reported here[0].
>>>
>>> What version of pulp are you using? What version of createrepo_c is
>>> installed?
>>>
>>> [0] https://pulp.plan.io/issues/2019
>>>
>>> On Mon, Aug 31, 2020 at 3:16 PM Konstantin M. Khankin <
>>> khankin.konstan...@gmail.com> wrote:
>>>
>>>> Hi!
>>>>
>>>> The issue still persists. Could someone take a look please?
>>>>
>>>> Thanks!
>>>>
>>>> ср, 19 авг. 2020 г., 13:15 Konstantin M. Khankin <
>>>> khankin.konstan...@gmail.com>:
>>>>
>>>>> Hi!
>>>>>
>>>>> I found that my pulp2-managed mirror of
>>>>> http://mirror.yandex.ru/centos/7/updates/x86_64 has not been
>>>>> successfully published since April 30. I ran publish manually and received
>>>>> an error:
>>>>>
>>>>> '''
>>>>> Generating sqlite files
>>>>> [/]
>>>>> ... failed
>>>>> Error occurred during 'sqliterepo_c' execution: Preparing sqlite
>>>>> DBs
>>>>>
>>>>> ::
>>>>> C_CREATEREPOLIB: Critical: cr_xml_parser_generic: parsing error
>>>>> '/var/cache/pulp/reserved_resource_worke...@hive.gsk.loc
>>>>> /12257118-33e7-4294-ad68
>>>>>
>>>>> -98e0515f2627/repodata/9e958c09c7880d130ef3321332000f43a4e4e701e416801a0f54313f5
>>>>> 766d03b-filelists.xml.gz': not well-formed (invalid token)
>>>>> Parse error
>>>>> '/var/cache/pulp/reserved_resource_worke...@hive.gsk.loc
>>>>> /12257118-33e7-4294-ad68
>>>>>
>>>>> -98e0515f2627/repodata/9e958c09c7880d130ef3321332000f43a4e4e701e416801a0f54313f5
>>>>> 766d03b-filelists.xml.gz' at line: 1884123 (not well-formed (invalid
>>>>> token))
>>>>> '''
>>>>>
>>>>> I can't open .sqlite files in /tmp from CLI either:
>>>>>
>>>>> '''
>>>>> -rw-

Re: [Pulp-list] Can't get CentOS 7 Updates repo published

[Konstantin M. Khankin]

# rpm -qa | grep pulp | sort
pulp-admin-client-2.21.3-1.el7.noarch
pulp-agent-2.21.3-1.el7.noarch
pulp-consumer-client-2.21.3-1.el7.noarch
pulp-deb-admin-extensions-1.10.1-1.el7.noarch
pulp-deb-plugins-1.10.1-1.el7.noarch
pulp-docker-admin-extensions-3.2.6-1.el7.noarch
pulp-docker-plugins-3.2.6-1.el7.noarch
pulp-puppet-consumer-extensions-2.21.3-1.el7.noarch
pulp-puppet-handlers-2.21.3-1.el7.noarch
pulp-python-admin-extensions-2.0.4-1.el7.noarch
pulp-python-plugins-2.0.4-1.el7.noarch
pulp-rpm-admin-extensions-2.21.3-1.el7.noarch
pulp-rpm-consumer-extensions-2.21.3-1.el7.noarch
pulp-rpm-handlers-2.21.3-1.el7.noarch
pulp-rpm-plugins-2.21.3-1.el7.noarch
pulp-rpm-yumplugins-2.21.3-1.el7.noarch
pulp-selinux-2.21.3-1.el7.noarch
pulp-server-2.21.3-1.el7.noarch
python-pulp-agent-lib-2.21.3-1.el7.noarch
python-pulp-bindings-2.21.3-1.el7.noarch
python-pulp-client-lib-2.21.3-1.el7.noarch
python-pulp-common-2.21.3-1.el7.noarch
python-pulp-deb-common-1.10.1-1.el7.noarch
python-pulp-docker-common-3.2.6-1.el7.noarch
python-pulp-oid_validation-2.21.3-1.el7.noarch
python-pulp-puppet-common-2.21.3-1.el7.noarch
python-pulp-python-common-2.0.4-1.el7.noarch
python-pulp-repoauth-2.21.3-1.el7.noarch
python-pulp-rpm-common-2.21.3-1.el7.noarch

# rpm -qa | grep createrepo
createrepo-0.9.9-28.el7.noarch
createrepo_c-0.10.0-20.el7.x86_64
createrepo_c-libs-0.10.0-20.el7.x86_64

# cat /etc/redhat-release
CentOS Linux release 7.8.2003 (Core)

пн, 31 авг. 2020 г. в 22:48, Dennis Kliban :

> This looks exactly like the issue that was reported here[0].
>
> What version of pulp are you using? What version of createrepo_c is
> installed?
>
> [0] https://pulp.plan.io/issues/2019
>
> On Mon, Aug 31, 2020 at 3:16 PM Konstantin M. Khankin <
> khankin.konstan...@gmail.com> wrote:
>
>> Hi!
>>
>> The issue still persists. Could someone take a look please?
>>
>> Thanks!
>>
>> ср, 19 авг. 2020 г., 13:15 Konstantin M. Khankin <
>> khankin.konstan...@gmail.com>:
>>
>>> Hi!
>>>
>>> I found that my pulp2-managed mirror of
>>> http://mirror.yandex.ru/centos/7/updates/x86_64 has not been
>>> successfully published since April 30. I ran publish manually and received
>>> an error:
>>>
>>> '''
>>> Generating sqlite files
>>> [/]
>>> ... failed
>>> Error occurred during 'sqliterepo_c' execution: Preparing sqlite
>>> DBs
>>>
>>> ::
>>> C_CREATEREPOLIB: Critical: cr_xml_parser_generic: parsing error
>>> '/var/cache/pulp/reserved_resource_worke...@hive.gsk.loc
>>> /12257118-33e7-4294-ad68
>>>
>>> -98e0515f2627/repodata/9e958c09c7880d130ef3321332000f43a4e4e701e416801a0f54313f5
>>> 766d03b-filelists.xml.gz': not well-formed (invalid token)
>>> Parse error
>>> '/var/cache/pulp/reserved_resource_worke...@hive.gsk.loc
>>> /12257118-33e7-4294-ad68
>>>
>>> -98e0515f2627/repodata/9e958c09c7880d130ef3321332000f43a4e4e701e416801a0f54313f5
>>> 766d03b-filelists.xml.gz' at line: 1884123 (not well-formed (invalid
>>> token))
>>> '''
>>>
>>> I can't open .sqlite files in /tmp from CLI either:
>>>
>>> '''
>>> -rw---. 1 apache apache  54066176 Aug 19 15:12
>>> filelists.211JP0.sqlite
>>> -rw---. 1 apache apache 157097984 Aug 19 15:12 primary.PQ3JP0.sqlite
>>> -rw---. 1 apache apache 0 Aug 19 15:12 other.X201JP0.sqlite
>>>
>>> [root@hive tmp]# sqlite3 primary.PQ3JP0.sqlite
>>> SQLite version 3.7.17 2013-05-20 00:56:22
>>> Enter ".help" for instructions
>>> Enter SQL statements terminated with a ";"
>>> sqlite> .databases
>>> Error: file is encrypted or is not a database
>>> '''
>>>
>>> I have multiple repos managed by pulp, some of which originate also from
>>> mirror.yandex.ru, and they are synced and published normally.
>>> --force-full doesn't help.
>>>
>>> What issue could be here?
>>>
>>> Thanks!
>>>
>>> --
>>> Konstantin Khankin
>>>
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
>

-- 
Ханкин Константин
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Can't get CentOS 7 Updates repo published

[Konstantin M. Khankin]

Hi!

The issue still persists. Could someone take a look please?

Thanks!

ср, 19 авг. 2020 г., 13:15 Konstantin M. Khankin <
khankin.konstan...@gmail.com>:

> Hi!
>
> I found that my pulp2-managed mirror of
> http://mirror.yandex.ru/centos/7/updates/x86_64 has not been successfully
> published since April 30. I ran publish manually and received an error:
>
> '''
> Generating sqlite files
> [/]
> ... failed
> Error occurred during 'sqliterepo_c' execution: Preparing sqlite
> DBs
>
> ::
> C_CREATEREPOLIB: Critical: cr_xml_parser_generic: parsing error
> '/var/cache/pulp/reserved_resource_worke...@hive.gsk.loc
> /12257118-33e7-4294-ad68
>
> -98e0515f2627/repodata/9e958c09c7880d130ef3321332000f43a4e4e701e416801a0f54313f5
> 766d03b-filelists.xml.gz': not well-formed (invalid token)
> Parse error
> '/var/cache/pulp/reserved_resource_worke...@hive.gsk.loc
> /12257118-33e7-4294-ad68
>
> -98e0515f2627/repodata/9e958c09c7880d130ef3321332000f43a4e4e701e416801a0f54313f5
> 766d03b-filelists.xml.gz' at line: 1884123 (not well-formed (invalid
> token))
> '''
>
> I can't open .sqlite files in /tmp from CLI either:
>
> '''
> -rw---. 1 apache apache  54066176 Aug 19 15:12 filelists.211JP0.sqlite
> -rw---. 1 apache apache 157097984 Aug 19 15:12 primary.PQ3JP0.sqlite
> -rw---. 1 apache apache 0 Aug 19 15:12 other.X201JP0.sqlite
>
> [root@hive tmp]# sqlite3 primary.PQ3JP0.sqlite
> SQLite version 3.7.17 2013-05-20 00:56:22
> Enter ".help" for instructions
> Enter SQL statements terminated with a ";"
> sqlite> .databases
> Error: file is encrypted or is not a database
> '''
>
> I have multiple repos managed by pulp, some of which originate also from
> mirror.yandex.ru, and they are synced and published normally.
> --force-full doesn't help.
>
> What issue could be here?
>
> Thanks!
>
> --
> Konstantin Khankin
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Can't get CentOS 7 Updates repo published

[Konstantin M. Khankin]

Hi!

I found that my pulp2-managed mirror of
http://mirror.yandex.ru/centos/7/updates/x86_64 has not been successfully
published since April 30. I ran publish manually and received an error:

'''
Generating sqlite files
[/]
... failed
Error occurred during 'sqliterepo_c' execution: Preparing sqlite
DBs

::
C_CREATEREPOLIB: Critical: cr_xml_parser_generic: parsing error
'/var/cache/pulp/reserved_resource_worke...@hive.gsk.loc
/12257118-33e7-4294-ad68
-98e0515f2627/repodata/9e958c09c7880d130ef3321332000f43a4e4e701e416801a0f54313f5
766d03b-filelists.xml.gz': not well-formed (invalid token)
Parse error
'/var/cache/pulp/reserved_resource_worke...@hive.gsk.loc
/12257118-33e7-4294-ad68
-98e0515f2627/repodata/9e958c09c7880d130ef3321332000f43a4e4e701e416801a0f54313f5
766d03b-filelists.xml.gz' at line: 1884123 (not well-formed (invalid token))
'''

I can't open .sqlite files in /tmp from CLI either:

'''
-rw---. 1 apache apache  54066176 Aug 19 15:12 filelists.211JP0.sqlite
-rw---. 1 apache apache 157097984 Aug 19 15:12 primary.PQ3JP0.sqlite
-rw---. 1 apache apache 0 Aug 19 15:12 other.X201JP0.sqlite

[root@hive tmp]# sqlite3 primary.PQ3JP0.sqlite
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .databases
Error: file is encrypted or is not a database
'''

I have multiple repos managed by pulp, some of which originate also from
mirror.yandex.ru, and they are synced and published normally. --force-full
doesn't help.

What issue could be here?

Thanks!

-- 
Konstantin Khankin
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Can't reinstate a replica from scratch after it was off for 6 months

[Konstantin M. Khankin]

I'm sorry for spam. It's so hard to choose the right mailing list when it's
night already.

вс, 9 авг. 2020 г., 01:11 Konstantin M. Khankin <
khankin.konstan...@gmail.com>:

> Hi!
>
> I run IPA on CentOS 7. I have two servers (Leader and Replica, though they
> changed roles couple times because of reinstalls), had ca and domain
> services on both of them, replication set up and working. I had to switch
> off Replica for 6 months. When I turned it on recently, I found expired
> certificates, couldn't fix them easily and lost the old Replica - at least
> I concluded it was easier to reinstate the Replica than to detange the mess
> I made while was trying to back out of outdated certs. I hit the same error
> as I do now though - Invalid Credentials (49).
>
> So I did the following:
>
> 1) on Replica - ipa-server-install --uninstall.
> 2) on Leader - ipa-replica-manage del --force --clean Replica.
> 3) removed obsolete replication agreement meToReplica from Leader.
> 4) removed all traces of Replica from DNS.
>
> Then I started to install Replica from scratch:
>
> 1) ipa-client-install
> 2) ipa-replica-install --setup-ca --setup-dns --forwarder X --forwarder Y
>
> Installation consistently fails with:
>
> '''
> Run connection check to master
> Connection check OK
> Configuring directory server (dirsrv). Estimated time: 30 seconds
> <...>
>   [29/42]: setting up initial replication
> Starting replication, please wait until this has completed.
> Update in progress, 16 seconds elapsed
> [ldap://Leader:389] reports: Update failed! Status: [Error (49)  - LDAP
> error: Invalid credentials]
>
>   [error] RuntimeError: Failed to start replication
> '''
>
> Logs from Leader, /var/log/dirsrv/slapd-DOMAIN/errors:
>
> '''
> [] - ERR - NSMMReplicationPlugin - bind_and_check_pwp -
> agmt="cn=meToReplica.domain" (Replica:389) - Replication bind with GSSAPI
> auth failed: LDAP error 49 (Invalid credentials) ()
> """
>
> I verified clocks on both Replica and Leader - they show the same time
> (within 1-2 seconds diff window). In fact, at some point I had Replica
> taking time straight from Leader, before they were set up to use the other
> common source. I dumped tracffic between Leader and Replica - indeed,
> Leader tried to authenticate on Replica and Replica replies "Invalid
> credentials".
>
> I googled this error and read multiple email threads but nothing helped so
> far. Replica works fine as IPA client but can't get promoted to a replica.
>
> What am I missing?
>
> Thanks!
>
> --
> Khankin Konstantin
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Can't reinstate a replica from scratch after it was off for 6 months

[Konstantin M. Khankin]

Hi!

I run IPA on CentOS 7. I have two servers (Leader and Replica, though they
changed roles couple times because of reinstalls), had ca and domain
services on both of them, replication set up and working. I had to switch
off Replica for 6 months. When I turned it on recently, I found expired
certificates, couldn't fix them easily and lost the old Replica - at least
I concluded it was easier to reinstate the Replica than to detange the mess
I made while was trying to back out of outdated certs. I hit the same error
as I do now though - Invalid Credentials (49).

So I did the following:

1) on Replica - ipa-server-install --uninstall.
2) on Leader - ipa-replica-manage del --force --clean Replica.
3) removed obsolete replication agreement meToReplica from Leader.
4) removed all traces of Replica from DNS.

Then I started to install Replica from scratch:

1) ipa-client-install
2) ipa-replica-install --setup-ca --setup-dns --forwarder X --forwarder Y

Installation consistently fails with:

'''
Run connection check to master
Connection check OK
Configuring directory server (dirsrv). Estimated time: 30 seconds
<...>
  [29/42]: setting up initial replication
Starting replication, please wait until this has completed.
Update in progress, 16 seconds elapsed
[ldap://Leader:389] reports: Update failed! Status: [Error (49)  - LDAP
error: Invalid credentials]

  [error] RuntimeError: Failed to start replication
'''

Logs from Leader, /var/log/dirsrv/slapd-DOMAIN/errors:

'''
[] - ERR - NSMMReplicationPlugin - bind_and_check_pwp -
agmt="cn=meToReplica.domain" (Replica:389) - Replication bind with GSSAPI
auth failed: LDAP error 49 (Invalid credentials) ()
"""

I verified clocks on both Replica and Leader - they show the same time
(within 1-2 seconds diff window). In fact, at some point I had Replica
taking time straight from Leader, before they were set up to use the other
common source. I dumped tracffic between Leader and Replica - indeed,
Leader tried to authenticate on Replica and Replica replies "Invalid
credentials".

I googled this error and read multiple email threads but nothing helped so
far. Replica works fine as IPA client but can't get promoted to a replica.

What am I missing?

Thanks!

--
Khankin Konstantin
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] [Pulp-dev] Pulp 3 CLI MVP

[Konstantin M. Khankin]

Is it an option to keep the Pulp 2 CLI syntax as much as possible?

чт, 7 мая 2020 г. в 15:28, Dennis Kliban :

> On Thu, May 7, 2020 at 7:13 AM Tatiana Tereshchenko 
> wrote:
>
>> +1 to `pulp` command.
>>
>> I think for me as a user, the most logical would be to have a plugin name
>> first and then follow the URL pattern.
>> The majority of commands are plugin specific. If I work with multiple
>> plugins, it also makes it easy for me as a user to just change the plugin
>> name in front for the commands which have the same structure in different
>> plugins.
>> It also makes it visually clear that I work with a specific plugin, in
>> comparison to when the plugin name is somewhere in the 3rd/4th place.
>> It will also allow not to guess in commands like the `pulp repositories
>> rpm rpm`  which one is the plugin name and which one is a repo type.
>>
>> I agree that this would make much more clear to the user which 'rpm' is
> the plugin type and which 'rpm' is the resource type.
>
>
>> +1 for
>> pulp rpm content packages
>> pulp rpm repositories rpm
>> pulp rpm repositories mirror
>> ...
>>
>> On Wed, May 6, 2020 at 7:58 PM Dennis Kliban  wrote:
>>
>>> On Wed, May 6, 2020 at 12:30 PM David Davis 
>>> wrote:
>>>
 Matthias and I met today to go over some plans for a prototype. I wrote
 some notes[0] down. As part of the prototype, we'd propose two deliverables
 (one this week and one next week):

 1. A set of ~2-3 click commands that use the bindings to interact with
 Pulp
 2. Some openapi-generator templates that will be able to generate such
 commands from the schema

 There is a question we had about how the commands for typed resources
 will be structured in the CLI. To illustrate with two endpoints:


>>> We should call the command 'pulp' instead of pulp-cli.
>>>
>>>
 # rpm.package content (/pulp/api/v3/content/rpm/packages/):
 - pulp-cli rpm content packages ...
 - pulp-cli content rpm packages ...
 - pulp-cli rpm packages content ...
 - ???

>>>
>>> I was thinking we should structure the commands similar to the URLs in
>>> the REST API.
>>>
>>> pulp content rpm packages
>>>
>>>

 # file.file repositories (/pulp/api/v3/repositories/file/file/):
 - pulp-cli file repositories file ...
 - pulp-cli repositories file file ...
 - pulp-cli file file repositories ...
 - ???

 pulp repositories file
>>>
>>> Plugins that provide multiple types of the same resource would need to
>>> be more descriptive. Though I can see a practical reason for requiring all
>>> resources to be that descriptive.
>>>
>>> pulp repositories rpm rpm
>>> pulp repositories rpm mirror
>>>
>>>
>>>
 [0] https://hackmd.io/aH9RqAS_TrGyxoi1UGRgig?view#Prototype

 David


 On Thu, Apr 30, 2020 at 1:42 PM David Davis 
 wrote:

> Today we met to discuss some ideas for a technical design for how the
> CLI would work. Here's a copy of our notes:
>
> https://hackmd.io/aH9RqAS_TrGyxoi1UGRgig#Technical-discussion
>
> And there is a rough design in the document as well:
>
> https://hackmd.io/aH9RqAS_TrGyxoi1UGRgig#Design
>
> I have also entered the CLI user stories from our meeting last week
> into redmine under the Pulp CLI project:
>
> https://pulp.plan.io/versions/93
>
> And I've filed a user story that we talked about today that would
> handle sync, publish, and distribution of repos. Feedback welcome:
>
> https://pulp.plan.io/issues/6626
>
> Matthias and I are planning to meet next week to look at creating a
> proof of concept that would provide 2-3 commands. If anyone is interested
> in joining us, please let me know and I can add you.
>
> David
>
>
> On Tue, Apr 28, 2020 at 8:06 AM David Davis 
> wrote:
>
>> I've also started working on some questions about how the CLI will
>> work. Feel free to add some of your own:
>>
>> https://hackmd.io/aH9RqAS_TrGyxoi1UGRgig?view#Technical-discussion
>>
>> David
>>
>>
>> On Tue, Apr 28, 2020 at 8:05 AM David Davis 
>> wrote:
>>
>>> I have set up a meeting to discuss the CLI technical design. Below
>>> are the details. I think a video conference might be easier for 
>>> technical
>>> discussion but am open to consider meeting on #pulp-meeting again.
>>>
>>> URL: https://meet.google.com/vgx-bzbb-wnh
>>> Date/time: April 30, 2020 at 9:00am ET (1pm UTC)
>>>
>>> David
>>>
>>>
>>> On Fri, Apr 24, 2020 at 10:29 AM David Davis 
>>> wrote:
>>>
 Today we met in #pulp-meeting on freenode to discuss the user
 stories for a Pulp 3 CLI MVP. The document with the user stories is
 available below. I'd like to ask for any feedback from users or plugin
 writers.

 The goal of the CLI MVP is to cover the pulp_file happy path 

Re: [Pulp-list] Can't import containers created with Buildah

[Konstantin M. Khankin]

Thank you Ina!

I manually applied this patch and looks like it actually works :)

ср, 25 сент. 2019 г. в 19:24, Ina Panova :

> Hi Konstantin,
>
> surprisingly today i was able to reproduce the issue and was able to find
> the root cause of the problem.
> Here's the PR that should fix the upload issue
> https://github.com/pulp/pulp_docker/pull/418
>
>
> 
> Regards,
>
> Ina Panova
> Senior Software Engineer| Pulp| Red Hat Inc.
>
> "Do not go where the path may lead,
>  go instead where there is no path and leave a trail."
>
>
> On Tue, Sep 17, 2019 at 11:58 AM Konstantin M. Khankin <
> khankin.konstan...@gmail.com> wrote:
>
>> Sorry for necroposting. I'm not sure if I still have the same issue, but
>> I have the same symptoms. Could you please have a look and advice? Thanks!
>>
>> Whenever I try to create and upload another version of a container to the
>> same repo, I get an error message like this:
>>
>> $ /usr/bin/pulp-admin docker repo uploads upload --repo-id gsk_postfix -f
>> /tmp/postfix.tar
>> ...
>> Uploading: postfix.tar
>> [==] 100%
>> 283176960/283176960 bytes
>> ... completed
>>
>> Importing into the repository...
>> This command may be exited via ctrl+c without affecting the request.
>>
>>
>> [-]
>> Running...
>>
>> Task Failed
>>
>> The importer docker_importer indicated a failed response when uploading
>> docker_manifest unit to repository gsk_postfix. Summary: 'Cannot resolve
>> field
>> "digest"'. Details: '{}'
>>
>> I run pulp 2.20.1 and use buildah 1.9.0 to build container images. First
>> image in the repository uploads fine, but the second always fails.
>>
>> Current repo status:
>> Id:  gsk_postfix
>> Display Name:None
>> Description: None
>> Content Unit Counts:
>>   Docker Blob: 4
>>   Docker Manifest: 1
>>   Docker Tag:  1
>>
>> # pulp-admin docker repo search image --repo-id gsk_postfix
>> 
>>
>> # pulp-admin docker repo search manifest --repo-id gsk_postfix
>> Created:  2019-08-20T22:58:50Z
>> Metadata:
>>   Config Layer:
>> sha256:d2b497c83fe099624567046432468d84a5fdb49139b5192c9b332f0b10784695
>>   Digest:
>> sha256:d17067c71cb229f703b0f43ecd8a85c55995f3007b99570f580f9f6ebc5dbccc
>>   Downloaded: True
>>   Fs Layers:
>> Blob Sum:
>> sha256:b05580fca2f9aabb2d8fa975b29146c9147c8418e559f197c54a4fac04babb95
>> Layer Type: application/vnd.docker.image.rootfs.diff.tar.gzip
>> Size:   210204160
>> Blob Sum:
>> sha256:204cafbfdb1d31cb427478102377a99ceccfd7cc7be19cc191568fb9008fd4af
>> Layer Type: application/vnd.docker.image.rootfs.diff.tar.gzip
>> Size:   12288
>> Blob Sum:
>> sha256:a85e1551e989545ca081c9ed0584e2db535a1484b91060eed88a17991e4b25cf
>> Layer Type: application/vnd.docker.image.rootfs.diff.tar.gzip
>> Size:   72946688
>>   Pulp User Metadata:
>>   Schema Version: 2
>> Repo Id:  gsk_postfix
>> Unit Id:  000c0b4a-fb98-4ca3-a3ec-3b2a42521949
>> Unit Type Id: docker_manifest
>> Updated:  2019-08-20T22:58:50Z
>>
>> # pulp-admin docker repo search tag --repo-id gsk_postfix
>> Created:  2019-08-20T22:58:55Z
>> Metadata:
>>   Manifest Digest:
>>  sha256:d17067c71cb229f703b0f43ecd8a85c55995f3007b99570f580f9f6ebc5dbccc
>>   Manifest Type:  image
>>   Name:   latest
>>   Pulp User Metadata:
>>   Repo Id:gsk_postfix
>>   Schema Version: 2
>> Repo Id:  gsk_postfix
>> Unit Id:  26c40bb1-4f3b-438f-8f50-cc5afe6baeb6
>> Unit Type Id: docker_tag
>> Updated:  2019-08-20T22:58:55Z
>>
>> To build and upload containers, I wrote a simple script. Here is the
>> command sequence:
>> ### STEP 1: Build an image from Dockerfile.
>> $ /usr/bin/buildah bud -t gsk_postfix .
>> ...
>> Writing manifest to image destination
>> Storing signatures
>> db008cf80da01afe8ca11898e566279a7765e682bf56070857d40ffd469e2486
>> ### STEP 2: Get the image digest.
>> $ /usr/bin/buildah images --quiet --digests --no-trunc
>> localhost/gsk_postfix:latest
>> sha256:db008cf80da01afe8ca11898e566279a7765e682bf56070857d40ffd469e2486
>> ### STEP 3: Pull blobs from the storage.
>> $ /usr/bin/skopeo copy --format v2s2
>> containers-storage:localhost/gsk_postfix:latest dir:/tmp/tmp.NYhxiiOsOe
>> Get

Re: [Pulp-list] Can't import containers created with Buildah

[Konstantin M. Khankin]

Sorry for necroposting. I'm not sure if I still have the same issue, but I
have the same symptoms. Could you please have a look and advice? Thanks!

Whenever I try to create and upload another version of a container to the
same repo, I get an error message like this:

$ /usr/bin/pulp-admin docker repo uploads upload --repo-id gsk_postfix -f
/tmp/postfix.tar
...
Uploading: postfix.tar
[==] 100%
283176960/283176960 bytes
... completed

Importing into the repository...
This command may be exited via ctrl+c without affecting the request.


[-]
Running...

Task Failed

The importer docker_importer indicated a failed response when uploading
docker_manifest unit to repository gsk_postfix. Summary: 'Cannot resolve
field
"digest"'. Details: '{}'

I run pulp 2.20.1 and use buildah 1.9.0 to build container images. First
image in the repository uploads fine, but the second always fails.

Current repo status:
Id:  gsk_postfix
Display Name:None
Description: None
Content Unit Counts:
  Docker Blob: 4
  Docker Manifest: 1
  Docker Tag:  1

# pulp-admin docker repo search image --repo-id gsk_postfix


# pulp-admin docker repo search manifest --repo-id gsk_postfix
Created:  2019-08-20T22:58:50Z
Metadata:
  Config Layer:
sha256:d2b497c83fe099624567046432468d84a5fdb49139b5192c9b332f0b10784695
  Digest:
sha256:d17067c71cb229f703b0f43ecd8a85c55995f3007b99570f580f9f6ebc5dbccc
  Downloaded: True
  Fs Layers:
Blob Sum:
sha256:b05580fca2f9aabb2d8fa975b29146c9147c8418e559f197c54a4fac04babb95
Layer Type: application/vnd.docker.image.rootfs.diff.tar.gzip
Size:   210204160
Blob Sum:
sha256:204cafbfdb1d31cb427478102377a99ceccfd7cc7be19cc191568fb9008fd4af
Layer Type: application/vnd.docker.image.rootfs.diff.tar.gzip
Size:   12288
Blob Sum:
sha256:a85e1551e989545ca081c9ed0584e2db535a1484b91060eed88a17991e4b25cf
Layer Type: application/vnd.docker.image.rootfs.diff.tar.gzip
Size:   72946688
  Pulp User Metadata:
  Schema Version: 2
Repo Id:  gsk_postfix
Unit Id:  000c0b4a-fb98-4ca3-a3ec-3b2a42521949
Unit Type Id: docker_manifest
Updated:  2019-08-20T22:58:50Z

# pulp-admin docker repo search tag --repo-id gsk_postfix
Created:  2019-08-20T22:58:55Z
Metadata:
  Manifest Digest:
 sha256:d17067c71cb229f703b0f43ecd8a85c55995f3007b99570f580f9f6ebc5dbccc
  Manifest Type:  image
  Name:   latest
  Pulp User Metadata:
  Repo Id:gsk_postfix
  Schema Version: 2
Repo Id:  gsk_postfix
Unit Id:  26c40bb1-4f3b-438f-8f50-cc5afe6baeb6
Unit Type Id: docker_tag
Updated:  2019-08-20T22:58:55Z

To build and upload containers, I wrote a simple script. Here is the
command sequence:
### STEP 1: Build an image from Dockerfile.
$ /usr/bin/buildah bud -t gsk_postfix .
...
Writing manifest to image destination
Storing signatures
db008cf80da01afe8ca11898e566279a7765e682bf56070857d40ffd469e2486
### STEP 2: Get the image digest.
$ /usr/bin/buildah images --quiet --digests --no-trunc
localhost/gsk_postfix:latest
sha256:db008cf80da01afe8ca11898e566279a7765e682bf56070857d40ffd469e2486
### STEP 3: Pull blobs from the storage.
$ /usr/bin/skopeo copy --format v2s2
containers-storage:localhost/gsk_postfix:latest dir:/tmp/tmp.NYhxiiOsOe
Getting image source signatures
Copying blob b05580fca2f9 done
Copying blob 8d136bb340b2 done
Copying blob 7a406e7184ab done
Copying config db008cf80d done
Writing manifest to image destination
Storing signatures
### STEP 4: Tar the archive.
$ tar cf /tmp/postfix.tar *
### STEP 5: Upload the archive to the repo.
$ /usr/bin/pulp-admin docker repo uploads upload --repo-id gsk_postfix -f
/tmp/postfix.tar


This happens no matter if I use v2s1 or v2s2. Resulting manifest.json
contains digests for all blobs (different image but built at the same time
with the same script and failed to get uploaded with the same error):
{'config': {'digest':
'sha256:fef5b5f9bb6fca5f1909303822ae34ae9d3ac1d88da84144fa1a7b83229652f3',
'mediaType': 'application/vnd.docker.container.image.v1+json',
'size': 2339},
 'layers': [{'digest':
'sha256:b05580fca2f9aabb2d8fa975b29146c9147c8418e559f197c54a4fac04babb95',
 'mediaType':
'application/vnd.docker.image.rootfs.diff.tar.gzip',
 'size': 210204160},
{'digest':
'sha256:bbd20206b01e4bf5d521890bf5ffe2d5f565bbf70eda987b059b267c80f4f9a1',
 'mediaType':
'application/vnd.docker.image.rootfs.diff.tar.gzip',
 'size': 39936}],
 'mediaType': 'application/vnd.docker.distribution.manifest.v2+json',
 'schemaVersion': 2}

ср, 24 окт. 2018 г. в 16:05, Konstantin M. Khankin <
khankin.konstan...@gmail.com>:

> Thank you, https://pulp.plan.io/issues/4095 filed (why didn't I wait a
> bit to get 4096...)
>
> ср, 24 окт. 2018 г. в 15:35, Austin Macdonald :
>
>> The other projects are sometimes hidden.
>> https://pul

Re: [Pulp-list] "Plugin "pulp-profile-update" can't be imported" at 2.20.0

[Konstantin M. Khankin]

Hi Dennis, the patch works, thank you!

пн, 15 июл. 2019 г. в 17:44, Dennis Kliban :

> Please try out the following patch;
> https://patch-diff.githubusercontent.com/raw/pulp/pulp/pull/3943.patch
> and let the list know if that fixes the problem for you.
>
> On Mon, Jul 15, 2019 at 10:42 AM Dennis Kliban  wrote:
>
>> Konstantin, I already filed an issue[0] myself so that we can attach it
>> to the patch that fixes it. Once the patch is merged, it will be associated
>> with the issue in pulp.plan.io.
>>
>>
>> [0] https://pulp.plan.io/issues/5126
>>
>> On Mon, Jul 15, 2019 at 10:05 AM Dennis Kliban 
>> wrote:
>>
>>> Please file an issue at https://pulp.plan.io/issues/new/. This is
>>> definitely a regression.
>>>
>>> On Mon, Jul 15, 2019 at 4:47 AM Konstantin M. Khankin <
>>> khankin.konstan...@gmail.com> wrote:
>>>
>>>> >>> d, m = os.path.split('/usr/lib/yum-plugins/pulp-profile-update.py')
>>>> >>> m = m.split('.py')[0]
>>>> >>> m
>>>> 'pulp-profile-update'
>>>> >>> d
>>>> '/usr/lib/yum-plugins'
>>>> >>> imp.find_module(m, [d])
>>>> (>>> 0x7f996857d540>, '/usr/lib/yum-plugins/pulp-profile-update.py', ('.py',
>>>> 'U', 1))
>>>> >>> f, p, d = imp.find_module(m, [d])
>>>> >>> imp.load_module(m, f, p, d)
>>>> Traceback (most recent call last):
>>>>   File "", line 1, in 
>>>>   File "/usr/lib/yum-plugins/pulp-profile-update.py", line 5, in
>>>> 
>>>> from pulp.bindings.server import PulpConnection
>>>>   File "/usr/lib/python2.7/site-packages/pulp/bindings/server.py", line
>>>> 18, in 
>>>> from pulp.common.constants import DEFAULT_CA_PATH
>>>>   File "/usr/lib/python2.7/site-packages/pulp/common/constants.py",
>>>> line 5, in 
>>>> PULP_VERSION = pkg_resources.require('pulp-server')[0].version
>>>>   File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 728,
>>>> in require
>>>> needed = self.resolve(parse_requirements(requirements))
>>>>   File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 626,
>>>> in resolve
>>>> raise DistributionNotFound(req)
>>>> pkg_resources.DistributionNotFound: pulp-server
>>>>
>>>> "PULP_VERSION = pkg_resources.require('pulp-server')[0].version" was
>>>> introduced in
>>>> https://github.com/pulp/pulp/commit/19a527036d4e1b3ed06037ada70a5de5cd556940
>>>>  and
>>>> was released with 2.20.0. I checked 2.19.1 RPM does not include this line.
>>>> Obviously client side will not have pulp-server package and will thus fail.
>>>>
>>>> пн, 15 июл. 2019 г. в 11:14, Konstantin M. Khankin <
>>>> khankin.konstan...@gmail.com>:
>>>>
>>>>> I tried to compare the content
>>>>> of pulp-rpm-yumplugins-2.19.1-1.el7.noarch.rpm
>>>>> and pulp-rpm-yumplugins-2.20.0-1.el7.noarch.rpm but the only difference
>>>>> found was:
>>>>>
>>>>> $ diff -Naur plug_19/pulp-profile-update.py
>>>>> plug_20/pulp-profile-update.py
>>>>> --- plug_19/pulp-profile-update.py 2019-05-30 15:16:40.0 +0300
>>>>> +++ plug_20/pulp-profile-update.py 2019-07-10 23:41:04.0 +0300
>>>>> @@ -41,7 +41,15 @@
>>>>>  cfg['filesystem']['id_cert_dir'],
>>>>>  cfg['filesystem']['id_cert_filename'])
>>>>>  ssl = cfg.parse_bool(cfg['server']['verify_ssl'])
>>>>> -connection = PulpConnection(host, port, cert_filename=cert,
>>>>> verify_ssl=ssl)
>>>>> +proxy_host = cfg['server']['proxy_host']
>>>>> +proxy_port = int(cfg['server']['proxy_port'])
>>>>> +connection = PulpConnection(
>>>>> +host,
>>>>> +port,
>>>>> +cert_filename=cert,
>>>>> +verify_ssl=ssl,
>>>>> +proxy_host=proxy_host,
>>>>> +proxy_port=proxy_port)
>>>>>  Bindings.__init__(self, connection)
>>>>>
>>>>> Tried to uncomment proxy_port and proxy_host in the config but it
>>>>> didn't help. I also tried to remov

Re: [Pulp-list] "Plugin "pulp-profile-update" can't be imported" at 2.20.0

[Konstantin M. Khankin]

>>> d, m = os.path.split('/usr/lib/yum-plugins/pulp-profile-update.py')
>>> m = m.split('.py')[0]
>>> m
'pulp-profile-update'
>>> d
'/usr/lib/yum-plugins'
>>> imp.find_module(m, [d])
(, '/usr/lib/yum-plugins/pulp-profile-update.py', ('.py',
'U', 1))
>>> f, p, d = imp.find_module(m, [d])
>>> imp.load_module(m, f, p, d)
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/yum-plugins/pulp-profile-update.py", line 5, in 
from pulp.bindings.server import PulpConnection
  File "/usr/lib/python2.7/site-packages/pulp/bindings/server.py", line 18,
in 
from pulp.common.constants import DEFAULT_CA_PATH
  File "/usr/lib/python2.7/site-packages/pulp/common/constants.py", line 5,
in 
PULP_VERSION = pkg_resources.require('pulp-server')[0].version
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 728, in
require
needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 626, in
resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: pulp-server

"PULP_VERSION = pkg_resources.require('pulp-server')[0].version" was
introduced in
https://github.com/pulp/pulp/commit/19a527036d4e1b3ed06037ada70a5de5cd556940
and
was released with 2.20.0. I checked 2.19.1 RPM does not include this line.
Obviously client side will not have pulp-server package and will thus fail.

пн, 15 июл. 2019 г. в 11:14, Konstantin M. Khankin <
khankin.konstan...@gmail.com>:

> I tried to compare the content
> of pulp-rpm-yumplugins-2.19.1-1.el7.noarch.rpm
> and pulp-rpm-yumplugins-2.20.0-1.el7.noarch.rpm but the only difference
> found was:
>
> $ diff -Naur plug_19/pulp-profile-update.py plug_20/pulp-profile-update.py
> --- plug_19/pulp-profile-update.py 2019-05-30 15:16:40.0 +0300
> +++ plug_20/pulp-profile-update.py 2019-07-10 23:41:04.0 +0300
> @@ -41,7 +41,15 @@
>  cfg['filesystem']['id_cert_dir'],
>  cfg['filesystem']['id_cert_filename'])
>  ssl = cfg.parse_bool(cfg['server']['verify_ssl'])
> -connection = PulpConnection(host, port, cert_filename=cert,
> verify_ssl=ssl)
> +proxy_host = cfg['server']['proxy_host']
> +proxy_port = int(cfg['server']['proxy_port'])
> +connection = PulpConnection(
> +host,
> +port,
> +cert_filename=cert,
> +verify_ssl=ssl,
> +proxy_host=proxy_host,
> +proxy_port=proxy_port)
>  Bindings.__init__(self, connection)
>
> Tried to uncomment proxy_port and proxy_host in the config but it didn't
> help. I also tried to remove .pyc and .pyo, but no effect.
>
> вс, 14 июл. 2019 г. в 23:54, Konstantin M. Khankin <
> khankin.konstan...@gmail.com>:
>
>> Hi!
>>
>> I've upgraded my clients to 2.20.0 and now each of them reports "Plugin
>> "pulp-profile-update" can't be imported" on any yum operation.
>>
>> I took one client which was not updated and didn't have this issue and
>> ran "yum update":
>>
>> # yum upgrade
>>
>> *Loaded plugins: fastestmirror, pulp-profile-update, versionlock*Loading
>> mirror speeds from cached hostfile
>> Resolving Dependencies
>> --> Running transaction check
>> ---> Package pulp-agent.noarch 0:2.19.1-1.el7 will be updated
>> ---> Package pulp-agent.noarch 0:2.20.0-1.el7 will be an update
>> ---> Package pulp-consumer-client.noarch 0:2.19.1-1.el7 will be updated
>> ---> Package pulp-consumer-client.noarch 0:2.20.0-1.el7 will be an update
>> ---> Package pulp-puppet-consumer-extensions.noarch 0:2.19.1-1.el7 will
>> be updated
>> ---> Package pulp-puppet-consumer-extensions.noarch 0:2.20.0-1.el7 will
>> be an update
>> ---> Package pulp-puppet-handlers.noarch 0:2.19.1-1.el7 will be updated
>> ---> Package pulp-puppet-handlers.noarch 0:2.20.0-1.el7 will be an update
>> ---> Package pulp-rpm-consumer-extensions.noarch 0:2.19.1-1.el7 will be
>> updated
>> ---> Package pulp-rpm-consumer-extensions.noarch 0:2.20.0-1.el7 will be
>> an update
>> ---> Package pulp-rpm-handlers.noarch 0:2.19.1-1.el7 will be updated
>> ---> Package pulp-rpm-handlers.noarch 0:2.20.0-1.el7 will be an update
>> ---> Package pulp-rpm-yumplugins.noarch 0:2.19.1-1.el7 will be updated
>> ---> Package pulp-rpm-yumplugins.noarch 0:2.20.0-1.el7 will be an update
>> ---> Package python-pulp-agent-lib.noarch 0:2.19.1-1.el7 will be updated
>> ---> Package python-pulp-agent-lib.noarch 0:2.20.0-1.el7 will be an update
>> ---> P

Re: [Pulp-list] "Plugin "pulp-profile-update" can't be imported" at 2.20.0

[Konstantin M. Khankin]

I tried to compare the content
of pulp-rpm-yumplugins-2.19.1-1.el7.noarch.rpm
and pulp-rpm-yumplugins-2.20.0-1.el7.noarch.rpm but the only difference
found was:

$ diff -Naur plug_19/pulp-profile-update.py plug_20/pulp-profile-update.py
--- plug_19/pulp-profile-update.py 2019-05-30 15:16:40.0 +0300
+++ plug_20/pulp-profile-update.py 2019-07-10 23:41:04.0 +0300
@@ -41,7 +41,15 @@
 cfg['filesystem']['id_cert_dir'],
 cfg['filesystem']['id_cert_filename'])
 ssl = cfg.parse_bool(cfg['server']['verify_ssl'])
-connection = PulpConnection(host, port, cert_filename=cert,
verify_ssl=ssl)
+proxy_host = cfg['server']['proxy_host']
+proxy_port = int(cfg['server']['proxy_port'])
+connection = PulpConnection(
+host,
+port,
+cert_filename=cert,
+verify_ssl=ssl,
+proxy_host=proxy_host,
+proxy_port=proxy_port)
 Bindings.__init__(self, connection)

Tried to uncomment proxy_port and proxy_host in the config but it didn't
help. I also tried to remove .pyc and .pyo, but no effect.

вс, 14 июл. 2019 г. в 23:54, Konstantin M. Khankin <
khankin.konstan...@gmail.com>:

> Hi!
>
> I've upgraded my clients to 2.20.0 and now each of them reports "Plugin
> "pulp-profile-update" can't be imported" on any yum operation.
>
> I took one client which was not updated and didn't have this issue and ran
> "yum update":
>
> # yum upgrade
>
> *Loaded plugins: fastestmirror, pulp-profile-update, versionlock*Loading
> mirror speeds from cached hostfile
> Resolving Dependencies
> --> Running transaction check
> ---> Package pulp-agent.noarch 0:2.19.1-1.el7 will be updated
> ---> Package pulp-agent.noarch 0:2.20.0-1.el7 will be an update
> ---> Package pulp-consumer-client.noarch 0:2.19.1-1.el7 will be updated
> ---> Package pulp-consumer-client.noarch 0:2.20.0-1.el7 will be an update
> ---> Package pulp-puppet-consumer-extensions.noarch 0:2.19.1-1.el7 will be
> updated
> ---> Package pulp-puppet-consumer-extensions.noarch 0:2.20.0-1.el7 will be
> an update
> ---> Package pulp-puppet-handlers.noarch 0:2.19.1-1.el7 will be updated
> ---> Package pulp-puppet-handlers.noarch 0:2.20.0-1.el7 will be an update
> ---> Package pulp-rpm-consumer-extensions.noarch 0:2.19.1-1.el7 will be
> updated
> ---> Package pulp-rpm-consumer-extensions.noarch 0:2.20.0-1.el7 will be an
> update
> ---> Package pulp-rpm-handlers.noarch 0:2.19.1-1.el7 will be updated
> ---> Package pulp-rpm-handlers.noarch 0:2.20.0-1.el7 will be an update
> ---> Package pulp-rpm-yumplugins.noarch 0:2.19.1-1.el7 will be updated
> ---> Package pulp-rpm-yumplugins.noarch 0:2.20.0-1.el7 will be an update
> ---> Package python-pulp-agent-lib.noarch 0:2.19.1-1.el7 will be updated
> ---> Package python-pulp-agent-lib.noarch 0:2.20.0-1.el7 will be an update
> ---> Package python-pulp-bindings.noarch 0:2.19.1-1.el7 will be updated
> ---> Package python-pulp-bindings.noarch 0:2.20.0-1.el7 will be an update
> ---> Package python-pulp-client-lib.noarch 0:2.19.1-1.el7 will be updated
> ---> Package python-pulp-client-lib.noarch 0:2.20.0-1.el7 will be an update
> ---> Package python-pulp-common.noarch 0:2.19.1-1.el7 will be updated
> ---> Package python-pulp-common.noarch 0:2.20.0-1.el7 will be an update
> ---> Package python-pulp-puppet-common.noarch 0:2.19.1-1.el7 will be
> updated
> ---> Package python-pulp-puppet-common.noarch 0:2.20.0-1.el7 will be an
> update
> ---> Package python-pulp-rpm-common.noarch 0:2.19.1-1.el7 will be updated
> ---> Package python-pulp-rpm-common.noarch 0:2.20.0-1.el7 will be an update
> --> Finished Dependency Resolution
> ...
>
> Now this client has this issue:
>
> # yum upgrade
>
> *Plugin "pulp-profile-update" can't be imported*Loaded plugins:
> fastestmirror, versionlock
> Loading mirror speeds from cached hostfile
> ^C
>
> Downgrade to 2.19.1 helps:
>
> # yum downgrade pulp-rpm-yumplugins python-pulp-bindings
> python-pulp-common python-pulp-puppet-common  pulp-consumer-client
> python-pulp-client-lib  pulp-agent  python-pulp-agent-lib
>  python-pulp-rpm-common  pulp-puppet-handlers
>  pulp-puppet-consumer-extensions  pulp-rpm-handlers
>  pulp-rpm-consumer-extensions
>
> *Plugin "pulp-profile-update" can't be imported*Loaded plugins:
> fastestmirror, versionlock
> Loading mirror speeds from cached hostfile
> Resolving Dependencies
> --> Running transaction check
> ---> Package pulp-agent.noarch 0:2.19.1-1.el7 will be a downgrade
> ---> Package pulp-agent.noarch 0:2.20.0-1.el7 will be erased
> ---> Package pulp-consumer-client.noarch 0:2.1

[Pulp-list] "Plugin "pulp-profile-update" can't be imported" at 2.20.0

[Konstantin M. Khankin]

Hi!

I've upgraded my clients to 2.20.0 and now each of them reports "Plugin
"pulp-profile-update" can't be imported" on any yum operation.

I took one client which was not updated and didn't have this issue and ran
"yum update":

# yum upgrade

*Loaded plugins: fastestmirror, pulp-profile-update, versionlock*Loading
mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package pulp-agent.noarch 0:2.19.1-1.el7 will be updated
---> Package pulp-agent.noarch 0:2.20.0-1.el7 will be an update
---> Package pulp-consumer-client.noarch 0:2.19.1-1.el7 will be updated
---> Package pulp-consumer-client.noarch 0:2.20.0-1.el7 will be an update
---> Package pulp-puppet-consumer-extensions.noarch 0:2.19.1-1.el7 will be
updated
---> Package pulp-puppet-consumer-extensions.noarch 0:2.20.0-1.el7 will be
an update
---> Package pulp-puppet-handlers.noarch 0:2.19.1-1.el7 will be updated
---> Package pulp-puppet-handlers.noarch 0:2.20.0-1.el7 will be an update
---> Package pulp-rpm-consumer-extensions.noarch 0:2.19.1-1.el7 will be
updated
---> Package pulp-rpm-consumer-extensions.noarch 0:2.20.0-1.el7 will be an
update
---> Package pulp-rpm-handlers.noarch 0:2.19.1-1.el7 will be updated
---> Package pulp-rpm-handlers.noarch 0:2.20.0-1.el7 will be an update
---> Package pulp-rpm-yumplugins.noarch 0:2.19.1-1.el7 will be updated
---> Package pulp-rpm-yumplugins.noarch 0:2.20.0-1.el7 will be an update
---> Package python-pulp-agent-lib.noarch 0:2.19.1-1.el7 will be updated
---> Package python-pulp-agent-lib.noarch 0:2.20.0-1.el7 will be an update
---> Package python-pulp-bindings.noarch 0:2.19.1-1.el7 will be updated
---> Package python-pulp-bindings.noarch 0:2.20.0-1.el7 will be an update
---> Package python-pulp-client-lib.noarch 0:2.19.1-1.el7 will be updated
---> Package python-pulp-client-lib.noarch 0:2.20.0-1.el7 will be an update
---> Package python-pulp-common.noarch 0:2.19.1-1.el7 will be updated
---> Package python-pulp-common.noarch 0:2.20.0-1.el7 will be an update
---> Package python-pulp-puppet-common.noarch 0:2.19.1-1.el7 will be updated
---> Package python-pulp-puppet-common.noarch 0:2.20.0-1.el7 will be an
update
---> Package python-pulp-rpm-common.noarch 0:2.19.1-1.el7 will be updated
---> Package python-pulp-rpm-common.noarch 0:2.20.0-1.el7 will be an update
--> Finished Dependency Resolution
...

Now this client has this issue:

# yum upgrade

*Plugin "pulp-profile-update" can't be imported*Loaded plugins:
fastestmirror, versionlock
Loading mirror speeds from cached hostfile
^C

Downgrade to 2.19.1 helps:

# yum downgrade pulp-rpm-yumplugins python-pulp-bindings python-pulp-common
python-pulp-puppet-common  pulp-consumer-client python-pulp-client-lib
 pulp-agent  python-pulp-agent-lib  python-pulp-rpm-common
 pulp-puppet-handlers  pulp-puppet-consumer-extensions  pulp-rpm-handlers
 pulp-rpm-consumer-extensions

*Plugin "pulp-profile-update" can't be imported*Loaded plugins:
fastestmirror, versionlock
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package pulp-agent.noarch 0:2.19.1-1.el7 will be a downgrade
---> Package pulp-agent.noarch 0:2.20.0-1.el7 will be erased
---> Package pulp-consumer-client.noarch 0:2.19.1-1.el7 will be a downgrade
---> Package pulp-consumer-client.noarch 0:2.20.0-1.el7 will be erased
---> Package pulp-puppet-consumer-extensions.noarch 0:2.19.1-1.el7 will be
a downgrade
---> Package pulp-puppet-consumer-extensions.noarch 0:2.20.0-1.el7 will be
erased
---> Package pulp-puppet-handlers.noarch 0:2.19.1-1.el7 will be a downgrade
---> Package pulp-puppet-handlers.noarch 0:2.20.0-1.el7 will be erased
---> Package pulp-rpm-consumer-extensions.noarch 0:2.19.1-1.el7 will be a
downgrade
---> Package pulp-rpm-consumer-extensions.noarch 0:2.20.0-1.el7 will be
erased
---> Package pulp-rpm-handlers.noarch 0:2.19.1-1.el7 will be a downgrade
---> Package pulp-rpm-handlers.noarch 0:2.20.0-1.el7 will be erased
---> Package pulp-rpm-yumplugins.noarch 0:2.19.1-1.el7 will be a downgrade
---> Package pulp-rpm-yumplugins.noarch 0:2.20.0-1.el7 will be erased
---> Package python-pulp-agent-lib.noarch 0:2.19.1-1.el7 will be a downgrade
---> Package python-pulp-agent-lib.noarch 0:2.20.0-1.el7 will be erased
---> Package python-pulp-bindings.noarch 0:2.19.1-1.el7 will be a downgrade
---> Package python-pulp-bindings.noarch 0:2.20.0-1.el7 will be erased
---> Package python-pulp-client-lib.noarch 0:2.19.1-1.el7 will be a
downgrade
---> Package python-pulp-client-lib.noarch 0:2.20.0-1.el7 will be erased
---> Package python-pulp-common.noarch 0:2.19.1-1.el7 will be a downgrade
---> Package python-pulp-common.noarch 0:2.20.0-1.el7 will be erased
---> Package python-pulp-puppet-common.noarch 0:2.19.1-1.el7 will be a
downgrade
---> Package python-pulp-puppet-common.noarch 0:2.20.0-1.el7 will be erased
---> Package python-pulp-rpm-common.noarch 0:2.19.1-1.el7 will be a
downgrade
---> Package python-pulp-rpm-common.noarch 

Re: [Pulp-list] Can't import containers created with Buildah

[Konstantin M. Khankin]

Thank you, https://pulp.plan.io/issues/4095 filed (why didn't I wait a bit
to get 4096...)

ср, 24 окт. 2018 г. в 15:35, Austin Macdonald :

> The other projects are sometimes hidden.
> https://pulp.plan.io/projects/pulp_docker
>
> On Wed, Oct 24, 2018 at 2:55 AM Konstantin M. Khankin <
> khankin.konstan...@gmail.com> wrote:
>
>> Dennis,
>>
>> Which project I should use? I don't see a separate project for
>> pulp_docker.
>>
>> Thanks!
>>
>> вт, 23 окт. 2018 г. в 23:58, Dennis Kliban :
>>
>>> Konstantin,
>>>
>>> Could you please file an issue in pulp.plan.io?
>>>
>>> Thanks,
>>> Dennis
>>>
>>> On Sat, Oct 20, 2018 at 5:40 AM Konstantin M. Khankin <
>>> khankin.konstan...@gmail.com> wrote:
>>>
>>>> *Forgot to mention versions:*
>>>>
>>>> pulp-admin-client-2.17.1-1.el7.noarch
>>>> pulp-agent-2.17.1-1.el7.noarch
>>>> pulp-consumer-client-2.17.1-1.el7.noarch
>>>> pulp-deb-admin-extensions-1.8.0-1.el7.noarch
>>>> pulp-deb-plugins-1.8.0-1.el7.noarch
>>>> pulp-docker-admin-extensions-3.2.1-1.el7.noarch
>>>> pulp-docker-plugins-3.2.1-1.el7.noarch
>>>> pulp-puppet-consumer-extensions-2.17.1-1.el7.noarch
>>>> pulp-puppet-handlers-2.17.1-1.el7.noarch
>>>> pulp-python-admin-extensions-2.0.3-1.el7.noarch
>>>> pulp-python-plugins-2.0.3-1.el7.noarch
>>>> pulp-rpm-admin-extensions-2.17.1-1.el7.noarch
>>>> pulp-rpm-consumer-extensions-2.17.1-1.el7.noarch
>>>> pulp-rpm-handlers-2.17.1-1.el7.noarch
>>>> pulp-rpm-plugins-2.17.1-1.el7.noarch
>>>> pulp-rpm-yumplugins-2.17.1-1.el7.noarch
>>>> pulp-selinux-2.17.1-1.el7.noarch
>>>> pulp-server-2.17.1-1.el7.noarch
>>>> python-pulp-agent-lib-2.17.1-1.el7.noarch
>>>> python-pulp-bindings-2.17.1-1.el7.noarch
>>>> python-pulp-client-lib-2.17.1-1.el7.noarch
>>>> python-pulp-common-2.17.1-1.el7.noarch
>>>> python-pulp-deb-common-1.8.0-1.el7.noarch
>>>> python-pulp-docker-common-3.2.1-1.el7.noarch
>>>> python-pulp-oid_validation-2.17.1-1.el7.noarch
>>>> python-pulp-puppet-common-2.17.1-1.el7.noarch
>>>> python-pulp-python-common-2.0.3-1.el7.noarch
>>>> python-pulp-repoauth-2.17.1-1.el7.noarch
>>>> python-pulp-rpm-common-2.17.1-1.el7.noarch
>>>>
>>>> buildah-1.2-2.gitbe87762.el7.x86_64
>>>>
>>>> *Manifest v2s1 from one of the images created by buildah:*
>>>> {'architecture': 'amd64',
>>>>  'fsLayers': [{'blobSum':
>>>> 'sha256:4d85b6fa2344380743db8ccf4f0404603f56c8289db0824c13bfc309b2009be8'},
>>>>   {'blobSum':
>>>> 'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4'},
>>>>   {'blobSum':
>>>> 'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4'},
>>>>   {'blobSum':
>>>> 'sha256:f972d139738dfcd1519fd2461815651336ee25a8b54c358834c50af094bb262f'}],
>>>>  'history': [{'v1Compatibility':
>>>> '{"architecture":"amd64","config":{"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Entrypoint":["/bin/bash"],"WorkingDir":"/","Labels":{"org.label-schema.build-date":"20181006","org.label-schema.license":"GPLv2","
>>>> org.label-schema.name":"CentOS Base
>>>> Image","org.label-schema.schema-version":"1.0","org.label-schema.vendor":"CentOS"}},"created":"2018-10-20T07:49:56.084883454Z","id":"4591c91f4c223fa61c800a84bcf711647786f3fe71e589606f17b4d11b9c5807","os":"linux","parent":"f871460fb057cfa08e7268df9b884eecedeed72bc6dbc3262deb19a25e66a904"}'},
>>>>  {'v1Compatibility':
>>>> '{"id":"f871460fb057cfa08e7268df9b884eecedeed72bc6dbc3262deb19a25e66a904","parent":"8a4f409d848e6f693d5a4e6c260fbd8ec6a4dc99fc9915e3210bd5ed9c22e700","created":"2018-10-09T18:19:48.447478476Z","container_config":{"Cmd":["/bin/sh
>>>> -c #(nop)  CMD [\\"/bin/bash\\"]"]},"throwaway":true}'},
>>>>  {'v1Compatibility':
>>>> '{"id":"8a4f409d848e6f693d5a4e6c260fbd8ec6a4dc99fc9915e3210bd5ed9c22e700","par

Re: [Pulp-list] Can't import containers created with Buildah

[Konstantin M. Khankin]

Dennis,

Which project I should use? I don't see a separate project for pulp_docker.

Thanks!

вт, 23 окт. 2018 г. в 23:58, Dennis Kliban :

> Konstantin,
>
> Could you please file an issue in pulp.plan.io?
>
> Thanks,
> Dennis
>
> On Sat, Oct 20, 2018 at 5:40 AM Konstantin M. Khankin <
> khankin.konstan...@gmail.com> wrote:
>
>> *Forgot to mention versions:*
>>
>> pulp-admin-client-2.17.1-1.el7.noarch
>> pulp-agent-2.17.1-1.el7.noarch
>> pulp-consumer-client-2.17.1-1.el7.noarch
>> pulp-deb-admin-extensions-1.8.0-1.el7.noarch
>> pulp-deb-plugins-1.8.0-1.el7.noarch
>> pulp-docker-admin-extensions-3.2.1-1.el7.noarch
>> pulp-docker-plugins-3.2.1-1.el7.noarch
>> pulp-puppet-consumer-extensions-2.17.1-1.el7.noarch
>> pulp-puppet-handlers-2.17.1-1.el7.noarch
>> pulp-python-admin-extensions-2.0.3-1.el7.noarch
>> pulp-python-plugins-2.0.3-1.el7.noarch
>> pulp-rpm-admin-extensions-2.17.1-1.el7.noarch
>> pulp-rpm-consumer-extensions-2.17.1-1.el7.noarch
>> pulp-rpm-handlers-2.17.1-1.el7.noarch
>> pulp-rpm-plugins-2.17.1-1.el7.noarch
>> pulp-rpm-yumplugins-2.17.1-1.el7.noarch
>> pulp-selinux-2.17.1-1.el7.noarch
>> pulp-server-2.17.1-1.el7.noarch
>> python-pulp-agent-lib-2.17.1-1.el7.noarch
>> python-pulp-bindings-2.17.1-1.el7.noarch
>> python-pulp-client-lib-2.17.1-1.el7.noarch
>> python-pulp-common-2.17.1-1.el7.noarch
>> python-pulp-deb-common-1.8.0-1.el7.noarch
>> python-pulp-docker-common-3.2.1-1.el7.noarch
>> python-pulp-oid_validation-2.17.1-1.el7.noarch
>> python-pulp-puppet-common-2.17.1-1.el7.noarch
>> python-pulp-python-common-2.0.3-1.el7.noarch
>> python-pulp-repoauth-2.17.1-1.el7.noarch
>> python-pulp-rpm-common-2.17.1-1.el7.noarch
>>
>> buildah-1.2-2.gitbe87762.el7.x86_64
>>
>> *Manifest v2s1 from one of the images created by buildah:*
>> {'architecture': 'amd64',
>>  'fsLayers': [{'blobSum':
>> 'sha256:4d85b6fa2344380743db8ccf4f0404603f56c8289db0824c13bfc309b2009be8'},
>>   {'blobSum':
>> 'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4'},
>>   {'blobSum':
>> 'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4'},
>>   {'blobSum':
>> 'sha256:f972d139738dfcd1519fd2461815651336ee25a8b54c358834c50af094bb262f'}],
>>  'history': [{'v1Compatibility':
>> '{"architecture":"amd64","config":{"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Entrypoint":["/bin/bash"],"WorkingDir":"/","Labels":{"org.label-schema.build-date":"20181006","org.label-schema.license":"GPLv2","
>> org.label-schema.name":"CentOS Base
>> Image","org.label-schema.schema-version":"1.0","org.label-schema.vendor":"CentOS"}},"created":"2018-10-20T07:49:56.084883454Z","id":"4591c91f4c223fa61c800a84bcf711647786f3fe71e589606f17b4d11b9c5807","os":"linux","parent":"f871460fb057cfa08e7268df9b884eecedeed72bc6dbc3262deb19a25e66a904"}'},
>>  {'v1Compatibility':
>> '{"id":"f871460fb057cfa08e7268df9b884eecedeed72bc6dbc3262deb19a25e66a904","parent":"8a4f409d848e6f693d5a4e6c260fbd8ec6a4dc99fc9915e3210bd5ed9c22e700","created":"2018-10-09T18:19:48.447478476Z","container_config":{"Cmd":["/bin/sh
>> -c #(nop)  CMD [\\"/bin/bash\\"]"]},"throwaway":true}'},
>>  {'v1Compatibility':
>> '{"id":"8a4f409d848e6f693d5a4e6c260fbd8ec6a4dc99fc9915e3210bd5ed9c22e700","parent":"961164371d3af72a151063f656820bc835c695fa44af66c2845d0850f01ce0d9","created":"2018-10-09T18:19:48.258784779Z","container_config":{"Cmd":["/bin/sh
>> -c #(nop)  LABEL org.label-schema.schema-version=1.0
>> org.label-schema.name=CentOS Base Image org.label-schema.vendor=CentOS
>> org.label-schema.license=GPLv2
>> org.label-schema.build-date=20181006"]},"throwaway":true}'},
>>  {'v1Compatibility':
>> '{"id":"961164371d3af72a151063f656820bc835c695fa44af66c2845d0850f01ce0d9","created":"2018-10-09T18:19:47.884079751Z","container_config":{"Cmd":["/bin/sh
>> -c #(nop) ADD
>> file:fbe9badfd2790f0747a25fbe5c94a6daa78969511ca08c8d4ac654f3442570de in /
>> "]}}'}],
>>  'name': '',
>>  'schemaVersion': 1,
>>

Re: [Pulp-list] Can't import containers created with Buildah

[Konstantin M. Khankin]

*Forgot to mention versions:*

pulp-admin-client-2.17.1-1.el7.noarch
pulp-agent-2.17.1-1.el7.noarch
pulp-consumer-client-2.17.1-1.el7.noarch
pulp-deb-admin-extensions-1.8.0-1.el7.noarch
pulp-deb-plugins-1.8.0-1.el7.noarch
pulp-docker-admin-extensions-3.2.1-1.el7.noarch
pulp-docker-plugins-3.2.1-1.el7.noarch
pulp-puppet-consumer-extensions-2.17.1-1.el7.noarch
pulp-puppet-handlers-2.17.1-1.el7.noarch
pulp-python-admin-extensions-2.0.3-1.el7.noarch
pulp-python-plugins-2.0.3-1.el7.noarch
pulp-rpm-admin-extensions-2.17.1-1.el7.noarch
pulp-rpm-consumer-extensions-2.17.1-1.el7.noarch
pulp-rpm-handlers-2.17.1-1.el7.noarch
pulp-rpm-plugins-2.17.1-1.el7.noarch
pulp-rpm-yumplugins-2.17.1-1.el7.noarch
pulp-selinux-2.17.1-1.el7.noarch
pulp-server-2.17.1-1.el7.noarch
python-pulp-agent-lib-2.17.1-1.el7.noarch
python-pulp-bindings-2.17.1-1.el7.noarch
python-pulp-client-lib-2.17.1-1.el7.noarch
python-pulp-common-2.17.1-1.el7.noarch
python-pulp-deb-common-1.8.0-1.el7.noarch
python-pulp-docker-common-3.2.1-1.el7.noarch
python-pulp-oid_validation-2.17.1-1.el7.noarch
python-pulp-puppet-common-2.17.1-1.el7.noarch
python-pulp-python-common-2.0.3-1.el7.noarch
python-pulp-repoauth-2.17.1-1.el7.noarch
python-pulp-rpm-common-2.17.1-1.el7.noarch

buildah-1.2-2.gitbe87762.el7.x86_64

*Manifest v2s1 from one of the images created by buildah:*
{'architecture': 'amd64',
 'fsLayers': [{'blobSum':
'sha256:4d85b6fa2344380743db8ccf4f0404603f56c8289db0824c13bfc309b2009be8'},
  {'blobSum':
'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4'},
  {'blobSum':
'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4'},
  {'blobSum':
'sha256:f972d139738dfcd1519fd2461815651336ee25a8b54c358834c50af094bb262f'}],
 'history': [{'v1Compatibility':
'{"architecture":"amd64","config":{"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Entrypoint":["/bin/bash"],"WorkingDir":"/","Labels":{"org.label-schema.build-date":"20181006","org.label-schema.license":"GPLv2","
org.label-schema.name":"CentOS Base
Image","org.label-schema.schema-version":"1.0","org.label-schema.vendor":"CentOS"}},"created":"2018-10-20T07:49:56.084883454Z","id":"4591c91f4c223fa61c800a84bcf711647786f3fe71e589606f17b4d11b9c5807","os":"linux","parent":"f871460fb057cfa08e7268df9b884eecedeed72bc6dbc3262deb19a25e66a904"}'},
 {'v1Compatibility':
'{"id":"f871460fb057cfa08e7268df9b884eecedeed72bc6dbc3262deb19a25e66a904","parent":"8a4f409d848e6f693d5a4e6c260fbd8ec6a4dc99fc9915e3210bd5ed9c22e700","created":"2018-10-09T18:19:48.447478476Z","container_config":{"Cmd":["/bin/sh
-c #(nop)  CMD [\\"/bin/bash\\"]"]},"throwaway":true}'},
 {'v1Compatibility':
'{"id":"8a4f409d848e6f693d5a4e6c260fbd8ec6a4dc99fc9915e3210bd5ed9c22e700","parent":"961164371d3af72a151063f656820bc835c695fa44af66c2845d0850f01ce0d9","created":"2018-10-09T18:19:48.258784779Z","container_config":{"Cmd":["/bin/sh
-c #(nop)  LABEL org.label-schema.schema-version=1.0
org.label-schema.name=CentOS
Base Image org.label-schema.vendor=CentOS org.label-schema.license=GPLv2
org.label-schema.build-date=20181006"]},"throwaway":true}'},
 {'v1Compatibility':
'{"id":"961164371d3af72a151063f656820bc835c695fa44af66c2845d0850f01ce0d9","created":"2018-10-09T18:19:47.884079751Z","container_config":{"Cmd":["/bin/sh
-c #(nop) ADD
file:fbe9badfd2790f0747a25fbe5c94a6daa78969511ca08c8d4ac654f3442570de in /
"]}}'}],
 'name': '',
 'schemaVersion': 1,
 'signatures': [{'header': {'alg': 'ES256',
'jwk': {'crv': 'P-256',
'kid': '...',
'kty': 'EC',
'x': '...',
'y': '...'}},
 'protected': '...',
 'signature': '...'}],
 'tag': ''}

*Manifest v2s2 for the same image:*
{'config': {'digest':
'sha256:c17d63d23edcbe46b14fbe531e2e9b4411483947669087e90bdba3372eb2684d',
'mediaType': 'application/vnd.docker.container.image.v1+json',
    'size': 1217},
 'layers': [{'digest':
'sha256:f972d139738dfcd1519fd2461815651336ee25a8b54c358834c50af094bb262f',
 'mediaType':
'application/vnd.docker.image.rootfs.diff.tar.gzip',
 'size': 208811520},
{'digest':
'sha256:4d85b6fa2344380743db8ccf4f0404603f56c8289db0824c13bfc309b2009be8',
 'mediaTyp

[Pulp-list] Can't import containers created with Buildah

[Konstantin M. Khankin]

Hi!

I tried the tool Buildah (
https://www.projectatomic.io/blog/2017/06/introducing-buildah/) to build
containers without having docker binary. Containers are built (not sure if
they work), but I can't upload them to the pulp repo:

1) I copy container from "container-storage:" to "dir:", then create tar
archive per
https://docs.pulpproject.org/plugins/pulp_docker/user-guide/recipes.html#upload-v2-schema-2-and-schema-1-images-to-pulp
2) pulp-admin -vvv docker repo uploads upload --repo-id docker_centos7 -f
centos7.tar

[2] results in:
  "traceback": "Traceback (most recent call last):\n  File
\"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 367, in
trace_task\nR = retval = fun(*args, **kwargs)\n  File
\"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 529,
in __call__\nreturn super(Task, self).__call__(*args, **kwargs)\n  File
\"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 107,
in __call__\nreturn super(PulpTask, self).__call__(*args, **kwargs)\n
File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 622, in
__protected_call__\nreturn self.run(*args, **kwargs)\n  File
\"/usr/lib/python2.7/site-packages/pulp/server/managers/content/upload.py\",
line 223, in import_uploaded_unit\nunit_type=unit_type_id,
summary=result['summary'], details=result['details']\nPulpCodedException:
The importer docker_importer indicated a failed response when uploading
docker_manifest unit to repository docker_centos7.\n",

  "error": {
"code": "PLP0047",
"data": {
  "unit_type": "docker_manifest",
  "importer_id": "docker_importer",
  "repo_id": "docker_centos7",
  "details": {},
  "summary": [
"Cannot resolve field \"digest\""
  ]
},
"description": "The importer docker_importer indicated a failed
response when uploading docker_manifest unit to repository
docker_centos7.",

I tried both v2s2 and v2s1, both don't work. I tried 2 different
Dockerfiles and both produce the same error.

Is it an issue in Buildah or in Pulp? Or am I doing something wrong?

Thanks!

-- 
Konstantin Khankin
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] docker cli 1.13.1 can't parse JSON returned by pulp+crane

[Konstantin M. Khankin]

Hi Ina,

Great, thank you very much :)

пн, 13 авг. 2018 г. в 13:16, Ina Panova :

> Hi Konstantin,
>
> after some investigation and collaboration with skopeo project, we managed
> to track down the root cause.
> The issue is filled [0] and a PR is on its way already.
> Fix will go into next upcoming bugfix release.
>
>
> [0] https://pulp.plan.io/issues/3910
>
>
>
> 
> Regards,
>
> Ina Panova
> Software Engineer| Pulp| Red Hat Inc.
>
> "Do not go where the path may lead,
>  go instead where there is no path and leave a trail."
>
> On Mon, Jul 23, 2018 at 11:22 AM, Ina Panova  wrote:
>
>> Hi Konstantin,
>>
>> It looks like it might be an issue with skopeo.
>> I was able to reproduce the error only if i convert with skopeo image to
>> v2s1.
>> Conversion and pull of v2s2 succeeds.
>> Docker pull from as a result of sync from registry succeeds.
>>
>> Pull of skopeo output v2s1:
>> $ docker pull localhost:1234/centos1:latest
>> Trying to pull repository localhost:1234/centos1 ...
>> unexpected end of JSON input
>>
>> Pull of skopeo output v2s2:
>> $ docker pull localhost:1234/centos2:latest
>> Trying to pull repository localhost:1234/centos2 ...
>> sha256:6ac335a2188c8fec30719d6fb137931f3b9b4c480f264116e05430455f7ad578:
>> Pulling from localhost:1234/centos2
>> Digest:
>> sha256:6ac335a2188c8fec30719d6fb137931f3b9b4c480f264116e05430455f7ad578
>> Status: Image is up to date for localhost:1234/centos2:latest
>>
>> I will reach skopeo devs and follow-up with you later once there are some
>> updates.
>> Thank you for pointing out this issue!
>>
>>
>>
>>
>

-- 
Konstantin Khankin
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] docker cli 1.13.1 can't parse JSON returned by pulp+crane

[Konstantin M. Khankin]

Hi!

I'm using docker 1.13.1 on CentOS 7.5 on the server named 'front'. On
another server named 'hive' I have docker API compatible registry based on
pulp 2.16.3 + crane 3.2.0, running on CentOS 7.5. I created custom
pulp-docker repository named 'gsk/centos7', dumped custom image from docker
server with skopeo in v2s1 format (v2s2 refused to work because of
incorrect redirect), uploaded and published an image. Repository has
manifests, blobs and tags:

[root@hive ~]# pulp-admin docker repo list
+--+
  Docker Repositories
+--+

Id:  docker_centos7
Display Name:None
Description: None
Content Unit Counts:
  Docker Blob: 5
  Docker Manifest: 1
  Docker Tag:  1

Crane correctly sees the repository and images inside it. Crane is serving
on the virtual host named 'docker'. URL
https://docker.gsk.loc/v2/gsk/centos7/manifests/latest redirects to
https://hive.gsk.loc/pulp/docker/v2/docker_centos7/manifests/1/latest (pulp
server is running under 'hive' virtual host) and produces seemingly correct
and parseable JSON.

But when I try to pull that image using docker cli, I get the following
error:
[root@front ~]# docker pull docker.gsk.loc/gsk/centos7
Using default tag: latest
Trying to pull repository docker.gsk.loc/gsk/centos7 ...
unexpected end of JSON input

Using 'docker -D' does not increase verbosity and I can't see in any logs
what exactly docker cli doesn't like.

I tried to dump https traffic with tcpdump but couldn't decrypt it in
wireshark. I also tried apache's module dumpio, but for some reasons
traffic to 'docker.gsk.loc' was not dumped.

Both 'docker' and 'hive' virtual hosts have correct X.509 certificates
accepted by 'front' server.

Could you please advice any troubleshooting path?

Thanks!

-- 
Konstantin Khankin
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Pulp CLI feedback

[Konstantin M. Khankin]

Hi David!

My deployment is quite small (10-15 repos, rpm/python, ~20 clients),
planning to also begin using puppet which is shipped with pulp-consumer
anyway and looking at using Docker registry in future.

- What commands or functionality in the CLI do you rely on the most?
>
rpm sync
schedules
managing own repositories (not synced with upstream)
managing python packages
managing subscribers (force remove, package installation)
managing repos

- Are there things you wish the CLI had or did?
>
I wish there was a command to remove packages from subscribers, not only
install. Another wish is package list differ for clients - would be happy
to dump custom scripts and use functionality shipped with the tools I
already use :)


> - Why do you use the CLI over using the REST API directly?
>
CLI is as simple as 'cmd arg arg' and is shipped with Pulp. API requires to
use other systems, write own scripts or play with wget/curl.


> - Do you strictly use the CLI or do you use other things like Katello or
> the REST API?
>
CLI only.


> - Would you prefer a CLI or a basic web UI?
>
CLI is good, basic web UI would be a good addition but is not necessary. I
had plans to create one myself but that in indefinite future still.

Thanks!

-- 
Konstantin Khankin
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] issues with pulp 2.9 on centos

[Konstantin M. Khankin]

It seems like Apache is looking for pulp wsgi file in /srv/pulp directory.
If it's empty as you say, it's expected that Apache cannot read anything
from there :) I would recommend to either verify your installation (which
packets are installed and where, how things are configured) or just
reinstall everything from scratch by this manual -
https://docs.pulpproject.org/user-guide/installation/index.html

2016-09-29 15:44 GMT+03:00 Reitmayer, Jeffrey S <je...@doit.tamu.edu>:

> Hey Konstantin,
>
>
>
> Thanks for the reply. I’m not sure which apache config you need. I didn’t
> build this server so the last guy left me with a mess. The /srv/pulp
> directory is empty right now and the directory permissions are set to
> root:root
>
>
>
> If you let me know what Apache configs you need I can provide them. It
> looks like he has everything set in vhost files in /etc/httpd/conf.d/
>
>
>
> Please let me know what you need I’ll be happy to provide.
>
>
>
> Thanks
>
>
>
> Jeff
>
>
>
> *From:* Konstantin M. Khankin [mailto:khankin.konstan...@gmail.com]
> *Sent:* Thursday, September 29, 2016 12:12 AM
> *To:* Reitmayer, Jeffrey S <je...@doit.tamu.edu>
> *Cc:* pulp-list@redhat.com
> *Subject:* Re: [Pulp-list] issues with pulp 2.9 on centos
>
>
>
> Hi Jeffrey,
>
>
>
> Could you please show your Apache config? Is your pulp instance installed
> in /srv/pulp? Are permissions set up so that Apach can read
> webservices.wsgi?
>
>
>
> 2016-09-28 22:59 GMT+03:00 Reitmayer, Jeffrey S <je...@doit.tamu.edu>:
>
> Good Afternoon.
>
> I am having issues with our reposrv running pulp. We cannot access and
> download any updates through yum. I am getting this error when I restart
> apache:
>
>
>
> ==> error_log <==
>
> [Wed Sep 28 14:54:09 2016] [notice] caught SIGTERM, shutting down
>
> [Wed Sep 28 14:54:09 2016] [notice] SELinux policy enabled; httpd running
> as context unconfined_u:system_r:httpd_t:s0
>
> [Wed Sep 28 14:54:09 2016] [notice] suEXEC mechanism enabled (wrapper:
> /usr/sbin/suexec)
>
> [Wed Sep 28 14:54:09 2016] [notice] Digest: generating secret for digest
> authentication ...
>
> [Wed Sep 28 14:54:09 2016] [notice] Digest: done
>
> [Wed Sep 28 14:54:09 2016] [notice] Apache/2.2.15 (Unix) DAV/2
> mod_ssl/2.2.15 OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.6.6 configured --
> resuming normal operations
>
> [Wed Sep 28 14:54:09 2016] [error] (2)No such file or directory: mod_wsgi
> (pid=16171, process='pulp', application='pulp'): Call to fopen() failed for
> '/srv/pulp/webservices.wsgi'.
>
> [Wed Sep 28 14:54:09 2016] [error] (2)No such file or directory: mod_wsgi
> (pid=16170, process='pulp', application='pulp'): Call to fopen() failed for
> '/srv/pulp/webservices.wsgi'.
>
> [Wed Sep 28 14:54:09 2016] [error] (2)No such file or directory: mod_wsgi
> (pid=16172, process='pulp', application='pulp'): Call to fopen() failed for
> '/srv/pulp/webservices.wsgi'.
>
>
>
> I don’t know if this is enough to start troubleshooting the process but I
> would greatly appreciate any help
>
>
> Thanks.
>
> Jeff
>
>
>
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com_mailman_listinfo_pulp-2Dlist=CwMFaQ=ODFT-G5SujMiGrKuoJJjVg=MFiN7iXU2x3gY25BmN9dtRxf6xeyq3fs5ZfQPm3sPdI=fjj3W-ZcbnrnFZqfUMbu8WoxalJA7mP2rL6utTJx83A=B0STBK0gS1fBBAbyaKmUfN5fPg5_EAI1Q6YB0CMyFGw=>
>
>
>
>
>
> --
>
> Ханкин Константин
>



-- 
Ханкин Константин
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] external authentication/authorization

[Konstantin M. Khankin]

You may try to use PAM to hook up authentication to any external source.
This is how I connected it to FreeIPA:

AuthType Basic
AuthBasicProvider PAM
AuthPAMService pulp
AuthName "Pulp"
Require valid-user


# cat /etc/pam.d/pulp
authrequired   pam_sss.so
account required   pam_sss.so

2016-09-02 0:50 GMT+03:00 Jay Medrano :

> I have the exact same issue... my cookbook/runbook instructions for
> setting up a pulp server require setting up users with passwords that are
> never actually used. The users are created that way so that they can be
> added to the admin group. If the LDAP feature is deprecated, there should
> be a better way to manage users via Apache auth groups, but at this point
> it doesn't seem that way.
>
>
>
> On a similar topic... Here is a code snippet related to some changes I
> made to the Apache auth section to allow LDAP auth when using the
> pulp-admin client. Notice that I'm using the User-Agent header to determine
> if LDAP auth is required, and I'm also defaulting apache auth when the
> login page is requested. This allows LDAP auth to work when requesting a
> cert from the pulp-admin client and also for the REST api. This also works
> when wget/curl calls submit data to pulp.
>
>
>
> 
>
> # pass everything that isn't a Basic auth request through to Pulp
>
> SetEnvIf Request_URI "^/pulp/api/v2/actions/login/" USE_APACHE_AUTH=1
>
> SetEnvIfNoCase ^User-Agent$ .+ USE_APACHE_AUTH=1
>
> Order allow,deny
>
> Allow from env=!USE_APACHE_AUTH
>
> Satisfy Any
>
>
>
>
>
> *From:* pulp-list-boun...@redhat.com [mailto:pulp-list-boun...@redhat.com]
> *On Behalf Of *Kodiak Firesmith
> *Sent:* Thursday, September 01, 2016 2:46 PM
> *To:* Vladimir Vasilev 
> *Cc:* pulp-list 
> *Subject:* Re: [Pulp-list] external authentication/authorization
>
>
>
> I'm pretty sure the answer in Pulp's current form is: no.
>
> But your request might be a great suggestion to make in an earlier (June?
> July?) thread requesting feedback on Pulp 3.x auth - it'll be completely
> different so it's a blank slate to work with.  Please check out the
> archives and reply to that thread with your auth needs and wants.
>
>
>
> As an Active Directory user (mod_auth_gssapi), I agree that being able to
> tie in AD names and groups in authorization would be a great improvement.
>
>
>
>  - Kodiak
>
>
>
> On Thu, Sep 1, 2016 at 3:47 PM, Vladimir Vasilev 
> wrote:
>
> Hi all,
>
> I'm trying to setup Pulp with external authentication and authorization
> against LDAP server.
> According to the docs direct LDAP access from pulp is deprecated so I
> followed "Apache Preauthentication" [1]
> Authentication works fine, pulp is trusting apache httpd with
> REMOTE_USER variable set.
> Problem is that the same LDAP user needs to exist in the internal pulp
> database as well.
>
> Is there a way to move both authentication and authorization to external
> provider like LDAP?
> At the end of the day I want to grant admin access to all LDAP accounts
> which are member of particular group (memberOf attribute) without making
> local pulp accounts.
>
> Thanks,
> Vova
>
> [1] https://docs.pulpproject.org/user-guide/authentication.html
>
>
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>
>
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>



-- 
Ханкин Константин
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Feedback needed: new user/auth system in 3.0

[Konstantin M. Khankin]

I use PAM auth, which is in turn authenticates login requests through
FreeIPA. I don't create any roles other than admin though

2016-07-15 22:29 GMT+03:00 Michael Hrivnak :

> As many of you know, we are switching from mongodb to postgres in Pulp
> 3.0. This will come with quite a few changes. For one in particular, we
> need your input about how you use Pulp's user and permission system.
> Anything you can tell us about how you use the current user/perm system
> would be very helpful. We are considering the use of Django's built-in
> user/auth system [0] as a replacement for what Pulp currently has.
>
> If we hear silence, we might be more likely to change things, so let us
> know what is important to you.
>
> Have you integrated Pulp with a separate authentication source? Which one?
>
> Do you assign permissions to specific users? How granular do you need that
> to be?
>
> Have you created "roles" in Pulp?
>
> Anything else you want us to know or to think about?
>
> If you would like to provide input confidentially, you are welcome to
> contact me directly.
>
> [0] https://docs.djangoproject.com/en/1.8/topics/auth/
>
> Thank you!
> Michael
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>



-- 
Ханкин Константин
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Updating pulp-consumer command on consumers

[Konstantin M. Khankin]

Hi Lutchy

I ended up mirroring pulp-2-stable repo and adding it to all consumers. My
pulp CA cert is broken now, so I cannot show how I did it, but it should
not be harder then mirroring any other repo

2016-05-02 17:05 GMT+03:00 Lutchy Horace (Mailing List) <
mailinglist.subscripti...@lhprojects.net>:

> Hello,
>
> Did some of my google-fu and could not find any information regarding
> updating 'pulp-consumer' automatically. I did find this post from late
> 2015 but not quite what I was looking for:
>
> https://www.redhat.com/archives/pulp-list/2015-December/msg9.html
>
> Is this done automatically without any human intervention or do I
> have to manually update pulp-consumer on every consumer? I am assuming
> at the moment to automate this, I would need to create a pulp repository
> and bind consumers to it?
>
> Regards, thankfully
>
> --
> Lutchy Horace
> Owner/Operator/Administrator [http://www.lhprojects.net]
> Owner/Operator/Administrator [http://www.bombshellz.net]
> Owner/Operator/Administrator [http://www.animehouse.club]
> About Me [http://about.me/lhprojects]
> USA
>
> 3c^39kvHMClWqKJ
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>



-- 
Ханкин Константин
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Existing sync schedules do not work after 2.7 -> 2.8 upgrade

[Konstantin M. Khankin]

Hi Brian,

Yes, this issue got resolved by restarting all Pulp processes on Pulp
server. Not sure why though. I could not reproduce it, so I think it was
some intermittent issue

"Did you try to turn it off and on again?" :)

2016-04-20 15:47 GMT+03:00 Brian Bouterse <bbout...@redhat.com>:

> Did this ever get resolved? If not, would you be willing to file a bug
> with reproducer instructions in the bug tracker[0].
>
> [0]: https://pulp.plan.io/
>
> -Brian
>
> On 04/02/2016 04:42 PM, Konstantin M. Khankin wrote:
> > Hello!
> >
> > I have sync schedules set up for repos I host locally with pulp. For
> > example:
> >
> > [root@nms ~]# pulp-admin rpm repo sync schedules list --repo-id
> centos7_base
> > +--+
> >Schedules
> > +--+
> >
> > Schedule: PT12H
> > Id:   558aa9b2cfbac056139ada41
> > Enabled:  True
> > Next Run: 2016-04-02T21:54:56Z
> >
> > Before 2.7->2.8 upgrade it worked well. Now when I didn't see any
> > updates on consumers for some time, I started looking into it and found
> > that automated syncs fail:
> >
> > [root@nms ~]# pulp-admin tasks list --state failed
> > ...
> > Operations:  sync
> > Resources:   centos7_base (repository)
> > State:   Failed
> > Start Time:  2016-04-02T09:54:56Z
> > Finish Time: 2016-04-02T09:54:56Z
> > Task Id: f0532278-0e62-4ce7-a59a-557e20308f3a
> >
> > Operations:  sync
> > Resources:   centos7_updates (repository)
> > State:   Failed
> > Start Time:  2016-04-02T09:55:22Z
> > Finish Time: 2016-04-02T09:55:23Z
> > Task Id: 87bdd3a3-e8e5-4c8e-80b6-09a09c293b47
> >
> > Operations:  sync
> > Resources:   centos7_extras (repository)
> > State:   Failed
> > Start Time:  2016-04-02T09:55:38Z
> > Finish Time: 2016-04-02T09:55:38Z
> > Task Id: cd53e254-75dd-4c56-a518-915fab1da057
> >
> > Operations:  sync
> > Resources:   google-chrome (repository)
> > State:   Failed
> > Start Time:  2016-04-02T20:00:00Z
> > Finish Time: 2016-04-02T20:00:00Z
> > Task Id: 712f41a0-439a-47e4-bf05-5a576a0ec97b
> >
> > When I started manual sync, it succeeded:
> > Operations:  sync
> > Resources:   centos7_base (repository)
> > State:   Successful
> > Start Time:  2016-04-02T20:29:55Z
> > Finish Time: 2016-04-02T20:30:01Z
> > Task Id: efe06e75-58d8-4bb6-8f9c-9044b772d83f
> >
> > Operations:  publish
> > Resources:   centos7_base (repository)
> > State:   Successful
> > Start Time:  2016-04-02T20:30:01Z
> > Finish Time: 2016-04-02T20:32:14Z
> > Task Id: c7eeff96-4bc5-487b-826f-14930e0c0418
> >
> > What can be wrong?
> >
> > Thanks!
> >
> > --
> > Konstantin Khankin
> >
> >
> > ___
> > Pulp-list mailing list
> > Pulp-list@redhat.com
> > https://www.redhat.com/mailman/listinfo/pulp-list
> >
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>



-- 
Ханкин Константин
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Pulp + FreeIPA = ?

[Konstantin M. Khankin]

led to give an access to IPA users and user
'admin' is configured as pulp admin. But for some reasons pulp is not able
to generate a certificate. I've tried to restart all pulp processes and
httpd multiple times already. Tried to disable SELinux (and enabled it back
since it didn't change anything). Do you have any idea why this error could
occur?

Thanks!

2016-04-15 22:29 GMT+03:00 Konstantin M. Khankin <
khankin.konstan...@gmail.com>:

> Hi!
>
> I'm trying to use FreeIPA PKI for pulp. I successfully installed all
> certificates/keys on consumers and set up httpd to use correct
> certificates/keys, but faced a problem - pulp wants to have CA superpowers
> to sign client certificates, even though config file and documentation both
> says that this function is deprecated
>
> Can I somehow disable certificate signing attempts in pulp so that it is
> able to use existing PKI?
>
> Thanks!
>
> --
> Konstantin Khankin
>



-- 
Konstantin Khankin
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Pulp + FreeIPA = ?

[Konstantin M. Khankin]

Hi!

I'm trying to use FreeIPA PKI for pulp. I successfully installed all
certificates/keys on consumers and set up httpd to use correct
certificates/keys, but faced a problem - pulp wants to have CA superpowers
to sign client certificates, even though config file and documentation both
says that this function is deprecated

Can I somehow disable certificate signing attempts in pulp so that it is
able to use existing PKI?

Thanks!

-- 
Konstantin Khankin
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] goferd on consumers cannot connect to pulp server's queue after 2.7 -> 2.8 upgrade

[Konstantin M. Khankin]

Documentation is still proposing to remove pulp-server package. I suppose
it should be fixed :)

2016-04-05 23:15 GMT+03:00 Jeff Ortel <jor...@redhat.com>:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
>
>
> On 04/05/2016 02:40 PM, Konstantin M. Khankin wrote:
> > On pulp-server:
> >
> > [root@nms pulp]# LANG=C ls -l total 16 -rw-r-. 1 root   apache 1021
> *Mar 24* 23:47 ca.crt -rw-r-.
> > 1 root   apache 1675 *Mar 24* 23:47 ca.key drwxr-xr-x. 2 apache apache
>  33 Mar 15 08:32 content
> > -rw-r-. 1 root   apache 1675 *Mar 24* 23:47 rsa.key -rw-r--r--. 1
> root   apache  451 *Mar 24* 23:47
> > rsa_pub.key
> >
> > [root@nms pulp]# grep pulp /var/log/yum.log Mar 24 23:45:53 Erased:
> > pulp-puppet-plugins-2.7.1-1.el7.noarch Mar 24 23:45:54 Erased:
> pulp-rpm-plugins-2.7.1-1.el7.noarch Mar 24
> > 23:46:13 Erased: pulp-server-2.7.1-1.el7.noarch Mar 24 23:47:00 Updated:
> > python-pulp-common-2.8.0-1.el7.noarch Mar 24 23:47:01 Updated:
> python-pulp-repoauth-2.8.0-1.el7.noarch
> > Mar 24 23:47:03 Updated: python-pulp-rpm-common-2.8.0-1.el7.noarch Mar
> 24 23:47:03 Updated:
> > python-pulp-client-lib-2.8.0-1.el7.noarch Mar 24 23:47:04 Updated:
> > python-pulp-puppet-common-2.8.0-1.el7.noarch Mar 24 23:47:04 Installed:
> > python-pulp-docker-common-2.0.0-1.el7.noarch Mar 24 23:47:05 Updated:
> > python-pulp-bindings-2.8.0-1.el7.noarch Mar 24 23:47:06 Updated:
> pulp-admin-client-2.8.0-1.el7.noarch
> > *Mar 24 23:47:14 Installed: pulp-server-2.8.0-1.el7.noarch* Mar 24
> 23:47:15 Installed:
> > pulp-docker-plugins-2.0.0-1.el7.noarch Mar 24 23:47:16 Updated:
> > pulp-rpm-admin-extensions-2.8.0-1.el7.noarch Mar 24 23:47:17 Updated:
> > pulp-puppet-admin-extensions-2.8.0-1.el7.noarch Mar 24 23:47:18 Updated:
> > python-pulp-oid_validation-2.8.0-1.el7.noarch Mar 24 23:49:01 Updated:
> pulp-selinux-2.8.0-1.el7.noarch
> > Mar 25 00:01:40 Installed: pulp-rpm-plugins-2.8.0-1.el7.noarch Mar 25
> 04:31:03 Обновлено:
> > python-kombu.noarch 1:3.0.33-4.pulp.el7 Mar 25 04:31:13 Установлено:
> pulp-docker-admin-extensions.noarch
> > 2.0.0-1.el7
> >
> > On pulp-consumer key was not changed since pulp-consumer installation
> time (June 2015)
> >
> > I think the root cause for this is here ->
> >
> http://pulp.readthedocs.org/en/latest/user-guide/release-notes/2.8.x.html#upgrade-instructions-for-2-7-x-2-8
> - -x.
> >
> >
> Removing python-semantic-version package causes removal of pulp-server
> package. Later on when I reinstalled
> > server, certificates were updated as well
>
> Yikes.  We should be advising users to do:
>
> $ rpm -e --nodeps python-semantic-version
>
> instead of using yum/dnf for just this reason when working around this
> upgrade problem.
>
> >
> > Discussion thread is here ->
> https://www.redhat.com/archives/pulp-list/2016-March/msg00078.html
> >
> >
> > 2016-04-05 16:32 GMT+03:00 Jeff Ortel <jor...@redhat.com  jor...@redhat.com>>:
> >
> >
> >
> > On 04/04/2016 09:53 AM, Konstantin M. Khankin wrote:
> >> Hi Jeff!
> >
> >> [root@drone ~]# pulp-consumer -v history
> >>
> +--+
> Consumer History [drone]
> >> +--+
> >
> >> 2016-04-04 19:46:18,164 - ERROR - Client-side exception occurred
> Traceback (most recent call last):
> >> File "/usr/lib/python2.7/site-packages/pulp/client/extensions/core.py",
> line 474, in run exit_code =
> >> Cli.run(self, args) File
> "/usr/lib/python2.7/site-packages/okaara/cli.py", line 974, in run exit_code
> >> = command_or_section.execute(self.prompt, remaining_args) File
> >>
> "/usr/lib/python2.7/site-packages/pulp/client/extensions/extensions.py",
> line 210, in execute return
> >> self.method(*arg_list, **clean_kwargs) File
> >> "/usr/lib/python2.7/site-packages/pulp/client/consumer/cli.py", line
> 367, in history
> >> kwargs['start-date'], kwargs['end-date']).response_body File
> >> "/usr/lib/python2.7/site-packages/pulp/bindings/consumer.py", line 199,
> in history return
> >> self.server.GET(path, queries) File
> "/usr/lib/python2.7/site-packages/pulp/bindings/server.py", line
> >> 92, in GET return self._request('GET', path, queries,
> ignore_prefix=ignore_prefix) File
> >> "/usr/lib/python2.7/site-packages/pulp/bindings/server.py", line 152,
> in _request response_code,
> >> response_body = self.se

[Pulp-list] goferd on consumers cannot connect to pulp server's queue after 2.7 -> 2.8 upgrade

[Konstantin M. Khankin]

Hello!

After 2.7->2.8 upgrade all consumers cannot talk to pulp server - goferd
returns the next error on any operation:
[WARNING][MainThread] pulp.agent.gofer.pulpplugin:107 - validate
registration failed: (None, 'tlsv1 alert decrypt error', None)

I didn't change any keys or configs. Logs also do not help. What could went
wrong?

Thanks!

-- 
Konstantin Khankin
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Dependencies are broken after 2.8 release

[Konstantin M. Khankin]

Thanks Michael! Will try this workaround

For the 1st issue I did some troubleshooting and found that
python-pulp-rpm-common-2.8.0 was really absent in my copy of pulp-2-stable
repo. I've resynced the repo and at least one consumer got updated
successfully

2016-03-23 23:36 GMT+02:00 Michael Hrivnak <mhriv...@redhat.com>:

> I've been able to reproduce this problem with python-semantic-version. The
> workaround is thankfully simple. After changing your repo file in
> /etc/yum.repos.d/ to point at pulp 2.8, do this:
>
> $ sudo yum remove python-semantic-version
>
> This will remove a number of pulp packages also. That's ok. Then do:
>
> $ sudo yum update
> $ sudo yum groupinstall pulp-server-qpid
>
>
> On Tue, Mar 22, 2016 at 10:05 PM, Konstantin M. Khankin <
> khankin.konstan...@gmail.com> wrote:
>
>> Hi!
>>
>> After Pulp 2,8 release dependencies got broken. This is what I see on
>> Pulp server:
>> file /usr/lib/python2.7/site-packages/semantic_version/__init__.py from
>> install of python-semantic_version-2.4.2-1.el7.noarch conflicts with
>> file from package python-semantic-version-2.2.0-3.el7.noarch
>> ...
>> (I've filed https://bugzilla.redhat.com/show_bug.cgi?id=1318318 to
>> address this)
>>
>> But also on consumers:
>> Failed to build transaction: python-pulp-rpm-common-2.7.1-1.el7.noarch
>> requires python-pulp-common = 2.7.1
>> pulp-rpm-handlers-2.8.0-1.el7.noarch requires python-pulp-rpm-common =
>> 2.8.0
>>
>> Could you please help?
>>
>> Thanks!
>>
>> --
>> Konstantin Khankin
>>
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>>
>
>


-- 
Ханкин Константин
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Dependencies are broken after 2.8 release

[Konstantin M. Khankin]

Hi!

After Pulp 2,8 release dependencies got broken. This is what I see on Pulp
server:
file /usr/lib/python2.7/site-packages/semantic_version/__init__.py from
install of python-semantic_version-2.4.2-1.el7.noarch conflicts with file
from package python-semantic-version-2.2.0-3.el7.noarch
...
(I've filed https://bugzilla.redhat.com/show_bug.cgi?id=1318318 to address
this)

But also on consumers:
Failed to build transaction: python-pulp-rpm-common-2.7.1-1.el7.noarch
requires python-pulp-common = 2.7.1
pulp-rpm-handlers-2.8.0-1.el7.noarch requires python-pulp-rpm-common = 2.8.0

Could you please help?

Thanks!

-- 
Konstantin Khankin
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list