Re: [Pulp-list] Pulp 3.20 content server

[Brian Bouterse]

Hi Bin,

I don't have an answer for you, but I'd like to invite you to start posting
questions to https://discourse.pulpproject.org/. There is a "support"
category there which would be a good fit for these types of questions.
Would that work for you? This mailing list will likely be put into read
only mode at some point.

Apologies I don't know the answer to your question.

All the best,
Brian

On Thu, Jul 28, 2022 at 9:00 PM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> I am trying to install content only server with pulp_content role and
> getting an error
>
> TASK [pulp_content : Get the database fields key from the
> pulp_database_config node]
> *
> fatal: [pulp-t14 -> pulp-t14]: FAILED! => {"changed": false, "msg": "file
> not found: /etc/pulp/certs/database_fields.symmetric.key"}
>
> Does anyone know how to fix this?
>
> Thanks
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Can't build PEX out of pulp-cli because of missing dependency

[Brian Bouterse]

I don't work with the CLI, but I have seen projects declare a dependency on
setuptools. That would make sense to me.

On Wed, Dec 15, 2021 at 6:12 AM Konstantin M. Khankin <
khankin.konstan...@gmail.com> wrote:

> Hello Matthias!
>
> This workaround is sufficient for me. However, I feel like the dependency
> should be added, yes. I don't have (and wouldn't want to have) GitHub
> account, so I can't file a PR or an issue, sorry. Not going to ask you to
> do this as I'm fine with a workaround :)
>
> Thanks!
>
> ср, 15 дек. 2021 г. в 13:01, Matthias Dellweg :
>
>> Hello!
>> Thank you for verifying this. Is this workaround sufficient for you?
>> Should we still add a dependency on setuptools (It kind of felt
>> implicit, because you cannot use setup.py without it anyway.)? If so,
>> can you file a PR for it?
>>
>>   Matthias
>>
>> On Wed, Dec 15, 2021 at 11:56 AM Konstantin M. Khankin
>>  wrote:
>> >
>> > Validated PEX build fails because of missing dependency with this
>> workaround:
>> >
>> > $ echo setuptools > pulp_cli.req
>> > $ pex -r pulp_cli.req -v -c pulp -o pulp.pex 'pulp-cli[pygments]'
>> > $ deactivate
>> > $ ./pulp.pex --version
>> > pulp3 command line interface, version 0.12.0
>> >
>> > ср, 15 дек. 2021 г. в 12:43, Konstantin M. Khankin <
>> khankin.konstan...@gmail.com>:
>> >>
>> >> Hello!
>> >>
>> >> I wanted to make pulp-cli portable and build PEX archive out of it but
>> the latest pex (2.1.56) fails to do so:
>> >>
>> >> ```
>> >> $ pex -v -c pulp 'pulp-cli[pygments]'
>> >> ...
>> >> Running PEX file at /tmp/tmpdy35exye with args []
>> >> pex: PEX.run invoking /home/hc/pex/bin/python3 -s -E /tmp/tmpdy35exye
>> >> pex: Re-executing: cmdline='/usr/bin/python3.10 /tmp/tmpdy35exye',
>> sys.executable='/home/hc/pex/bin/python3', PEX_PYTHON=None,
>> PEX_PYTHON_PATH=None, interpreter_constraints=[]
>> >> pex: Found site-library: /usr/local/lib/python3.10/site-packages
>> >> pex: Found site-library: /usr/lib/python3.10/site-packages
>> >> pex: Found site-library: /usr/local/lib64/python3.10/site-packages
>> >> pex: Found site-library: /usr/lib64/python3.10/site-packages
>> >> pex: Tainted path element: /usr/lib64/python3.10/site-packages
>> >> pex: Tainted path element: /usr/lib/python3.10/site-packages
>> >> pex: Scrubbing from user site:
>> /home/hc/.local/lib/python3.10/site-packages
>> >> pex: Scrubbing from site-packages: /usr/lib64/python3.10/site-packages
>> >> pex: Scrubbing from site-packages: /usr/lib/python3.10/site-packages
>> >> pex: New sys.path: ['/tmp/tmpdy35exye/.bootstrap', '/tmp/tmpdy35exye',
>> '/usr/lib64/python310.zip', '/usr/lib64/python3.10',
>> '/usr/lib64/python3.10/lib-dynload']
>> >> pex: Activating PEX virtual environment from /tmp/tmpdy35exye: 54.7ms
>> >> pex: Bootstrap complete, performing final sys.path modifications...
>> >> pex: PYTHONPATH contains:
>> >> pex: /tmp/tmpdy35exye
>> >> pex:   * /usr/lib64/python310.zip
>> >> pex: /usr/lib64/python3.10
>> >> pex: /usr/lib64/python3.10/lib-dynload
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/7679cb4a5378680b93c1f33b463094a55cf8c547/pulp_cli-0.12.0-py3-none-any.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/23fff3a23c744ec73d03f9639bec17d3745b762c/click-8.0.3-py3-none-any.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/bc1215388f22b27f973bc1bc3d74a96373404ac9/packaging-21.3-py3-none-any.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/33b6478f39bfd332868d887340fd44164373dfb4/pyparsing-3.0.6-py3-none-any.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/5aa8517e7212729e4c0b95fab8b765c01e48f97c/PyYAML-5.4.1-cp310-cp310-linux_x86_64.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/cd6d3dfd86e52a31a3664cff69214a5dd94161e7/schema-0.7.4-py2.py3-none-any.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/c29688d87604522b7f9aa8a178a6a21b6a9fe775/contextlib2-21.6.0-py2.py3-none-any.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/d20e2df000dd43249c3a9eed041f08a812a93423/requests-2.26.0-py2.py3-none-any.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/601e0a29cd7d663eb8ea40c3ba7a52200e2544c0/urllib3-1.26.7-py2.py3-none-any.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/883fca1b3199ac78822a54c4af0a01c1f7a7ea74/certifi-2021.10.8-py2.py3-none-any.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/7e4deb9e3d26e447cfdd8c00b13805d8d2a65fca/charset_normalizer-2.0.9-py3-none-any.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/5485a373e7aae71ef1d2c02b8b09c330b75c5248/idna-3.3-py3-none-any.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/a55ae166e643e6c7a28c16fe005efc32ee98ee76/toml-0.10.2-py2.py3-none-any.whl
>> >> pex:
>>  
>> /home/hc/.pex/installed_wheels/8dd8170137ab3f4dc94d5006227635731b451dd1/Pygments-2.10.0-py3-none-any.whl
>> >> pex: /tmp/tmpdy35exye/.bootstrap
>> >> pex:   * - paths that do not exist or will be imported via zipimport
>> >> Traceback (most recent call last):
>> >>   File "/tmp/tmpdy35exye/.bootstrap/pex/pex.py", line 476, in execute

Re: [Pulp-list] redis on a passive backup server

[Brian Bouterse]

On Mon, Sep 20, 2021 at 5:40 PM Matthias Dellweg 
wrote:

> I am not sure if the cache invalidations will be pushed to all redis
> instances if deployed that way. Maybe there is a little bit of extra
> work needed.
>
I imagined the backup Pulp would have its Redis empty because that Pulp
wasn't in use. If that were the case cache invalidations not reaching it
would be fine because the cache would be empty at failover time. As always
though, there could be more needed to handle some specific use cases.


> On Mon, Sep 20, 2021 at 10:31 PM Brian Bouterse 
> wrote:
> >
> > I believe with 3.14+ and new style workers, you can have an independant
> redis on the backup server.
> >
> > With the new-style tasking system (the default in 3.14) you no longer
> need Redis for tasking. It's only purpose then is to help speedup the
> content app which caches info about requests to make answering subsequent
> content-app requests easier. If those caches miss, e.g. if you had a
> failover event, Pulp will continue to work fine, just building the Redis
> cache data as it goes.
> >
> > On Mon, Sep 20, 2021 at 4:22 PM Bin Li (BLOOMBERG/ 120 PARK) <
> bli...@bloomberg.net> wrote:
> >>
> >> We configured a redis slave on a passive backup server which shares the
> same external database with the primary pulp server. I noticed the task
> queues are now being tracked in the database. Is it still necessary to
> configure redis as a slave or we can have an independent instance of redis
> on the backup server?
> >>
> >> Thanks
> >> ___
> >> Pulp-list mailing list
> >> Pulp-list@redhat.com
> >> https://listman.redhat.com/mailman/listinfo/pulp-list
> >
> > ___
> > Pulp-list mailing list
> > Pulp-list@redhat.com
> > https://listman.redhat.com/mailman/listinfo/pulp-list
>
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] redis on a passive backup server

[Brian Bouterse]

I believe with 3.14+ and new style workers, you can have an independant
redis on the backup server.

With the new-style tasking system (the default in 3.14) you no longer need
Redis for tasking. It's only purpose then is to help speedup the content
app which caches info about requests to make answering subsequent
content-app requests easier. If those caches miss, e.g. if you had a
failover event, Pulp will continue to work fine, just building the Redis
cache data as it goes.

On Mon, Sep 20, 2021 at 4:22 PM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> We configured a redis slave on a passive backup server which shares the
> same external database with the primary pulp server. I noticed the task
> queues are now being tracked in the database. Is it still necessary to
> configure redis as a slave or we can have an independent instance of redis
> on the backup server?
>
> Thanks
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] pulpcore 3.14.0 is Generally Available

[Brian Bouterse]

This release contains a lot of exciting features. See the blog announcement
for a great recap of the major highlights, or or the github discussion
announcement for info on upgrading and changelog links.

https://pulpproject.org/2021/07/01/pulpcore-3.14-release-announcement/
https://github.com/pulp/community/discussions/40

Thanks to everyone who helped put together this release!
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Content server Performance

[Brian Bouterse]

On Tue, Jun 22, 2021 at 11:56 AM Danny Sauer  wrote:

> You can certainly run multiple instances of the content server.  It just
> needs a connection to the database and access to the storage.
>
Agreed, you could deploy additional content servers and have your
nginx/apache load balance them.


> Have you tuned the number of worker processes in Gunicorn?  It defaults to
> 1, but should almost certainly be increased for any sort of volume.
> https://docs.gunicorn.org/en/stable/settings.html#worker-processes
>
Pulp changed the default gunicorn worker processes to 8 maybe a release or
two ago. See the `pulp_content_workers` variable in the installer here
https://pulp-installer.readthedocs.io/en/latest/roles/pulp_content/#role-variables

>
> There are several moving pieces, but that's really all I had to touch here.
>
> --Danny
>
With pulpcore==3.14 there is a significant performance improvement being
reviewed now  https://pulp.plan.io/issues/8805  . In addition to resolving
it with methods like ^, when 3.14 comes out (scheduled for June 29th) it
would be great if you could report on if the improvements helped you.

>
> On Tue, Jun 22, 2021 at 10:34 AM Bin Li (BLOOMBERG/ 120 PARK) <
> bli...@bloomberg.net> wrote:
>
>> We recently add more clients to use the pulp content server. The
>> processes run out the file descriptor first. We then increased both nginx
>> and pulp-content by creating a override.conf
>> /etc/systemd/system/pulpcore-content.service.d # cat override.conf
>> [Service]
>> LimitNOFILE=65536
>>
>> and updated nginx.conf
>> # Gunicorn docs suggest this value.
>> worker_processes 1;
>> events {
>> worker_connections 1; # increase if you have lots of clients
>> accept_mutex off; # set to 'on' if nginx worker_processes > 1
>> }
>>
>> worker_rlimit_nofile 2;
>>
>>
>> Now we are keep getting this error.
>> 2021/06/22 11:26:36 [error] 78373#0: *112823 upstream timed out (110:
>> Connection timed out) while connecting to upstream, client:
>>
>> It looks like pulp-content server cannot keep up with requests. Is there
>> anything we could do to increase the performance of the content server?
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://listman.redhat.com/mailman/listinfo/pulp-list
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulpcore 3.13.0 is Generally Available

[Brian Bouterse]

Yes it's here:  https://github.com/fao89/pdc

On Thu, Jun 3, 2021 at 12:11 PM Daniel Alley  wrote:

> I think that the resolver will have a hard time detecting package
>> inconsistencies prior to upgrade when the installer calls pip for all the
>> pulp modules individually.
>>
>
> Yup, you're definitely right about that...
>
> We do actually have a tool for checking the compatibility of all of the
> plugins (it was one of fao89's hobby projects) but haven't integrated it
> into the upgrade workflow.  Maybe we should, to avoid issues like this.
> Although - not having missing releases, would be preferable.
>
> The ansible plugin is released, the migration plugin is not (yet) but I'll
> let you know once it is.
>
> On Tue, Jun 1, 2021 at 6:59 PM Ben Stanley  wrote:
>
>> The way I read [0], the new resolver is already enabled and the legacy
>> resolver is already removed!
>>
>> I think that the resolver will have a hard time detecting package
>> inconsistencies prior to upgrade when the installer calls pip for all the
>> pulp modules individually.
>>
>> With rpm, you have a better experience by passing all the upgrade rpms in
>> one transaction, to avoid dependency analysing intermediate states. I want
>> to know if the final configuration is inconsistent before making any
>> changes to my installation (which is the claimed behavior of the pulp
>> installer...).
>>
>> Ben.
>>
>> On 2 June 2021 8:38:46 am Daniel Alley  wrote:
>>
>>> It turns out that pulp-2to3-migration and pulp-ansible are not yet
 compatible with pulpcore-3.13.0.
>>>
>>>
>>> Unfortunately there were some delays with the pulp_ansible release, but
>>> it should be up tomorrow, and the pulp-2to3-migration release some time
>>> this week.  Sincere apologies for the frustration.
>>>
>>> This brings up a good point though.  The reason this is a problem is
>>> that Python has historically done a bad job of resolving dependencies and
>>> avoiding updates that break packages.  They have recently done a lot of
>>> work improving that [0] but until it's enabled by default we should be more
>>> careful about our releasing, or adjust the instructions to avoid upgrade
>>> issues.
>>>
>>> Maybe we can opt-in to the new resolver in the installer?  Probably
>>> something to investigate.
>>>
>>> [0] https://pyfound.blogspot.com/2020/11/pip-20-3-new-resolver.html
>>>
>>> On Tue, Jun 1, 2021 at 3:12 AM Ben Stanley 
>>> wrote:
>>>
 Hello Daniel,

 I attempted to upgrade my pulp installation using these instructions.

 I upgraded the pulp_installer as described.

 Note that I am behind a proxy server, and I have to edit the file
 ~/.ansible/collections/ansible_collections/pulp_installer/roles/pulp_common/tasks/repos.yml
 to remove the rpm_key module (which doesn't work with my proxy) and replace
 it with the raw module (which does work with my proxy). I have described
 this previously on this mailing list.

 I have properly set my proxy environment variables (proxy, proxy_http,
 proxy_https and no_proxy) before attempting this procedure.

 I ran the pulp_installer as:

 ansible-playbook pulp_install.yml -l honeybee

 

 TASK [pulp.pulp_installer.pulp_common : Install pulpcore via PyPI]
 *
 fatal: [honeybee]: FAILED! => {"changed": false, "cmd":
 ["/usr/local/lib/pulp/bin/pip", "install", "pulpcore==3.13.0"], "msg":
 "\n:stderr: WARNING: Retrying (Retry(total=4, connect=None, read=None,
 redirect=None, status=None)) after connection broken by 'ProxyError('Cannot
 connect to proxy.', OSError('Tunnel connection failed: 407 Proxy
 Authentication Required',))': /simple/pulpcore/\nWARNING: Retrying
 (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after
 connection broken by 'ProxyError('Cannot connect to proxy.',
 OSError('Tunnel connection failed: 407 Proxy Authentication Required',))':
 /simple/pulpcore/\nWARNING: Retrying (Retry(total=2, connect=None,
 read=None, redirect=None, status=None)) after connection broken by
 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed:
 407 Proxy Authentication Required',))': /simple/pulpcore/\nWARNING:
 Retrying (Retry(total=1, connect=None, read=None, redirect=None,
 status=None)) after connection broken by 'ProxyError('Cannot connect to
 proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication
 Required',))': /simple/pulpcore/\nWARNING: Retrying (Retry(total=0,
 connect=None, read=None, redirect=None, status=None)) after connection
 broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel
 connection failed: 407 Proxy Authentication Required',))':
 /simple/pulpcore/\nERROR: Could not find a version that satisfies the
 requirement pulpcore==3.13.0 (from versions: none)\nERROR: No matching
 distribution found for pulpcore==3.13.0\n"}

 It seems that the 

Re: [Pulp-list] pulp mongodb upgrade

[Brian Bouterse]

Hi Venkataramana,

Thanks for the logs. In there, I only see one error, but I see it a huge
number of times:   ServerSelectionTimeoutError: localhost:27017: [Errno
111] Connection refusedTo me that means environmentally celery and/or
the WSGI environments cannot connect to your mongod. Unfortunately, I don't
have more info than that.

All the best,
Brian


On Fri, Jun 4, 2021 at 2:29 PM Venkataramana Bora 
wrote:

> Hi Brian, here have attached "pulp:"  logs from /var/log/messages .Sorry
> for attaching bit lengthy text files.
> Other than those, I did not find  any other pulp logs . Thank you !
>
>
>
> Sincerely,
> Ramana Bora
>
>
>
> - Original message -
> From: Brian Bouterse 
> To: Venkataramana Bora 
> Cc: pulp-list , Donal Keane 
> Subject: [EXTERNAL] Re: [Pulp-list] pulp mongodb upgrade
> Date: Wed, Jun 2, 2021 9:29 PM
>
> Hi Venkataramana and Donal, I looked at the logs, but unfortunately I
> don't see any exceptions related to Pulp's use of Mongo. Mostly I just see
> a mongodb error:  `FileRenameFailed Renaming file
> /var/log/mongodb/mongodb.log to
> /var/log/mongodb/mongodb.log.2021-05-27T07-33-08 ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
> ZjQcmQRYFpfptBannerEnd
> Hi Venkataramana and Donal,
>
> I looked at the logs, but unfortunately I don't see any exceptions related
> to Pulp's use of Mongo. Mostly I just see a mongodb error:
> `FileRenameFailed Renaming file /var/log/mongodb/mongodb.log to
> /var/log/mongodb/mongodb.log.2021-05-27T07-33-08 failed; Cannot verify
> whether destination already exists`.
>
> I believe the pulp logs themselves would be needed.
>
> -Brian
>
>
> On Fri, May 28, 2021 at 12:18 PM Venkataramana Bora 
> wrote:
>
>
> Hi Brian,
>
> Please find attached  MongoDB logs from /var/log/mongodb/. There are no
> logs in /var/log/pulp. Please let us know if any other logs required .
> Thanks a lot for your reply!
>
>
> Sincerely,
> Ramana Bora
>
>
>
> - Original message -
> From: Donal Keane/Ireland/IBM
> To: bmbou...@redhat.com
> Cc: pulp-list@redhat.com, Venkataramana Bora/India/IBM@IBM
> Subject: Re: [EXTERNAL] Re: [Pulp-list] pulp mongodb upgrade
> Date: Fri, May 28, 2021 2:19 AM
>
> Hi Brian,
>
> Thank you kindly for your response.
>
> The plan here originally was to move to Pulp 3 - and definitely still is.
> However, our brand within IBM has recently been sold off and it is
> happening sooner rather than later. As part of this acquisition we are
> losing a good proportion of expertise around our Linux operating systems.
> We have been tasked as a high priority to make sure all of our open source
> tools are patched to the latest security level before the acquisition goes
> through. This is why Ramana is attempting to get us to 2.21.x asap and from
> there, concentrate on getting to 3 when time and resources allow. I hope
> this explains our current situation and why any help would be graciously
> accepted!
>
> Ramana can you post relevant log info when you get a chance tomorrow?
>
> Thanks,
>
> Donal
>
>
> - Original message -
> From: Brian Bouterse 
> To: Venkataramana Bora 
> Cc: pulp-list , Donal Keane 
> Subject: [EXTERNAL] Re: [Pulp-list] pulp mongodb upgrade
> Date: Thu, May 27, 2021 8:12 PM
>
> I don't have ready advice, but it might be helpful to post a snippet of
> the logs. Without that, all we know is there is a 500 server error, not
> what the cause could be. Have you considered migrating to Pulp3? Pulp2 is
> in maintenance mode, ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
> ZjQcmQRYFpfptBannerEnd
> I don't have ready advice, but it might be helpful to post a snippet of
> the logs. Without that, all we know is there is a 500 server error, not
> what the cause could be.
>
> Have you considered migrating to Pulp3? Pulp2 is in maintenance mode, and
> likely has had its terminal release. Pulp3 is at 3.13 and has been out
> since Dec 2019. Here's more information on how to upgrade
> https://pulpproject.org/migrate-to-pulp-3/  We would be very interested
> in helping you do that.
>
> On Thu, May 27, 2021 at 12:03 PM Venkataramana Bora 
> wrote:
>
>
> Hi team,
> We are using pulp-server-2.21.5-1.el7 with
> mongodb-server-2.6.12-6.el7.x86_64 , this version of mongodb 2.6.12-16 got
> installed from our config manage server puppet .
> Time and again pulp-admin commands run, throwing below errors , I think it
> is due to old version of mongodb. After running /var/lib/mongodb --repair
> able to access pulp-admin commands fo

Re: [Pulp-list] pulp3 remote mirrorlist

[Brian Bouterse]

Yes I believe so. I also see some entries about it in the changelog
https://pulp-rpm.readthedocs.io/en/latest/changes.html#id94


On Thu, Jun 3, 2021 at 1:58 PM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> Does the pulp support a mirrorlist as the remote url? If not, will this
> feature be added in the future?
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] [Pulp-dev] Github Discussions

[Brian Bouterse]

I did this also for the pulpcore meeting:
https://github.com/pulp/community/discussions/8

My format was a little different, but the same idea.

On Thu, Jun 3, 2021 at 10:13 AM Grant Gainey  wrote:

> On Thu, Jun 3, 2021 at 9:40 AM David Davis  wrote:
>
>> Based on feedback, I've moved discussions to its own repo:
>> https://github.com/pulp/community/discussions.
>>
>
> Brillliant!
>
> One discovery I made this week - for 'Meetings' threads that exist to have
> meeting-minutes posted, the first entry should be a description of what the
> meeting you're recording is for, and each set of minutes should be a
> comment. This lets the reader sort by "Newest" and get
> most-recent-minutes-first.The initial message in a discussion is always at
> the top, no matter how you sort - so if it's your first meeting-minutes,
> they'll always be first.
>
> I redid the katello/pulp and community/pulp integration discussion-threads
> (in their new location) in light of this, apologies to anyone who got some
> notification-spam as a result this morning.
>
>- https://github.com/pulp/community/discussions/7
>- https://github.com/pulp/community/discussions/4
>
> G
>
>>
>> David
>>
>>
>> On Tue, May 25, 2021 at 1:49 PM David Davis 
>> wrote:
>>
>>> We've heard from the community about the amount of friction involved in
>>> getting help with Pulp and one of the areas I think we could improve is
>>> user communications. We currently run two mailing lists: pulp-list and
>>> pulp-dev.
>>>
>>> At today's open floor meeting, we talked about using Github's new
>>> Discussions feature[0] to host these conversations instead. I've set up a
>>> Discussion against pulpcore[1] for us to try but here's also an example of
>>> a project that has a lot of threads[2].
>>>
>>> I think the consensus was that we'd just keep pulpcore as our one and
>>> only Github Discussions instance, which would serve as a replacement for
>>> pulp-list and pulp-dev. I'd propose that we try this out for a bit and
>>> eventually decommission our mailing lists.
>>>
>>> [0] https://docs.github.com/en/discussions
>>> [1] https://github.com/pulp/pulpcore/discussions
>>> [2] https://github.com/vercel/next.js/discussions
>>>
>>> David
>>>
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://listman.redhat.com/mailman/listinfo/pulp-list
>
>
>
> --
> Grant Gainey
> Principal Software Engineer, Red Hat System Management Engineering
> ___
> Pulp-dev mailing list
> pulp-...@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulp mongodb upgrade

[Brian Bouterse]

Hi Venkataramana and Donal,

I looked at the logs, but unfortunately I don't see any exceptions related
to Pulp's use of Mongo. Mostly I just see a mongodb error:
`FileRenameFailed Renaming file /var/log/mongodb/mongodb.log to
/var/log/mongodb/mongodb.log.2021-05-27T07-33-08 failed; Cannot verify
whether destination already exists`.

I believe the pulp logs themselves would be needed.

-Brian


On Fri, May 28, 2021 at 12:18 PM Venkataramana Bora 
wrote:

>
> Hi Brian,
>
> Please find attached  MongoDB logs from /var/log/mongodb/. There are no
> logs in /var/log/pulp. Please let us know if any other logs required .
> Thanks a lot for your reply!
>
>
> Sincerely,
> Ramana Bora
>
>
>
> - Original message -
> From: Donal Keane/Ireland/IBM
> To: bmbou...@redhat.com
> Cc: pulp-list@redhat.com, Venkataramana Bora/India/IBM@IBM
> Subject: Re: [EXTERNAL] Re: [Pulp-list] pulp mongodb upgrade
> Date: Fri, May 28, 2021 2:19 AM
>
> Hi Brian,
>
> Thank you kindly for your response.
>
> The plan here originally was to move to Pulp 3 - and definitely still is.
> However, our brand within IBM has recently been sold off and it is
> happening sooner rather than later. As part of this acquisition we are
> losing a good proportion of expertise around our Linux operating systems.
> We have been tasked as a high priority to make sure all of our open source
> tools are patched to the latest security level before the acquisition goes
> through. This is why Ramana is attempting to get us to 2.21.x asap and from
> there, concentrate on getting to 3 when time and resources allow. I hope
> this explains our current situation and why any help would be graciously
> accepted!
>
> Ramana can you post relevant log info when you get a chance tomorrow?
>
> Thanks,
>
> Donal
>
>
> - Original message -
> From: Brian Bouterse 
> To: Venkataramana Bora 
> Cc: pulp-list , Donal Keane 
> Subject: [EXTERNAL] Re: [Pulp-list] pulp mongodb upgrade
> Date: Thu, May 27, 2021 8:12 PM
>
> I don't have ready advice, but it might be helpful to post a snippet of
> the logs. Without that, all we know is there is a 500 server error, not
> what the cause could be. Have you considered migrating to Pulp3? Pulp2 is
> in maintenance mode, ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
> ZjQcmQRYFpfptBannerEnd
> I don't have ready advice, but it might be helpful to post a snippet of
> the logs. Without that, all we know is there is a 500 server error, not
> what the cause could be.
>
> Have you considered migrating to Pulp3? Pulp2 is in maintenance mode, and
> likely has had its terminal release. Pulp3 is at 3.13 and has been out
> since Dec 2019. Here's more information on how to upgrade
> https://pulpproject.org/migrate-to-pulp-3/  We would be very interested
> in helping you do that.
>
> On Thu, May 27, 2021 at 12:03 PM Venkataramana Bora 
> wrote:
>
>
> Hi team,
> We are using pulp-server-2.21.5-1.el7 with
> mongodb-server-2.6.12-6.el7.x86_64 , this version of mongodb 2.6.12-16 got
> installed from our config manage server puppet .
> Time and again pulp-admin commands run, throwing below errors , I think it
> is due to old version of mongodb. After running /var/lib/mongodb --repair
> able to access pulp-admin commands for some time/few days as shown here
> below.
> Some one kindly help us on how to upgrade mongodb-server-2.6.12-6 to
> higher mongodb version that is compatible with pulp-server-2.21.5-1 ? This
> specific pulp-server we are using instead of higher versions due to some
> internal dependencies. Thanks a lot in advance.
>
> [root@drtp1opsndbx04 ~]# pulp-admin tasks list
> +--+
>  Tasks
> +--+
> There was an internal server error while trying to access the Pulp
> application.
> One possible cause is that the database needs to be migrated to the latest
> version. If this is the case, run pulp-manage-db and restart the services.
> More
> information may be found in Apache's log.
> root@drtp1opsndbx04 ~]# pulp-admin repo list
> +--+
>   Repositories
> +--+
> There was an internal server error while trying to access the Pulp
> application.
> One possible cause is that the database needs to be migrated to the latest
> version. If this is the case, run pulp-manage-db and rest

Re: [Pulp-list] pulp mongodb upgrade

[Brian Bouterse]

I don't have ready advice, but it might be helpful to post a snippet of the
logs. Without that, all we know is there is a 500 server error, not what
the cause could be.

Have you considered migrating to Pulp3? Pulp2 is in maintenance mode, and
likely has had its terminal release. Pulp3 is at 3.13 and has been out
since Dec 2019. Here's more information on how to upgrade
https://pulpproject.org/migrate-to-pulp-3/  We would be very interested in
helping you do that.

On Thu, May 27, 2021 at 12:03 PM Venkataramana Bora 
wrote:

>
> Hi team,
> We are using pulp-server-2.21.5-1.el7 with
> mongodb-server-2.6.12-6.el7.x86_64 , this version of mongodb 2.6.12-16 got
> installed from our config manage server puppet .
> Time and again pulp-admin commands run, throwing below errors , I think it
> is due to old version of mongodb. After running /var/lib/mongodb --repair
> able to access pulp-admin commands for some time/few days as shown here
> below.
> Some one kindly help us on how to upgrade mongodb-server-2.6.12-6 to
> higher mongodb version that is compatible with pulp-server-2.21.5-1 ? This
> specific pulp-server we are using instead of higher versions due to some
> internal dependencies. Thanks a lot in advance.
>
> [root@drtp1opsndbx04 ~]# pulp-admin tasks list
> +--+
>  Tasks
> +--+
> There was an internal server error while trying to access the Pulp
> application.
> One possible cause is that the database needs to be migrated to the latest
> version. If this is the case, run pulp-manage-db and restart the services.
> More
> information may be found in Apache's log.
> root@drtp1opsndbx04 ~]# pulp-admin repo list
> +--+
>   Repositories
> +--+
> There was an internal server error while trying to access the Pulp
> application.
> One possible cause is that the database needs to be migrated to the latest
> version. If this is the case, run pulp-manage-db and restart the services.
> More
> information may be found in Apache's log.
>
> After running /var/lib/mongodb --repair working a little while:
>
> [root@drtp1opsndbx04 ~]# pulp-admin tasks list
> +--+
>  Tasks
> +--+
> No tasks found
>
> [root@drtp1opsndbx04 ~]# pulp-admin repo list
> +--+
>   Repositories
> +--+
> Id:  virtualbox
> Display Name:virtualbox
> Description: virtualbox
> Content Unit Counts:
>   Rpm: 107
> .
>
>
>
>
> Sincerely,
> Venkataramana Bora
> IBM - TMS cloud operations
> Plot #1, Hill # 3, Apiic & Ites Sez, Rishikonda, Madhuravada
> Visakhapatnam, AP  530045, India.
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Pulpcore 3.11.2 Released

[Brian Bouterse]

Since Pulp is trying GitHub Discussions, the actual announcement is here:
https://github.com/pulp/pulpcore/discussions/1369
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Django v2.2.21 breaking changes

[Brian Bouterse]

tl;dr You should temporarily downgrade Django for now back to 2.2.20. See
#workaround below.

# The issue

It's unfortunate but Django's 2.2.21 breaks Pulp which given that Django
2.2.z is the Django LTS is extremely surprising to us. It actually broke a
lot of projects. I've been investigating this
https://pulp.plan.io/issues/8691. The 2.2.21 Django release has a bug in it
( https://code.djangoproject.com/ticket/32718 ) which is also actively
being investigated.  Based on testing I completed yesterday, even if Django
resolves their bug, Pulp will still need a compatibility release.

Overall I expect all of this to be worked out with the upcoming pulpcore
3.13.0, so my advice is to use the workaround below and then upgrade to
3.13.0 when it's released.

# The temporary workaround

Downgrade Django back to 2.2.20. You can do this manually by:

1. Activating the virtualenv of Pulp `source
/usr/local/lib/pulp/bin/activate`
2. Uninstalling your Django, `pip uninstall django`
3. pip install django==2.2.20

Feedback, questions, and suggestions are welcome.

-Brian

On Thu, May 6, 2021 at 1:50 AM Sathasivam, Pradeep <
pradeep.sathasi...@hpe.com> wrote:

> Hi All,
>
>
>
> We are using pulp project with below specified plug-in versions:
>
>
>
> {
>
>   "component": "pulpcore",
>
>   "version": "3.9.0"
>
> },
>
> {
>
>   "component": "pulp_rpm",
>
>   "version": "3.8.0"
>
> },
>
> {
>
>   "component": "pulp_file",
>
>   "version": "1.5.0"
>
> },
>
> {
>
>   "component": "pulp_container",
>
>   "version": "2.2.0"
>
> }
>
>
>
> Pulpcore-3.9.0 has dependency on Django with version ‘2.2.17’.
>
>
>
> Looks like Django-admin version ‘2.2.21’ has breaking changes with respect
> to ‘Remote to local’ repo sync call.
>
>
>
> There is no issue with till version ‘2.2.20’, but with ‘2.2.21’, sync call
> is failing with below error;
>
>
>
>
>
> "traceback": " File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line
> 1008, in perform_job\n rv = job.perform()\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 706,
> in perform\n self._result = self._execute()\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 729,
> in _execute\n result = self.func(*self.args, **self.kwargs)\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py\",
> line 274, in synchronize\n dv.create()\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\",
> line 148, in create\n loop.run_until_complete(pipeline)\n File
> \"/usr/lib64/python3.6/asyncio/base_events.py\", line 484, in
> run_until_complete\n return future.result()\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py\",
> line 225, in create_pipeline\n await asyncio.gather(*futures)\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py\",
> line 43, in __call__\n await self.run()\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py\",
> line 219, in run\n d_artifact.artifact for d_artifact in da_to_save\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/content.py\",
> line 82, in bulk_get_or_create\n return super().bulk_create(objs,
> batch_size=batch_size)\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py\",
> line 82, in manager_method\n return getattr(self.get_queryset(),
> name)(*args, **kwargs)\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py\",
> line 468, in bulk_create\n self._batched_insert(objs_with_pk, fields,
> batch_size, ignore_conflicts=ignore_conflicts)\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py\",
> line 1204, in _batched_insert\n ignore_conflicts=ignore_conflicts,\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py\",
> line 1186, in _insert\n return
> query.get_compiler(using=using).execute_sql(return_id)\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py\",
> line 1376, in execute_sql\n for sql, params in self.as_sql():\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django_readonly_field/compiler.py\",
> line 31, in as_sql\n return super(ReadonlySQLCompilerMixin,
> self).as_sql()\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py\",
> line 1320, in as_sql\n for obj in self.query.objs\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py\",
> line 1320, in \n for obj in self.query.objs\n File
> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py\",
> line 1319, in \n [self.prepare_value(field,
> self.pre_save_val(field, obj)) for field in fields]\n File
> 

Re: [Pulp-list] using ansible to install azure backend

[Brian Bouterse]

I agree with @daviddavis. I think the installer would benefit from docs on
how to configure django-storages for S3 and Azure as basic examples. I
think the process is roughly:

1) Make sure the pulpcore[azure] pip extras is installed in the
virtualenv:
https://github.com/pulp/pulpcore/blob/483abb5c4e8a048db4a8bd39f88c12617cf4b047/setup.py#L25-L30
2) Configure django-storages, which you can set via settings through the
pulp_settings var entirely through the

I believe (1) is not available in the installer today and I'm hoping
@mikedep333 can reply about what can be done there. As a workaround you
could install it manually into the virtualenv onto the target system, let
us know if you want more info on that.


On Wed, Mar 24, 2021 at 11:15 AM David Davis  wrote:

> To clarify, you're talking about having to install the django-storages
> packages[0]?
>
> I don't know if this exists today in the installer (maybe someone from the
> pulp_installer team can confirm) but it would be great to have support for
> setting up pulp to use non-fs storage backends.
>
> With using pip, I think you just need to be sure you're using the pulp
> virtualenv that the installer creates ($ workon pulp).
>
> [0]
> https://docs.pulpproject.org/pulpcore/installation/storage.html#configuring-pulp
>
> David
>
>
> On Tue, Mar 23, 2021 at 9:55 AM Briand, Sheldon <
> sheldon.bri...@nrc-cnrc.gc.ca> wrote:
>
>> Hi,
>>
>>
>>
>> I see instructions for using pip to install storage back ends.  I’m
>> wondering if there is any way to install the storage back ends using
>> ansible.  If I have used ansible to install pulp3 are there any tricks (or
>> dangers) I need to know when using pip to install the storage back ends?
>>
>>
>>
>> Thanks,
>>
>> -Sheldon
>>
>>
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://listman.redhat.com/mailman/listinfo/pulp-list
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulp 3.7.3 sync with checksum error

[Brian Bouterse]

h_size=batch_size)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/manager.py\",
>>> line 82, in manager_method\n return getattr(self.get_queryset(),
>>> name)(*args, **kwargs)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/query.py\",
>>> line 468, in bulk_create\n self._batched_insert(objs_with_pk, fields,
>>> batch_size, ignore_conflicts=ignore_conflicts)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/query.py\",
>>> line 1204, in _batched_insert\n ignore_conflicts=ignore_conflicts,\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/query.py\",
>>> line 1186, in _insert\n return
>>> query.get_compiler(using=using).execute_sql(return_id)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/sql/compiler.py\",
>>> line 1376, in execute_sql\n for sql, params in self.as_sql():\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django_readonly_field/compiler.py\",
>>> line 31, in as_sql\n return super(ReadonlySQLCompilerMixin,
>>> self).as_sql()\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/sql/compiler.py\",
>>> line 1320, in as_sql\n for obj in self.query.objs\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/sql/compiler.py\",
>>> line 1320, in \n for obj in self.query.objs\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/sql/compiler.py\",
>>> line 1319, in \n [self.prepare_value(field,
>>> self.pre_save_val(field, obj)) for field in fields]\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/sql/compiler.py\",
>>> line 1270, in pre_save_val\n return field.pre_save(obj, add=True)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/app/models/fields.py\",
>>> line 68, in pre_save\n return super().pre_save(model_instance, add)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/fields/files.py\",
>>> line 288, in pre_save\n file.save(file.name, file.file, save=False)\n
>>> File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/fields/files.py\",
>>> line 87, in save\n self.name = self.storage.save(name, content,
>>> max_length=self.field.max_length)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/files/storage.py\",
>>> line 52, in save\n return self._save(name, content)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/storages/backends/s3boto3.py\",
>>> line 447, in _save\n obj.upload_fileobj(content, ExtraArgs=params)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/boto3/s3/inject.py\",
>>> line 621, in object_upload_fileobj\n ExtraArgs=ExtraArgs,
>>> Callback=Callback, Config=Config)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/boto3/s3/inject.py\",
>>> line 539, in upload_fileobj\n return future.result()\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/s3transfer/futures.py\",
>>> line 106, in result\n return self._coordinator.result()\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/s3transfer/futures.py\",
>>> line 265, in result\n raise self._exception\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/s3transfer/tasks.py\",
>>> line 126, in __call__\n return self._execute_main(kwargs)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/s3transfer/tasks.py\",
>>> line 150, in _execute_main\n return_value = self._main(**kwargs)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/s3transfer/upload.py\",
>>> line 692, in _main\n client.put_object(Bucket=bucket, Key=key, Body=body,
>>> **extra_args)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/botocore/client.py\",
>>> line 357, in _api_call\n return self._make_api_call(operation_name,
>>> kwargs)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/botocore/client.py\",
>>> line 676, in _make_api_call\n raise error_class(par

Re: [Pulp-list] PulpCon 2021

[Brian Bouterse]

On Mon, Mar 15, 2021 at 11:12 AM David Davis  wrote:

> The topic of PulpCon came up today as spring is usually the time we begin
> to plan PulpCon. The main question I think is whether we should hold
> PulpCon again virtually this year or not.
>
> Optimally, we'd like to meet in person but given the uncertainty of our
> current situation, I think we should consider going virtual again this
> year. I noticed that other conferences such as DevConf.us (September 2021)
> are already planning to be virtual this year. And also, if we meet
> virtually this year, we could do an in-person PulpCon in early 2022 perhaps.
>
+1 to planning on virtual again for fall 2021. I think it allowed for more
attendance. +1 to also having an in-person event in early 2022.


> As for time frame, we'd need at least 3 months to prepare if it's virtual.
> So we'd be looking at sometime between July and December.
>
> Thoughts?
>
> David
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulp 3.7.3 sync with checksum error

[Brian Bouterse]

Did this happen inside a task? Did you see a traceback for it also?

On Fri, Mar 5, 2021 at 12:00 PM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> The sync process gave an error "A file failed validation due to checksum".
> Is this error caused by remote repo? Is there a way to find out which file
> cause the issue?
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Removing MD5 and SHA-1 as default available checksums in 3.11

[Brian Bouterse]

tl;dr With pulpcore 3.11, the plan is to remove MD5 and SHA-1 from the list
of default available checksums.  RPM and Migration plugin users will need
to add this back in at 3.11 upgrade time for your systems to continue
working. Please give on-list feedback on this change.

## Background

Pulp has the ALLOWED_CONTENT_CHECKSUMS setting [0] which, by default,
currently includes md5, sha-1, sha-224, sha-256, sha-384, and sha-512. Pulp
code is restricted to only using hashers from this list. This feature gives
admins the ability to prohibit hashers they do not trust. Pulp uses these
checksums for package integrity verification purposes when syncing and
publishing content.

## Motivation

We need to make Pulp secure by default. MD5 is known to be insecure, and
therefore it is unsafe for Pulp to allow its use for calculating package
integrity by default. SHA-1 is widely believed to be insecure, or will be
soon, and should not be allowed by default for the same reason.

## Proposal

Pulpcore 3.11 would remove md5 and sha-1 from the default list of allowed
checksums, leaving sha-224..sha-512. Specifically this change is occuring
in the `ALLOWED_CONTENT_CHECKSUMS` setting [0]. This is only a change to
the default settings; any specific system can be configured as desired.
Nothing is "being taken away".

## Required User Action with 3.11

We believe both RPM plugin users and Migration plugin users will be
impacted by this and mostly from the SHA-1 removal. SHA-1 is still used on
a variety of CDNs including Red Hat's. Also as data is migrated from Pulp2
systems, this also likely uses SHA-1 and MD5 as the migration plugin runs.

If users are using the defaults for `ALLOWED_CONTENT_CHECKSUMS` and want to
continue using SHA-1, they will need to update `ALLOWED_CONTENT_CHECKSUMS`
in their settings file. Alternatively, users will need to run
`pulpcore-manager handle-artifact-checksums` after upgrade to update any
existing artifacts after upgrading.

## Why not automate this?

We do not take manual user action at upgrade time lightly. However, this is
a security change, and we believe we need each Pulp system to opt-in for
themselves.

[0]:
https://docs.pulpproject.org/pulpcore/settings.html#allowed-content-checksums

Thanks!
The Pulpcore Team
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Possible to disable all client certificate checking?

[Brian Bouterse]

Hi Don,

Pulp by default doesn't use client certificate checking, so it's very
possible. Really though if you're using Pulp through Katello, this is
highly dependant on how Katello is a) configuring the webservers and b) if
they would allow that configuration to occur. Unfortunately I don't know
either of those things. :/

Sorry I can't be of more help.
-Brian


On Tue, Feb 9, 2021 at 10:08 AM Don Hoover  wrote:

> I am using pulp/katello and the way katello is setup is all "protected"
> repos are shared via https and setup with client cert checking
> (subscription-manager), while all "unprotected" repos are shared via
> unencrypted-http but no certificate checking.
>
> By default katello wants to use unprotected/unencrypted http for sharing
> "kickstart" repos for clients to access to boot off of during the first
> phase of kickstart.  Anaconda/kickstart can't use self-signed SSL certs
> which I assume is what they were thinking, but https it works fine for
> commercial certs.
>
> So I was wondering if there is a way to just disable all client cert
> checking and then I can point the kickstart at the protected copy of the
> repo instead.
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Apache 502 proxy errors when performing Pulp 3 to Pulp 3 syncs

[Brian Bouterse]

Hi Adam,

Thanks for sharing all of that and for opening that issue. Would you be
willing to do one or two more things to help move it forward? Could you
post the diff of the Apache config changes you made on the issue? Also if
you'd be willing to open a PR I think it would go against this file.
https://github.com/pulp/pulp_installer/blob/master/roles/pulp_webserver/templates/pulp-vhost.conf.j2
Can you link us to either of these you're able to do?

Regarding improving the content app's performance, @dalley has done some
investigation there and we hope to build on that work to make some
improvements.

Cheers,
Brian


On Fri, Feb 5, 2021 at 12:47 PM Winberg Adam  wrote:

> I raised a similar question here about pulp-content performance a short
> while ago, and also created https://pulp.plan.io/issues/8180 since there
> may be a lack of documentation about this.
>
>
> In short, I've made a number of adjustments and among them was setting the
> 'disablereuse=on' setting in my proxypass declaration. I have not seen
> any performance issue, probably because the proxypass target is on
> localhost and not a remote machine.
>
>
> //Adam
>
>
> --
> *From:* pulp-list-boun...@redhat.com  on
> behalf of Eric Helms 
> *Sent:* 05 February 2021 17:29
> *To:* pulp-list
> *Subject:* [Pulp-list] Apache 502 proxy errors when performing Pulp 3 to
> Pulp 3 syncs
>
> Howdy,
>
> Some quick background, over in the Katello project we deploy the
> pulpcore-content service via a unix socket with Apache serving as a reverse
> proxy. Today, we deploy pulpcore-content with two gunicorn workers.
> Tomorrow we are considering changing this to 2 * CPU + 1 per gunicorn
> documentation.
>
> The issue we are running into is intermittent 502s from Apache caused by
> being unable to make a connection to the underlying pulpcore-content app.
> This manifests itself primarily during a Pulp 3 to Pulp 3 sync. That is,
> when we sync our content proxy from the main servers Pulp 3. The sync can
> result in a large number of parallel connections back to the
> pulpcore-content application running on the main server.
>
> In an issue for aiohttp which is used by the project, and whose worker is
> used for gunicorn [1] they talk about the issues with Apache and aiohttp.
> In that issue there are two suggestions that I could extract:
>
>  1) set disablereuse=on the Apache reverse proxy declarations for the
> content app
>  2) change the default Apache worker type to be more like Nginx
>
> There are performance tradeoffs with #1, however, I do not fully grasp if
> they are relative to our primary use case when it comes to the content app.
>
> So, I am coming to the experts here to try to get some insight into what
> changes we should pursue to ensure the optimal default performance for our
> deployment. And to, as best as we can, limit these kind of intermittent
> failures to extreme cases. Because today, we see this intermittent failure
> with Pulp 3 running the same test suite we did with Pulp 2.
>
> Related, is there retry support built into syncing?
>
> Thanks!
>
> [1] https://github.com/aio-libs/aiohttp/issues/2687
>
> --
> Eric Helms
> Principal Software Engineer
> Satellite
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Content plugin for Conan packages

[Brian Bouterse]

Hi Marcin,

There isn't currently; the current list of plugins is here:
https://pulpproject.org/content-plugins/#pulp-3-content-plugins-information

Our goal is to make plugin writing as easy as possible, and to support them
through their development. If you know someone who is interested in
developing Conan support, have them reach out through
https://www.redhat.com/mailman/listinfo/pulp-dev or on #pulp-dev on IRC.

Cheers,
Brian


On Mon, Jan 18, 2021 at 5:54 AM Marcin Kotarba  wrote:

> Hi,
>
> Are there any plans for supporting Conan packages in Pulp3 server?
>
> Best regards!
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Database migration from 3.3 to 3.7

[Brian Bouterse]

Maybe it's selinux labeling related. What step is it stuck on in the
installer's output? What OS are you using?

On Sat, Dec 5, 2020 at 9:48 AM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> We are upgrading pulp from 3.3 to 3.7. Ansible stuck on database migration
> for more that 50 mins. We have 1.5T of artifact in file system. The
> database size is about 25G. Is this normal? Is there a way we can check the
> progress log?
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulp2 migration: AccessPolicy matching query does not exist

[Brian Bouterse]

I've filed this issue tracking the improvement which would allow users to
run `flush` and not experience this problem.

https://pulp.plan.io/issues/7710

On Wed, Oct 14, 2020 at 12:14 PM Tatiana Tereshchenko 
wrote:

> Adam,
> I agree we lack documentation on resetting the environment and the
> database, probably because we never ask to wipe out the database for
> upgrades.
> The instructions are usually provided with release and ask you basically
> to run the latest pulp_installer.
>
> There is some data which is provided with migrations and which has to be
> present in the database, that's why dropping the database and applying
> migrations work and `flush` does not.
> `flush` just removes data and keeps all the tables in place, so migrations
> are not re-applied.
> So for now, please do not use `flush` if you want to start using pulp 3 db
> from scratch. We potentially can provide a separate command or find some
> other way to fill in the essential data back.
> Please file an issue in our tracker if such feature/command is helpful for
> you. https://pulp.plan.io/projects/pulp/issues/new
>
> As a side note, if you are interested, reset_db command drops database.
> It's provided as a part of django-extensions package
> https://django-extensions.readthedocs.io/en/latest/command_extensions.html
>
> I'm glad that the migration  runs for you now,
> Tanya
>
>
> On Wed, Oct 14, 2020 at 3:52 PM Winberg Adam  wrote:
>
>> Thanks for the reply!
>>
>>
>> I use 'flush' to clear the db, I don't have a 'reset_db' command.
>> Otherwise that's pretty much my process.
>>
>>
>> The migrate command returns 'No migrations to apply'.
>>
>> Running 'pulpcore-manager show-migrations' shows that all migrations,
>> including 'guardian.0001/guardian.0002' has checkmarks ([X]).
>>
>> guardian
>>  [X] 0001_initial
>>  [X] 0002_generic_permissions_index
>>
>> The creation of the migration plan works so I assume my admin user is
>> ok.
>>
>>
>> I am using an rpm based installation on RHEL8, with
>>
>> python3-pulp-2to3-migration-0.4.0-1.el8.noarch
>> python3-pulp-rpm-3.7.0-1.el8.noarch
>> python3-pulpcore-3.7.1-3.el8.noarch
>>
>>
>> I don't know what went wrong, but I surrendered and dropped my DB and
>> redid the migrations from scratch - and now it works.
>> Is there a documented instruction on upgrading existing installations?
>>
>> //Adam
>>
>>
>>
>> --
>> *From:* Tatiana Tereshchenko 
>> *Sent:* 14 October 2020 14:15
>> *To:* Winberg Adam
>> *Cc:* pulp-list@redhat.com
>> *Subject:* Re: [Pulp-list] pulp2 migration: AccessPolicy matching query
>> does not exist
>>
>>
>>
>> On Wed, Oct 14, 2020 at 2:12 PM Tatiana Tereshchenko 
>> wrote:
>>
>>> Hi Adam,
>>>
>>> My understanding is that you did the following:
>>>  * stop pulp services
>>>  * pulpcore-manager (or django-admin) reset_db
>>>  * pulpcore-manager migrate
>>>  * pulpcore-manager reset-admin-password --password password
>>>  * start services
>>>  * http POST :/pulp/api/v3/migration-plans/ < your_migraiton_plan.json
>>>  * http POST
>>> :/pulp/api/v3/migration-plans/48d03a72-96a1-4d36-9f8b-9a57e97846ef/run/
>>>
>>
>> Sent too early :)
>> I can't reproduce it so far, so any hints about what can be special about
>> your environment or installation would be appreciated.
>> Make sure that you have at least one user which has admin privileges and
>> that the guardian migrations ran indeed.
>>   Applying guardian.0001_initial... OK
>>   Applying guardian.0002_generic_permissions_index... OK
>>
>> Tanya
>>
>>
>>
>>
>>
>>>
>>> On Wed, Oct 14, 2020 at 8:02 AM Winberg Adam 
>>> wrote:
>>>
 Hello,


 so I updated my pulp3 installation from 3.4 to 3.7 and tried to rerun
 my pulp2 migration - but it errors out with "AccessPolicy matching
 query does not exist". Anyone know why?


 I flushed my db, reran the 'migrate' job, created a pulp2migration plan
 (which worked fine) and then tried to run it. Here's the complete error:


 Oct 14 05:43:26  gunicorn[2150852]: pulp: django.request:ERROR:
 Internal Server Error:
 /pulp/api/v3/migration-plans/48d03a72-96a1-4d36-9f8b-9a57e97846ef/run/
 Oct 14 05:43:26  gunicorn[2150852]: Traceback (most recent call last):
 Oct 14 05:43:26  gunicorn[2150852]:   File
 "/usr/lib/python3.6/site-packages/django/core/handlers/exception.py", line
 34, in inner
 Oct 14 05:43:26  gunicorn[2150852]: response = get_response(request)
 Oct 14 05:43:26  gunicorn[2150852]:   File
 "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 115,
 in _get_response
 Oct 14 05:43:26  gunicorn[2150852]: response =
 self.process_exception_by_middleware(e, request)
 Oct 14 05:43:26  gunicorn[2150852]:   File
 "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 113,
 in _get_response
 Oct 14 05:43:26  gunicorn[2150852]: response =
 wrapped_callback(request, *callback_args, **callback_kwargs)

Re: [Pulp-list] [Pulp-dev] Pulp 3 CLI PoC Feedback

[Brian Bouterse]

I'm excited to use this. I went to try it, but the CLI's requests wouldn't
trust the server which uses self-signed certificates. When running `pulp
status` I get this error:

requests.exceptions.SSLError: HTTPSConnectionPool(host='localhost',
port=443): Max retries exceeded with url: /pulp/api/v3/docs/api.json
(Caused by SSLError(SSLCertVerificationError(1, '[SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local
issuer certificate (_ssl.c:1123)'))

Let me know if I should file this somewhere more specifically.

On Mon, Oct 5, 2020 at 5:08 AM Melanie Corr  wrote:

> Hi all,
>
> An initial Pulp 3 CLI PoC is now available. We are hoping to gather as
> much feedback as possible from the Pulp community so that future
> development can progress with your opinions and requirements in mind.
>
> The following blog post has instructions on how to download and install
> the Pulp 3 CLI PoC, as well as a demo and a feedback link. Please test out
> the Pulp 3 CLI PoC and let us know your thoughts. We will have some SWAG
> for participants :)
>
> https://pulpproject.org/2020/09/28/pulp-3-cli-poc-call-for-feedback/
>
> If you've any questions or comments, feel free to ask.
>
> Melanie
>
> --
>
> Melanie Corr, RHCE
>
> Community Manager
>
> Red Hat 
>
> Remote, Ireland
>
> mc...@redhat.com
> M: +353857774436 IM: mcorr
> 
>
> ___
> Pulp-dev mailing list
> pulp-...@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] pulp-certguard 1.0.3 is Generally Available

[Brian Bouterse]

pulp-certguard 1.0.3 has been released. It is compatible with pulpcore 3.3
through 3.8.

This is a strict compatibility release and has no meaningful changes except
declaring compatibility with pulpcore through 3.8.

PyPI: https://pypi.org/project/pulp-certguard/1.0.3/
Changelog:
https://github.com/pulp/pulp-certguard/blob/master/CHANGES.rst#103-2020-09-25
Docs: https://pulp-certguard.readthedocs.io/
Python bindings: https://pypi.org/project/pulp-certguard-client/1.0.3/
Ruby bindings:
https://rubygems.org/gems/pulp_certguard_client/versions/1.0.3/
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] pulp_ansible 0.4.0 is Generally Available

[Brian Bouterse]

pulp_ansible 0.4.0 has been released. It is compatible with pulpcore 3.7
and pulpcore 3.8 (not yet released).

It contains only bugfixes, but we don't want to change the pulpcore version
a plugin uses in the z-stream. See the changelog for all changes (link
below).

PyPI: https://pypi.org/project/pulp-ansible/0.4.0/
Changelog: https://pulp-ansible.readthedocs.io/en/latest/changes.html#id1
Docs: https://pulp-ansible.readthedocs.io/
Python bindings: https://pypi.org/project/pulp-ansible-client/0.4.0/
Ruby bindings: https://rubygems.org/gems/pulp_ansible_client/versions/0.4.0
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] pulpcore 3.7.0 and pulp_file 1.3.0 are Generally Available

[Brian Bouterse]

Pulpcore 3.7.0[0] and pulp_file 1.3.0[1] have been released.

Details of the most important changes are available in our blog[2]. For a
full list of changes, please check the changelog for pulpcore[3] and
pulp_file[4].

Starting with pulpcore 3.7 plugins can now be compatible with both 3.y and
3.y+1, this should allow users to upgrade pulpcore more easily without
waiting for every plugin you use to release a compatibility release.

# Installation and Upgrade

Users should use the 3.7.0 release of pulp_installer[5] to install or
upgrade their installations. This version of the installer will check
compatibility of all installed plugins with pulpcore 3.7. The installer
will abort if any plugin is incompatible.

The pulp_installer collection can be installed from Ansible Galaxy with the
following command:

ansible-galaxy collection  install --force pulp.pulp_installer

The --force flag will upgrade the collection if you had a previous version
installed.

# Plugin API

Plugin writers can see the API changes here[6].

A significant change includes the 1-cycle deprecation policy for
pulpcore.plugin which allows you to safely declare dependency on
pulpcore==3.y+1 when you declare dependency on pulpcore==3.y.

With this change, the recommended strategy is to pin plugins to a 3.y and
3.y+1 version of pulpcore. So for a compatibility release with 3.7 use:
"pulpcore>=3.7,<3.9".

# Thanks

Thank you to everyone who worked so hard to make this release possible!

[0] https://pypi.org/project/pulpcore/3.7.0/
[1] https://pypi.org/project/pulp-file/1.3.0/
[2] https://pulpproject.org/2020/09/20/pulp-3.7.0-is-generally-available/
[3] https://docs.pulpproject.org/en/3.7/changes.html#id1
[4] https://pulp-file.readthedocs.io/en/latest/changes.html#id1
[5] https://galaxy.ansible.com/pulp/pulp_installer
[6] https://docs.pulpproject.org/en/3.7/changes.html#plugin-api
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Simple use case of mirroring external repos in pulp3

[Brian Bouterse]

@Jim Davis   My takeaway from this is that we need a
verbatim mirror feature for pulp_rpm. Thanks for sharing that use case.

On Tue, Sep 22, 2020 at 11:28 AM Jim Davis  wrote:

> I do think it is important that it is possible to have a verbatim repo
> replication happen in a true mirror fashion rather than massaging it. We
> have audits we need to pass and a file by file “bits is bits” comparison
> from what we have and what is available in the official repo may likely
> fail.
>
>
>
> I have been looking at the workflows docs and all the associated script
> samples over the last few days. It is an interesting read but no where near
> the rpm repo create and rpm repo sync that will give me an exact mirror
> (honestly, I was looking for the easy button to mirror an external repo).
> My requirements are also that I sync only when the patching process needs
> refreshed and everything gets tested with that snapshot. Outside of that
> manual sync, the repo would remain static.
>
>
>
> I haven’t looked at squeezer yet but I will. Everything else I do is
> ansible based.
>
>
>
> Many thanks!
>
>
>
> Jim
>
>
>
> *From:* Matthias Dellweg 
> *Sent:* Tuesday, September 22, 2020 5:11 AM
> *To:* Jim Davis 
> *Cc:* pulp-list@redhat.com
> *Subject:* Re: [Pulp-list] Simple use case of mirroring external repos in
> pulp3
>
>
>
> Hi Jim,
>
> thanks for reaching out. I think the use case you are describing here is
> best described in
> https://pulp-rpm.readthedocs.io/en/latest/workflows/create_sync_publish.html#sync-publish-workflow
> .
>
> And yes it is necessary to sync, publish and distribute that repository to
> have a mirror. Also this mirror will AFAIK republish the synched repository
> in its own layout. (Anyone, please help me out if i'm wrong!)
>
> We have recently discussed the possibility to republish synched repository
> versions in a verbatim manner (including all files and original metadata in
> their original relative location). Let us know if that is an important use
> case for you.
>
> As of the CLI for pulp3, it is in a very ealy poc/planning phase. But
> personally, i would love to hear if you made any experience with using the
> ansible modules in https://galaxy.ansible.com/pulp/squeezer (also
> incomplete, but there are some for use with RPM facilities).
>
>   Matthias
>
>
>
> On Mon, Sep 21, 2020 at 10:27 PM Jim Davis  wrote:
>
> For some time now I have built up several pulp2 mirrors of various
> upstream rpm repos to make local updates run faster and to segregate
> environments. I do not require client registration so it is a dirt simple
> setup for basic OS patching. It is trivial to rebuild my rpm repositories
> if needed and to sync them as needed via scripts using pulp_admin.
>
>
>
> I’m trying to figure out how to do this with pulp3 – and I’m still a bit
> puzzled. The pulp3 ansible deploy method is really quite slick. One issue
> down. Simply mirroring outside repos is quite common with pulp2 usage and
> documented everywhere in the webnets but not so much for pulp3. I realize
> the CLI went away but surely there must be some pulp2 to pulp3 task
> equivalence somewhere or maybe someone has real world examples written up
> already and hasn’t yet published them. Searching/reading/searching gives me
> lots of information on complex issues around document versioning, history,
> plugins, API usage, pulp3 architecture, etc. For me this makes a simple
> migration to pulp3 more difficult for probably the easiest use case out
> there. Am I too early to the game to consider a move or is it all in front
> of me but in a million pieces!
>
>
>
> Thanks
>
>
>
> Jim
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] pulpcore 3.6.4 is Generally Available

[Brian Bouterse]

pulpcore 3.6.4 is available on PyPI[0]. This release fixes bindings issues
caused by a dependency that released backwards incompatible changes[1].

# Installation and Upgrade

Users should use the 3.6.4 release of pulp_installer[2] to install or
upgrade their installations. This version of the installer will check
compatibility of all installed plugins with pulpcore 3.6. The installer
will abort if any plugin is incompatible.

The pulp_installer collection can be installed from Ansible Galaxy with the
following command:

ansible-galaxy collection  install --force pulp.pulp_installer

The --force flag will upgrade the collection if you had a previous version
installed.

# Plugin API

There are no plugin API changes.

# Client libraries

Both the Python[3] and Ruby[4] clients are available also.

[0] https://pypi.org/project/pulpcore/3.6.4/
[1] https://docs.pulpproject.org/pulpcore/en/3.6.4/changes.html#id1
[2] https://galaxy.ansible.com/pulp/pulp_installer
[3] https://pypi.org/project/pulpcore-client/3.6.4/
[4] https://rubygems.org/gems/pulpcore_client/versions/3.6.4/
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Can't get CentOS 7 Updates repo published

[Brian Bouterse]

I don't know much about it, but I believe the sqlite db is used by some
client types in some cases when fetching packages from Pulp. IIRC, if the
sqlite db is not present it falls back to a less-efficient form of package
fetching from Pulp.

On Tue, Sep 1, 2020 at 4:53 AM Konstantin M. Khankin <
khankin.konstan...@gmail.com> wrote:

> Thank you, that obviously helped. Still would be useful to know the root
> cause.
>
> OTOH, if sqlite creation functionality is not that critical - why would we
> ever enable something which only consumes time during sync for no benefit?
>
> вт, 1 сент. 2020 г. в 00:26, Dennis Kliban :
>
>> Looks like everything is up to date. I have no idea what the root cause
>> is, but according to this comment[0], you can work around the problem by
>> disabling the generation of sqlite db. I am not sure what the exact effect
>> of this will be for the clients that are consuming this content, but the
>> repository will be usable.
>>
>> [0] https://pulp.plan.io/issues/2019#note-2
>>
>> On Mon, Aug 31, 2020 at 4:10 PM Konstantin M. Khankin <
>> khankin.konstan...@gmail.com> wrote:
>>
>>> # rpm -qa | grep pulp | sort
>>> pulp-admin-client-2.21.3-1.el7.noarch
>>> pulp-agent-2.21.3-1.el7.noarch
>>> pulp-consumer-client-2.21.3-1.el7.noarch
>>> pulp-deb-admin-extensions-1.10.1-1.el7.noarch
>>> pulp-deb-plugins-1.10.1-1.el7.noarch
>>> pulp-docker-admin-extensions-3.2.6-1.el7.noarch
>>> pulp-docker-plugins-3.2.6-1.el7.noarch
>>> pulp-puppet-consumer-extensions-2.21.3-1.el7.noarch
>>> pulp-puppet-handlers-2.21.3-1.el7.noarch
>>> pulp-python-admin-extensions-2.0.4-1.el7.noarch
>>> pulp-python-plugins-2.0.4-1.el7.noarch
>>> pulp-rpm-admin-extensions-2.21.3-1.el7.noarch
>>> pulp-rpm-consumer-extensions-2.21.3-1.el7.noarch
>>> pulp-rpm-handlers-2.21.3-1.el7.noarch
>>> pulp-rpm-plugins-2.21.3-1.el7.noarch
>>> pulp-rpm-yumplugins-2.21.3-1.el7.noarch
>>> pulp-selinux-2.21.3-1.el7.noarch
>>> pulp-server-2.21.3-1.el7.noarch
>>> python-pulp-agent-lib-2.21.3-1.el7.noarch
>>> python-pulp-bindings-2.21.3-1.el7.noarch
>>> python-pulp-client-lib-2.21.3-1.el7.noarch
>>> python-pulp-common-2.21.3-1.el7.noarch
>>> python-pulp-deb-common-1.10.1-1.el7.noarch
>>> python-pulp-docker-common-3.2.6-1.el7.noarch
>>> python-pulp-oid_validation-2.21.3-1.el7.noarch
>>> python-pulp-puppet-common-2.21.3-1.el7.noarch
>>> python-pulp-python-common-2.0.4-1.el7.noarch
>>> python-pulp-repoauth-2.21.3-1.el7.noarch
>>> python-pulp-rpm-common-2.21.3-1.el7.noarch
>>>
>>> # rpm -qa | grep createrepo
>>> createrepo-0.9.9-28.el7.noarch
>>> createrepo_c-0.10.0-20.el7.x86_64
>>> createrepo_c-libs-0.10.0-20.el7.x86_64
>>>
>>> # cat /etc/redhat-release
>>> CentOS Linux release 7.8.2003 (Core)
>>>
>>> пн, 31 авг. 2020 г. в 22:48, Dennis Kliban :
>>>
 This looks exactly like the issue that was reported here[0].

 What version of pulp are you using? What version of createrepo_c is
 installed?

 [0] https://pulp.plan.io/issues/2019

 On Mon, Aug 31, 2020 at 3:16 PM Konstantin M. Khankin <
 khankin.konstan...@gmail.com> wrote:

> Hi!
>
> The issue still persists. Could someone take a look please?
>
> Thanks!
>
> ср, 19 авг. 2020 г., 13:15 Konstantin M. Khankin <
> khankin.konstan...@gmail.com>:
>
>> Hi!
>>
>> I found that my pulp2-managed mirror of
>> http://mirror.yandex.ru/centos/7/updates/x86_64 has not been
>> successfully published since April 30. I ran publish manually and 
>> received
>> an error:
>>
>> '''
>> Generating sqlite files
>> [/]
>> ... failed
>> Error occurred during 'sqliterepo_c' execution: Preparing sqlite
>> DBs
>>
>> ::
>> C_CREATEREPOLIB: Critical: cr_xml_parser_generic: parsing error
>> '/var/cache/pulp/reserved_resource_worke...@hive.gsk.loc
>> /12257118-33e7-4294-ad68
>>
>> -98e0515f2627/repodata/9e958c09c7880d130ef3321332000f43a4e4e701e416801a0f54313f5
>> 766d03b-filelists.xml.gz': not well-formed (invalid token)
>> Parse error
>> '/var/cache/pulp/reserved_resource_worke...@hive.gsk.loc
>> /12257118-33e7-4294-ad68
>>
>> -98e0515f2627/repodata/9e958c09c7880d130ef3321332000f43a4e4e701e416801a0f54313f5
>> 766d03b-filelists.xml.gz' at line: 1884123 (not well-formed (invalid
>> token))
>> '''
>>
>> I can't open .sqlite files in /tmp from CLI either:
>>
>> '''
>> -rw---. 1 apache apache  54066176 Aug 19 15:12
>> filelists.211JP0.sqlite
>> -rw---. 1 apache apache 157097984 Aug 19 15:12
>> primary.PQ3JP0.sqlite
>> -rw---. 1 apache apache 0 Aug 19 15:12
>> other.X201JP0.sqlite
>>
>> [root@hive tmp]# sqlite3 primary.PQ3JP0.sqlite
>> SQLite version 3.7.17 2013-05-20 00:56:22
>> Enter ".help" for instructions
>> Enter SQL statements terminated with a ";"
>> sqlite> .databases
>> Error: file is encrypted or is not a database
>> '''
>>
>> I 

Re: [Pulp-list] pulpcore 3.6.0 and pulp_file 1.2.0 are generally available

[Brian Bouterse]

pulp-certguard 1.0.2 has been released. It is compatible with pulpcore 3.6.

This is a strict compatibility release and has no meaningful changes except
declaring compatibility with 3.6

PyPI: https://pypi.org/project/pulp-certguard/1.0.2/
Changelog:
https://github.com/pulp/pulp-certguard/blob/master/CHANGES.rst#102-2020-08-18
Docs: https://pulp-certguard.readthedocs.io/
Python bindings: https://pypi.org/project/pulp-certguard-client/1.0.2/
Ruby bindings:
https://rubygems.org/gems/pulp_certguard_client/versions/1.0.2

On Tue, Aug 18, 2020 at 9:14 AM Matthias Dellweg 
wrote:

> pulp_container 2.0.0 has been released. It is compatible with pulpcore 3.6.
>
> For the full list of features and bug fixes, see the changelog.
> There is a blogpost with highlights:
> https://pulpproject.org/2020/08/18/pulp-container-2.0-is-generally-available/
>
> PyPI: https://pypi.org/project/pulp-container/2.0.0/
> Changelog:
> https://pulp-container.readthedocs.io/en/latest/changes.html#id1
> Docs: https://pulp-container.readthedocs.io/
> Python bindings: https://pypi.org/project/pulp-container-client/2.0.0/
> Ruby bindings:
> https://rubygems.org/gems/pulp_container_client/versions/2.0.0
>
> On Tue, Aug 18, 2020 at 9:32 AM Matthias Dellweg 
> wrote:
>
>> Following the release of pulpcore 3.6 that had breaking changes around
>> the api documentation (used by those ansible modules) we have just
>> released version 0.0.3 of squeezer [0].
>> This new release can talk to both pulp 3.5 (using swagger/openapiv2)
>> and 3.6 (using openapiv3).
>> We will try to keep that compatibility around until pulpcore 3.7 arrives.
>>
>> [0] https://galaxy.ansible.com/pulp/squeezer
>>
>>
>> On Mon, Aug 17, 2020 at 10:23 PM Fabricio Aguiar <
>> fabricio.agu...@redhat.com> wrote:
>>
>>> pulp_ansible 0.2.0 has been released. It is compatible with pulpcore 3.6.
>>>
>>> For the full list of features and bug fixes, see the changelog.
>>> Here are some highlights:
>>>  * Allow a Remote to be associated with a Repository
>>>  * Changed the role remote path
>>>  * Replaced Artifact with PulpTemporaryFile on import_collection task
>>>
>>> PyPI: https://pypi.org/project/pulp-ansible/0.2.0/
>>> Changelog:
>>> https://pulp-ansible.readthedocs.io/en/latest/changes.html#id1
>>> Docs: https://pulp-ansible.readthedocs.io/
>>> Python bindings: https://pypi.org/project/pulp-ansible-client/0.2.0/
>>> Ruby bindings:
>>> https://rubygems.org/gems/pulp_ansible_client/versions/0.2.0
>>>
>>> Best regards,
>>> Fabricio Aguiar
>>> Software Engineer, Pulp Project
>>> Red Hat Brazil - Latam 
>>> +55 11 999652368
>>>
>>>
>>> On Mon, Aug 17, 2020 at 2:53 PM Tatiana Tereshchenko <
>>> ttere...@redhat.com> wrote:
>>>
 pulp_rpm 3.6.0 has been released. It is compatible with pulpcore 3.6.

 For the full list of features and bug fixes, see the changelog.
 Here are some highlights:
  * Improved support for advisories
  * Added import/export support for distribution trees
  * Added import/export support for advisories
  * Added ability to associate a remote with a repository

 PyPI: https://pypi.org/project/pulp-rpm/3.6.0/
 Changelog: https://pulp-rpm.readthedocs.io/en/3.6/changes.html#id1
 Docs: https://pulp-rpm.readthedocs.io/
 Python bindings: https://pypi.org/project/pulp-rpm-client/3.6.0/
 Ruby bindings:
 https://rubygems.org/gems/pulp_rpm_client/versions/3.6.0/

 On Fri, Aug 14, 2020 at 1:35 AM Dennis Kliban 
 wrote:

> Pulpcore 3.6.0[0] and pulp_file 1.2.0[1] have been released.
>
> Details of the most important changes are available in our blog[2].
> For a full list of changes, please check the changelog for pulpcore[3] and
> pulp_file[4].
>
> There are breaking changes for users of the client bindings and
> OpenAPI schema users.
>
> # Installation and Upgrade
>
> Users should use the 3.6.0 release of pulp_installer[5] to install or
> upgrade their installations. This version of the installer will check
> compatibility of all installed plugins with pulpcore 3.6. The
> installer will abort if any plugin is incompatible.
>
> The pulp_installer collection can be installed from Ansible Galaxy
> with the following command:
>
> ansible-galaxy collection  install --force pulp.pulp_installer
>
> The --force flag will upgrade the collection if you had a previous
> version installed.
>
> # Plugin API
>
> Plugin writers can see the API changes here[6].  Some highlights:
>
>  * Role Based Access Control support.
>
> And in keeping with the recommended strategy to pin plugins to a 3.y
> version of pulpcore, plugins should release compatibility releases with
> 3.5 as soon as they can. We recommend using "pulpcore>=3.6,<3.7".
>
> [0] https://pypi.org/project/pulpcore/3.6.0/
> [1] https://pypi.org/project/pulp-file/1.2.0/
> [2]
> 

Re: [Pulp-list] Pulp 3 distribution trees

[Brian Bouterse]

If you have a use case you'd like to share that is valuable to you, please
do!

On Thu, Jul 23, 2020 at 11:04 AM Dennis Kliban  wrote:

> Distribution trees are currently considered a tech preview feature. There
> are multiple bugs that are actively being fixed at this time[0,1,2]. There
> is currently no way for users to create new Distribution Trees. They can
> only be synced from remote repositories.
>
> [0] https://pulp.plan.io/issues/7150
> [1] https://pulp.plan.io/issues/7115
> [2] https://pulp.plan.io/issues/7046
>
>
> On Mon, Jul 20, 2020 at 4:21 PM Bin Li (BLOOMBERG/ 120 PARK) <
> bli...@bloomberg.net> wrote:
>
>> Hi,
>> I am trying to understand how to create a distribution tree. I was able
>> to create a repo and sync a kickstart tree to this repo. Should I create a
>> publication and a distribution, then use the distribution base_url to
>> access the the tree just as a regular repo? Is there anything else we need
>> add to distribution tree? Can we create a distribution tree from a iso or
>> local directory?
>>
>> Thanks
>> Bin
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] pulp-certguard 1.0.1 is available

[Brian Bouterse]

pulp-certguard 1.0.1 is available on PyPI. This release is the same as
1.0.0, but now compatible with pulpcore 3.5.0

Changelog:
https://github.com/pulp/pulp-certguard/blob/master/CHANGES.rst#101-2020-07-20
PyPI: https://pypi.org/project/pulp-certguard/1.0.1/
Docs: https://pulp-certguard.readthedocs.io/en/latest/
Python bindings: https://pypi.org/project/pulp-certguard-client/1.0.1/
Ruby bindings:
https://rubygems.org/gems/pulp_certguard_client/versions/1.0.1
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] General Availability of pulp-certguard 1.0.0!

[Brian Bouterse]

Certificate based client authorization is now generally available with
pulp-certguard 1.0.0. The pypi release is here [0] and the docs are here
[1]*. There are no changes since 0.1.0rc5 so there is no meaningful, new
changelog to see. The 1.0.0 release is compatible with pulpcore 3.3.z and
3.4.z.

The existing installer can install/upgrade this plugin if your pulpcore is
at 3.3.z or 3.4.z. See the "pulp-certguard" example entry in the installer
quickstart [2].

The bump from 0.1.0 to 1.0.0 is intentional to indicate the pulp-certguard
is now stable and ready for production use. It's gone through a lot of
testing with Katello's use and also has many functional tests.

Thanks to everyone who has contributed to this release. Please report bugs
and issues here [3]. General questions are best sent to this mailing list
or to #pulp on freenode IRC.

[0]: https://pypi.org/project/pulp-certguard/
[1]:
https://pulp-certguard.readthedocs.io/en/latest/changes.html#rc5-2020-05-22
[2]: https://pulp-installer.readthedocs.io/en/latest/quickstart/
[3]: https://pulp.plan.io/projects/certguard/issues/new

*: The docs are erroring in the build on readthedocs, but the changelog is
actually empty since there have not been changes since 0.1.0rc5. See the
actual changelog here:
https://github.com/pulp/pulp-certguard/blob/master/CHANGES.rst#100-2020-07-01
We will work on resolving this.

Cheers,
Brian
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Tuning Pulpcore Worker Count in Katello

[Brian Bouterse]

On Mon, Jun 15, 2020 at 2:27 PM Eric Helms  wrote:

> Are Pulp 3 workers running in a threaded manner ? There is a related
> concern, that is more of a concern for Katello installations, around the
> number of database connections needed to prevent starvation and ensure
> PostgreSQL is tuned correctly to handle this. For Foreman/Katello tihs
> means we need to count across Foreman, Foreman's task handler, Candlepin
> and Pulp connections.
>
Generally there are no threads but there are caveats. First, I don't know
of any Pulp code that uses threads, but it's possible code in the future
would. I know of no plans currently though. Second, aiofiles (a dependency
of Pulp) does use threads to move files around. These threads should not
make db connections though. Third, plugins can ship their own tasks that
can be run so this isn't entirely in our control.



>
> If you can also speak to the requirements for API and Content app that
> would be helpful as well rounding out the information.
>
Sure. The Content App and the API are expected to not load system resources
significantly, so for those it's mainly about how much capacity you want.
For example, if you want to serve lots of clients binary data yourself with
the content app (when not using S3 or Azure like pulp supports), then
you'll want "more" content app workers. For the API, the capacity there is
about the # API operations per second. More workers the more concurrent API
operations per second you can do. Generally I expect you'll want a small
number of API workers and a variable (potentially large) number of content
app workers. You'll also need to make sure your reverse proxy configs can
match the connection throughput desired also.

Workers in these cases are typically gunicorn workers in a single gunicorn
process, but to really scale a system we'll need to have both more gunicorn
workers per proces to vertically scale on a single node, and more gunicorn
processes themselves (each with N gunicorn workers) to horizontally scale.
Both vertical and horizontal scaling can be deployed manually and fully
today. The installer currently can vertically scale the number of content
and API gunicorn workers per gunicorn process, we cannot currently perform
clustered installations for horizontal scaling well. We are working on that
clustered install capability now actually.

More questions are welcome; this topic is important and can be a bit
complicated with our lack of docs. :/

>
> On Mon, Jun 15, 2020 at 2:14 PM Brian Bouterse 
> wrote:
>
>>
>>
>> On Mon, Jun 15, 2020 at 1:09 PM William Clark  wrote:
>>
>>> Hello Pulp Community!
>>>
>>> I'm working on a feature to allow the foreman-installer to set the
>>> number of Pulpcore workers deployed on Katello or Content Proxy, and I
>>> require some assistance from the Pulp community in setting sane defaults
>>> and limits.
>>>
>>> With Pulp 2 in Katello, the default behavior was that the worker count
>>> would match the number of logical CPUs up to a soft limit of 8. We advised
>>> that users could tune the worker count higher but it was expected to cause
>>> performance degradation in most cases due to I/O blocking. The largest
>>> scale installation to my knowledge uses a Pulp 2 worker count of 16.
>>>
>> I believe having one pulpcore-worker per CPU is still a good practice. We
>> haven't gotten a lot of feedback on right-sizing an installation so I won't
>> claim it's the absolute best practice, but it's what I recommend currently.
>> We have an issue to document sizing recommendations here that also has some
>> similar/more info:  https://pulp.plan.io/issues/6856
>>
>> The I/O blocking concern is roughly about the same as Pulp2; during sync
>> operations the workload could be I/O bound.
>>
>> A lot more CPU processing has been moved into postgresql which auto forks
>> postgresql processes per client connection, which in this case is a 1-1
>> pairing with each pulpcore-worker. So when it's under heavy load I expect
>> postresql to scale out a process and the workload could become constrained
>> on the postgresql CPU itself. In that case, lowering the worker to half of
>> the available processors would likely improve throughput. Moving postgresql
>> to another, dedicated box is also an option.
>>
>> If you'd be willing to share your findings with the Pulp community that
>> would be really great.
>>
>>
>>> With Pulp 2 being replaced and rebuilt with Pulpcore, I'm looking to
>>> understand what are the tuning best practices for the new technology, so
>>> that we could apply them to Katello.
>>>
>> Please let us know what other questions we can help answer.
&

Re: [Pulp-list] Tuning Pulpcore Worker Count in Katello

[Brian Bouterse]

On Mon, Jun 15, 2020 at 1:09 PM William Clark  wrote:

> Hello Pulp Community!
>
> I'm working on a feature to allow the foreman-installer to set the number
> of Pulpcore workers deployed on Katello or Content Proxy, and I require
> some assistance from the Pulp community in setting sane defaults and limits.
>
> With Pulp 2 in Katello, the default behavior was that the worker count
> would match the number of logical CPUs up to a soft limit of 8. We advised
> that users could tune the worker count higher but it was expected to cause
> performance degradation in most cases due to I/O blocking. The largest
> scale installation to my knowledge uses a Pulp 2 worker count of 16.
>
I believe having one pulpcore-worker per CPU is still a good practice. We
haven't gotten a lot of feedback on right-sizing an installation so I won't
claim it's the absolute best practice, but it's what I recommend currently.
We have an issue to document sizing recommendations here that also has some
similar/more info:  https://pulp.plan.io/issues/6856

The I/O blocking concern is roughly about the same as Pulp2; during sync
operations the workload could be I/O bound.

A lot more CPU processing has been moved into postgresql which auto forks
postgresql processes per client connection, which in this case is a 1-1
pairing with each pulpcore-worker. So when it's under heavy load I expect
postresql to scale out a process and the workload could become constrained
on the postgresql CPU itself. In that case, lowering the worker to half of
the available processors would likely improve throughput. Moving postgresql
to another, dedicated box is also an option.

If you'd be willing to share your findings with the Pulp community that
would be really great.


> With Pulp 2 being replaced and rebuilt with Pulpcore, I'm looking to
> understand what are the tuning best practices for the new technology, so
> that we could apply them to Katello.
>
Please let us know what other questions we can help answer.


> I am looking forward to hearing from you,
>
Thank you for reaching out.


> --
>
> William Clark, RHCA
>
> He/Him/His
>
> Software Engineer
>
> Red Hat 
>
> IM: wclark
> 
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Pulp Core Support on SUSE

[Brian Bouterse]

It should work anywhere Python 3.6+ and the dependencies are available so I
expect it would. We currently do not test the CI against SLES like we do
other distros. See the Supported Platform docs for more details [0]

[0]:
https://docs.pulpproject.org/installation/instructions.html#supported-platforms

On Fri, Jun 5, 2020 at 8:22 PM Sathasivam, Pradeep <
pradeep.sathasi...@hpe.com> wrote:

> Hi All,
>
>
>
> Is PulpCore3 supported on SLES 15?
>
>
>
> Regards.
>
> Pradeep. S
>
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Syncing Red hat Repos entitlement issue

[Brian Bouterse]

One idea to track down which process is editing those certs/files would be
to use auditd or systemtap https://unix.stackexchange.com/a/99091  Just a
thought I wanted to share.

On Thu, May 28, 2020 at 9:18 AM Gravel Bone  wrote:

> In this case the entitlement certs themselves aren't expired from a date
> perspective, they just no longer work connecting to Red Hat.It's more
> like they've been revoked because the server they are on got new
> entitlement certs which is happening automatically, I just have not figured
> out how to prevent that.   I've tried turning of rhsmcertd, disabled
> subscription management, and combinations in between.
>
> On Wed, May 27, 2020 at 2:23 PM Brian Bouterse 
> wrote:
>
>> If the certs are short-lived, then there isn't much to do except ask the
>> issuer to give you longer ones. You could inspect the certs more closely I
>> believe using the `rct cat-crt` command. Pulp-certguard has some docs
>> showing an example with that tool
>> https://pulp-certguard.readthedocs.io/en/latest/debugging.html#checking-authorized-urls-in-rhsm-certificates
>>
>> On Wed, May 27, 2020 at 11:20 AM Myers, Mike  wrote:
>>
>>> We’ve faced that too.  I’ve love some deeper insight, but what I’ve
>>> found so far is that “rhsmcertd” process does some sort of check/update on
>>> those certs.  We’ve just set a process to pull those from
>>> /etc/pki/entitlement into Pulp when such a failure occurs.  It would be
>>> nice if there were a Pulp native way to address this (short of running the
>>> whole Satellite suite)
>>>
>>>
>>>
>>> Cheers,
>>>
>>> *Mike Myers*
>>>
>>>
>>>
>>> *From: * on behalf of Gravel Bone <
>>> gravelb...@gmail.com>
>>> *Date: *Wednesday, May 27, 2020 at 5:48 AM
>>> *To: *"pulp-list@redhat.com" 
>>> *Subject: *[Pulp-list] Syncing Red hat Repos entitlement issue
>>>
>>>
>>>
>>> This is probably something straight forward, but my searches have found
>>> nothing...
>>>
>>>
>>>
>>> I pull an entitlement files from our server (well three for three
>>> different subscriptions) and create repos using them to sync the
>>> corresponding Red Hat repository.The problem is, the entitlements seem
>>> to expire about every month.   I'm sure it's something I'm missing that
>>> stupid obvious, but google has not been my friend nor has the
>>> documentation...help would be appreciated...
>>> ___
>>> Pulp-list mailing list
>>> Pulp-list@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>
>>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Syncing Red hat Repos entitlement issue

[Brian Bouterse]

If the certs are short-lived, then there isn't much to do except ask the
issuer to give you longer ones. You could inspect the certs more closely I
believe using the `rct cat-crt` command. Pulp-certguard has some docs
showing an example with that tool
https://pulp-certguard.readthedocs.io/en/latest/debugging.html#checking-authorized-urls-in-rhsm-certificates

On Wed, May 27, 2020 at 11:20 AM Myers, Mike  wrote:

> We’ve faced that too.  I’ve love some deeper insight, but what I’ve found
> so far is that “rhsmcertd” process does some sort of check/update on those
> certs.  We’ve just set a process to pull those from /etc/pki/entitlement
> into Pulp when such a failure occurs.  It would be nice if there were a
> Pulp native way to address this (short of running the whole Satellite suite)
>
>
>
> Cheers,
>
> *Mike Myers*
>
>
>
> *From: * on behalf of Gravel Bone <
> gravelb...@gmail.com>
> *Date: *Wednesday, May 27, 2020 at 5:48 AM
> *To: *"pulp-list@redhat.com" 
> *Subject: *[Pulp-list] Syncing Red hat Repos entitlement issue
>
>
>
> This is probably something straight forward, but my searches have found
> nothing...
>
>
>
> I pull an entitlement files from our server (well three for three
> different subscriptions) and create repos using them to sync the
> corresponding Red Hat repository.The problem is, the entitlements seem
> to expire about every month.   I'm sure it's something I'm missing that
> stupid obvious, but google has not been my friend nor has the
> documentation...help would be appreciated...
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Bug triage is moving to #pulp-meeting on Freenode

[Brian Bouterse]

Thank you for bringing this up. Do we know what needs to be done to resolve
this?

I filed this task to fix the minutes not being recorded, and added it to
the open floor agenda also.  https://pulp.plan.io/issues/6791

On Tue, May 19, 2020 at 6:06 PM Fabricio Aguiar 
wrote:

> I just noticed one thing, supybot probably was configured to pulp-dev
> It records log on pulp-dev:
> https://pulpadmin.fedorapeople.org/triage/pulp-dev/2020/
>
> The triage held on pulp-meeting could not log:
>
> https://pulpadmin.fedorapeople.org/triage/pulp-meeting/2020/pulp-meeting.2020-05-19-14.30.html
> and we have nothing on:
> https://pulpadmin.fedorapeople.org/triage/pulp-meeting/2020
>
>
> From my logs I could recover the AIs:
> May 19 11:53:06  !action dkliban will do the 3.4.0 on May 27th
> May 19 11:59:36  !action tag 6421 and yet-to-be-filed pulpcore
> write_only issue as 3.4.0 blocker
> May 19 12:04:29  !action remove 6460 as blocker, katello has a
> workaround and the patch isn't ready
> May 19 12:05:56  !action add 3.4.0 blocker tag to
> https://pulp.plan.io/issues/6369
> May 19 12:07:13  !action dkliban to file a bug about not being
> able to upload content in single call using bindings
>
> Best regards,
> Fabricio Aguiar
> Software Engineer, Pulp Project
> Red Hat Brazil - Latam 
> +55 11 999652368
>
>
> On Mon, May 18, 2020 at 9:53 AM Dennis Kliban  wrote:
>
>> The decision to do this was actually made a while ago when we noticed
>> that holding meetings in #pulp-dev interrupted discussions that were
>> happening in that channel around the same time as the meeting. However, I
>> did not follow through to actually announce this decision and update the
>> website. @bmbouter brought this up again on Friday during open floor.
>>
>> On Mon, May 18, 2020 at 8:35 AM David Davis 
>> wrote:
>>
>>> Any chance we can get some background on how, why, where this decision
>>> was made? I'm not opposed to it but having some more information in this
>>> announcement would be helpful.
>>>
>>> David
>>>
>>>
>>> On Fri, May 15, 2020 at 11:21 AM Dennis Kliban 
>>> wrote:
>>>
 Starting on May 19th, bug triage will be held in #pulp-meeting on
 Freenode[0].

 [0] https://pulpproject.org/get_involved/#meetings
 ___
 Pulp-list mailing list
 Pulp-list@redhat.com
 https://www.redhat.com/mailman/listinfo/pulp-list
>>>
>>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] [Pulp-dev] RFC: pulp command line interface

[Brian Bouterse]

Thank you for sharing this! Can a basic README be added showing a few
things a user could try after installing it from source? I also put a few
comments inline also.

On Mon, May 11, 2020 at 1:53 PM David Davis  wrote:

> Adding pulp-list to hopefully get user feedback on this.
>
> David
>
>
> On Mon, May 11, 2020 at 6:54 AM Matthias Dellweg 
> wrote:
>
>> A first draft of the architecture that should eventually govern the pulp
>> cli has been completed [0].
>> The feature set is naturally very limited, since we want to autotemplate
>> most of this after getting good feedback about the architecture.
>>
>> Questions we want to settle at this point are:
>>   - Is the command structure comprehensive and useable?
>> 'pulp   [--type resource_type] 
>> <...>'
>> 'pulp file repository list' === 'pulp file repository --type File
>> list'
>>
> Does ^ mean the user would run `pulp file repository list`? If so then +1

  - Is the implemented plugin autoloading useful?
>>
> It is useful, but why use importlib with naming conventions rather than
each cli package declare a python entrypoint? I think entrypoints are a
more common mechanism of discovery in Python.

  - Should we put the cli (plugin-)packages in the corresponding pulp
>> plugin repos as subpackages?
>>
> I'm not entirely sure which is better. I *think* the decision should
mainly come from how we want to test the CLI packages in CI. If we want to
test them with every plugin change then we probably do want them in the
same repo. This is similar reasoning to why we keep the bindings "with the
plugin". What do you think?


>> [0] https://github.com/mdellweg/pulp-cli
>> ___
>> Pulp-dev mailing list
>> pulp-...@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> pulp-...@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] [Pulp-dev] Pulp 3 CLI MVP

[Brian Bouterse]

I think having the commands namespaced by the plugin name would be an
organized way to see what capabilities a given plugin is shipping. I
imagine for pulpcore's commands they could be namespaced under 'core' or
'pulpcore'.

Also +1 to 'pulp' command name.

On Mon, May 11, 2020 at 6:42 AM David Davis  wrote:

> In some places, the API in Pulp 3 is very different from Pulp 2 but where
> it's possible and makes sense, I think we will want to do this. Perhaps
> this is a good argument for having plugin name after the "pulp" command (eg
> "pulp rpm ...", "pulp file ...").
>
> David
>
>
> On Thu, May 7, 2020 at 8:47 AM Konstantin M. Khankin <
> khankin.konstan...@gmail.com> wrote:
>
>> Is it an option to keep the Pulp 2 CLI syntax as much as possible?
>>
>> чт, 7 мая 2020 г. в 15:28, Dennis Kliban :
>>
>>> On Thu, May 7, 2020 at 7:13 AM Tatiana Tereshchenko 
>>> wrote:
>>>
 +1 to `pulp` command.

 I think for me as a user, the most logical would be to have a plugin
 name first and then follow the URL pattern.
 The majority of commands are plugin specific. If I work with multiple
 plugins, it also makes it easy for me as a user to just change the plugin
 name in front for the commands which have the same structure in different
 plugins.
 It also makes it visually clear that I work with a specific plugin, in
 comparison to when the plugin name is somewhere in the 3rd/4th place.
 It will also allow not to guess in commands like the `pulp repositories
 rpm rpm`  which one is the plugin name and which one is a repo type.

 I agree that this would make much more clear to the user which 'rpm' is
>>> the plugin type and which 'rpm' is the resource type.
>>>
>>>
 +1 for
 pulp rpm content packages
 pulp rpm repositories rpm
 pulp rpm repositories mirror
 ...

 On Wed, May 6, 2020 at 7:58 PM Dennis Kliban 
 wrote:

> On Wed, May 6, 2020 at 12:30 PM David Davis 
> wrote:
>
>> Matthias and I met today to go over some plans for a prototype. I
>> wrote some notes[0] down. As part of the prototype, we'd propose two
>> deliverables (one this week and one next week):
>>
>> 1. A set of ~2-3 click commands that use the bindings to interact
>> with Pulp
>> 2. Some openapi-generator templates that will be able to generate
>> such commands from the schema
>>
>> There is a question we had about how the commands for typed resources
>> will be structured in the CLI. To illustrate with two endpoints:
>>
>>
> We should call the command 'pulp' instead of pulp-cli.
>
>
>> # rpm.package content (/pulp/api/v3/content/rpm/packages/):
>> - pulp-cli rpm content packages ...
>> - pulp-cli content rpm packages ...
>> - pulp-cli rpm packages content ...
>> - ???
>>
>
> I was thinking we should structure the commands similar to the URLs in
> the REST API.
>
> pulp content rpm packages
>
>
>>
>> # file.file repositories (/pulp/api/v3/repositories/file/file/):
>> - pulp-cli file repositories file ...
>> - pulp-cli repositories file file ...
>> - pulp-cli file file repositories ...
>> - ???
>>
>> pulp repositories file
>
> Plugins that provide multiple types of the same resource would need to
> be more descriptive. Though I can see a practical reason for requiring all
> resources to be that descriptive.
>
> pulp repositories rpm rpm
> pulp repositories rpm mirror
>
>
>
>> [0] https://hackmd.io/aH9RqAS_TrGyxoi1UGRgig?view#Prototype
>>
>> David
>>
>>
>> On Thu, Apr 30, 2020 at 1:42 PM David Davis 
>> wrote:
>>
>>> Today we met to discuss some ideas for a technical design for how
>>> the CLI would work. Here's a copy of our notes:
>>>
>>> https://hackmd.io/aH9RqAS_TrGyxoi1UGRgig#Technical-discussion
>>>
>>> And there is a rough design in the document as well:
>>>
>>> https://hackmd.io/aH9RqAS_TrGyxoi1UGRgig#Design
>>>
>>> I have also entered the CLI user stories from our meeting last week
>>> into redmine under the Pulp CLI project:
>>>
>>> https://pulp.plan.io/versions/93
>>>
>>> And I've filed a user story that we talked about today that would
>>> handle sync, publish, and distribution of repos. Feedback welcome:
>>>
>>> https://pulp.plan.io/issues/6626
>>>
>>> Matthias and I are planning to meet next week to look at creating a
>>> proof of concept that would provide 2-3 commands. If anyone is 
>>> interested
>>> in joining us, please let me know and I can add you.
>>>
>>> David
>>>
>>>
>>> On Tue, Apr 28, 2020 at 8:06 AM David Davis 
>>> wrote:
>>>
 I've also started working on some questions about how the CLI will
 work. Feel free to add some of your own:

 

Re: [Pulp-list] 3.3.1 migration error

[Brian Bouterse]

On Mon, May 11, 2020 at 5:04 AM Tatiana Tereshchenko 
wrote:

> Dennis,
> FWIW, I believe there are a couple of problems here but neither of them is
> related to your advice to manually modify a migration.
> Problem #1.  `django-admin makemigrations` was run, so this 0009_auto_...
> migration got created. I think the installer doesn't run this command, or
> does it?
>
The installer does not run makemigrations.

Problem #2. The migration 0008 was not in sync with the model, so
> makemigraitons will create a new migraiton if run. Fixed in 3.3.1.
> https://pulp.plan.io/issues/6665
>
Great, then 3.3.1 as the current GA should resolve it.

>
> Tanya
>
> On Fri, May 8, 2020 at 11:15 PM Bin Li (BLOOMBERG/ 120 PARK) <
> bli...@bloomberg.net> wrote:
>
>> No idea where it came from. The timestamp is from 4/25. I run pip
>> uninstall pulp-rpm and it didn't remove it.
>>
>> # ls -ld
>> /opt/utils/venv/pulp/3.7.3/lib/python3.7/site-packages/pulp_rpm/app/migrations/0009_auto_20200425_1932.py
>> -rw-rw-r-- 1 pulp mse-python 492 Apr 25 15:32
>> /opt/utils/venv/pulp/3.7.3/lib/python3.7/site-packages/pulp_rpm/app/migrations/0009_auto_20200425_1932.py
>>
>> # more
>> /opt/utils/venv/pulp/3.7.3/lib/python3.7/site-packages/pulp_rpm/app/migrations/0009_auto_20200425_1932.py
>> # Generated by Django 2.2.12 on 2020-04-25 19:32
>>
>> from django.db import migrations, models
>>
>>
>> class Migration(migrations.Migration):
>>
>> dependencies = [
>> ('rpm', '0008_advisory_pkg_sumtype_as_int'),
>> ]
>>
>> operations = [
>> migrations.AlterField(
>> model_name='updatecollectionpackage',
>> name='sum_type',
>> field=models.PositiveIntegerField(choices=[(0, 0), (1, 1), (2, 2), (3,
>> 3), (4, 4), (5, 5), (6, 6), (7, 7)], null=True),
>> ),
>>
>>
>> From: ggai...@redhat.com At: 05/08/20 16:21:45
>> To: Bin Li (BLOOMBERG/ 120 PARK ) 
>> Cc: pulp-list@redhat.com
>> Subject: Re: [Pulp-list] 3.3.1 migration error
>>
>> Hey there!
>>
>> On Fri, May 8, 2020 at 3:53 PM Bin Li (BLOOMBERG/ 120 PARK) <
>> bli...@bloomberg.net> wrote:
>>
>>> Hi All,
>>> Getting an error to upgrade from 3.3. run 'python manage.py
>>> makemigrations --merge' gave more errors. Anyidea how this can be fixed?
>>>
>>> TASK [pulp-database : Run database auth migrations]
>>> **
>>> fatal: [pulp3-2]: FAILED! => {"changed": true, "cmd":
>>> ["/opt/utils/venv/pulp/3.7.3/bin/django-admin", "migrate", "auth",
>>> "--no-input"], "delta": "0:00:03.534845", "end": "2020-05-08
>>> 15:44:20.362670", "msg": "non-zero return code", "rc": 1, "start":
>>> "2020-05-08 15:44:16.827825", "stderr": "CommandError: Conflicting
>>> migrations detected; multiple leaf nodes in the migration graph:
>>> (0009_revision_null, 0009_auto_20200425_1932 in rpm).\nTo fix them run
>>> 'python manage.py makemigrations --merge'", "stderr_lines": ["CommandError:
>>> Conflicting migrations detected; multiple leaf nodes in the migration
>>> graph: (0009_revision_null, 0009_auto_20200425_1932 in rpm).", "To fix them
>>> run 'python manage.py makemigrations --merge'"], "stdout": "",
>>> "stdout_lines": []}
>>>
>>
>> Where did migration "0009_auto_20200425_1932" come from? Looks like it's
>> conflicting with the delivered 0009_revision_null and causing your problem.
>>
>> G
>> --
>> Grant Gainey
>> Principal Software Engineer, Red Hat System Management Engineering
>>
>>
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] upgrade to 3.3

[Brian Bouterse]

Hi Bin,

Another user reported a similar error to you, and I've filed this issue
https://pulp.plan.io/issues/6623  The "pre-flight" check is designed to
verify that the pulpcore version being installed/upgraded to is compatible
with all plugins that will be installed prior to any bits installing and
stop the user from continuing if there is a conflict. In your case the
conflict is occuring after all the installation has occurred, yet it can't
finish due to the conflict. I believe the pre-flight test does not work
correctly in all cases. I'm going to advocate for us to fix this bug soon.

Thanks,
Brian


On Wed, Apr 29, 2020 at 3:47 PM Mike DePaulo  wrote:

> Hi Bin,
>
> Usually whenever you upgrade pulpcore (X.Y release), you need to upgrade
> the plugins.
>
> If you run it right after pulpcore release, your updated plugins may not
> be available yet on PyPI.
>
> You can upgrade plugins by specifying one of these 2 sub-variables under
> `pulp_install_plugins.plugin-name`[1]:
> `upgrade: true`
> `version: "3.3.0"`
> Where "3.3.0" is an actual released version of the plugin.[2]
>
> The latter is safer (but more work) since `upgrade` tries the latest
> version no matter what, pulpcore compatible or not.
>
> We are also wondering why this task didn't fail. It should fail as a
> precaution; so as to stop the installer early on & not break your install:
> `pulp: Run pip-compile to check pulpcore/plugin compatibility`
>
> Do you have any output from that task?
>
> As for the external postgresql database, It shouldn't affect pip installs.
> But there are the django migrations to run within the role. Without the
> migrations, the pip install would work, but Pulp would not function. You
> should include the role, but set `pulp_install_db` to false.[3]
>
> -Mike
>
> [1]
> https://github.com/pulp/pulp_installer/tree/master/roles/pulp#role-variables
> [2] https://pypi.org/search/?q=pulp=
> [3]
> https://github.com/pulp/pulp_installer/tree/master/roles/pulp-database#pulp-database
>
> On Wed, Apr 29, 2020 at 1:33 PM Bin Li (BLOOMBERG/ 120 PARK) <
> bli...@bloomberg.net> wrote:
>
>> We are using the latest version of installer. It upgraded the pulpcore
>> but not plugins.
>> # grep 3.3 README.md
>> This version of the installer, 3.3.0, installs Pulp 3.3.0 specifically.
>> If run against an older version of Pulp 3, it will upgrade it to 3.3.0.
>>
>> We are using external postrgresql database so we exclude the
>> pulp-database role from playbook. Could this cause the issue?
>>
>> From: dkli...@redhat.com At: 04/29/20 11:00:14
>> To: Bin Li (BLOOMBERG/ 120 PARK ) 
>> Cc: fabricio.agu...@redhat.com, pulp-list@redhat.com
>> Subject: Re: [Pulp-list] upgrade to 3.3
>>
>> It seems like you are using a very old version of pulp_installer
>> (formerly known as ansible-pulp). What commit are you at?
>>
>> You should be using pulp_installer 3.3.0 to do the install[0].
>>
>> [0] https://github.com/pulp/pulp_installer/releases/tag/3.3.0
>>
>> On Wed, Apr 29, 2020 at 8:12 AM Bin Li (BLOOMBERG/ 120 PARK) <
>> bli...@bloomberg.net> wrote:
>>
>>> Hi Fabricio,
>>>
>>> I got the errors below. It looks it only update pulpcore but not the
>>> plugins.
>>>
>>> RUNNING HANDLER [pulp : Collect static content]
>>> 
>>> fatal: [pulp3]: FAILED! => {"changed": true, "cmd":
>>> ["/opt/utils/venv/pulp/3.7.3/bin/django-admin", "collectstatic",
>>> "--noinput", "--link"], "delta": "0:00:00.250852", "end": "2020-04-29
>>> 08:08:45.007362", "msg": "non-zero return code", "rc": 1, "start":
>>> "2020-04-29 08:08:44.756510", "stderr": "Traceback (most recent call
>>> last):\n File \"/opt/utils/venv/pulp/3.7.3/bin/django-admin\", line 8, in
>>> \n sys.exit(execute_from_command_line())\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/__init__.py\",
>>> line 381, in execute_from_command_line\n utility.execute()\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/__init__.py\",
>>> line 325, in execute\n settings.INSTALLED_APPS\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/conf/__init__.py\",
>>> line 79, in __getattr__\n self._setup(name)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/conf/__init__.py\",
>>> line 66, in _setup\n self._wrapped = Settings(settings_module)\n File
>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/conf/__init__.py\",
>>> line 157, in __init__\n mod =
>>> importlib.import_module(self.SETTINGS_MODULE)\n File
>>> \"/opt/python/3.7.3/lib64/python3.7/importlib/__init__.py\", line 127, in
>>> import_module\n return _bootstrap._gcd_import(name[level:], package,
>>> level)\n File \"\", line 1006, in
>>> _gcd_import\n File \"\", line 983, in
>>> _find_and_load\n File \"\", line 967, in
>>> _find_and_load_unlocked\n File \"\", line 677,
>>> in _load_unlocked\n File \"\", line
>>> 728, in exec_module\n File \"\", line 219, in
>>> 

Re: [Pulp-list] Pulp-list Digest, Vol 125, Issue 19

[Brian Bouterse]

Hi Bin,

Related to this issue, I have some questions I'm interested in. I posted
them here https://pulp.plan.io/issues/6463#note-1 If you're able to post
any info you have that would be great.

Thanks.
Brian

On Mon, Apr 20, 2020 at 7:12 PM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> We are still seeing the same errors on 8 repos out of total of 270. The
> sync job failed immediately when we tried to resync. However, if we delete
> and recreate the repo, we don't see the errors when we sync.
>
>
> From: pulp-list@redhat.com At: 04/20/20 12:01:43
> To: pulp-list@redhat.com
> Subject: Pulp-list Digest, Vol 125, Issue 19
>
> Send Pulp-list mailing list submissions to
> pulp-list@redhat.com
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://www.redhat.com/mailman/listinfo/pulp-list
> or, via email, send a message with subject or body 'help' to
> pulp-list-requ...@redhat.com
>
> You can reach the person managing the list at
> pulp-list-ow...@redhat.com
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Pulp-list digest..."
>
>
> Today's Topics:
>
> 1. Re: pulp 3.2.1 duplicate key error (Dennis Kliban)
>
>
> --
>
> Message: 1
> Date: Mon, 20 Apr 2020 11:25:42 -0400
> From: Dennis Kliban 
> To: Bin Li 
> Cc: pulp-list 
> Subject: Re: [Pulp-list] pulp 3.2.1 duplicate key error
> Message-ID:
> 
> Content-Type: text/plain; charset="utf-8"
>
> The issue does not have any steps to reproduce it. Have you been able to
> consistently reproduce the issue with a specific repository? Does it go
> away the next time you perform the sync?
>
> On Wed, Apr 8, 2020 at 9:03 AM Bin Li (BLOOMBERG/ 120 PARK) <
> bli...@bloomberg.net> wrote:
>
> > Thanks Brian. I filed an issue https://pulp.plan.io/issues/6463
> >
> > From: bmbou...@redhat.com At: 04/07/20 15:59:11
> > To: Bin Li (BLOOMBERG/ 120 PARK ) 
> > Cc: pulp-list@redhat.com
> > Subject: Re: [Pulp-list] pulp 3.2.1 duplicate key error
> >
> > I heard another developer report a similar issue, but we couldn't
> > reproduce it. Could you file this as an issue also please?
> >
> > On Tue, Apr 7, 2020 at 3:42 PM Bin Li (BLOOMBERG/ 120 PARK) <
> > bli...@bloomberg.net> wrote:
> >
> >> Noticed we have a few errors when running sync.
> >>
> >> "error": {
> >> "description": "duplicate key value violates unique constraint
> >> \"core_repositoryversion_repository_id_number_3c54ce50_uniq\"\nDETAIL:
> Key
> >> (repository_id, number)=(59eb02b1-edab-46e3-a69b-d69a8b314f20, 2)
> already
> >> exists.\n",
> >>
> >> What could be the cause of this? How can we resolve it?
> >> ___
> >> Pulp-list mailing list
> >> Pulp-list@redhat.com
> >> https://www.redhat.com/mailman/listinfo/pulp-list
> >
> >
> > ___
> > Pulp-list mailing list
> > Pulp-list@redhat.com
> > https://www.redhat.com/mailman/listinfo/pulp-list
> -- next part --
> An HTML attachment was scrubbed...
> URL:
> <
> https://www.redhat.com/archives/pulp-list/attachments/20200420/6c0bc1a7/atta
> chment.html>
>
> --
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>
> End of Pulp-list Digest, Vol 125, Issue 19
> **
>
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulp 3 publication required

[Brian Bouterse]

Also regarding the order of the creation of these things, I believe in all
cases a Distribution (of any type) can be created before it's corresponding
repository_version or in other cases a publication (depending on the
content type's needs). The use case is that a user may want to design their
Distribution base_path URL layout and receive sections of it even before
they have content to serve from those Distributions.

On Fri, Apr 17, 2020 at 10:52 AM Dennis Kliban  wrote:

> Some content types require publications and others don't. It all depends
> on the requirements of the client that consumes content. For example,
> yum/dnf expects to find metadata about all the packages in the repository.
> Since a RepositoryVersion only contains the packages, you must create a
> Publication to generate the metadata. On the other hand 'podman' and
> 'docker' interact with a REST API that serves content without any special
> metadata. So Container repositories and repository versions can be directly
> associated with ContainerDistributions.
>
> On Fri, Apr 17, 2020 at 9:59 AM Bin Li (BLOOMBERG/ 120 PARK) <
> bli...@bloomberg.net> wrote:
>
>> I was under impression that the publication has to be created before
>> creating a distribution. It looks like now I can create a distribution by
>> specify a repository or repository version. Is the publication created
>> automatically in this case? If not, in what scenario we need a create a
>> publication before create a distribution?
>>
>> Thanks
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulp 3.2.1 duplicate key error

[Brian Bouterse]

I heard another developer report a similar issue, but we couldn't reproduce
it. Could you file this as an issue also please?

On Tue, Apr 7, 2020 at 3:42 PM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> Noticed we have a few errors when running sync.
>
> "error": {
> "description": "duplicate key value violates unique constraint
> \"core_repositoryversion_repository_id_number_3c54ce50_uniq\"\nDETAIL: Key
> (repository_id, number)=(59eb02b1-edab-46e3-a69b-d69a8b314f20, 2) already
> exists.\n",
>
> What could be the cause of this? How can we resolve it?
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Pulp-list Digest, Vol 125, Issue 10

[Brian Bouterse]

Thanks Bin! I put some info about the resource-manager inline.

On Tue, Apr 7, 2020 at 3:28 PM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> Thanks Brian. I was able to cancel all waiting tasks. I started over
> hundred tasks today and so far we have no issues.
>
> How many resource manger are we supposed to run at the same time? I only
> notice single instance. Is it configurable?
>
You have to run at least 1 resource manager, or Pulp will log errors. You
can run multiple of them for high-availability, but one one will be active
and the others will wait as hot-spares. See these docs
https://docs.pulpproject.org/components.html#tasking-system

In my reading of the configuration of the installer in this area (
https://github.com/pulp/pulp_installer/blob/master/roles/pulp-resource-manager/README.md#pulp-resource-manager
) it looks like the installer does not have the ability to create multiple
resource managers. If you'd like that feature (for more high availability)
you could file a feature request and link us to the issue so we see it.


>
>
> From: pulp-list@redhat.com At: 04/06/20 17:25:01
> To: pulp-list@redhat.com
> Subject: Pulp-list Digest, Vol 125, Issue 10
>
> Send Pulp-list mailing list submissions to
> pulp-list@redhat.com
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://www.redhat.com/mailman/listinfo/pulp-list
> or, via email, send a message with subject or body 'help' to
> pulp-list-requ...@redhat.com
>
> You can reach the person managing the list at
> pulp-list-ow...@redhat.com
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Pulp-list digest..."
>
>
> Today's Topics:
>
> 1. Re: Pulp 3 waiting tasks (Daniel Alley)
>
>
> ------
>
> Message: 1
> Date: Mon, 6 Apr 2020 17:24:11 -0400
> From: Daniel Alley 
> To: Brian Bouterse 
> Cc: pulp-list , Bin Li 
> Subject: Re: [Pulp-list] Pulp 3 waiting tasks
> Message-ID:
> 
> Content-Type: text/plain; charset="utf-8"
>
> On the few occasions when I've had issues (usually doing something like
> deleting the database while a task was still running), a "redis-cli
> FLUSHALL" has solved my problems as well. So if ^ does not resolve things,
> try that.
>
> On Mon, Apr 6, 2020 at 1:37 PM Brian Bouterse  wrote:
>
> > Thank you. We will look into this bug you've filed.
> >
> > I believe you can recover your current installation by canceling the
> tasks
> > stuck in the "waiting" state. To cancel use this API call
> > https://docs.pulpproject.org/restapi.html#operation/tasks_cancel
> >
> > Let me know if this doesn't help get your system back on track.
> >
> > Thanks,
> > Brian
> >
> >
> > On Mon, Apr 6, 2020 at 8:04 AM Bin Li (BLOOMBERG/ 120 PARK) <
> > bli...@bloomberg.net> wrote:
> >
> >> Brian,
> >>
> >> I filed a bug to track this issue "https://pulp.plan.io/issues/6449;.
> In
> >> the meantime, is it possible to recover from this issue or we need to
> erase
> >> the database and reinstall?
> >>
> >> Thanks
> >>
> >>
> >> From: bmbou...@redhat.com At: 04/03/20 16:45:00
> >> To: Bin Li (BLOOMBERG/ 120 PARK ) 
> >> Cc: pulp-list@redhat.com
> >> Subject: Re: [Pulp-list] Pulp 3 waiting tasks
> >>
> >> So the problematic thing I see in this output is the "resource-manager |
> >> 0". This tells me that Pulp's record of the task is in postgresql (and
> was
> >> never run), but RQ has lost the task from the "resource-manager" queue
> in
> >> Redis. So the next question is how did that happen?
> >>
> >> Would you be willing to file a bug and link it here so that I could try
> >> to reproduce it on our end?
> >>
> >> Thanks!
> >> Brian
> >>
> >>
> >> On Fri, Apr 3, 2020 at 4:35 PM Bin Li (BLOOMBERG/ 120 PARK) <
> >> bli...@bloomberg.net> wrote:
> >>
> >>> Brian,
> >>>
> >>> Here is rq info output. Thanks for look into this.
> >>>
> >>> # rq --version
> >>> rq, version 1.2.2
> >>>
> >>> # rq info
> >>> 134692@pulpmaster |?? 7
> >>> 182536@pulpmaster | 0
> >>> 134343@pulpmaster |?? 7
> >>> 191144@pulpmaster | 0
> >>> 130945@pulpmaster |?? 7
> >>> 135922@pulpmaster | 0
> >>> 182528@pulpmaster | 0
> >>> 182532@pulpmaster | 0
>

Re: [Pulp-list] Pulp 3 waiting tasks

[Brian Bouterse]

Thank you. We will look into this bug you've filed.

I believe you can recover your current installation by canceling the tasks
stuck in the "waiting" state. To cancel use this API call
https://docs.pulpproject.org/restapi.html#operation/tasks_cancel

Let me know if this doesn't help get your system back on track.

Thanks,
Brian


On Mon, Apr 6, 2020 at 8:04 AM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> Brian,
>
> I filed a bug to track this issue "https://pulp.plan.io/issues/6449;. In
> the meantime, is it possible to recover from this issue or we need to erase
> the database and reinstall?
>
> Thanks
>
>
> From: bmbou...@redhat.com At: 04/03/20 16:45:00
> To: Bin Li (BLOOMBERG/ 120 PARK ) 
> Cc: pulp-list@redhat.com
> Subject: Re: [Pulp-list] Pulp 3 waiting tasks
>
> So the problematic thing I see in this output is the "resource-manager |
> 0". This tells me that Pulp's record of the task is in postgresql (and was
> never run), but RQ has lost the task from the "resource-manager" queue in
> Redis. So the next question is how did that happen?
>
> Would you be willing to file a bug and link it here so that I could try to
> reproduce it on our end?
>
> Thanks!
> Brian
>
>
> On Fri, Apr 3, 2020 at 4:35 PM Bin Li (BLOOMBERG/ 120 PARK) <
> bli...@bloomberg.net> wrote:
>
>> Brian,
>>
>> Here is rq info output. Thanks for look into this.
>>
>> # rq --version
>> rq, version 1.2.2
>>
>> # rq info
>> 134692@pulpmaster |██ 7
>> 182536@pulpmaster | 0
>> 134343@pulpmaster |██ 7
>> 191144@pulpmaster | 0
>> 130945@pulpmaster |██ 7
>> 135922@pulpmaster | 0
>> 182528@pulpmaster | 0
>> 182532@pulpmaster | 0
>> 191145@pulpmaster | 0
>> 135796@pulpmaster | 0
>> 191148@pulpmaster | 0
>> 191152@pulpmaster | 0
>> 191151@pulpmaster | 0
>> 135306@pulpmaster | 0
>> 135679@pulpmaster | 0
>> 182539@pulpmaster | 0
>> 182547@pulpmaster | 0
>> 182530@pulpmaster | 0
>> 134332@pulpmaster |██ 7
>> 191147@pulpmaster | 0
>> 131701@pulpmaster |██ 5
>> 134330@pulpmaster |██ 7
>> 134688@pulpmaster |██ 5
>> 182548@pulpmaster | 0
>> 134929@pulpmaster | 0
>> 135180@pulpmaster | 0
>> 135503@pulpmaster | 0
>> 182546@pulpmaster | 0
>> 131485@pulpmaster |██ 7
>> 131269@pulpmaster |██ 7
>> 32603@pulpmaster | 0
>> 191146@pulpmaster | 0
>> 131053@pulpmaster |██ 7
>> 134339@pulpmaster |██ 7
>> 134336@pulpmaster |██ 7
>> 191150@pulpmaster | 0
>> 182542@pulpmaster | 0
>> 182540@pulpmaster | 0
>> 32609@pulpmaster | 0
>> 191153@pulpmaster | 0
>> 131593@pulpmaster |████████ 21
>> 135051@pulpmaster | 0
>> 134696@pulpmaster |██ 7
>> 191149@pulpmaster | 0
>> 131377@pulpmaster |██ 5
>> 134694@pulpmaster |██ 7
>> 134690@pulpmaster |██ 7
>> 131161@pulpmaster |██ 5
>> 136342@pulpmaster | 0
>> 32626@pulpmaster | 0
>> 131810@pulpmaster |██ 7
>> 136462@pulpmaster | 0
>> 130836@pulpmaster |██ 5
>> resource-manager | 0
>> 54 queues, 144 jobs total
>>
>> 191146@pulpmaster (b'pulpp-ob-581' 191146): idle 191146@pulpmaster
>> 191147@pulpmaster (b'pulpp-ob-581' 191147): idle 191147@pulpmaster
>> 191153@pulpmaster (b'pulpp-ob-581' 191153): idle 191153@pulpmaster
>> resource-manager (b'pulpp-ob-581' 187238): idle resource-manager
>> 191144@pulpmaster (b'pulpp-ob-581' 191144): idle 191144@pulpmaster
>> 191151@pulpmaster (b'pulpp-ob-581' 191151): idle 191151@pulpmaster
>> 191149@pulpmaster (b'pulpp-ob-581' 191149): idle 191149@pulpmaster
>> 191145@pulpmaster (b'pulpp-ob-581' 191145): idle 191145@pulpmaster
>> 191148@pulpmaster (b'pulpp-ob-581' 191148): idle 191148@pulpmaster
>> 191150@pulpmaster (b'pulpp-ob-581' 191150): idle 191150@pulpmaster
>> 191152@pulpmaster (b'pulpp-ob-581' 191152): idle 191152@pulpmaster
>> 11 workers, 54 queues
>>
>> Updated: 2020-04-03 16:30:23.373244
>>
>> From: bmbou...@redhat.com At: 04/03/20 16:23:22
>> To: Bin Li (BLOOMBERG/ 120 PARK ) 
>> Cc: pulp-list@redhat.com
>> Subject: Re: [Pulp-list] Pulp 3 waiting tasks
>>
>> Since the task that is stalled has a "worker" unassigned it tells me it
>> has not traveled through the resource-manager yet. All tests in Pulp3
>> (currently) go through the resource-manager. I can see from your ps output
>> there is 1 resource-manager running (which is good), and the status API
>> agrees with that (also good).
>>
>> So what does RQ thing the situation is? Can you paste the output of `rq
>> info` please?
>>
>> Also what version of RQ are do you have installed?
>>
>> Thanks,
>> Brian
>>
>>
>> On Fri, Apr 3, 2020 at 9:39 AM Bin Li (BLOOMBERG/ 120 PARK) <
>> bli...@bloomberg.net> wrote:
>>
>>> Here is the more info. Log is very big. I will send you shortly.
>>>
>>> # ./sget status
>>> {
>>> "database_connection": {
>>> "connected": true
>>> },
>>> "online_content_apps": [
>>> {
>>> "last_heartbeat": "2020-04-03T13:10:30.135954Z",
>>> "name": "187254@pulpp-ob-581"
>>> },
>>> {
>>> "last_heartbeat": "2020-04-03T13:10:30.132849Z",
>>> "name": "187257@pulpp-ob-581"
>>> }
>>> ],
>>> "online_workers": [
>>> {
>>> 

Re: [Pulp-list] Pulp 3 waiting tasks

[Brian Bouterse]

So the problematic thing I see in this output is the "resource-manager |
0". This tells me that Pulp's record of the task is in postgresql (and was
never run), but RQ has lost the task from the "resource-manager" queue in
Redis. So the next question is how did that happen?

Would you be willing to file a bug and link it here so that I could try to
reproduce it on our end?

Thanks!
Brian


On Fri, Apr 3, 2020 at 4:35 PM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> Brian,
>
> Here is rq info output. Thanks for look into this.
>
> # rq --version
> rq, version 1.2.2
>
> # rq info
> 134692@pulpmaster |██ 7
> 182536@pulpmaster | 0
> 134343@pulpmaster |██ 7
> 191144@pulpmaster | 0
> 130945@pulpmaster |██ 7
> 135922@pulpmaster | 0
> 182528@pulpmaster | 0
> 182532@pulpmaster | 0
> 191145@pulpmaster | 0
> 135796@pulpmaster | 0
> 191148@pulpmaster | 0
> 191152@pulpmaster | 0
> 191151@pulpmaster | 0
> 135306@pulpmaster | 0
> 135679@pulpmaster | 0
> 182539@pulpmaster | 0
> 182547@pulpmaster | 0
> 182530@pulpmaster | 0
> 134332@pulpmaster |██ 7
> 191147@pulpmaster | 0
> 131701@pulpmaster |██ 5
> 134330@pulpmaster |██ 7
> 134688@pulpmaster |██ 5
> 182548@pulpmaster | 0
> 134929@pulpmaster | 0
> 135180@pulpmaster | 0
> 135503@pulpmaster | 0
> 182546@pulpmaster | 0
> 131485@pulpmaster |██ 7
> 131269@pulpmaster |██ 7
> 32603@pulpmaster | 0
> 191146@pulpmaster | 0
> 131053@pulpmaster |██ 7
> 134339@pulpmaster |██ 7
> 134336@pulpmaster |██ 7
> 191150@pulpmaster | 0
> 182542@pulpmaster | 0
> 182540@pulpmaster | 0
> 32609@pulpmaster | 0
> 191153@pulpmaster | 0
> 131593@pulpmaster |████████ 21
> 135051@pulpmaster | 0
> 134696@pulpmaster |██ 7
> 191149@pulpmaster | 0
> 131377@pulpmaster |██ 5
> 134694@pulpmaster |██ 7
> 134690@pulpmaster |██ 7
> 131161@pulpmaster |██ 5
> 136342@pulpmaster | 0
> 32626@pulpmaster | 0
> 131810@pulpmaster |██ 7
> 136462@pulpmaster | 0
> 130836@pulpmaster |██ 5
> resource-manager | 0
> 54 queues, 144 jobs total
>
> 191146@pulpmaster (b'pulpp-ob-581' 191146): idle 191146@pulpmaster
> 191147@pulpmaster (b'pulpp-ob-581' 191147): idle 191147@pulpmaster
> 191153@pulpmaster (b'pulpp-ob-581' 191153): idle 191153@pulpmaster
> resource-manager (b'pulpp-ob-581' 187238): idle resource-manager
> 191144@pulpmaster (b'pulpp-ob-581' 191144): idle 191144@pulpmaster
> 191151@pulpmaster (b'pulpp-ob-581' 191151): idle 191151@pulpmaster
> 191149@pulpmaster (b'pulpp-ob-581' 191149): idle 191149@pulpmaster
> 191145@pulpmaster (b'pulpp-ob-581' 191145): idle 191145@pulpmaster
> 191148@pulpmaster (b'pulpp-ob-581' 191148): idle 191148@pulpmaster
> 191150@pulpmaster (b'pulpp-ob-581' 191150): idle 191150@pulpmaster
> 191152@pulpmaster (b'pulpp-ob-581' 191152): idle 191152@pulpmaster
> 11 workers, 54 queues
>
> Updated: 2020-04-03 16:30:23.373244
>
> From: bmbou...@redhat.com At: 04/03/20 16:23:22
> To: Bin Li (BLOOMBERG/ 120 PARK ) 
> Cc: pulp-list@redhat.com
> Subject: Re: [Pulp-list] Pulp 3 waiting tasks
>
> Since the task that is stalled has a "worker" unassigned it tells me it
> has not traveled through the resource-manager yet. All tests in Pulp3
> (currently) go through the resource-manager. I can see from your ps output
> there is 1 resource-manager running (which is good), and the status API
> agrees with that (also good).
>
> So what does RQ thing the situation is? Can you paste the output of `rq
> info` please?
>
> Also what version of RQ are do you have installed?
>
> Thanks,
> Brian
>
>
> On Fri, Apr 3, 2020 at 9:39 AM Bin Li (BLOOMBERG/ 120 PARK) <
> bli...@bloomberg.net> wrote:
>
>> Here is the more info. Log is very big. I will send you shortly.
>>
>> # ./sget status
>> {
>> "database_connection": {
>> "connected": true
>> },
>> "online_content_apps": [
>> {
>> "last_heartbeat": "2020-04-03T13:10:30.135954Z",
>> "name": "187254@pulpp-ob-581"
>> },
>> {
>> "last_heartbeat": "2020-04-03T13:10:30.132849Z",
>> "name": "187257@pulpp-ob-581"
>> }
>> ],
>> "online_workers": [
>> {
>> "last_heartbeat": "2020-04-03T13:10:29.898377Z",
>> "name": "191...@pulpp-ob-581.bloomberg.com",
>> "pulp_created": "2020-04-02T13:36:11.796937Z",
>> "pulp_href": "/pulp/api/v3/workers/268261b9-f46d-4d37-ab47-0b50ca382637/"
>> },
>> {
>> "last_heartbeat": "2020-04-03T13:10:19.087502Z",
>> "name": "191...@pulpp-ob-581.bloomberg.com",
>> "pulp_created": "2020-04-02T13:36:11.807418Z",
>> "pulp_href": "/pulp/api/v3/workers/4fb4d87c-2c3c-4f64-b6f3-e05d9aaf6fc0/"
>> },
>> {
>> "last_heartbeat": "2020-04-03T13:10:29.498852Z",
>> "name": "191...@pulpp-ob-581.bloomberg.com",
>> "pulp_created": "2020-04-02T13:36:11.810402Z",
>> "pulp_href": "/pulp/api/v3/workers/7b15b6bd-1437-47b8-9832-0b44b326e0fa/"
>> },
>> {
>> "last_heartbeat": "2020-04-03T13:10:29.798941Z",
>> "name": "191...@pulpp-ob-581.bloomberg.com",
>> "pulp_created": "2020-04-02T13:36:11.817391Z",
>> "pulp_href": "/pulp/api/v3/workers/62523740-e109-4828-bcbb-e8459c0944c5/"
>> },
>> 

Re: [Pulp-list] Pulp 3 waiting tasks

[Brian Bouterse]

Since the task that is stalled has a "worker" unassigned it tells me it has
not traveled through the resource-manager yet. All tests in Pulp3
(currently) go through the resource-manager. I can see from your ps output
there is 1 resource-manager running (which is good), and the status API
agrees with that (also good).

So what does RQ thing the situation is? Can you paste the output of `rq
info` please?

Also what version of RQ are do you have installed?

Thanks,
Brian


On Fri, Apr 3, 2020 at 9:39 AM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> Here is the more info. Log is very big. I will send you shortly.
>
> # ./sget status
> {
> "database_connection": {
> "connected": true
> },
> "online_content_apps": [
> {
> "last_heartbeat": "2020-04-03T13:10:30.135954Z",
> "name": "187254@pulpp-ob-581"
> },
> {
> "last_heartbeat": "2020-04-03T13:10:30.132849Z",
> "name": "187257@pulpp-ob-581"
> }
> ],
> "online_workers": [
> {
> "last_heartbeat": "2020-04-03T13:10:29.898377Z",
> "name": "191...@pulpp-ob-581.bloomberg.com",
> "pulp_created": "2020-04-02T13:36:11.796937Z",
> "pulp_href": "/pulp/api/v3/workers/268261b9-f46d-4d37-ab47-0b50ca382637/"
> },
> {
> "last_heartbeat": "2020-04-03T13:10:19.087502Z",
> "name": "191...@pulpp-ob-581.bloomberg.com",
> "pulp_created": "2020-04-02T13:36:11.807418Z",
> "pulp_href": "/pulp/api/v3/workers/4fb4d87c-2c3c-4f64-b6f3-e05d9aaf6fc0/"
> },
> {
> "last_heartbeat": "2020-04-03T13:10:29.498852Z",
> "name": "191...@pulpp-ob-581.bloomberg.com",
> "pulp_created": "2020-04-02T13:36:11.810402Z",
> "pulp_href": "/pulp/api/v3/workers/7b15b6bd-1437-47b8-9832-0b44b326e0fa/"
> },
> {
> "last_heartbeat": "2020-04-03T13:10:29.798941Z",
> "name": "191...@pulpp-ob-581.bloomberg.com",
> "pulp_created": "2020-04-02T13:36:11.817391Z",
> "pulp_href": "/pulp/api/v3/workers/62523740-e109-4828-bcbb-e8459c0944c5/"
> },
> {
> "last_heartbeat": "2020-04-03T13:10:29.598962Z",
> "name": "191...@pulpp-ob-581.bloomberg.com",
> "pulp_created": "2020-04-02T13:36:11.818322Z",
> "pulp_href": "/pulp/api/v3/workers/02e33d62-797d-4797-8fdc-b999efc8cd12/"
> },
> {
> "last_heartbeat": "2020-04-03T13:10:16.685771Z",
> "name": "191...@pulpp-ob-581.bloomberg.com",
> "pulp_created": "2020-04-02T13:36:11.831154Z",
> "pulp_href": "/pulp/api/v3/workers/23e2a484-a877-4083-bcd4-38a0e89fcb49/"
> },
> {
> "last_heartbeat": "2020-04-03T13:10:18.487964Z",
> "name": "191...@pulpp-ob-581.bloomberg.com",
> "pulp_created": "2020-04-02T13:36:11.869871Z",
> "pulp_href": "/pulp/api/v3/workers/9e63708f-bbc0-473d-8de1-8788a1c91f51/"
> },
> {
> "last_heartbeat": "2020-04-03T13:10:29.898354Z",
> "name": "191...@pulpp-ob-581.bloomberg.com",
> "pulp_created": "2020-04-02T13:36:11.880995Z",
> "pulp_href": "/pulp/api/v3/workers/ddd49126-5531-471a-bea1-3aab07bcf8b4/"
> },
> {
> "last_heartbeat": "2020-04-03T13:10:18.887949Z",
> "name": "191...@pulpp-ob-581.bloomberg.com",
> "pulp_created": "2020-04-02T13:36:11.893280Z",
> "pulp_href": "/pulp/api/v3/workers/2ef1e562-845f-4ae7-8007-9b7db8cf73a0/"
> },
> {
> "last_heartbeat": "2020-04-03T13:10:29.798877Z",
> "name": "191...@pulpp-ob-581.bloomberg.com",
> "pulp_created": "2020-04-02T13:36:11.917095Z",
> "pulp_href": "/pulp/api/v3/workers/6e2cf918-af8e-4c5d-bc8f-bef3d3a83dca/"
> },
> {
> "last_heartbeat": "2020-04-03T13:10:15.684710Z",
> "name": "resource-manager",
> "pulp_created": "2020-01-23T18:24:49.246717Z",
> "pulp_href": "/pulp/api/v3/workers/d46e4da0-9735-445b-a502-2aff7ce13ef7/"
> }
> ],
> "redis_connection": {
> "connected": true
> },
> "storage": {
> "free": 32543019880448,
> "total": 33521607376896,
> "used": 978587496448
> },
> "versions": [
> {
> "component": "pulpcore",
> "version": "3.2.1"
> },
> {
> "component": "pulp_rpm",
> "version": "3.2.0"
> },
> {
> "component": "pulp_file",
> "version": "0.2.0"
> }
> ]
>
>
> # ps -awfux |grep pulp
> root 180078 0.0 0.0 107992 616 pts/1 S+ Apr02 0:00 | \_ tail -f
> /var/log/pulp/pulp-config.log
> root 184836 0.0 0.0 124448 2044 pts/2 S+ Apr02 0:00 | \_ vi bbpulp3.py
> root 43270 0.0 0.0 112708 984 pts/3 S+ 09:11 0:00 \_ grep --color=auto pulp
> pulp 187224 0.0 0.0 228600 19188 ? Ss Apr02 0:04
> /opt/utils/venv/pulp/3.7.3/bin/python3
> /opt/utils/venv/pulp/3.7.3/bin/gunicorn pulpcore.app.wsgi:application
> --bind 127.0.0.1:24817 --access-logfile -
> pulp 187251 1.4 0.0 528708 109752 ? S Apr02 20:48 \_
> /opt/utils/venv/pulp/3.7.3/bin/python3
> /opt/utils/venv/pulp/3.7.3/bin/gunicorn pulpcore.app.wsgi:application
> --bind 127.0.0.1:24817 --access-logfile -
> pulp 187231 0.0 0.0 269476 27976 ? Ss Apr02 0:05
> /opt/utils/venv/pulp/3.7.3/bin/python3
> /opt/utils/venv/pulp/3.7.3/bin/gunicorn pulpcore.content:server --bind
> 127.0.0.1:24816 --worker-class aiohttp.GunicornWebWorker -w 2
> --access-logfile -
> pulp 187254 0.0 0.0 485860 68592 ? S Apr02 0:18 \_
> /opt/utils/venv/pulp/3.7.3/bin/python3
> /opt/utils/venv/pulp/3.7.3/bin/gunicorn pulpcore.content:server --bind
> 127.0.0.1:24816 --worker-class aiohttp.GunicornWebWorker -w 

Re: [Pulp-list] Pulp 3 waiting tasks

[Brian Bouterse]

While you are experiencing the issue, can you capture the status API output?

Also can you paste an output of the workers on that system with `ps -awfux
| grep pulp`.

Also do you see any errors in the log? Could you share a copy of the log?

On Fri, Apr 3, 2020 at 9:01 AM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> We have been seeing many waiting tasks. They seem to be stuck forever.
> e.g.
> pulpp-ob-581 /home/bli4/pulp3-script # ./get
> /pulp/api/v3/tasks/14b76b27-9f34-4297-88ed-5ec13cbe5e50/
> HTTP/1.1 200 OK
> Allow: GET, PATCH, DELETE, HEAD, OPTIONS
> Connection: keep-alive
> Content-Length: 323
> Content-Type: application/json
> Date: Fri, 03 Apr 2020 12:56:02 GMT
> Server: nginx/1.16.1
> Vary: Accept, Cookie
> X-Frame-Options: SAMEORIGIN
>
> {
> "created_resources": [],
> "error": null,
> "finished_at": null,
> "name": "pulpcore.app.tasks.base.general_update",
> "progress_reports": [],
> "pulp_created": "2020-04-02T13:00:14.881212Z",
> "pulp_href": "/pulp/api/v3/tasks/14b76b27-9f34-4297-88ed-5ec13cbe5e50/",
> "reserved_resources_record": [],
> "started_at": null,
> "state": "waiting",
> "worker": null
> }
>
> What could be the reason for these stuck waiting tasks? How should we
> troubleshot the issue?
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Pulp 3.2 Scale Testing to 27K Repositories and 173K Repository Versions

[Brian Bouterse]

I recently did some scale testing for Pulp3, and I posted the performance
results and reproducer scripts to the Pulp blog today.

https://pulpproject.org/2020/03/05/pulp-3-scale-testing/

Any feedback, or questions are welcome.

Thanks!
Brian
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulp 3 django.request:WARNING

[Brian Bouterse]

I think maybe this issue is related to https://pulp.plan.io/issues/6057
When a fix for 6057 is released maybe you can confirm if this is happening
still or not.

Thanks!
Brian


On Fri, Jan 31, 2020 at 9:31 AM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> Hi All,
>
> Just noticed we continues get the following messages in the logs. Has
> anyone seen this before? How can we fix this?
>
> Jan 31 09:26:01 pulp gunicorn[147915]: 127.0.0.1 [31/Jan/2020:14:26:01
> +] "GET / HTTP/1.0" 404 172 "-" "-"
> Jan 31 09:26:01 pulp gunicorn[147883]: pulp: django.request:WARNING: Not
> Found: /
> Jan 31 09:26:01 pulp gunicorn[147883]: 127.0.0.1 - - [31/Jan/2020:14:26:01
> +] "GET / HTTP/1.0" 404 77 "-" "-"
> Jan 31 09:26:03 pulp gunicorn[147883]: pulp: django.request:WARNING: Not
> Found: /
> Jan 31 09:26:03 pulp gunicorn[147883]: 127.0.0.1 - - [31/Jan/2020:14:26:03
> +] "GET / HTTP/1.0" 404 77 "-" "-"
> Jan 31 09:26:03 pulp gunicorn[147915]: 127.0.0.1 [31/Jan/2020:14:26:03
> +] "GET / HTTP/1.0" 404 172 "-" "-"
> Jan 31 09:26:06 pulpp-ob-581 gunicorn[147915]: 127.0.0.1
> [31/Jan/2020:14:26:06 +] "GET / HTTP/1.0" 404 172 "-" "-"
> Jan 31 09:26:06 pulp gunicorn[147883]: pulp: django.request:WARNING: Not
> Found: /
> Jan 31 09:26:06 pulp gunicorn[147883]: 127.0.0.1 - - [31/Jan/2020:14:26:06
> +] "GET / HTTP/1.0" 404 77 "-" "-"
> Jan 31 09:26:08 pulp gunicorn[147883]: pulp: django.request:WARNING: Not
> Found: /
> Jan 31 09:26:08 pulp gunicorn[147883]: 127.0.0.1 - - [31/Jan/2020:14:26:08
> +] "GET / HTTP/1.0" 404 77 "-" "-"
> Jan 31 09:26:08 pulp gunicorn[147915]: 127.0.0.1 [31/Jan/2020:14:26:08
> +] "GET / HTTP/1.0" 404 172 "-" "-"
> Jan 31 09:26:11 pulp gunicorn[147915]: 127.0.0.1 [31/Jan/2020:14:26:11
> +] "GET / HTTP/1.0" 404 172 "-" "-"
> Jan 31 09:26:11 pulp gunicorn[147883]: pulp: django.request:WARNING: Not
> Found: /
> Jan 31 09:26:11 pulp gunicorn[147883]: 127.0.0.1 - - [31/Jan/2020:14:26:11
> +] "GET / HTTP/1.0" 404 77 "-" "-"
> Jan 31 09:26:13 pulp gunicorn[147883]: pulp: django.request:WARNING: Not
> Found: /
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] pulpcore 3.1.1 is Generally Available

[Brian Bouterse]

Pulpcore 3.1.1 is available and contains bugfixes for database connection
loss errors, S3 integration, and more. See the release notes for all the
details. https://docs.pulpproject.org/en/3.1.1/changes.html


Pulpcore 3.1.1 contains only bugfixes and no breaking changes for either
users or plugin developers. This means all existing released plugins
compatible with 3.1.0 should be able to upgrade onto pulpcore==3.1.1
without issue.

## New Installations

If you are using a new install, you can use the latest ansible installer
and you'll receive 3.1.1. The quick start guides below are useful for that.

* pulp_rpm installation --
https://pulp-rpm.readthedocs.io/en/latest/installation.html
* pulp_container installation --
https://pulp-container.readthedocs.io/en/latest/installation.html
* pulp_file installation --
https://pulp-file.readthedocs.io/en/latest/installation.html

## Upgrading from 3.0.z

You should first upgrade your installer to the latest one from PyPI and
re-run it. If you get the new installer, you'll get a newer pulpcore
release. If you rerun an older version of the installer, you'll get an
older version (downgrading is not supported).

## Quickstart

Pulp_rpm offers a quickstart guide (link below) which uses pulplift (link
below) to create a not-long-living installation just to try it out. These
same instructions can be adapted to the other plugins also.

https://pulp-rpm.readthedocs.io/en/latest/quickstart.html
https://github.com/pulp/pulplift

## Migrating from Pulp2 -> Pulp3

A Pulp 2 -> Pulp 3 migration tool is now at RC1 and can migrate docker and
iso content and repositories ( https://github.com/pulp/pulp-2to3-migration
). RPM migrations are coming soon, but are not ready yet.

## Getting Help or Reporting Bugs

Find where to get help or report bugs via the help page on the website (
https://pulpproject.org/help/ ). If you're unsure where to start, we
recommend emailing your question to the Pulp user mailing list (
https://www.redhat.com/mailman/listinfo/pulp-list ).

## Feedback

Let us know what you think! Please send feedback to either:

* Pulp user mailing list --
https://www.redhat.com/mailman/listinfo/pulp-list
* Pulp developer mailing list --
https://www.redhat.com/mailman/listinfo/pulp-dev

## Thanks!

Thank you to all the developers, integrators, testers, and users who have
contributed to this bugfix release.
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulp3 High availability and disaster recovery

[Brian Bouterse]

Hi Bin Li,

When you perform the failover to the passive standby Pulp what does the
status API show for its workers before, during, and after failover? Note
the workers and webservers all need to be using the same Redis because the
task data flows through Redis.

I'm wondering if maybe your passive Pulp after failover is indeed receiving
the workers that belong to it and that all "initially active" workers are
declared missing/dead. Then if the right workers are being shown they all
need to be using the same Redis instance (the webservers and the workers).

Also the "stalling" task should show the "worker" it was assigned to.
Sharing that info would help to know where the work was being routed to
also.



On Tue, Feb 18, 2020 at 9:28 AM Dennis Kliban  wrote:

> The redis cluster support needs to be added to rq actually[0,1]. Looks
> like there is an open PR but it hasn't moved forward in a long time[2].
>
> [0] https://github.com/rq/rq/issues/862
> [1] https://github.com/rq/rq/issues/1048
> [2] https://github.com/rq/rq/pull/942
>
> On Tue, Feb 18, 2020 at 9:07 AM Dennis Kliban  wrote:
>
>> How many instances of Redis are involved? Is every pulpcore-api instance
>> and pulpcore-worker instance pointing to the same redis instance? This is
>> necessary for the work to be routed correctly.
>>
>> Pulpcore currently uses redis-py, which does not support connecting to a
>> Redis Cluster[0]. However, we should investigate if it's viable to switch
>> to using redis-py-cluster[1].
>>
>> [0] https://github.com/andymccurdy/redis-py/issues/931
>> [1] https://github.com/Grokzen/redis-py-cluster
>>
>> On Thu, Feb 13, 2020, 6:28 PM Bin Li (BLOOMBERG/ 120 PARK) <
>> bli...@bloomberg.net> wrote:
>>
>>> Hi Brian,
>>> I did a quick test on a active passive pulp 3.1 setup. Two pulp servers
>>> are pointing to the same external postgres database. Only one server is
>>> active at any time. Redis queue resides on the localhost. The /var/lib/pulp
>>> are synced from primary to the contingency host.
>>> After I shutdown primary host, I was able to bring up the contingency
>>> pulp server and created a repo. Deleting any repo stuck in a waiting state.
>>> Then I started primary host and shutdown contingency host, I was able to
>>> delete repos I created on the contingency host but all previous delete job
>>> continually stuck in the waiting state.
>>> I am wonder if anything I could do to make this work on contingency host
>>> or this setup is not going to work?
>>>
>>> Thanks
>>>
>>>
>>> From: pulp-list@redhat.com At: 01/03/20 12:01:44
>>> To: pulp-list@redhat.com
>>> Subject: Pulp-list Digest, Vol 122, Issue 1
>>>
>>> Send Pulp-list mailing list submissions to
>>> pulp-list@redhat.com
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>> or, via email, send a message with subject or body 'help' to
>>> pulp-list-requ...@redhat.com
>>>
>>> You can reach the person managing the list at
>>> pulp-list-ow...@redhat.com
>>>
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of Pulp-list digest..."
>>>
>>>
>>> Today's Topics:
>>>
>>> 1. Re: pulp3 High availability and disaster recovery (Brian Bouterse)
>>>
>>>
>>> --
>>>
>>> Message: 1
>>> Date: Thu, 2 Jan 2020 16:10:29 -0500
>>> From: Brian Bouterse 
>>> To: JASON STELZER 
>>> Cc: pulp-list 
>>> Subject: Re: [Pulp-list] pulp3 High availability and disaster recovery
>>> Message-ID:
>>> 
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> Sorry for the late reply. Each component of Pulp itself can be deployed
>>> in
>>> HA configurations. Of the services Pulp's processes depend on, Redis is
>>> the
>>> one service that can't run as a full cluster because RQ doesn't support
>>> that yet, so the best you can do is a hot-spare Redis that auto-fails
>>> over.
>>> That isn't graceful failover so when traffic routes to your hot-spare
>>> Redis
>>> it has to data and doesn't have the tasking system's data. Those Pulp
>>> tasks
>>> would be cancelled, and Pulp would be immediately ready to accept new
>>> tasks
>>> so they could be resubmitted, e.g. Katello resubm

Re: [Pulp-list] pulpcore 3.1 is Generally Available

[Brian Bouterse]

pulp_file 0.1.1 is released and is compatible with pulpcore 3.1. It
contained only bugfixes so it is a z-release.

https://pulp-file.readthedocs.io/en/0.1.1/changes.html#id1
https://pypi.org/project/pulp-file/



On Fri, Jan 31, 2020 at 6:41 PM Ina Panova  wrote:

> pulp_container 1.1.0 is GA with the set of new features and bugfixes. It
> is also compatible with pulpcore 3.1.
>
> https://pulp-container.readthedocs.io/en/1.1/changes.html
> https://pypi.org/project/pulp-container/1.1.0/
>
>
> 
> Regards,
>
> Ina Panova
> Senior Software Engineer| Pulp| Red Hat Inc.
>
> "Do not go where the path may lead,
>  go instead where there is no path and leave a trail."
>
>
> On Fri, Jan 31, 2020 at 1:12 PM Brian Bouterse 
> wrote:
>
>> Pulpcore 3.1.0 is the first feature release of Pulp 3 and contains some
>> significant features and fixes. See the release notes for all the details.
>> https://docs.pulpproject.org/en/3.1.0/changes.html Here are some
>> highlights:
>>
>> * Azure support
>> * signing services for plugins to use
>> * restriction of file:// paths and a new setting to mark filesystem areas
>> as safe to import from
>> * improved webserver auth support
>> * various bugfixes including resolution of a task hanging issue, S3
>> filename fix, and more
>>
>> Pulpcore 3.1.0 contains no user API breaking changes (ever), but plugins
>> will require a compatibility release due to the plugin API bringing
>> breaking changes in with each Y-release. Users should watch for
>> announcements from plugins soon and upgrade when all of your plugins are
>> compatible. Plugin writers can see their changelog to see what has changed
>> in the plugin API here:
>>
>> ## New Installations
>>
>> If you are using a new install, you can use the ansible-installer
>> directly and you'll receive 3.1.0. The quick start guides below are
>> useful for that.
>>
>> * pulp_rpm installation -- https://pulp-rpm.readthedocs.io/en/3.0
>> /installation.html
>> * pulp_container installation --
>> https://pulp-container.readthedocs.io/en/1.0/installation.html
>> * pulp_file installation -- https://pulp-file.readthedocs.io/en/0.1
>> .0/installation.html
>>
>> ## Updating from 3.0.z
>>
>> You should first upgrade your installer to the latest one from PyPI and
>> re-run it. Starting with 3.1.0 each release will push a new version of the
>> installer. If you get the new installer, you'll get a newer pulpcore
>> release.
>>
>> ## Quickstart
>>
>> Pulp_rpm offers a quickstart guide (link below) which uses pulplift (link
>> below) to create a not-long-living installation just to try it out. These
>> same instructions can be adapted to the other plugins also.
>>
>> https://pulp-rpm.readthedocs.io/en/3.0/quickstart.html
>> https://github.com/pulp/pulplift
>>
>> ## Migrating from Pulp2 -> Pulp3
>>
>> A Pulp 2 -> Pulp 3 migration tool is now in beta and can migrate docker
>> and iso content ( https://github.com/pulp/pulp-2to3-migration ). RPM
>> migrations are coming soon, but are not ready yet.
>>
>> ## Getting Help or Reporting Bugs
>>
>> Find where to get help or report bugs via the help page on the website (
>> https://pulpproject.org/help/ ). If you're unsure where to start, we
>> recommend emailing your question to the Pulp user mailing list (
>> https://www.redhat.com/mailman/listinfo/pulp-list ).
>>
>> ## Feedback
>>
>> Let us know what you think! Please send feedback to either:
>>
>> * Pulp user mailing list --
>> https://www.redhat.com/mailman/listinfo/pulp-list
>> * Pulp developer mailing list --
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>> ## Thanks!
>>
>> Thank you to all the developers, integrators, testers, and users who have
>> contributed to this feature release.
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] pulpcore 3.1 is Generally Available

[Brian Bouterse]

Pulpcore 3.1.0 is the first feature release of Pulp 3 and contains some
significant features and fixes. See the release notes for all the details.
https://docs.pulpproject.org/en/3.1.0/changes.html Here are some highlights:

* Azure support
* signing services for plugins to use
* restriction of file:// paths and a new setting to mark filesystem areas
as safe to import from
* improved webserver auth support
* various bugfixes including resolution of a task hanging issue, S3
filename fix, and more

Pulpcore 3.1.0 contains no user API breaking changes (ever), but plugins
will require a compatibility release due to the plugin API bringing
breaking changes in with each Y-release. Users should watch for
announcements from plugins soon and upgrade when all of your plugins are
compatible. Plugin writers can see their changelog to see what has changed
in the plugin API here:

## New Installations

If you are using a new install, you can use the ansible-installer directly
and you'll receive 3.1.0. The quick start guides below are useful for that.

* pulp_rpm installation -- https://pulp-rpm.readthedocs.io/en/3.0
/installation.html
* pulp_container installation -- https://pulp-container.readthedocs.io/en/1
.0/installation.html
* pulp_file installation -- https://pulp-file.readthedocs.io/en/0.1
.0/installation.html

## Updating from 3.0.z

You should first upgrade your installer to the latest one from PyPI and
re-run it. Starting with 3.1.0 each release will push a new version of the
installer. If you get the new installer, you'll get a newer pulpcore
release.

## Quickstart

Pulp_rpm offers a quickstart guide (link below) which uses pulplift (link
below) to create a not-long-living installation just to try it out. These
same instructions can be adapted to the other plugins also.

https://pulp-rpm.readthedocs.io/en/3.0/quickstart.html
https://github.com/pulp/pulplift

## Migrating from Pulp2 -> Pulp3

A Pulp 2 -> Pulp 3 migration tool is now in beta and can migrate docker and
iso content ( https://github.com/pulp/pulp-2to3-migration ). RPM migrations
are coming soon, but are not ready yet.

## Getting Help or Reporting Bugs

Find where to get help or report bugs via the help page on the website (
https://pulpproject.org/help/ ). If you're unsure where to start, we
recommend emailing your question to the Pulp user mailing list (
https://www.redhat.com/mailman/listinfo/pulp-list ).

## Feedback

Let us know what you think! Please send feedback to either:

* Pulp user mailing list --
https://www.redhat.com/mailman/listinfo/pulp-list
* Pulp developer mailing list --
https://www.redhat.com/mailman/listinfo/pulp-dev

## Thanks!

Thank you to all the developers, integrators, testers, and users who have
contributed to this feature release.
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulp 3 list packages

[Brian Bouterse]

What do the logs say about why the gunicorn process serving pulp-api is
dying? Would you want to file an issue https://pulp.plan.io/issues/new so
we can do some testing?

As an aside, I recommend using paging when pulling so many items from an
API. You could decompose your large request to more, smaller requests like:

http GET localhost/pulp/api/v3/content/rpm/packages/ offset=0 limit==1
repository_version==/pulp/api/v3/repositories/rpm/rpm/2f46d319-7997-4e86-b159-8babee4aba19/versions/1/
--timeout=200
http GET localhost/pulp/api/v3/content/rpm/packages/ offset=1
limit==1
repository_version==/pulp/api/v3/repositories/rpm/rpm/2f46d319-7997-4e86-b159-8babee4aba19/versions/1/
--timeout=200

What's interesting about more, smaller requests is you can likely get the
data out of Pulp a lot faster since you can engage more gunicorn processes
in parallel. Conceptually one large query is attractive though, so maybe we
could improve that if you file it.

Another idea is to limit which fields are being returned to get at the data
you need faster.

All the best,
Brian



On Thu, Jan 30, 2020 at 2:46 PM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> The rhel 7 servers rpm repo has more than 26k packages. I got an "502 Bad
> Gateway" error if I tried to list all of them
>
> http GET localhost/pulp/api/v3/content/rpm/packages/ limit==2
> repository_version==/pulp/api/v3/repositories/rpm/rpm/2f46d319-7997-4e86-b159-8babee4aba19/versions/1/
> --timeout=200
>
> What could cause this? Is there a fix?
>
> Thanks
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] pulpcore 3.0.1 is Generally Available

[Brian Bouterse]

Pulpcore 3.0.1 is the first bugfix release and contains some significant
fixes. See the release notes for all the details.
https://docs.pulpproject.org/en/3.0.1/changes.html

Pulpcore 3.0.1 contains only bugfixes and no breaking changes for either
users or plugin developers. This means all existing released plugins should
be able to upgrade onto pulpcore==3.0.1 without issue.

## New Installations

If you are using a new install, you can use the ansible-installer directly
and you'll receive 3.0.1. The quick start guides below are useful for that.

* pulp_rpm installation --
https://pulp-rpm.readthedocs.io/en/3.0/installation.html
* pulp_container installation --
https://pulp-container.readthedocs.io/en/1.0/installation.html
* pulp_file installation --
https://pulp-file.readthedocs.io/en/0.1.0/installation.html

## Updating from 3.0.0

For safety reasons, if you rerun the installer exactly as you did before
you won't be updated. There are a various scenarios a user may want to
rerun the installer, e.g. settings changes, fixing drift, etc, and you
wouldn't want to unexpectedly update.

Two new boolean variables are added to the installer, `upgrade` (under
`pulp_install_plugins`) and `pulp_upgrade`. The former will update the
plugin it is specified under; the latter will update pulpcore itself.

To update you should:

1. Download the latest installer from https://github.com/pulp/ansible-pulp
2. Set the `pulp_upgrade` variable to `True`
3. Run the installer

Note: Later on, additional variables will be added to differentiate updates
vs upgrades.

## Quickstart

Pulp_rpm offers a quickstart guide (link below) which uses pulplift (link
below) to create a not-long-living installation just to try it out. These
same instructions can be adapted to the other plugins also.

https://pulp-rpm.readthedocs.io/en/3.0/quickstart.html
https://github.com/pulp/pulplift

## Migrating from Pulp2 -> Pulp3

A Pulp 2 -> Pulp 3 migration tool is in-progress (
https://github.com/pulp/pulp-2to3-migration ), but not fully available yet.
Until it is, users are recommended to **not** install Pulp3 on top of their
Pulp2 system until it is available.

This tool is progressing well. We hope this tool will be available within a
month or two to migrate the plugin types that are released.

## Getting Help or Reporting Bugs

Find where to get help or report bugs via the help page on the website (
https://pulpproject.org/help/ ). If you're unsure where to start, we
recommend emailing your question to the Pulp user mailing list (
https://www.redhat.com/mailman/listinfo/pulp-list ).

## Feedback

Let us know what you think! Please send feedback to either:

* Pulp user mailing list --
https://www.redhat.com/mailman/listinfo/pulp-list
* Pulp developer mailing list --
https://www.redhat.com/mailman/listinfo/pulp-dev

## Thanks!

Thank you to all the developers, integrators, testers, and users who have
contributed to this bugfix release.
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulp3 s3 migration path

[Brian Bouterse]

It's a bit manual, but a second pulp system could be setup with the new
type of storage and it could sync from the first system.

On Tue, Jan 7, 2020 at 2:46 PM David Davis  wrote:

> No, there is not. You cannot change your storage once you have set up and
> started to use pulp.
>
> I'm not sure what would be involved in migrating a Pulp system from one
> storage type to another but you can open a story in our tracker if you'd
> like.
>
> David
>
>
> On Tue, Jan 7, 2020 at 2:31 PM Bin Li (BLOOMBERG/ 120 PARK) <
> bli...@bloomberg.net> wrote:
>
>>
>> Is there a migration path to move from local file system to s3 after
>> creating repos, publications and distributions?
>>
>> Thanks
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulp3 High availability and disaster recovery

[Brian Bouterse]

Sorry for the late reply. Each component of Pulp itself can be deployed in
HA configurations. Of the services Pulp's processes depend on, Redis is the
one service that can't run as a full cluster because RQ doesn't support
that yet, so the best you can do is a hot-spare Redis that auto-fails over.
That isn't graceful failover so when traffic routes to your hot-spare Redis
it has to data and doesn't have the tasking system's data. Those Pulp tasks
would be cancelled, and Pulp would be immediately ready to accept new tasks
so they could be resubmitted, e.g. Katello resubmits some job failures I
believe.

More docs about this are here:
https://docs.pulpproject.org/components.html#architecture-and-deploying
More questions are welcome; sorry for the slow response. If you can see any
way to improve the docs and want to get involved, PRs are welcome!

-Brian


On Mon, Nov 18, 2019 at 7:37 AM JASON STELZER 
wrote:

> For what it is worth, at heart pulp3 is a django app. So, following the
> advice for HA and django apps generally works. A lot of it is driven by the
> particulars of your use case.
>
> My use case is a little different than yours I'm sure. But in terms of HA
> for now I'm good with a balancer and nodes in multiple azs, an RDS db with
> failover, and regular db backups.
>
> In my case, the pulp3 server is far enough behind the scenes that even if
> there were to be a several hour outage, the impact would be minimal. YMMV.
>
> Others can chime in with pulp3 specifics.
>
> On Fri, Nov 15, 2019 at 11:41 AM Bin Li (BLOOMBERG/ 120 PARK) <
> bli...@bloomberg.net> wrote:
>
>> Does pulp3 support active/active or active/passive configuration? What is
>> the strategy to restore the pulp3 service on a different server if the
>> primary is down? Do we have any documentation on this topic?
>>
>> Thanks
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
>
>
> --
> J.
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] RPM, Container, and File plugins for Pulp 3.0 are Generally Available

[Brian Bouterse]

The Pulp developer community is excited to announce the general
availability of the following:

* pulp_rpm 3.0 to manage RPM content --
https://pulp-rpm.readthedocs.io/en/3.0/
* pulp_container 1.0 to manage Docker content --
https://pulp-container.readthedocs.io/en/1.0/
* pulp_file 0.1 to manage Files --
https://pulp-file.readthedocs.io/en/0.1.0/

These plugins are compatible with the pulpcore 3.0.0 (
https://docs.pulpproject.org/ ) release which is also Generally Available
today!

### RPM Features

pulp_rpm can sync, organize, and distribute the following content types
currently:

* RPMs
* SRPMs
* Advisory Content, i.e. Errata
* Distribution Trees (kickstart trees)
* Yum Metadata, e.g. comps.xml
* modularity content

You can upload the following to pulp_rpm:

* RPM content
* SRPM
* Advisory

### Container Features

pulp_container can sync, organize, and distribute Docker content.

### File Features

pulp_file can sync, organize, upload, and distribute File content.


## Installation

See the installation docs (links below) for each plugin on how to install.
They all use the same
installer so they are very similar. Plugins are capable of being installed
all together on one
installation.

* pulp_rpm installation --
https://pulp-rpm.readthedocs.io/en/3.0/installation.html
* pulp_container installation --
https://pulp-container.readthedocs.io/en/1.0/installation.html
* pulp_file installation --
https://pulp-file.readthedocs.io/en/0.1.0/installation.html


## Quickstart

Pulp_rpm offers a quickstart guide (link below) which uses pulplift (link
below) to create a not-long-living installation just to try it out. These
same instructions can be adapted to the other plugins also.

https://pulp-rpm.readthedocs.io/en/3.0/quickstart.html
https://github.com/pulp/pulplift


## Migrating from Pulp2 -> Pulp3

A Pulp 2 -> Pulp 3 migration tool is in-progress (
https://github.com/pulp/pulp-2to3-migration ),
but not fully available yet. Until it is, users are recommended to **not**
install Pulp3 on top of their Pulp2 system until it is available.

We hope this tool will be available within a month or two to migrate the
plugin types that are
released.


## Why these version numbers?

These Pulp plugins use semantic versioning, so the different version
numbers have different meanings summarized below.

* pulp_rpm is 3.0 to indicate future releases will not contain API breaking
changes, and it's the
  next major release after pulp_rpm 2.y
* pulp_container uses 1.0 to indicate future releases will not contain API
breaking changes, but it
  is overall a new plugin.
* pulp_file uses 0.1 to indicate future Y-releases may contain API
backwards incompatible breaking
  changes, and it is a new plugin.


## Getting Help or Reporting Bugs

Find where to get help or report bugs via the help page on the website (
https://pulpproject.org/help/ ). If you're unsure where to start, we
recommend emailing your question to the Pulp user mailing list (
https://www.redhat.com/mailman/listinfo/pulp-list ).


## Feedback

Let us know what you think! Please send feedback to either:

* Pulp user mailing list --
https://www.redhat.com/mailman/listinfo/pulp-list
* Pulp developer mailing list --
https://www.redhat.com/mailman/listinfo/pulp-dev


## Thanks!

Thank You to all the developers, integrators, testers, and users who have
contributed to this huge milestone for the Pulp project. The collaboration
that created this has been excellent.
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Pulpcore RC9

[Brian Bouterse]

The pulpcore 3.0.0rc9 and pulp_file 0.1.0rc2 packages are available
on PyPI [0][1]. There are a lot of changes that have happened; see the
release notes for all the details. There were a few user-facing breaking
changes, please especially read that part. Note this is expected to be the
final code for 3.0.0 which will formally release on Dec 12th.

# Release Notes

pulpcore -
https://docs.pulpproject.org/en/3.0.0rc9/changes.html#rc9-2019-12-03
pulp_file -
https://github.com/pulp/pulp_file/blob/master/CHANGES.rst#010rc2-2019-12-03


# Can I upgrade from earlier RCs?

No, you should install RC9 on a fresh to fresh database. Upgrading from
earlier Release Candidates is not supported.


# What plugins are compatible with RC9?

There were breaking changes introduced in RC9 so plugin releases prior to
RC9 will likely not be compatible with RC9. Please follow the Pulp list for
more information about plugin releases.


# How do I try this?

* Use pulplift ( https://github.com/pulp/pulplift ) which creates a VM for
you locally and runs the Pulp installer.
* Use the Pulp Installer ( https://github.com/pulp/ansible-pulp ) on a
machine you provisioned.


# What about client bindings?

Python [2] and Ruby [3] bindings are built+published to PyPI and
rubgems.org respectively.
See [2][3] for details on usage.


# Where do I get help?

Ask questions via pulp-list@redhat.com or come into the #pulp channel on
Freenode for help.


[0]: https://pypi.org/project/pulpcore/
[1]: https://pypi.org/project/pulp-file/
[2]:
https://docs.pulpproject.org/en/3.0.0rc9/client_bindings.html#python-client-for-pulpcore-s-rest-api
[3]:
https://docs.pulpproject.org/en/3.0.0rc9/client_bindings.html#ruby-client-for-pulpcore-s-rest-api
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Pulp 3.0.0 Release Date, December 12

[Brian Bouterse]

title correction: Dec 12th

On Tue, Nov 26, 2019 at 11:08 AM Brian Bouterse  wrote:

> We're planning one final release candidate 3.0.0rc9 for Dec 3rd, and a
> generally available release with improved documentation on Dec 12th. This
> will include pulp_rpm, pulp_container, and pulp_file as well. This is a
> delay from the initial GA release date of Dec 3rd.
>
> Read the blog post for more details on the schedule change.
> https://pulpproject.org/2019/11/25/pulp-3-GA-update/
>
> Any questions or concerns are welcome! Please share.
>
> -Brian
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Pulp 3.0.0 Release Date, December 13

[Brian Bouterse]

We're planning one final release candidate 3.0.0rc9 for Dec 3rd, and a
generally available release with improved documentation on Dec 12th. This
will include pulp_rpm, pulp_container, and pulp_file as well. This is a
delay from the initial GA release date of Dec 3rd.

Read the blog post for more details on the schedule change.
https://pulpproject.org/2019/11/25/pulp-3-GA-update/

Any questions or concerns are welcome! Please share.

-Brian
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Role Based Access Control for Pulp 3.y -- Use Cases

[Brian Bouterse]

Here are the use cases and notes
<https://docs.google.com/document/d/1byC3kXNqK0b4vA7iwkS9FDUdU60gfofixzxCMfLBt8E/edit?usp=sharing>
from the last meeting, and here is the full transcript
<https://pulpadmin.fedorapeople.org/triage/pulp-dev/2019/pulp-dev.2019-11-20-14.01.log.html>.
Feel free to add more use cases into it ^. We'll have another chat meeting
in early December, see details below:

When: Tuesday, December 2  @  9:00 – 10:00am EST or in your time zone
<https://www.worldtimebuddy.com/?qm=1=4460243,100,3448439,3078610=4460243=2019-12-02=9-10>
.
Where: #pulp-dev on freenode
Agenda: We'll discuss the use cases in this doc
<https://docs.google.com/document/d/1byC3kXNqK0b4vA7iwkS9FDUdU60gfofixzxCMfLBt8E/edit?usp=sharing>



On Tue, Nov 19, 2019 at 4:54 PM Brian Bouterse  wrote:

> The RBAC use case discussion is happening tomorrow! Details are below.
>
> On Tue, Oct 29, 2019 at 11:59 AM Brian Bouterse 
> wrote:
>
>> I'd like to host a chat meeting to gather use cases for Role Based Access
>> Control needs in Pulp 3.y. The meeting details and agenda link are below.
>> Both users and plugin developers are invited. Please join if you're
>> interested!
>>
>> When: Wednesday, November 20⋅9:00 – 10:00am EST or in your time zone
>> <https://www.worldtimebuddy.com/?qm=1=4460243,100,3448439,3078610=4460243=2019-11-20=9-10>
>> .
>> Where: #pulp-dev on freenode
>> Agenda: https://etherpad.net/p/Pulp_-_RBAC_Use_Cases
>>
>> Questions and feedback are also welcome ahead of time.
>>
>> Thanks!
>> Brian
>>
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] How to troubleshoot: Cannot sync EPEL

[Brian Bouterse]

On Mon, Nov 11, 2019 at 7:38 AM Tatiana Tereshchenko 
wrote:

> Unintentionally, this conversation went off-list on Friday. I'm sending
> this note to let everyone know that the thread is closed.
>
> JFYI, the summary of the discussion:
>  - .treeinfo error is not an error, it's just a way of Pulp to find
> distribution(kickstart) tree. Maybe this message should be logged at debug
> level not to confuse people.
>
+1 we should log this at info or debug.

 - Joey's issue was mostly environmental
>  - EPEL7URL http://download.fedoraproject.org/pub/epel/7/x86_64/ doesn't
> work for sync, but https://dl.fedoraproject.org/pub/epel/7/x86_64/ does.
> Needs investigation.
>
> Tanya
>
>
>
> On Fri, Nov 8, 2019 at 8:18 PM Dumont, Joey 
> wrote:
>
>> Hi,
>>
>>
>> Versions:
>>
>>
>> "versions": [
>> {
>> "component": "pulpcore",
>> "version": "3.0.0rc7"
>> },
>> {
>> "component": "pulpcore-plugin",
>> "version": "0.1.0rc7"
>> },
>> {
>> "component": "pulp_rpm",
>> "version": "3.0.0b7"
>> },
>> {
>> "component": "pulp_file",
>> "version": "0.1.0b4"
>> }
>> ]
>> }
>>
>> Relevant error messages I found in journalctl:
>>
>> pulp: backoff:ERROR: Giving up _run(...) after 1 tries
>> (aiohttp.client_exceptions.ClientResponseError: 404, message='Not Found',
>> url=URL('
>> https://d2lzkl7pfhq30w.cloudfront.net/pub/epel/7/x86_64/treeinfo'))
>> Giving up _run(...) after 1 tries
>> (aiohttp.client_exceptions.ClientResponseError: 404, message='Not Found',
>> url=URL('
>> https://d2lzkl7pfhq30w.cloudfront.net/pub/epel/7/x86_64/treeinfo'))
>> pulp: backoff:ERROR: Giving up _run(...) after 1 tries
>> (aiohttp.client_exceptions.ClientResponseError: 404, message='Not Found',
>> url=URL('
>> https://d2lzkl7pfhq30w.cloudfront.net/pub/epel/7/x86_64/.treeinfo'))
>> Giving up _run(...) after 1 tries
>> (aiohttp.client_exceptions.ClientResponseError: 404, message='Not Found',
>> url=URL('
>> https://d2lzkl7pfhq30w.cloudfront.net/pub/epel/7/x86_64/.treeinfo'))
>> pulp: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing:
>> repository=test-rpm remote=test-rpm-remote
>> pulp: rq.worker:INFO: Cleaning registries for queue: resource-manager
>> pulp: rq.worker:INFO: resource-manager: Job OK
>> (6cc2e593-83b5-4ef9-9bcf-84175707670c)
>> pulp: rq.worker:INFO:
>> reserved-resource-worke...@ip-100-96-160-204.opsaws.nrc.ca:
>> e8c293fe-e5f2-402f-a3e3-a33f421ed8cb
>> 127.0.0.1 - admin [06/Nov/2019:14:15:38 +] "POST
>> /pulp/api/v3/remotes/rpm/rpm/fa1c9782-11cb-415f-b710-17df275a82de/sync/
>> HTTP/1.1" 202 67 "-" "HTTPie/0.9.4"
>> pulp: rq.worker:INFO: resource-manager:
>> 6cc2e593-83b5-4ef9-9bcf-84175707670c
>>
>> It's looking for a .treeinfo file, without success, then failing. What is
>> a .treeinfo file?
>>
>>
>> Joey Dumont
>>
>> Technical Advisor, Knowledge, Information, and Technology Services
>> National Research Council Canada / Governement of Canada
>> joey.dum...@nrc-cnrc.gc.ca / Tel: 613-990-8152 / Cell: 438-340-7436
>>
>> Conseiller technique, Services du savoir, de l'information et de la
>> technologie
>> Conseil national de recherches Canada / Gouvernement du Canada
>> joey.dum...@nrc-cnrc.gc.ca / Tél.: 613-990-8152 / Tél. cell.:
>> 438-340-7436
>> --
>> *From:* Tatiana Tereshchenko 
>> *Sent:* 08 November 2019 14:02
>> *To:* Dumont, Joey
>> *Cc:* pulp-list@redhat.com
>> *Subject:* Re: [Pulp-list] How to troubleshoot: Cannot sync EPEL
>>
>> Hi Joey,
>>
>> Can you please share which version you are running?
>> http GET :24817/pulp/api/v3/status/
>>
>> Logs should be available in the standard location depending on your
>> system and configuration.
>> E.g. if you have journalctl, you can see pulp logs there.
>>
>> Tanya
>>
>>
>> On Wed, Nov 6, 2019 at 3:18 PM Dumont, Joey 
>> wrote:
>>
>>> I am trying to sync the EPEL repo using the pulp-rpm plugin. Here's the
>>> repo info:
>>>
>>>
>>> http GET
>>> :24817/pulp/api/v3/remotes/rpm/rpm/fa1c9782-11cb-415f-b710-17df275a82de/
>>> {
>>> "download_concurrency": 20,
>>> "name": "test-rpm-remote",
>>> "policy": "immediate",
>>> "proxy_url": null,
>>> "pulp_created": "2019-10-31T15:51:43.651250Z",
>>> "pulp_href":
>>> "/pulp/api/v3/remotes/rpm/rpm/fa1c9782-11cb-415f-b710-17df275a82de/",
>>> "pulp_last_updated": "2019-11-05T19:56:38.242131Z",
>>> "ssl_ca_certificate": null,
>>> "ssl_client_certificate": null,
>>> "ssl_client_key": null,
>>> "ssl_validation": true,
>>> "url": "http://download.fedoraproject.org/pub/epel/7/x86_64/;
>>> }
>>>
>>>
>>> Here's the repo used to do the sync:
>>>
>>> http GET
>>> :24817/pulp/api/v3/repositories/086a95b2-6aaa-4786-b317-f0e91fa2b800/
>>> {
>>> "description": null,
>>> "latest_version_href": null,
>>> "name": "test-rpm",
>>> "plugin_managed": false,
>>> "pulp_created": "2019-10-31T15:41:08.128783Z",

Re: [Pulp-list] pulp3 publication

[Brian Bouterse]

Hi Bin,

Thank you for your question. We talked this over some during the "open
floor" part of developer triage. There are a few use cases that motivate
allowing multiple publications of a single RepositoryVersion.

The main reason: Publishers could take various options which could make the
two publications meaningfully different even though they came from one
RepositoryVersion with the same content. For example, signing the
publication assets with two different signing keys. Or perhaps filtering
options that limit the types of content being published, e.g. skip drpms,
etc.

Small secondary reason: Even if the options aren't different, the metadata
timestamps would be so they aren't exactly identical. This is a technical
difference, not really a useful one, but we include it here for
completeness.

Also FYI, plugins could adjust this behavior, plugin-by-plugin so if ^
doesn't make sense for a specific plugin, the plugin developer could
enforce each RepositoryVersion has no more than 1 RepositoryVersion.

Please let us know how you think it could be improved (either in a specific
plugin or more broadly).

Best,
Brian


On Thu, Nov 7, 2019 at 4:20 PM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> It looks like I could keep creating publications on the same repository
> version. I thought the publications are the same base on the same
> repository version. Just wondering why this is allowed?
>
> Thanks
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Role Based Access Control for Pulp 3.y -- Use Cases

[Brian Bouterse]

I'd like to host a chat meeting to gather use cases for Role Based Access
Control needs in Pulp 3.y. The meeting details and agenda link are below.
Both users and plugin developers are invited. Please join if you're
interested!

When: Wednesday, November 20⋅9:00 – 10:00am EST or in your time zone

.
Where: #pulp-dev on freenode
Agenda: https://etherpad.net/p/Pulp_-_RBAC_Use_Cases

Questions and feedback are also welcome ahead of time.

Thanks!
Brian
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Pulpcore RC7

[Brian Bouterse]

The pulpcore 3.0.0rc7, pulpcore-plugin 0.1.0rc7, and pulp_file 0.1.0b4
packages are available on PyPI [0][1][2]. There are a lot of smaller
changes that have happened; see the release notes for all the details. One
noticeable change is the field names starting with an underscore were
renamed. Commonly like _id became pulp_id.

# Release Notes

pulpcore -
https://docs.pulpproject.org/en/3.0/nightly/changes.html#rc7-2019-10-15
pulpcore-plugin  -
https://docs.pulpproject.org/en/pulpcore-plugin/nightly/changes.html#rc7-2019-10-15
pulp_file -
https://github.com/pulp/pulp_file/blob/master/CHANGES.rst#010b4-2019-10-15


# Can I upgrade from earlier RCs?

No, you should install RC7 on a fresh to fresh database. Upgrading from
earlier Release Candidates is not supported.


# What plugins are compatible with RC7?

There were breaking changes introduced in RC7 so plugin releases prior to
RC7 will likely not be compatible with RC7. Please follow the Pulp list for
more information about plugin releases.


# How do I try this?

* Use pulplift ( https://github.com/pulp/pulplift ) which creates a VM for
you locally and runs the Pulp installer.
* Use the Pulp Installer ( https://github.com/pulp/ansible-pulp ) on a
machine you provisioned.


# What about client bindings?

Python [3] and Ruby [4] bindings are built+published to PyPI and
rubgems.org respectively.
See [3][4] for details on usage.


# Where do I get help?

Ask questions via pulp-list@redhat.com or come into the #pulp channel on
Freenode for help.


[0]: https://pypi.org/project/pulpcore/
[1]: https://pypi.org/project/pulpcore-plugin/
[2]: https://pypi.org/project/pulp-file/
[3]:
https://docs.pulpproject.org/en/3.0/rc/integration-guide/index.html#python-client-for-pulpcore-s-rest-api
[4]:
https://docs.pulpproject.org/en/3.0/rc/integration-guide/index.html#ruby-client-for-pulpcore-s-rest-api


Brian
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] pulp3 sync epel repo

[Brian Bouterse]

I agree a single mirror as the source URL is what is expected currently.

Neither pulp_rpm for either Pulp2 or Pulp3 (not GA yet) implement metalink
support currently. You can file a feature request against Pulp3 to request
this feature here:  https://pulp.plan.io/projects/pulp_rpm/issues/new



On Sat, Oct 12, 2019 at 9:03 PM aaron.t.wyllie 
wrote:

> EPEL metalinks point to any number of hosted repositories with the closest
> and fastest being chosen each time.
>
> I would suggest chosing a single provider from the EPEL mirrors list as
> your source URL.  You could also point it at the primary repository that
> will be commented out in the epel.repo file.
>
> I don't know the specifics behind why Pulp would fail but I suspect that
> it is not able to arbitrarily change the source URL every time it is sync'd
> as provided through a metalink value.
>
>
>  Original message 
> From: "Bin Li (BLOOMBERG/ 120 PARK)" 
> Date: 10/12/19 20:30 (GMT-05:00)
> To: pulp-list@redhat.com
> Subject: [Pulp-list] pulp3 sync epel repo
>
> We are able to use "reposync" command to sync epel repo with the metalink,
> metalink=
> https://mirrors.fedoraproject.org/metalink?repo=epel-source-7=$basearch,
> defined in the yum.repo.d. Pulp3 failed to sync to the same url which we
> defined in the remote. Is this expected behavior? What url should we use to
> sync the epel repo?
>
> Thanks
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Pulp 3.0 GA Timeline

[Brian Bouterse]

We are very excited to announce a timeline for the GA releases of Pulp3
RPM, Docker, ISO/File, and pulpcore. The target date is Dec 3rd, 2019.

See the full details in the blog post:
https://pulpproject.org/2019/09/30/pulp-3-GA-timeline/

Thanks,
Brian
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] pulp_ansible 0.2 Beta 3 is released

[Brian Bouterse]

The beta 3 release of pulp_ansible 0.2.0 for Pulp 3 is now available (
https://pypi.org/project/pulp-ansible/0.2.0b3/ ). Here's some of the big
changes:

* replacement of `mazer` with the new `ansible-galaxy collections` CLI in
the Ansible 2.9 release
* easier out-of-the-box installer experience
* extensive improvements in the Galaxy V3 API

For more information, see the release notes:
https://pulp-ansible.readthedocs.io/en/latest/changes.html#b3-2019-09-18

Thanks to all the contributors and testers who worked towards this release.

Pulp Ansible Team
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] New install: v2 vs v3

[Brian Bouterse]

+1 to @kodiak's assessment. Also in terms of the use cases you identified;
I believe those are all functional so the published bits (still pre-GA)
would work to meet your needs.

Asking questions via this list if you run into problems is good, or you
could report issues via the the issue tracker and we can try to get you on
the right track.

Thanks for asking!
-Brian

On Thu, Sep 12, 2019 at 10:43 AM Kodiak Firesmith 
wrote:

> Hi Joey,
> My point of view comes from being a long term end user of Pulp 2x only
> (not a developer, much less a Pulp dev, FWIW):
>
> Despite Pulp 2x being great, starting fresh on a tech evaluation, I'd 100%
> pick Pulp 3.  Because:
> 1.  Pulp 3x/PulpCore is the next long term support code base, so it's the
> branch with a long lifespan out in front of if within Satellite 6.
> 2.  Pulp 2x was (AFAICT) the branch where the vast majority of lessons
> were learned throughout the course of the project, so it might carry an
> amount of technical debt common with such ongoing lessons learned.
> 3.  Pulp 3x is more than an evolution of Pulp 2x, it's a completely new
> codebase that uses Django rather than homebrew.  Adopting 3x from the
> outset will save you a potentially difficult migration in the future.
>
> Hope this helps, and I'm sure the core devs can hop in and correct me on
> any mistakes in my reasoning.
>
>  - Kodiak, a very happy Pulp 2x user
>
> On Thu, Sep 12, 2019 at 10:20 AM Dumont, Joey 
> wrote:
>
>> ​Hi,
>>
>>
>> My group woud like to assess Pulp to see if it fits our needs. Would you
>> recommend installing v3 over v2?
>>
>>
>> We need to mirror python packages, RPM repos, and spin up a private
>> Docker registry.
>>
>>
>> Thanks!
>>
>>
>> Joey Dumont
>>
>> Technical Advisor, Knowledge, Information, and Technology Services
>> National Research Council Canada / Governement of Canada
>> joey.dum...@nrc-cnrc.gc.ca / Tel: 613-990-8152 / Cell: 438-340-7436
>>
>> Conseiller technique, Services du savoir, de l'information et de la
>> technologie
>> Conseil national de recherches Canada / Gouvernement du Canada
>> joey.dum...@nrc-cnrc.gc.ca / Tél.: 613-990-8152 / Tél. cell.:
>> 438-340-7436
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] pulp_ansible 0.2 Beta 2 is released

[Brian Bouterse]

The beta 2 release of pulp_ansible 0.2.0 release for Pulp 3 is now
available ( https://pypi.org/project/pulp-ansible/0.2.0b2/ ). Here's some
of the big changes:

* full mirroring support for Collection sync
* full-text search of collection using the 'q' query param
* significantly more attributes available about a Collection from the API
* A new V3 Galaxy api (alpha) for next-gen clients
* various bugfixes

For more information, see the release notes:
https://pulp-ansible.readthedocs.io/en/latest/changes.html#b2-2019-08-12

# Whitelist Removal
The  "whitelist" collection sync was removed so we can offer a richer
version of it in a future release that does things like this:

https://pulp.plan.io/issues/5250
https://pulp.plan.io/issues/5251

Thanks to all the contributors and testers who worked towards this release.
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] ISE when getting distribution task state

[Brian Bouterse]

Hi Jan,

You're encountering ( https://pulp.plan.io/issues/4945 ) which was an issue
in pulpcore. If you upgrade pulpcore to the latest released bits, it
includes a change in the setting's default which should fix this for you.

If you still encounter the issue after upgrading pulpcore, can you specify
the versions you're using.

Thanks!
Brian


On Thu, Aug 1, 2019 at 8:10 AM Jan Hutař  wrote:

> [CCing bbouters]
>
> On 2019-07-28 22:00 +0200, Jan Hutař wrote:
> >Hello,
> >I'm out of ideas on how to create a distribution. Could you please help
> >me? I have this publication:
> >
> >   [root@gprfc075 ~]# http GET
> http://localhost:24817/pulp/api/v3/publications/file/file/add43bfc-05b4-43cb-8423-dc41c144a44c/
> >   HTTP/1.1 200 OK
> >   Allow: GET, DELETE, HEAD, OPTIONS
> >   Connection: close
> >   Content-Length: 356
> >   Content-Type: application/json
> >   Date: Sun, 28 Jul 2019 19:48:07 GMT
> >   Server: gunicorn/19.9.0
> >   Vary: Accept, Cookie
> >   X-Frame-Options: SAMEORIGIN
> >
> >   {
> >   "_created": "2019-07-27T20:59:03.645609Z","_href":
> "/pulp/api/v3/publications/file/file/add43bfc-05b4-43cb-8423-dc41c144a44c/",
>
> >"_type": "file.file","distributions": [
> >
>  "/pulp/api/v3/distributions/file/file/0dc4f191-c874-4e55-868a-e6e83dcb2e6b/"
> >   ],"publisher": null,"repository_version":
> "/pulp/api/v3/repositories/33e56022-89c3-42d2-94c0-ebaa38cb5689/versions/1/"
> >   }
> >
> >and I schedule its distribution like this:
> >
> >   [root@gprfc075 ~]# http POST
> http://localhost:24817/pulp/api/v3/distributions/file/file/ name=xyz123
> base_path=abc987
> publication=/pulp/api/v3/publications/file/file/add43bfc-05b4-43cb-8423-dc41c144a44c/
> >   HTTP/1.1 202 Accepted
> >   Allow: GET, POST, HEAD, OPTIONS
> >   Connection: close
> >   Content-Length: 67
> >   Content-Type: application/json
> >   Date: Sun, 28 Jul 2019 19:49:34 GMT
> >   Server: gunicorn/19.9.0
> >   Vary: Accept, Cookie
> >   X-Frame-Options: SAMEORIGIN
> >
> >   {
> >   "task": "/pulp/api/v3/tasks/9fa6a404-830e-4b8b-9c15-52220f5ffb15/"
> >   }
> >
> >but after some time, getting task details fails with ISE:
> >
> >   [root@gprfc075 ~]# http GET
> http://localhost:24817/pulp/api/v3/tasks/9fa6a404-830e-4b8b-9c15-52220f5ffb15/
> >   HTTP/1.1 500 Internal Server Error
> >   Connection: close
> >   Content-Length: 27
> >   Content-Type: text/html
> >   Date: Sun, 28 Jul 2019 19:50:02 GMT
> >   Server: gunicorn/19.9.0
> >   Vary: Cookie
> >   X-Frame-Options: SAMEORIGIN
> >
> >   Server Error (500)
> >
> >Maybe I have misunderstood what the publication and distribution is (I
> >assume in Satellite terminology publication is an content view which
> >needs to be published and promoted into some live cycle environment via
> >distribution and that makes it's content downloadable)?
> >
> >Feel free to look around and experiment in my setup:
> >
> >   r...@gprfc075.sbu.lab.eng.bos.redhat.com   # passwd is redhat
> >
> >Traceback I got in journal is attached.
> >
> >Thank you in advance,
> >Jan
> >
> >
> >
> >--
> >Jan HutarSatellite QA
> >jhu...@redhat.com   Red Hat, Inc.
>
> >Jul 28 15:49:55 gprfc075.sbu.lab.eng.bos.redhat.com dhclient[7957]:
> DHCPREQUEST on em1 to 10.19.43.29 port 67 (xid=0x7c416cc4)
> >Jul 28 15:50:02 gprfc075.sbu.lab.eng.bos.redhat.com gunicorn[28351]:
> pulp: django.request:ERROR: Internal Server Error:
> /pulp/api/v3/tasks/9fa6a404-830e-4b8b-9c15-52220f5ffb15/
> >Jul 28 15:50:02 gprfc075.sbu.lab.eng.bos.redhat.com gunicorn[28351]:
> Traceback (most recent call last):
> >Jul 28 15:50:02 gprfc075.sbu.lab.eng.bos.redhat.com gunicorn[28351]:
> File
> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/exception.py",
> line 34, in inner
> >Jul 28 15:50:02 gprfc075.sbu.lab.eng.bos.redhat.com gunicorn[28351]:
> response = get_response(request)
> >Jul 28 15:50:02 gprfc075.sbu.lab.eng.bos.redhat.com gunicorn[28351]:
> File
> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/base.py",
> line 115, in _get_response
> >Jul 28 15:50:02 gprfc075.sbu.lab.eng.bos.redhat.com gunicorn[28351]:
> response = self.process_exception_by_middleware(e, request)
> >Jul 28 15:50:02 gprfc075.sbu.lab.eng.bos.redhat.com gunicorn[28351]:
> File
> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/base.py",
> line 113, in _get_response
> >Jul 28 15:50:02 gprfc075.sbu.lab.eng.bos.redhat.com gunicorn[28351]:
> response = wrapped_callback(request, *callback_args, **callback_kwargs)
> >Jul 28 15:50:02 gprfc075.sbu.lab.eng.bos.redhat.com gunicorn[28351]:
> File
> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/views/decorators/csrf.py",
> line 54, in wrapped_view
> >Jul 28 15:50:02 gprfc075.sbu.lab.eng.bos.redhat.com gunicorn[28351]:
> return view_func(*args, **kwargs)
> >Jul 28 15:50:02 gprfc075.sbu.lab.eng.bos.redhat.com gunicorn[28351]:
> File
> 

[Pulp-list] Pulpcore RC4 is Available

[Brian Bouterse]

The pulpcore 3.0.0rc4 and pulpcore-plugin 0.1.0rc4 packages are available
on PyPI [0][1]. This release was prompted to resolve an incompatibility
with djangorestframework 3.10.0 that released on July 15th.

RC4 includes a new chunked/parallel upload feature and various bugfixes. We
encourage users to try RC4, through an RC4 compatible plugin on a
non-production system.


# Release Notes

pulpcore (user docs)  -
https://docs.pulpproject.org/en/3.0/nightly/changes.html#rc4-2019-07-25
pulpcore-plugin (plugin writer docs) -
https://docs.pulpproject.org/en/pulpcore-plugin/nightly/changes.html#rc4-2019-07-25


# Can I upgrade from earlier RCs?

You should install RC4 on a fresh to fresh database. Upgrading from earlier
Release Candidates is not supported.


# What plugins are compatible with RC4?

All RC3 compatible plugins should be possible with RC4.


# How do I try this?

* Use pulplift ( https://github.com/pulp/pulplift ) which creates a VM for
you locally and runs the Pulp installer.
* Use the Pulp Installer ( https://github.com/pulp/ansible-pulp ) on a
machine you provisioned.


# What about client bindings?

Python [2] and Ruby [3] bindings are built+published to PyPI and rubgems.org
respectively. See [2][3] for details on usage.


# Where do I get help?

Ask questions via pulp-list@redhat.com or come into the #pulp channel on
Freenode for help.


[0]: https://pypi.org/project/pulpcore/
[1]: https://pypi.org/project/pulpcore-plugin/
[2]:
https://docs.pulpproject.org/en/3.0/rc/integration-guide/index.html#python-client-for-pulpcore-s-rest-api
[3]:
https://docs.pulpproject.org/en/3.0/rc/integration-guide/index.html#ruby-client-for-pulpcore-s-rest-api
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Pulpcore RC3 is Available

[Brian Bouterse]

The pulpcore 3.0.0rc3 and pulpcore-plugin 0.1.0rc3 packages are available
on PyPI [0][1]. It includes many bugfixes, important usability
improvements, and a few backwards incompatible changes. We encourage users
to try RC3, through an RC3 compatible plugin on a non-production system.

# Release Notes

Check out the new, improved release notes!

pulpcore (user docs)  -
https://docs.pulpproject.org/en/3.0/nightly/changes.html#rc3-2019-06-28

pulpcore-plugin (plugin writer docs) -
https://docs.pulpproject.org/en/pulpcore-plugin/nightly/changes.html#rc3-2019-06-28

# Can I upgrade from earlier Release Candidates?

The migration had to be squashed again so you'll need to install RC3 on a
fresh database. Upgrading from earlier Release Candidates is not supported.

# What plugins are compatible with rc3?

Due to breaking changes, most plugins probably need to release compatible
versions against RC3. These new versions should be released next week.
Announcements
are expected to go out on pulp-list@redhat.com as the compatible plugins
release.

# How do I try this?

* Use pulplift ( https://github.com/pulp/pulplift ) which creates a VM for
you locally and runs the Pulp installer.
* Use the Pulp Installer ( https://github.com/pulp/ansible-pulp ) on a
machine you provisioned.

# What about client bindings?

Python [2] and Ruby [3] bindings are built+published to PyPI and rubgems.org
respectively. See [2][3] for details on usage.

# Where do I get help?

Ask questions via pulp-list@redhat.com or come into the #pulp channel on
Freenode for help.


Thank you to everyone who helped make RC3, and especially the push to
resolve the blockers at the end. Happy testing!


[0]: https://pypi.org/project/pulpcore/
[1]: https://pypi.org/project/pulpcore-plugin/
[2]:
https://docs.pulpproject.org/en/3.0/rc/integration-guide/index.html#python-client-for-pulpcore-s-rest-api
[3]:
https://docs.pulpproject.org/en/3.0/rc/integration-guide/index.html#ruby-client-for-pulpcore-s-rest-api
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] create remote to redhat on pulp 3

[Brian Bouterse]

It says "Timeout on reading data from socket" so either this is a bug, or
the remote server is not delivering you data over the socket connection,
perhaps due to some rate limiting. The remote has a download_concurrency
option [0] so perhaps reducing that will cause the server to give you all
of the data. You can also open an issue with reproducer details and we can
try to reproduce it.

[0]:
https://pulp-rpm.readthedocs.io/en/latest/restapi.html#operation/remotes_rpm_rpm_create

On Thu, Jun 27, 2019 at 10:48 AM Bin Li (BLOOMBERG/ 120 PARK) <
bli...@bloomberg.net> wrote:

> I am now able to sync the redhat repo through proxy but the task failed
> eventually on a timeout after sync most of contents. I tried sync both
> rhel7 and rhel6 os repo and both gave the same error. Any idea what could
> cause this?
>
>
> # ./get /pulp/api/v3/tasks/738fc1bc-463f-4d7e-9ac9-ebb1e629b6f7/
> HTTP/1.1 200 OK
> Allow: GET, PATCH, DELETE, HEAD, OPTIONS
> Connection: close
> Content-Length: 3512
> Content-Type: application/json
> Date: Thu, 27 Jun 2019 14:38:12 GMT
> Server: gunicorn/19.9.0
> Vary: Accept, Cookie
> X-Frame-Options: SAMEORIGIN
>
> {
> "_created": "2019-06-26T20:35:38.514762Z",
> "_href": "/pulp/api/v3/tasks/738fc1bc-463f-4d7e-9ac9-ebb1e629b6f7/",
> "created_resources": [],
> "error": {
> "code": null,
> "description": "Timeout on reading data from socket",
> "traceback": " File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/worker.py\", line 812,
> in perform_job\n rv = job.perform()\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/job.py\", line 588, in
> perform\n self._result = self._execute()\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/job.py\", line 594, in
> _execute\n return self.func(*self.args, **self.kwargs)\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py\",
> line 68, in synchronize\n dv.create()\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\",
> line 169, in create\n loop.run_until_complete(pipeline)\n File
> \"/opt/python/3.6.5/lib64/python3.6/asyncio/base_events.py\", line 468, in
> run_until_complete\n return future.result()\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py\",
> line 209, in create_pipeline\n await asyncio.gather(*futures)\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py\",
> line 43, in __call__\n await self.run()\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py\",
> line 132, in run\n pb.done += task.result() # download_count\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py\",
> line 155, in _handle_content_unit\n await
> asyncio.gather(*downloaders_for_content)\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/models.py\",
> line 78, in download\n download_result = await
> downloader.run(extra_data=self.extra_data)\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/download/base.py\",
> line 212, in run\n return await self._run(extra_data=extra_data)\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/backoff/_async.py\", line
> 131, in retry\n ret = await target(*args, **kwargs)\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/download/http.py\",
> line 184, in _run\n to_return = await self._handle_response(response)\n
> File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/download/http.py\",
> line 158, in _handle_response\n chunk = await
> response.content.read(1048576) # 1 megabyte\n File
> \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/streams.py\",
> line 369, in read\n await self._wait('read')\n File
> \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/streams.py\",
> line 297, in _wait\n await waiter\n"
> },
> "finished_at": "2019-06-26T21:57:33.499085Z",
> "name": "pulp_rpm.app.tasks.synchronizing.synchronize",
> "non_fatal_errors": [],
> "parent": null,
> "progress_reports": [
> {
> "done": 3757,
> "message": "Downloading Artifacts",
> "state": "failed",
> "suffix": null,
> "total": 3757
> },
> {
> "done": 5,
> "message": "Downloading Metadata Files",
> "state": "canceled",
> "suffix": null,
> "total": 5
> },
> {
> "done": 11350,
> "message": "Associating Content",
> "state": "canceled",
> "suffix": null,
> "total": 11350
> },
> {
> "done": 3582,
> "message": "Parsed Erratum",
> "state": "running",
> "suffix": null,
> "total": 3582
> },
> {
> "done": 8201,
> "message": "Parsed Packages",
> "state": "running",
> "suffix": null,
> "total": 24392
> }
> ],
> "spawned_tasks": [],
> "started_at": "2019-06-26T20:35:38.650635Z",
> "state": "failed",
> "worker": "/pulp/api/v3/workers/d1db2594-52b6-402e-8ef1-7c0a5635c3c4/"
> }
>
>
>
> From: dkli...@redhat.com At: 06/23/19 07:14:50
> To: Bin Li (BLOOMBERG/ 120 PARK ) 
> Cc: pulp-list@redhat.com
> Subject: 

Re: [Pulp-list] [Pulp-dev] Pulp 2 to 3 Migration meeting on Wednesday June 19th

[Brian Bouterse]

On Thu, Jun 20, 2019 at 7:54 AM Dennis Kliban  wrote:

> On Wed, Jun 19, 2019 at 2:10 PM Eric Helms  wrote:
>
>> Thanks for these updates.
>>
>> On Wed, Jun 19, 2019, 1:59 PM Tatiana Tereshchenko 
>> wrote:
>>
>>> The logs from the meeting can be viewed here[0].
>>>
>>> We covered:
>>>   - hard links approach for "migrating" files, good or bad; alternatives
>>> * current plan: Pulp 3's migration task will migrate content by
>>> either creating a hard link in the Pulp 3 artifact storage location or by
>>> copying it there if hard links are not supported. There will be a way to
>>> see how much content has been migrated and how much is left.
>>>
>>
>> In the copy case, will Pulp 3 migrations prevent a user from potentially
>> filling their disk up and ending up in a broken state?
>>
>>
> Definitely. Pulp will check the type of filesystem the user has. If it
> supports hardlinks it will check that there is some sane amount of space
> available. If it doesn't support hard links it will check that there is at
> least as much free space as space already used by /var/lib/pulp. Pulp will
> notify the user if there is not enough space.
>
This sounds good. I want to point out that Pulp2 could still be writing to
the filesystem while the migration plan is applied by Pulp3 causing the
initial "we're good" check to possibly later fail. I'm ok w/ that I just
wanted to point out this case and that we are not planning to handle it.

I'm also contemplating what would happen if we don't support migrations on
filesystems that don't support hardlinks. To my knowledge, and from the
feedback recently about what filesystems our users use in the community
survey, all of them support hardlinks. Does it make any sense to wait to
develop the "copy fallback" when we know a user will need it? Does anyone
know that we will need to migrate on filesystems without hard links?


>
>
>>>   - protected repos migration
>>> * current plan: CertGuards witll be created and associated with
>>> Distributions in Pulp3. The Migration Plan will provide the ability to
>>> configure which repositories content protection is migrated for to Pulp 3.
>>>
>>> The etherpad [1] is updated with the suggested changes to the Migration
>>> Plan structure, you can see it in the examples for use cases on lines L316,
>>> L325.
>>>
>>> [0]
>>> https://pulpadmin.fedorapeople.org/triage/pulp-2to3-sig/2019/pulp-2to3-sig.2019-06-19-16.00.log.html
>>> [1] https://etherpad.net/p/pulp-2to3-migration
>>>
>>> On Tue, Jun 18, 2019 at 6:34 PM Tatiana Tereshchenko <
>>> ttere...@redhat.com> wrote:
>>>
 An IRC meeting will be held on Wednesday June 19th at 12pm Eastern[0].
 The meeting will take place in #pulp-2to3-sig on Freenode IRC network.

 Tentative agenda:
   - continue on use cases, see etherpad L18+
 * especially distributors migration part
 * verify that everything is covered by the current MP structure
 * discuss any questions around use cases that can affect the MP
 structure
  - hardlinks approach for "migrating" files, good or bad; alternatives

 Please put any questions or ideas into the etherpad [1] before the
 meeting.

 [0]
 https://www.worldtimebuddy.com/?qm=1=100,4487042,3078610=100=2019-6-19=16-17
 [1] https://etherpad.net/p/pulp-2to3-migration

>>> ___
>>> Pulp-list mailing list
>>> Pulp-list@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
> ___
> Pulp-dev mailing list
> pulp-...@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] pulp_rpm Python and Ruby bindings available

[Brian Bouterse]

The pulp_rpm plugin for Pulp3 now has Python and Ruby bindings available.
See the new section of the docs here:
https://pulp-rpm.readthedocs.io/en/latest/bindings.html  The bindings will
publish nightly, and with each tagged release.

Thanks to @dkliban for adding the binding publishing to pulp_rpm.

-Brian
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] rpm distribution not deleted when deleting a repo

[Brian Bouterse]

Distributions can be reused (and updated with a PATCH call versus a POST
which creates a new one). Note for rc2 specifically, because of the
breaking changes with rc2, you'll need to make any Distributions and
Publications new with rc2.

Hopefully this is helpful and not confusing. More questions are welcome!


On Wed, May 22, 2019 at 11:45 AM Juan Cabrera 
wrote:

> I see in the ticket that:
>
> "We decided that we'll allow users to create distributions without
> distributing anything. This could be to reserve a path namespace, take a
> publication offline quickly without having to delete the distribution, etc"
>
> So, if I'm not wrong, the distribution can be reused and recreating the
> repo and the publication and doing again
>
> http POST http://localhost:24817/pulp/api/v3/distributions/rpm/rpm/
> name='baz' base_path='foo' publication=$PUBLICATION_HREF
>
^ would create a new Distribution because it's a POST

> no new distribution will be created and the publication will be set to the
> already existing distribution, is that correct?
>
> Juan
>
> On 22/05/19 16:52, David Davis wrote:
>
> Actually I think this is the expected behavior. Distributions can exist
> without serving a publication[0] so I don't think deleting a repository or
> publication should in turn delete the distribution.
>
> [0] https://pulp.plan.io/issues/4840
>
> David
>
>
> On Wed, May 22, 2019 at 10:46 AM Dennis Kliban  wrote:
>
>> No it is not. Please file an issue at https://pulp.plan.io/issues/new
>>
>> On Wed, May 22, 2019 at 10:13 AM Juan Cabrera 
>> wrote:
>>
>>> Hi,
>>>
>>> I'm updating my ansible/vagrant project to take into account the new
>>> release 3.0.0rc2.
>>>
>>> I use now:
>>>
>>> pulp_source_dir: "git+https://github.com/pulp/pulpcore.git@3.0.0rc2;
>>> https://github.com/pulp/pulpcore.git@3.0.0rc2>
>>> pulp_plugin_source_dir:
>>> "git+https://github.com/pulp/pulpcore-plugin.git@0.1.0rc2;
>>> https://github.com/pulp/pulpcore-plugin.git@0.1.0rc2>
>>>   pulp-rpm:
>>> app_label: "rpm"
>>> source_dir: "git+https://github.com/pulp/pulp_rpm.git@3.0.0b3;
>>> https://github.com/pulp/pulp_rpm.git@3.0.0b3>
>>>
>>> I realized that when removing a repository, the publication that depends
>>> on it are removed but that the distribution that depends on the publication
>>> are still there.
>>>
>>> Is this the expected result?
>>>
>>> sincerely
>>>
>>> Juan
>>>
>>> PS: Some details:
>>>
>>> - After doing all steps in the documentation
>>> 
>>> I delete the repo
>>>
>>> [vagrant@dev-pulp-server ~]$ http DELETE $PORT$REPO_HREF
>>> HTTP/1.1 202 Accepted
>>> Allow: GET, PUT, PATCH, DELETE, HEAD, OPTIONS
>>> Connection: close
>>> Content-Length: 67
>>> Content-Type: application/json
>>> Date: Wed, 22 May 2019 14:05:42 GMT
>>> Server: gunicorn/19.9.0
>>> Vary: Accept, Cookie
>>> X-Frame-Options: SAMEORIGIN
>>>
>>> {
>>> "task": "/pulp/api/v3/tasks/d5e34bc4-f9fc-4184-b991-7619a888a57b/"
>>> }
>>>
>>> The distribution is still there but with *"publication": null*
>>>
>>> [vagrant@dev-pulp-server ~]$ http
>>> $PORT/pulp/api/v3/distributions/rpm/rpm/
>>> HTTP/1.1 200 OK
>>> Allow: GET, POST, HEAD, OPTIONS
>>> Connection: close
>>> Content-Length: 308
>>> Content-Type: application/json
>>> Date: Wed, 22 May 2019 14:05:47 GMT
>>> Server: gunicorn/19.9.0
>>> Vary: Accept, Cookie
>>> X-Frame-Options: SAMEORIGIN
>>>
>>> {
>>> "count": 1,
>>> "next": null,
>>> "previous": null,
>>> "results": [
>>> {
>>> "_created": "2019-05-22T14:04:14.749447Z",
>>> "_href":
>>> "/pulp/api/v3/distributions/rpm/rpm/2b2ad506-1ea4-48ec-8406-d941554ca54e/",
>>> "base_path": "foo",
>>> "base_url": "dev-pulp-server.ptci.dev:8080/pulp/content/foo",
>>>
>>> "content_guard": null,
>>> "name": "baz",
>>> *"publication": null*
>>> }
>>> ]
>>> }
>>>
>>> The publication has been removed:
>>>
>>> [vagrant@dev-pulp-server ~]$ http
>>> $PORT/pulp/api/v3/publications/rpm/rpm/
>>> HTTP/1.1 200 OK
>>> Allow: GET, POST, HEAD, OPTIONS
>>> Connection: close
>>> Content-Length: 52
>>> Content-Type: application/json
>>> Date: Wed, 22 May 2019 14:06:17 GMT
>>> Server: gunicorn/19.9.0
>>> Vary: Accept, Cookie
>>> X-Frame-Options: SAMEORIGIN
>>>
>>> {
>>> "count": 0,
>>> "next": null,
>>> "previous": null,
>>> "results": []
>>> }
>>>
>>> --
>>>
>>> Juan CABRERA
>>> Correspondant informatique
>>> Département de Mathématiques
>>>
>>> T. 081724919
>>> juan.cabr...@unamur.be
>>> http://staff.unamur.be/jbcabrer
>>>
>>> Université de Namur ASBL
>>> Rue de Bruxelles 61 - 5000 Namur
>>> Belgique
>>>
>>> Let’s respect the environment together.
>>> Only print this message if necessary!
>>> ___
>>> Pulp-list mailing list
>>> Pulp-list@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>
>> 

Re: [Pulp-list] Pulpcore RC2 is Available

[Brian Bouterse]

The beta 11 release of pulp_file 0.0.1 for Pulp 3 is now available:

https://pypi.org/project/pulp-file/0.0.1b11/

This release is compatible with pulpcore 3.0.0rc2.

On Mon, May 20, 2019 at 3:11 PM Brian Bouterse  wrote:

> The pulpcore 3.0.0rc2 and pulpcore-plugin 0.1.0rc2 packages are available
> on PyPI [0][1]. It includes many bugfixes, important usability
> improvements, and a few backwards incompatible changes. We encourage users
> to try RC2, through an RC2 compatible plugin on a non-production system.
>
> # Release Notes
>
> pulpcore (user docs)  -
> https://docs.pulpproject.org/en/3.0/rc/release-notes/pulpcore/3.0.x.html#rc2
>
> pulpcore-plugin (plugin writer docs) -
> https://docs.pulpproject.org/en/pulpcore-plugin/rc/release-notes/index.html#rc2
>
> # Can I upgrade from RC1?
>
> Upgrading from RC1 is not supported. You will need to connect your RC2
> install to a fresh, uninitialized database.
>
> # What plugins are compatible with rc2?
>
> Due to breaking changes, all plugins will need to release compatible
> versions against RC2. These new versions are expected to be released
> shortly after RC2. Announcements are expected to go out on pulp-list@
> redhat.com as the compatible plugins release.
>
> # How do I try this?
>
> * Use pulplift ( https://github.com/pulp/pulplift ) which creates a VM
> for you locally and runs the Pulp installer.
> * Use the Pulp Installer ( https://github.com/pulp/ansible-pulp ) on a
> machine you provisioned.
>
> # What about client bindings?
>
> Python [2] and Ruby [3] bindings are built+published to PyPI and
> rubgems.org respectively. See [2][3] for details on usage.
>
> # Where do I get help?
>
> Ask questions via pulp-list@redhat.com or come into the #pulp channel on
> Freenode for help.
>
>
> Thank you to all the contributors, testers, and users who have given their
> time, energy, and feedback into making RC2. Happy testing!
>
>
> [0]: https://pypi.org/project/pulpcore/
> [1]: https://pypi.org/project/pulpcore-plugin/
> [2]:
> https://docs.pulpproject.org/en/3.0/rc/integration-guide/index.html#python-client-for-pulpcore-s-rest-api
> [3]:
> https://docs.pulpproject.org/en/3.0/rc/integration-guide/index.html#ruby-client-for-pulpcore-s-rest-api
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Pulpcore RC2 is Available

[Brian Bouterse]

The rc5 release of pulp_ansible 0.1.0rc5 for Pulp3 is now available:

https://pypi.org/project/pulp-ansible/

It supports pulpcore 3.0.0rc2.

On Tue, May 21, 2019 at 1:02 AM Simon Baatz  wrote:

> The beta 3 release of pulp_cookbook 0.0.4 for Pulp 3 is now
> available:
>
> https://pypi.org/project/pulp-cookbook/0.0.4b3/
>
> It supports pulpcore 3.0.0rc2. For more information, see the release
> notes:
>
> https://github.com/gmbnomis/pulp_cookbook/blob/master/CHANGES.rst
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Pulpcore RC2 is Available

[Brian Bouterse]

The pulpcore 3.0.0rc2 and pulpcore-plugin 0.1.0rc2 packages are available
on PyPI [0][1]. It includes many bugfixes, important usability
improvements, and a few backwards incompatible changes. We encourage users
to try RC2, through an RC2 compatible plugin on a non-production system.

# Release Notes

pulpcore (user docs)  -
https://docs.pulpproject.org/en/3.0/rc/release-notes/pulpcore/3.0.x.html#rc2

pulpcore-plugin (plugin writer docs) -
https://docs.pulpproject.org/en/pulpcore-plugin/rc/release-notes/index.html#rc2

# Can I upgrade from RC1?

Upgrading from RC1 is not supported. You will need to connect your RC2
install to a fresh, uninitialized database.

# What plugins are compatible with rc2?

Due to breaking changes, all plugins will need to release compatible
versions against RC2. These new versions are expected to be released
shortly after RC2. Announcements are expected to go out on pulp-list@
redhat.com as the compatible plugins release.

# How do I try this?

* Use pulplift ( https://github.com/pulp/pulplift ) which creates a VM for
you locally and runs the Pulp installer.
* Use the Pulp Installer ( https://github.com/pulp/ansible-pulp ) on a
machine you provisioned.

# What about client bindings?

Python [2] and Ruby [3] bindings are built+published to PyPI and rubgems.org
respectively. See [2][3] for details on usage.

# Where do I get help?

Ask questions via pulp-list@redhat.com or come into the #pulp channel on
Freenode for help.


Thank you to all the contributors, testers, and users who have given their
time, energy, and feedback into making RC2. Happy testing!


[0]: https://pypi.org/project/pulpcore/
[1]: https://pypi.org/project/pulpcore-plugin/
[2]:
https://docs.pulpproject.org/en/3.0/rc/integration-guide/index.html#python-client-for-pulpcore-s-rest-api
[3]:
https://docs.pulpproject.org/en/3.0/rc/integration-guide/index.html#ruby-client-for-pulpcore-s-rest-api
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Create and remove repo with ansible

[Brian Bouterse]

We don't have one for Pulp3 yet, but we do want to make one. I created a
stub issue here to start tracking that effort.

https://pulp.plan.io/issues/4658

On Tue, Apr 9, 2019 at 10:58 AM Juan Cabrera  wrote:

> Thanks.
>
> I suppose this does not work with pup 3.x , isn't it ?
> On 9/04/19 16:00, Kodiak Firesmith wrote:
>
> For Pulp 2.x, I use this plus some extra commands:
> https://docs.ansible.com/ansible/latest/modules/pulp_repo_module.html
>
> On Tue, Apr 9, 2019 at 9:48 AM Juan Cabrera 
> wrote:
>
>> Hi,
>>
>> Is there an ansible module to add or remove a rpm repository ?
>>
>> I send in attachment the scripts I use for the moment to create and
>> remove the rpm repositories.
>>
>> I use this:
>>
>> vars:
>>
>>- pulp_repos:
>>   - repo: epel_ptci
>> remote: epel
>> remote_url: https://epel.mirror.it2go.eu/7/x86_64/
>> state: present
>>
>> TODO: to become idempotent, check if repo already exists
>>
>> - name: config | Add repositories.
>>   command: ~/create_repo --repo "{{ item.repo }}" --remote "{{
>> item.remote }}" --href "{{ item.remote_url }}"
>>   loop: "{{ pulp_repos }}"
>>   when: item.state is not defined or item.state == 'present'
>>   become: yes
>>   become_user: "{{ pulp_repo_user }}"
>>
>>
>> - name: config | Remove repositories
>>   command: ~/remove_repo --repo "{{ item.repo }}" --remote "{{
>> item.remote }}"
>>   loop: "{{ pulp_repos }}"
>>   when: item.state is defined and item.state == 'absent'
>>   become: yes
>>   become_user: "{{ pulp_repo_user }}"
>>
>> What is the best way to check if a repo already exist?
>>
>> Juan
>> --
>>
>> Juan CABRERA
>> Correspondant informatique
>> Département de Mathématiques
>>
>> T. 081724919
>> juan.cabr...@unamur.be
>> http://staff.unamur.be/jbcabrer
>>
>> Université de Namur ASBL
>> Rue de Bruxelles 61 - 5000 Namur
>> Belgique
>>
>> Let’s respect the environment together.
>> Only print this message if necessary!
>> ___
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
> --
>
> Juan CABRERA
> Correspondant informatique
> Département de Mathématiques
>
> T. 081724919
> juan.cabr...@unamur.be
> http://staff.unamur.be/jbcabrer
>
> Université de Namur ASBL
> Rue de Bruxelles 61 - 5000 Namur
> Belgique
>
> Let’s respect the environment together.
> Only print this message if necessary!
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Anyone have a configuration dump/load tool for Pulp 2.x?

[Brian Bouterse]

I don't have any existing tooling, but this sounds related to a migration
idea that came up recently...

For migration from Pulp2 -> Pulp3 specifically, I think having a Pulp2
feature that outputs a config of all Pulp2 repositories for example. Then
the user could edit this config file which would be declaring how to
handle/replicate that repo into your Pulp3 install by editing the template.
The migration tool would validate and consume this template.

FYI the Pulp2 -> Pulp3 epic in Redmine is here
https://pulp.plan.io/issues/3821 I haven't put this idea into that yet (and
I should).


On Mon, Apr 8, 2019 at 11:04 AM Kodiak Firesmith 
wrote:

> Hello Pulp-List!
>
> I've frequently found myself in need of propagating various in-Pulp
> settings out to other Pulp servers without doing a full database
> dump/restore.  Examples might be dumping a set of organizations from my
> main server and loading them into a test host.  I have not yet written
> something like this do do the work more easily.  I figured before digging
> any deeper that I'd reach out to the list to see if someone else might have
> already done the hard work, especially as I see this being useful for
> potentially dumping configs from Pulp 2.x in preparation for moving to Pulp
> 3.0, though understandably those configs won't be 1:1 by any stretch, but
> if dumps could be done into a common format like YAML, perhaps a new load
> tool for Pulp 3.x could translate those into Pulp 3x API.
>
> Thanks!
>  - Kodiak
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] [Pulp-dev] Documentation Systemd clarification?

[Brian Bouterse]

Good point that last service wasn't in there. I applied some additional
updates today which can be seen here:
https://docs.pulpproject.org/en/3.0/nightly/installation/instructions.html#systemd

Any other feedback or issue discussion is welcome.

On Thu, Apr 4, 2019 at 7:01 PM Juan Cabrera  wrote:

> Hi,
>
> Very clear now. I like the idea to link the ansible templates and the
> README.
>
> I see that a service is still missing in the documentation. Is it normal ?
>
>
> https://github.com/pulp/ansible-pulp/blob/master/roles/pulp/templates/pulp-api.service.j2
>
> sincerely
>
> Juan
>
>
> On 5/04/19 00:12, Brian Bouterse wrote:
>
> See the updated docs here for the systemd templates:
> https://docs.pulpproject.org/en/3.0/nightly/installation/instructions.html#systemd
>
> On Tue, Apr 2, 2019 at 5:33 PM Brian Bouterse  wrote:
>
>> I updated the path on the docs as you identified. Also, we are
>> considering linking to the ansible-pulp templates directly since those
>> contain all 3 of the service definitions we need. I'm hoping to fix this
>> tomorrow: https://pulp.plan.io/issues/4629#note-4  Feedback on what we
>> should do is welcome.
>>
>> Let us know if you run into more issues. Thanks!
>>
>>
>> On Tue, Apr 2, 2019 at 12:01 PM Brian Bouterse 
>> wrote:
>>
>>> I moved this convo to pulp-list since it's focused on user usage of
>>> Pulp3 versus the development of Pulp itself (on pulp-...@redhat.com).
>>> See some answers to Pulp3 usage questions inline. More questions are
>>> welcome!
>>>
>>>
>>>
>>> On Tue, Apr 2, 2019 at 4:49 AM Juan Cabrera 
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Thank you for the modifications. I'm more gitlab user than github so
>>>> I'm not familiar with PR procedure.
>>>>
>>> It is not clear if the PULP_SETTINGS environment is not used any more.
>>>> In this case the paragraph
>>>>
>>>> "Make sure to substitute
>>>> Environment=PULP_SETTINGS=/path/to/pulp/server.yaml with the real
>>>> location of configuration file
>>>> <https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html#id1>
>>>> ."
>>>>
>>>> Should be changed by
>>>>
>>>> Make sure that the configuration file
>>>> <https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html#id1>
>>>> /etc/pulp/settings.py exist.
>>>>
>>> The /etc/pulp/settings.py file isn't needed at all anymore. This change
>>> ( https://github.com/pulp/pulpcore/pull/63 ) hopefully removes
>>> reference to it leaving this section (
>>> https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html
>>> ) to be the main authority. My PR tries to clarify that section also.
>>>
>>>>
>>>> Another difference I see with the ansible provision of my Vagran VM is
>>>> that  -c 'pulpcore.rqconfig' is missing. It is not needed any more ?
>>>>
>>>> On the VM I have this in pulp-resource-manager.service :
>>>>
>>>> ExecStart=/usr/local/lib/pulp/bin/rq worker \
>>>>   -w pulpcore.tasking.worker.PulpWorker -n resource-manager@%%h \
>>>>   --pid=/var/run/pulp-resource-manager/resource-manager.pid
>>>>
>>>> I think we should update the docs to use the ansible-pulp defaults I
>>> can do this in my PR also after checking with @asmacdo on it.
>>>
>>>> And the last difference I see, is that the systemd services are
>>>> different. There are no worker in /etc/systemd/system/ but an pulp-api
>>>> service
>>>>
>>>> [root@dev-pulp-server ~]# ls /etc/systemd/system/pulp-*
>>>> /etc/systemd/system/pulp-api.service
>>>> /etc/systemd/system/pulp-resource-manager.service
>>>>
>>>> The worker service seems to be moved to /usr/lib/systemd/system/
>>>>
>>>> [root@dev-pulp-server ~]# ls /usr/lib/systemd/system/pulp-*
>>>> /usr/lib/systemd/system/pulp-content-app.service
>>>> /usr/lib/systemd/system/pulp-worker@.service
>>>>
>>> I'll look more into this, but I wanted to get what I had out to the
>>> list.
>>>
>>>> Sincerely
>>>>
>>>> Juan
>>>>
>>>> On 1/04/19 23:13, Brian Bouterse wrote:
>>>>
>>>> I made a docs issue [0] and a PR to adjust the systemd settings this
>>>>

Re: [Pulp-list] [Pulp-dev] Documentation Systemd clarification?

[Brian Bouterse]

See the updated docs here for the systemd templates:
https://docs.pulpproject.org/en/3.0/nightly/installation/instructions.html#systemd

On Tue, Apr 2, 2019 at 5:33 PM Brian Bouterse  wrote:

> I updated the path on the docs as you identified. Also, we are considering
> linking to the ansible-pulp templates directly since those contain all 3 of
> the service definitions we need. I'm hoping to fix this tomorrow:
> https://pulp.plan.io/issues/4629#note-4  Feedback on what we should do is
> welcome.
>
> Let us know if you run into more issues. Thanks!
>
>
> On Tue, Apr 2, 2019 at 12:01 PM Brian Bouterse 
> wrote:
>
>> I moved this convo to pulp-list since it's focused on user usage of Pulp3
>> versus the development of Pulp itself (on pulp-...@redhat.com). See some
>> answers to Pulp3 usage questions inline. More questions are welcome!
>>
>>
>>
>> On Tue, Apr 2, 2019 at 4:49 AM Juan Cabrera 
>> wrote:
>>
>>> Hi,
>>>
>>> Thank you for the modifications. I'm more gitlab user than github so I'm
>>> not familiar with PR procedure.
>>>
>> It is not clear if the PULP_SETTINGS environment is not used any more.
>>> In this case the paragraph
>>>
>>> "Make sure to substitute
>>> Environment=PULP_SETTINGS=/path/to/pulp/server.yaml with the real
>>> location of configuration file
>>> <https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html#id1>
>>> ."
>>>
>>> Should be changed by
>>>
>>> Make sure that the configuration file
>>> <https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html#id1>
>>> /etc/pulp/settings.py exist.
>>>
>> The /etc/pulp/settings.py file isn't needed at all anymore. This change (
>> https://github.com/pulp/pulpcore/pull/63 ) hopefully removes reference
>> to it leaving this section (
>> https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html
>> ) to be the main authority. My PR tries to clarify that section also.
>>
>>>
>>> Another difference I see with the ansible provision of my Vagran VM is
>>> that  -c 'pulpcore.rqconfig' is missing. It is not needed any more ?
>>>
>>> On the VM I have this in pulp-resource-manager.service :
>>>
>>> ExecStart=/usr/local/lib/pulp/bin/rq worker \
>>>   -w pulpcore.tasking.worker.PulpWorker -n resource-manager@%%h \
>>>   --pid=/var/run/pulp-resource-manager/resource-manager.pid
>>>
>>> I think we should update the docs to use the ansible-pulp defaults I can
>> do this in my PR also after checking with @asmacdo on it.
>>
>>> And the last difference I see, is that the systemd services are
>>> different. There are no worker in /etc/systemd/system/ but an pulp-api
>>> service
>>>
>>> [root@dev-pulp-server ~]# ls /etc/systemd/system/pulp-*
>>> /etc/systemd/system/pulp-api.service
>>> /etc/systemd/system/pulp-resource-manager.service
>>>
>>> The worker service seems to be moved to /usr/lib/systemd/system/
>>>
>>> [root@dev-pulp-server ~]# ls /usr/lib/systemd/system/pulp-*
>>> /usr/lib/systemd/system/pulp-content-app.service
>>> /usr/lib/systemd/system/pulp-worker@.service
>>>
>> I'll look more into this, but I wanted to get what I had out to the list.
>>
>>> Sincerely
>>>
>>> Juan
>>>
>>> On 1/04/19 23:13, Brian Bouterse wrote:
>>>
>>> I made a docs issue [0] and a PR to adjust the systemd settings this
>>> [1]. Since dynaconf configures it now, I believe removing this from the
>>> systemd file is the best resolution.
>>>
>>> [0]: https://pulp.plan.io/issues/4622
>>> [1]: https://github.com/pulp/pulpcore/pull/63
>>>
>>> Please let us know if there is anything else we can improve on.
>>>
>>> On Fri, Mar 29, 2019 at 1:11 PM Mike DePaulo 
>>> wrote:
>>>
>>>> On Fri, Mar 29, 2019 at 4:44 AM Juan Cabrera 
>>>> wrote:
>>>>
>>>>> At the section about Systemd :
>>>>>
>>>>>
>>>>> https://docs.pulpproject.org/en/3.0/nightly/installation/instructions.html#systemd
>>>>>
>>>>> It is said that the default config file is /etc/pulp/server.yaml.
>>>>>
>>>>> In the installed VM there is not a 
>>>>> Environment=PULP_SETTINGS=/path/to/pulp/server.yaml
>>>>> line in the pulp-resource-manager.service file

Re: [Pulp-list] [Pulp-dev] Documentation Systemd clarification?

[Brian Bouterse]

I updated the path on the docs as you identified. Also, we are considering
linking to the ansible-pulp templates directly since those contain all 3 of
the service definitions we need. I'm hoping to fix this tomorrow:
https://pulp.plan.io/issues/4629#note-4  Feedback on what we should do is
welcome.

Let us know if you run into more issues. Thanks!


On Tue, Apr 2, 2019 at 12:01 PM Brian Bouterse  wrote:

> I moved this convo to pulp-list since it's focused on user usage of Pulp3
> versus the development of Pulp itself (on pulp-...@redhat.com). See some
> answers to Pulp3 usage questions inline. More questions are welcome!
>
>
>
> On Tue, Apr 2, 2019 at 4:49 AM Juan Cabrera 
> wrote:
>
>> Hi,
>>
>> Thank you for the modifications. I'm more gitlab user than github so I'm
>> not familiar with PR procedure.
>>
> It is not clear if the PULP_SETTINGS environment is not used any more. In
>> this case the paragraph
>>
>> "Make sure to substitute
>> Environment=PULP_SETTINGS=/path/to/pulp/server.yaml with the real
>> location of configuration file
>> <https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html#id1>
>> ."
>>
>> Should be changed by
>>
>> Make sure that the configuration file
>> <https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html#id1>
>> /etc/pulp/settings.py exist.
>>
> The /etc/pulp/settings.py file isn't needed at all anymore. This change (
> https://github.com/pulp/pulpcore/pull/63 ) hopefully removes reference to
> it leaving this section (
> https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html
> ) to be the main authority. My PR tries to clarify that section also.
>
>>
>> Another difference I see with the ansible provision of my Vagran VM is
>> that  -c 'pulpcore.rqconfig' is missing. It is not needed any more ?
>>
>> On the VM I have this in pulp-resource-manager.service :
>>
>> ExecStart=/usr/local/lib/pulp/bin/rq worker \
>>   -w pulpcore.tasking.worker.PulpWorker -n resource-manager@%%h \
>>   --pid=/var/run/pulp-resource-manager/resource-manager.pid
>>
>> I think we should update the docs to use the ansible-pulp defaults I can
> do this in my PR also after checking with @asmacdo on it.
>
>> And the last difference I see, is that the systemd services are
>> different. There are no worker in /etc/systemd/system/ but an pulp-api
>> service
>>
>> [root@dev-pulp-server ~]# ls /etc/systemd/system/pulp-*
>> /etc/systemd/system/pulp-api.service
>> /etc/systemd/system/pulp-resource-manager.service
>>
>> The worker service seems to be moved to /usr/lib/systemd/system/
>>
>> [root@dev-pulp-server ~]# ls /usr/lib/systemd/system/pulp-*
>> /usr/lib/systemd/system/pulp-content-app.service
>> /usr/lib/systemd/system/pulp-worker@.service
>>
> I'll look more into this, but I wanted to get what I had out to the list.
>
>> Sincerely
>>
>> Juan
>>
>> On 1/04/19 23:13, Brian Bouterse wrote:
>>
>> I made a docs issue [0] and a PR to adjust the systemd settings this [1].
>> Since dynaconf configures it now, I believe removing this from the systemd
>> file is the best resolution.
>>
>> [0]: https://pulp.plan.io/issues/4622
>> [1]: https://github.com/pulp/pulpcore/pull/63
>>
>> Please let us know if there is anything else we can improve on.
>>
>> On Fri, Mar 29, 2019 at 1:11 PM Mike DePaulo 
>> wrote:
>>
>>> On Fri, Mar 29, 2019 at 4:44 AM Juan Cabrera 
>>> wrote:
>>>
>>>> At the section about Systemd :
>>>>
>>>>
>>>> https://docs.pulpproject.org/en/3.0/nightly/installation/instructions.html#systemd
>>>>
>>>> It is said that the default config file is /etc/pulp/server.yaml.
>>>>
>>>> In the installed VM there is not a 
>>>> Environment=PULP_SETTINGS=/path/to/pulp/server.yaml
>>>> line in the pulp-resource-manager.service file and the configuration
>>>> file is named `/etc/pulp/settings.py`. Some thing must be updated in
>>>> the documentation?
>>>>
>>>> The file contents in the VM are:
>>>>
>>>> [root@dev-pulp-server system]# cat
>>>> /etc/systemd/system/pulp-resource-manager.service
>>>> [Unit]
>>>> Description=Pulp Resource Manager
>>>> After=network-online.target
>>>> Wants=network-online.target
>>>>
>>>> # This service will break if left running while PostgreSQL restarts.
>>>

Re: [Pulp-list] [Pulp-dev] Documentation Systemd clarification?

[Brian Bouterse]

I moved this convo to pulp-list since it's focused on user usage of Pulp3
versus the development of Pulp itself (on pulp-...@redhat.com). See some
answers to Pulp3 usage questions inline. More questions are welcome!



On Tue, Apr 2, 2019 at 4:49 AM Juan Cabrera  wrote:

> Hi,
>
> Thank you for the modifications. I'm more gitlab user than github so I'm
> not familiar with PR procedure.
>
It is not clear if the PULP_SETTINGS environment is not used any more. In
> this case the paragraph
>
> "Make sure to substitute
> Environment=PULP_SETTINGS=/path/to/pulp/server.yaml with the real
> location of configuration file
> <https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html#id1>
> ."
>
> Should be changed by
>
> Make sure that the configuration file
> <https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html#id1>
> /etc/pulp/settings.py exist.
>
The /etc/pulp/settings.py file isn't needed at all anymore. This change (
https://github.com/pulp/pulpcore/pull/63 ) hopefully removes reference to
it leaving this section (
https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html
) to be the main authority. My PR tries to clarify that section also.

>
> Another difference I see with the ansible provision of my Vagran VM is
> that  -c 'pulpcore.rqconfig' is missing. It is not needed any more ?
>
> On the VM I have this in pulp-resource-manager.service :
>
> ExecStart=/usr/local/lib/pulp/bin/rq worker \
>   -w pulpcore.tasking.worker.PulpWorker -n resource-manager@%%h \
>   --pid=/var/run/pulp-resource-manager/resource-manager.pid
>
> I think we should update the docs to use the ansible-pulp defaults I can
do this in my PR also after checking with @asmacdo on it.

>  And the last difference I see, is that the systemd services are
> different. There are no worker in /etc/systemd/system/ but an pulp-api
> service
>
> [root@dev-pulp-server ~]# ls /etc/systemd/system/pulp-*
> /etc/systemd/system/pulp-api.service
> /etc/systemd/system/pulp-resource-manager.service
>
> The worker service seems to be moved to /usr/lib/systemd/system/
>
> [root@dev-pulp-server ~]# ls /usr/lib/systemd/system/pulp-*
> /usr/lib/systemd/system/pulp-content-app.service
> /usr/lib/systemd/system/pulp-worker@.service
>
I'll look more into this, but I wanted to get what I had out to the list.

> Sincerely
>
> Juan
>
> On 1/04/19 23:13, Brian Bouterse wrote:
>
> I made a docs issue [0] and a PR to adjust the systemd settings this [1].
> Since dynaconf configures it now, I believe removing this from the systemd
> file is the best resolution.
>
> [0]: https://pulp.plan.io/issues/4622
> [1]: https://github.com/pulp/pulpcore/pull/63
>
> Please let us know if there is anything else we can improve on.
>
> On Fri, Mar 29, 2019 at 1:11 PM Mike DePaulo 
> wrote:
>
>> On Fri, Mar 29, 2019 at 4:44 AM Juan Cabrera 
>> wrote:
>>
>>> At the section about Systemd :
>>>
>>>
>>> https://docs.pulpproject.org/en/3.0/nightly/installation/instructions.html#systemd
>>>
>>> It is said that the default config file is /etc/pulp/server.yaml.
>>>
>>> In the installed VM there is not a 
>>> Environment=PULP_SETTINGS=/path/to/pulp/server.yaml
>>> line in the pulp-resource-manager.service file and the configuration
>>> file is named `/etc/pulp/settings.py`. Some thing must be updated in
>>> the documentation?
>>>
>>> The file contents in the VM are:
>>>
>>> [root@dev-pulp-server system]# cat
>>> /etc/systemd/system/pulp-resource-manager.service
>>> [Unit]
>>> Description=Pulp Resource Manager
>>> After=network-online.target
>>> Wants=network-online.target
>>>
>>> # This service will break if left running while PostgreSQL restarts.
>>> BindsTo=postgresql.service
>>> After=postgresql.service
>>>
>>> [Service]
>>> Environment="DJANGO_SETTINGS_MODULE=pulpcore.app.settings"
>>> User=pulp
>>> WorkingDirectory=/var/run/pulp-resource-manager/
>>> RuntimeDirectory=pulp-resource-manager
>>> ExecStart=/usr/local/lib/pulp/bin/rq worker \
>>>   -w pulpcore.tasking.worker.PulpWorker -n resource-manager@%%h
>>> \
>>>   --pid=/var/run/pulp-resource-manager/resource-manager.pid
>>>
>>> [Install]
>>> WantedBy=multi-user.target
>>>
>> Hi Juan,
>>
>> Sorry you ran into this issue with our docs.
>>
>> I think that specific documentation page[1] is was not sufficiently
>> updated to reflect the migrati

Re: [Pulp-list] [Pulp-dev] Pulp 3.0 RC release

[Brian Bouterse]

A new pulp-certguard has been released and is compatible with pulpcore
3.0.0rc1.

pulp-certguard is an X.509 Certificate Based Content protection for Pulp3.
This ensures only clients with valid certificates receive content from
Pulp. It's compatible with various types of content including RPM, File,
etc.

https://pypi.org/project/pulp-certguard/

-Brian


On Fri, Mar 29, 2019 at 1:32 PM David Davis  wrote:

> A new release of pulp_file 0.0.1b10 has been pushed to PyPI. This release
> supports pulpcore 3.0.0rc1.
>
> https://pypi.org/project/pulp-file/0.0.1b10/
>
> David
>
>
> On Thu, Mar 28, 2019 at 2:03 PM David Davis  wrote:
>
>> Today we are excited to announce an important milestone of the Pulp 3
>> project, the release of pulpcore 3.0.0rc1 and pulpcore-plugin 0.1.0rc1.
>> These packages constitute the core of Pulp 3 and its plugin API. Both are
>> now available on PyPI.
>>
>> In the coming days, we expect plugins to announce releases that are
>> compatible with this new RC release of pulpcore. We suggest that they reply
>> to this email thread to announce when they have a compatible release.
>>
>> For more information about this RC release, please visit our blog post at
>> https://pulpproject.org/2019/03/28/pulp-3-rc-release/.
>>
>> Special thanks to all the contributors, stakeholders, and community
>> members who made this release possible.
>> David
>>
> ___
> Pulp-dev mailing list
> pulp-...@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] What Pulp2 RPM sync or publish features do you most use?

[Brian Bouterse]

I'm trying to get some feedback to inform pulp3 RPM development.

If you used the RPM plugin for Pulp 2.y, what sync or publish features did
you use most? These are features like "force-full" syncing, retain X
versions of an rpm, sync or publish only specific types, e.g. "errata".

If you can reply on-list so the rest of the rpm developers can also see
that would be helpful.

Thanks!
Brian
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Pulp Community Demo - Dec 5th

[Brian Bouterse]

Due to technical issues, this is moving to Thursday at 16:00 UTC. Sorry for
the last minute change.

On Tue, Dec 4, 2018 at 12:22 PM Brian Bouterse  wrote:

> On Wednesday, the Pulp community demo will be broadcast live via YouTube.
> The tentative agenda is:
>
> * Pulp3 Lazy Sync, bmbouter, 3.0
> * Pulp3 Docker Sync and Publish, dkliban, 3.0
>
> When: Dec 5th at 3pm UTC [0]
> Where: https://www.youtube.com/watch?v=IeY4lI8jG38
> Interact: Chat during the event on the watch page ^ or leave comments on
> the page after
>
> This video will be available on the Pulp YouTube Channel[1] after
> recording. Consider following the Pulp YouTube channel where we post all
> Pulp videos. Any feedback or suggestions are welcome.
>
> [0]: https://bit.ly/2Q7d5uw <https://bit.ly/2ANCtfn>
> [1]: https://www.youtube.com/channel/UCI43Ffs4VPDv7awXvvBJfRQ
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Pulp Community Demo - Dec 5th

[Brian Bouterse]

On Wednesday, the Pulp community demo will be broadcast live via YouTube.
The tentative agenda is:

* Pulp3 Lazy Sync, bmbouter, 3.0
* Pulp3 Docker Sync and Publish, dkliban, 3.0

When: Dec 5th at 3pm UTC [0]
Where: https://www.youtube.com/watch?v=IeY4lI8jG38
Interact: Chat during the event on the watch page ^ or leave comments on
the page after

This video will be available on the Pulp YouTube Channel[1] after
recording. Consider following the Pulp YouTube channel where we post all
Pulp videos. Any feedback or suggestions are welcome.

[0]: https://bit.ly/2Q7d5uw 
[1]: https://www.youtube.com/channel/UCI43Ffs4VPDv7awXvvBJfRQ
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Pulp Community Demo - Nov 7th

[Brian Bouterse]

The Pulp Community Demo video is available on the Pulp YouTube Channel [0]
and on the Pulp blog [1].


Sections from the demo:

* Community Update (bmbouter) -
http://www.youtube.com/watch?v=Cl9v5WYM-og=0m25s

* Human-friendly specified fields for serialization (dalley) (3.0) -
http://www.youtube.com/watch?v=Cl9v5WYM-og=5m15s

* Unified Pulp 3 Ansible Installer (asmacdo) (3.0) -
http://www.youtube.com/watch?v=Cl9v5WYM-og=11m15s

* Pulplift demo to easily run Pulp3 (bmbouter) (3.0) -
http://www.youtube.com/watch?v=Cl9v5WYM-og=26m38s

* Stages API Performance Data Collector (bmbouter) (3.0) -
http://www.youtube.com/watch?v=Cl9v5WYM-og=30m18s


You can find the presenter IRC nicknames in the links above along with the
version numbers they are being released in. You can ask questions via the
mailing list or come chat on IRC.

[0]: https://www.youtube.com/channel/UCI43Ffs4VPDv7awXvvBJfRQ
[1]: https://pulpproject.org/2018/11/07/community-demo/


On Tue, Nov 6, 2018 at 8:03 PM Brian Bouterse  wrote:

> On Wednesday, the Pulp community demo will be broadcast live via YouTube.
> The tentative agenda is:
>
> * Human-friendly specified fields for serialization, (dalley), 3.0
> * Unified Pulp 3 Ansible Installer, (asmacdo), 3.0
> * Pulplift demo to easily run Pulp3, (bmbouter), 3.0
> * Stages API Performance Data Collector, (bmbouter), 3.0
>
> When: Nov 7th at 3pm UTC [0]
> Where: https://www.youtube.com/watch?v=Cl9v5WYM-og
> <https://www.youtube.com/watch?v=xniMgTodmnc>
> Interact: Chat during the event on the watch page ^ or leave comments on
> the page after
>
> This video will be available on the Pulp YouTube Channel[1] after
> recording. Consider following the Pulp YouTube channel where we post all
> Pulp videos. Any feedback or suggestions are welcome.
>
> [0]: https://bit.ly/2ANCtfn
> [1]: https://www.youtube.com/channel/UCI43Ffs4VPDv7awXvvBJfRQ
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Pulp Community Demo - Nov 7th

[Brian Bouterse]

On Wednesday, the Pulp community demo will be broadcast live via YouTube.
The tentative agenda is:

* Human-friendly specified fields for serialization, (dalley), 3.0
* Unified Pulp 3 Ansible Installer, (asmacdo), 3.0
* Pulplift demo to easily run Pulp3, (bmbouter), 3.0
* Stages API Performance Data Collector, (bmbouter), 3.0

When: Nov 7th at 3pm UTC [0]
Where: https://www.youtube.com/watch?v=Cl9v5WYM-og

Interact: Chat during the event on the watch page ^ or leave comments on
the page after

This video will be available on the Pulp YouTube Channel[1] after
recording. Consider following the Pulp YouTube channel where we post all
Pulp videos. Any feedback or suggestions are welcome.

[0]: https://bit.ly/2ANCtfn
[1]: https://www.youtube.com/channel/UCI43Ffs4VPDv7awXvvBJfRQ
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Pulp Community Demo - September 19th

[Brian Bouterse]

The Pulp Community Demo video is available on the Pulp YouTube Channel [0]
and on the Pulp blog [1].


Sections from the demo:

* Community Update (bmbouter) -
http://www.youtube.com/watch?v=xniMgTodmnc=0m51s

* pulp_rpm sync + publish demo for Pulp3 (daviddavis) (3.0) -
http://www.youtube.com/watch?v=xniMgTodmnc=7m34s

* new related model hooks added to ContentUnitSaver stage for Pulp3
(bmbouter) (3.0) - http://www.youtube.com/watch?v=xniMgTodmnc=14m20s

* swagger-codegen bindings for Pulp3 (dkliban) (3.0) -
http://www.youtube.com/watch?v=xniMgTodmnc=18m18s

* Pulp3 templates for Openshift (bmbouter) (3.0) -
http://www.youtube.com/watch?v=xniMgTodmnc=24m30s


You can find the presenter IRC nicknames in the links above along with the
version numbers they are being released in. You can ask questions via the
mailing list or come chat on IRC.

[0]: https://www.youtube.com/channel/UCI43Ffs4VPDv7awXvvBJfRQ
[1]: https://pulpproject.org/2018/09/19/community-demo/


On Tue, Sep 18, 2018 at 7:08 PM Brian Bouterse  wrote:

> On Wednesday, the Pulp community demo will be broadcast live via YouTube.
> The tentative agenda is:
>
> * pulp_rpm sync + publish demo for Pulp3 (daviddavis)
> * new related model hooks added to ContentUnitSaver stage for Pulp3
> (bmbouter)
> * swagger-codegen bindings for Pulp3 (dkliban)
> * Pulp3 templates for Openshift (bmbouter)
>
> When: Sept 19th at 2pm UTC [0]
> Where: https://www.youtube.com/watch?v=xniMgTodmnc
> Interact: Chat during the event on the watch page ^ or leave comments on
> the page after
>
> This video will be available on the Pulp YouTube Channel[1] after
> recording. Consider following the Pulp YouTube channel where we post all
> Pulp videos. Any feedback or suggestions are welcome.
>
> [0]: https://bit.ly/2NkFf3A
> [1]: https://www.youtube.com/channel/UCI43Ffs4VPDv7awXvvBJfRQ
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list