Re: [Pulp-dev] Hanging tasks (reloaded)

2020-01-21 Thread Simon Baatz
27;m not sure why I can't reproduce the issue on either of my machines >but I think it would be proper to re-pin redis until we get it figured >out. PR here: [1]https://github.com/pulp/pulpcore/pull/513 > >On Tue, Jan 21, 2020 at 4:37 PM Simon Baatz <[2]gm

Re: [Pulp-dev] Hanging tasks (reloaded)

2020-01-21 Thread Simon Baatz
14. http://pulp3-source-fedora30.anubis.example.com/ > 15. http://pulp3-source-fedora30.anubis.example.com/ > 16. http://pulp3-source-fedora30.anubis.example.com/ > 17. http://pulp3-source-fedora30.anubis.example.com/ > 18. http://pulp3-source-fedora30.anubis.example.com/ > 19. http://pulp3-source-fedora30.anubis.example.com/ > 20. http://pulp3-source-fedora30.anubis.example.com/ > 21. http://pulp3-source-fedora30.anubis.example.com/ > 22. http://pulp3-source-fedora30.anubis.example.com/ > 23. http://pulp3-source-fedora30.anubis.example.com/ > 24. http://pulp3-source-fedora30.anubis.example.com/ > 25. http://pulp3-source-fedora30.anubis.example.com/ > 26. http://pulp3-source-fedora30.anubis.example.com/ > 27. http://pulp3-source-fedora30.anubis.example.com/ > 28. http://pulp3-source-fedora30.anubis.example.com/ > 29. http://pulp3-source-fedora30.anubis.example.com/ > 30. http://pulp3-source-fedora30.anubis.example.com/ > 31. http://pulp3-source-fedora30.anubis.example.com/ > 32. http://pulp3-source-fedora30.anubis.example.com/ > 33. http://pulp3-source-fedora30.anubis.example.com/ > 34. http://pulp3-source-fedora30.anubis.example.com/ > 35. http://pulp3-source-fedora30.anubis.example.com/ > 36. http://pulp3-source-fedora30.anubis.example.com/ > 37. http://pulp3-source-fedora30.anubis.example.com/ > 38. http://pulp3-source-fedora30.anubis.example.com/ > 39. http://pulp3-source-fedora30.anubis.example.com/ > 40. http://pulp3-source-fedora30.anubis.example.com/ > 41. http://pulp3-source-fedora30.anubis.example.com/ > 42. http://pulp3-source-fedora30.anubis.example.com/ > 43. http://pulp3-source-fedora30.anubis.example.com/ > 44. http://pulp3-source-fedora30.anubis.example.com/ > 45. http://pulp3-source-fedora30.anubis.example.com/ > 46. http://pulp3-source-fedora30.anubis.example.com/ > 47. http://pulp3-source-fedora30.anubis.example.com/ > 48. http://pulp3-source-fedora30.anubis.example.com/ > 49. http://pulp3-source-fedora30.anubis.example.com/ > 50. http://pulp3-source-fedora30.anubis.example.com/ > 51. mailto:dal...@redhat.com > 52. https://github.com/rq/rq/issues/1044 > 53. mailto:dal...@redhat.com > 54. > https://github.com/andymccurdy/redis-py/issues/1136#issuecomment-571168161 > 55. mailto:dell...@atix.de > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev -- Simon Baatz ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Proposal: Changing in 3.1 that Artifact.save() will hard-link/copy, not move

2020-01-12 Thread Simon Baatz
On Tue, Jan 07, 2020 at 04:45:54PM -0500, Brian Bouterse wrote: >We had two bugs filed recently [0][1] which suggest that when using the >default backend for Pulp, i.e. pulpcore.app.models.storage.FileSystem >Pulp should not be "moving" files. This is the default behavior Django >gi

Re: [Pulp-dev] 3.1 GA plan -- Jan 30th

2020-01-12 Thread Simon Baatz
On Fri, Jan 10, 2020 at 12:10:34PM -0500, Brian Bouterse wrote: >David and I are coordinating the 3.1 pulpcore release. We are proposing >we release 3.1 on Jan 30th, and have it be a time-based release. >Tentatively, we hope to release about a pulpcore y-release every month >for the

Re: [Pulp-dev] Proposal: Changing in 3.1 that Artifact.save() will hard-link/copy, not move

2020-01-08 Thread Simon Baatz
On Tue, Jan 07, 2020 at 04:45:54PM -0500, Brian Bouterse wrote: >We had two bugs filed recently [0][1] which suggest that when using the >default backend for Pulp, i.e. pulpcore.app.models.storage.FileSystem >Pulp should not be "moving" files. This is the default behavior Django >gi

Re: [Pulp-dev] Required fields for models at the DB level

2019-12-02 Thread Simon Baatz
On Mon, Dec 02, 2019 at 03:53:06PM -0500, Brian Bouterse wrote: >If anyone has concerns with us enabling Model validation by default on >all models please let us know soon. I don't know (yet) if I have concerns, but DRF seems to have, see https://www.django-rest-framework.org/community/3.0

Re: [Pulp-dev] Solving the "callback problem" ... aka how pulpcore will stop finalizing RepositoryVersion

2019-11-05 Thread Simon Baatz
Hi Brian, On Tue, Nov 05, 2019 at 02:46:18PM -0500, Brian Bouterse wrote: >... >Note that the context manager is only syntactic sugar. The pulp_file >sync code could also just as easily be as shown below. This is >incomplete, but I think you'll get the idea. >[3]https://github.

Re: [Pulp-dev] Merging pulpcore.plugin into pulp/pulpcore repo?

2019-10-17 Thread Simon Baatz
+1 On Wed, Oct 16, 2019 at 03:50:54PM -0400, Mike DePaulo wrote: >+1. > >On Wed, Oct 16, 2019 at 3:26 PM Pavel Picka <[1]ppi...@redhat.com> >wrote: > >+1 > >On Wed, Oct 16, 2019 at 9:23 PM Dana Walker <[2]dawal...@redhat.com> >wrote: > >+1, more straightforward > >

[Pulp-dev] SingleArtifactContentUploadSerializer and task's created_resources

2019-10-01 Thread Simon Baatz
Using the new SingleArtifactContentUploadSerializer I noticed two things that I would like to get feedback on: 1. If a repo version is created (using the "repository" parameter), "created_resources" of the respective task contains the content unit and the new repo version, e.g. { "_crea

Re: [Pulp-dev] black

2019-06-27 Thread Simon Baatz
On Thu, Jun 27, 2019 at 07:58:38AM -0400, David Davis wrote: >Follow up question to adding support for black: should we drop flake8? >We shouldn't need it anymore since black is pep8 compliant but I'm >happy to keep it around at least temporarily if people prefer? Please do not drop it

Re: [Pulp-dev] black

2019-06-04 Thread Simon Baatz
On Tue, Jun 04, 2019 at 08:25:47AM -0400, David Davis wrote: >I wanted to get feedback from the Pulp community on using black[0] to >auto-format our Pulp 3 code. I have some mixed feelings about it as I >see some potential benefits of using it but also some downsides as >well. +1 f

Re: [Pulp-dev] uniqueness constraints within a repository version

2019-06-03 Thread Simon Baatz
On Mon, Jun 03, 2019 at 09:11:07AM -0400, David Davis wrote: >@Simon I like the idea behind the repo_key solution you came up with. >Can you be more specific around cases you think that it couldn't >handle? I imagine that plugin writers could use properties or >denormailzation (ie a

Re: [Pulp-dev] uniqueness constraints within a repository version

2019-05-31 Thread Simon Baatz
On Fri, May 31, 2019 at 01:12:58PM +0200, Tatiana Tereshchenko wrote: >A while ago RemoveDuplicates stage [0] was introduced to solve the >problem of enforcing uniqueness constraints within a repository version >at sync time. >The same problem ought to be solved when content which a

Re: [Pulp-dev] problems with publishers in Pulp 3

2019-05-05 Thread Simon Baatz
On Mon, Apr 29, 2019 at 02:48:29PM -0400, Dennis Kliban wrote: >+1 ... Want to write up a story? Yes! I created the story 4785 [1] and a draft PR [2] that contains the basic functionality. Comments welcome! [1] https://pulp.plan.io/issues/4785 [2] https://github.com/pulp/pulpcore/pull/117 _

Re: [Pulp-dev] problems with publishers in Pulp 3

2019-04-29 Thread Simon Baatz
On Fri, Apr 12, 2019 at 04:51:00PM -0400, Brian Bouterse wrote: >After some discussion the proposal has been adjusted to leave >publishers as-is and only introduce Master/Detail Publications with >this change. [1]https://pulp.plan.io/issues/4678#note-6 >Please provide more feedback

Re: [Pulp-dev] Pulp 3.0 RC release

2019-03-31 Thread Simon Baatz
Version 0.0.4b2 of the pulp_cookbook plugin is now available on PyPI as pulp-cookbook 0.0.4b2 [0] and on GitHub [1] The plugin supports pulpcore 3.0.0rc1 now. There have been no functional changes since the previous release. This means that the plugin currently supports PostgreSQL only, MySQL/Ma

Re: [Pulp-dev] Pulp 3 Licensing

2019-02-05 Thread Simon Baatz
Hi Dana, On Tue, Feb 05, 2019 at 03:55:38PM -0500, Dana Walker wrote: >Thus far, Pulp 3 has been operating under the GPLv2 license. Given the >way the GPL defines derivative works, this means that the plugins >should also be licensed as GPLv2. Take a look at this FAQ to further >

[Pulp-dev] pulp_cookbook 0.0.4b1 available

2019-02-05 Thread Simon Baatz
Version 0.0.4ab1 of the pulp_cookbook plugin is now available on PyPI as pulp-cookbook 0.0.4b1 [0] and on GitHub [1] Changes: - Supports 'lazy' remote policies ('on_demand', 'streaming') - live universe API migrated to new content app - Implements "repo isolation". Content is shared between repos

Re: [Pulp-dev] Concerns about bulk_create and PostgreSQL

2019-01-09 Thread Simon Baatz
rmarket, this would cause thousands of requests during an initial repo sync even when using on_demand policy) >David > >On Tue, Jan 8, 2019 at 2:50 PM Simon Baatz <[1]gmbno...@gmail.com> >wrote: > > On 08.01.2019 17:16, Jeff Ortel wrote: > > >

Re: [Pulp-dev] Concerns about bulk_create and PostgreSQL

2019-01-08 Thread Simon Baatz
On 08.01.2019 17:16, Jeff Ortel wrote: > > > On 1/3/19 1:28 PM, Simon Baatz wrote: >> On Thu, Jan 03, 2019 at 01:02:57PM -0500, David Davis wrote: >>>     I don't think that using integer ids with bulk_create and >>> supporting >>>     mysql/mari

Re: [Pulp-dev] Concerns about bulk_create and PostgreSQL

2019-01-03 Thread Simon Baatz
4290 >3. mailto:dal...@redhat.com >4. mailto:bbout...@redhat.com >5. mailto:pcre...@redhat.com >6. https://docs.djangoproject.com/en/2.1/ref/models/querysets/#bulk-create >7. https://github.com/pulp/pulp/pull/3764#discussio

Re: [Pulp-dev] Uniqueness constraints on Content in Pulp 3

2018-11-13 Thread Simon Baatz
On Tue, Nov 13, 2018 at 02:38:33PM -0500, David Davis wrote: >I want to point out that the RPM example is not correct. RPMs are >unique in Pulp by checksum (aka pkgId in our code and createrepo_c): >[1]https://github.com/pulp/pulp_rpm/blob/44f97560533379ad8680055edff9c3 >c5bd4e859f/

Re: [Pulp-dev] Uniqueness constraints on Content in Pulp 3

2018-11-08 Thread Simon Baatz
On Tue, Nov 06, 2018 at 11:40:35AM -0500, Brian Bouterse wrote: >These are great questions. I'll try to keep my responses short to >promote more discussion. >On Mon, Nov 5, 2018 at 3:21 PM Simon Baatz <[1]gmbno...@gmail.com> >wrote: > > I apologiz

[Pulp-dev] Uniqueness constraints on Content in Pulp 3

2018-11-05 Thread Simon Baatz
I apologize for the lengthy post, but I did not know where to file an issue for this. It is a generic problem affecting most Pulp 3 plugins. I am puzzled for some time now about the natural keys used for content in plugins. Examples are: pulp_python: 'filename' pulp_ansible: 'version', 'role' (fo

Re: [Pulp-dev] pulp-cookbook 0.0.2a2 released

2018-09-18 Thread Simon Baatz
On Sat, Sep 15, 2018 at 05:40:15PM -0400, Dennis Kliban wrote: >That's great news Simon! It should be added to the plugins page of the >docs[0]. What do you think? Good idea! PR is at [0] [0] https://github.com/pulp/pulp/pull/3646 ___ Pulp-dev

[Pulp-dev] pulp-cookbook 0.0.2a2 released

2018-09-14 Thread Simon Baatz
Version 0.0.2a2 of the pulp_cookbook plugin is now available on PyPI as pulp-cookbook 0.0.2a2 [0] and on GitHub [1] Notable changes: - sync is implemented now (using a custom stages API pipeline) - sync supports "specifier whitelisting" à la pulp_python (currently restricted to cookbook name m

Re: [Pulp-dev] Proposal to drop support of Python 3.5 for Pulp 3

2018-09-14 Thread Simon Baatz
lt;[3]ttere...@redhat.com> wrote: > >Big +1 >Tanya > >On Wed, Sep 12, 2018 at 4:50 PM, David Davis <[4]davidda...@redhat.com> >wrote: > >+1 >David > >On Tue, Sep 11, 2018 at 10:35 AM Jeff Ortel <[5]jor...@redhat.com> >w

Re: [Pulp-dev] Commit message validation

2018-09-11 Thread Simon Baatz
Hi David, On Tue, Sep 11, 2018 at 10:15:22AM -0400, David Davis wrote: >I want to announce that commit messages will be validated for PRs >against pulp’s master branch per [0]. This validation will check two >things. >First, the validation script checks that there’s an attached

[Pulp-dev] Proposal to drop support of Python 3.5 for Pulp 3

2018-09-06 Thread Simon Baatz
I had a discussion on IRC with Brian yesterday which led to the question whether we can drop support for Python 3.5. I think there are good reasons for this, see the rationale below. Brian proposed to initiate a vote on this topic (and find out whether this "community thing" works :-) ). Please s

Re: [Pulp-dev] 'id' versus 'pulp_id' on Content

2018-06-14 Thread Simon Baatz
My 2 cents (in my role as a user, not plugin writer): I think the most important argument in the entire discussion is this (not sure who said this): >* plugin users (not writers) who are familiar with 'id' as part of the >erratum data type would then have to also understand this field na

Re: [Pulp-dev] Pulp 3 plugin for Chef cookbooks

2018-05-17 Thread Simon Baatz
On Wed, May 16, 2018 at 04:14:33PM -0400, David Davis wrote: >This is great. I had a chance to look at your plugin and am really >excited by having chef support in Pulp. >Some responses below. >On Tue, May 15, 2018 at 2:31 PM, Simon Baatz <[1]gmbno...@gmail.

[Pulp-dev] Pulp 3 plugin for Chef cookbooks

2018-05-15 Thread Simon Baatz
I created the beginnings of a Pulp 3 plugin to manage Chef cookbooks [1]. Currently, it supports to create repos, create cookbook content units, and publish repos. A published & distributed repo will offer a "universe" API endpoint for tools like Berkshelf. I did not implement sync yet. I am wa

[Pulp-dev] Pulp 3 content import

2018-04-18 Thread Simon Baatz
I am looking at Pulp 3 for the first time and have a question on creating/importing content units. In the file plugin, a file content unit can be created using POST@.../content/file/ by passing the meta-data ('relative_path') and the artifact (from which 'digest' is obtained as additional meta-dat

Re: [Pulp-dev] Crane redirects - internal and external content

2017-12-21 Thread Simon Baatz
On Tue, Dec 19, 2017 at 12:41:08PM -0500, Dennis Kliban wrote: >Crane cannot perform a rewrite of the redirect URL at this time. This >seems like a reasonable feature request. I recommend filing a story - >we can discuss the feature details on there. > That would be a nice feature inde