Re: [Pulp-dev] Replace pulp_template repo with cookiecutter template

2018-10-12 Thread Daniel Alley 
Cookiecutter looks really nice and I'm not opposed to switching, but I
don't expect we would save much maintenance work in doing so. We aren't
doing / haven't needed to do too much maintenance on boostrap.py that I'm
aware of, it seems like it's relatively complete and tested.  Most of the
maintenance work is on the template itself.

Also in my brief lookthrough of the docs and some of the examples, I didn't
see anything about doing case substitutions like we're doing (e.g. creating
names RpmContent, RPM_CONTENT_PATH, pulp_rpm.app... from just "rpm").  Our
template is much bigger than any of the examples I looked at and it kind of
feels like it's focused towards a totally separate use case (filling out
many different small variables instead of the same one in many different
contexts).

I don't know, I feel like this is an "if it isn't broke" situation.



On Fri, Oct 12, 2018 at 11:13 AM, Dana Walker  wrote:

> Cookiecutter certainly looks very thorough, an active community, BSD
> license.  I'm fairly unfamiliar with our existing plugin_template for
> Pulp.  I suspect we would gain hours by not having to maintain as much for
> the template ourselves since our repo is fairly active.  Is there anything
> we would lose through this approach?
>
> --Dana
>
>
> Dana Walker
>
> Associate Software Engineer
>
> Red Hat
>
> 
> 
>
> On Fri, Oct 12, 2018 at 11:03 AM, Brian Bouterse 
> wrote:
>
>> A plugin writer suggested we do this, and I agree with them:
>>
>> https://pulp.plan.io/issues/4082
>>
>> Please comment on if this is a good idea or not.
>>
>> Thanks!
>> Brian
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] pulpcore-3.0.0b13 and pulpcore-plugin-0.1.0b11 available

2018-10-12 Thread Brian Bouterse 
The following packages are now available on PyPI:
   - pulpcore 3.0.0b13 [0] with its release notes here [1]
   - pulpcore-plugin 0.1.0b11 [2] with its release notes here [3]

The beta documentation is available here[4].

[0]: https://pypi.org/project/pulpcore/3.0.0b13/
[1]:
https://docs.pulpproject.org/en/3.0/nightly/release-notes/pulpcore/3.0.x.html#b13

[2]: https://pypi.org/project/pulpcore-plugin/0.1.0b11/
[3]:
https://docs.pulpproject.org/en/3.0/nightly/release-notes/plugin_api/0.1.z.html#b11
[4] https://docs.pulpproject.org/en/3.0/beta/
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] To integrate Fus or not to....

2018-10-12 Thread Jeff Ortel 



On 10/12/2018 11:37 AM, Milan Kovacik wrote:



On Fri, Oct 12, 2018 at 5:17 PM Jeff Ortel > wrote:




On 10/12/2018 09:53 AM, Milan Kovacik wrote:



On Fri, Oct 12, 2018 at 3:59 PM Jeff Ortel mailto:jor...@redhat.com>> wrote:



On 10/10/2018 08:59 AM, Milan Kovacik wrote:

...that might be the question we should ask ourselves once
again when it comes to recursive copying of units between
repositories.

I'd like to poll folks opinions about the possibilities that
we may have when it comes to integrating third party solvers
in Pulp. My yesterday's chat with the #fedora-modularity
folks about us integrating the Fus[1] solver in order to
reuse the Fus algorithm ran into a couple of bumps:

* it would be laborous to create a programmatic Python API
between Fus and Pulp because we can't directly use the
libsolv thingies (pools, solvables and friends) in such an
API because Fus is written utilizing GObject, which is
incompatible with Swig, which in turn is used in libsolv to
expose the python bindings. One would have to either re-wrap
libsolv code in Fus to work with pygobject or submit PRs
against libsolv to support GObject introspection. I dunno
the details of either approach (yet) but from the sad faces
on the IRC and the Fus PR[1] it seemed like a lot of work
but it's still an option

* we still should be able to integrate thru a pipe into Fus,
that would make it possible to dump modular and ursine
metadata into Fus to perform the dependency solving in a
separate subprocess. We should probably re-check the reasons
behind our previous decision not to do the same with DNF[2].


How is Integration with Fus via pipe (CLI) easier than with
gobject?  Either way, you "can't directly use the libsolv
thingies (pools, solvables and friends)".  Right?  What am I
missing?


Right, a publish-like operation would be required every time, for
all repositories involved in the copy to dump the metadata to the
pipe(s); sample of this interface is can be found in Pungi:
https://pagure.io/pungi/blob/master/f/pungi/wrappers/fus.py the
"query" is passed thru command line.
I just learnt Fedora will keep modules and their ursine deps in
separate repos, so the source repo won't necessarily be closed on
dependencies thus multiple source repos would be needed.


This be done using the Fus gobject interface as well?


we'd just dump the XML (and YAML) metadata and run: fus --repo 
source1,1,/path/to/pipe1 --repo source2,2,/path/to/pipe2 --repo 
target,system,/path/to/target_pipe  "module(walrus)" "penguin:1-2.3" etc

then parse the textual output of fus such as:


Can't this ^ be done with Fus through gobject as well and instead of 
parsing textual output, inspect the objects returned?




# ---%>-
  - nothing provides policycoreutils-python-utils needed by 
container-selinux-2:2.69-3.git452b90d.module_2040+0e96cf1b.noarch

Problem 1 / 1:
  - conflicting requests
  - nothing provides libpthread.so.0(GLIBC_2.2.5)(64bit) needed by 
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides libc.so.6(GLIBC_2.2.5)(64bit) needed by 
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides libpthread.so.0(GLIBC_2.3.2)(64bit) needed by 
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides /bin/bash needed by 
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides /usr/bin/python3 needed by 
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides python3-dateutil needed by 
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides dbus needed by 
atomic-1.22.1-2.module_1637+1872e86a.x86_64

# >%--
(fus:8524): fus-WARNING **: 15:13:09.350: Can't resolve all solvables
module:docker:2017.0:20180816194539:3ff668f0.x86_64@f29
module:container-tools:2017.0:20180816194450:80bd9113.x86_64@f29
*docker-devel-2:1.13.1-61.git9cb56fd.module_2109+7c83ead1.noarch@f29
*containers-common-0.1.31-14.dev.gitb0b750d.module_2040+0e96cf1b.x86_64@f29





* we should be able to extend current libsolv solver in
Pulp, reimplementing the algorithm from Fus. This might be
as laborous as the first option. It would probably give us
more flexibility as well as more room for screwing things up
but the responsibility would be ours alone.

Please let me know what option seems more appealing to you;
other option suggestion are welcome  too.

Cheers,
milan

[1] https://github.com/fedora-modularity/fus/pull/46
[2] https://pulp.plan.io/issues/3528#note-7


___
Pulp-dev mailing list
Pulp-dev@redhat.com

Re: [Pulp-dev] To integrate Fus or not to....

2018-10-12 Thread Milan Kovacik 
On Fri, Oct 12, 2018 at 5:17 PM Jeff Ortel  wrote:

>
>
> On 10/12/2018 09:53 AM, Milan Kovacik wrote:
>
>
>
> On Fri, Oct 12, 2018 at 3:59 PM Jeff Ortel  wrote:
>
>>
>>
>> On 10/10/2018 08:59 AM, Milan Kovacik wrote:
>>
>> ...that might be the question we should ask ourselves once again when it
>> comes to recursive copying of units between repositories.
>>
>> I'd like to poll folks opinions about the possibilities that we may have
>> when it comes to integrating third party solvers in Pulp. My yesterday's
>> chat with the #fedora-modularity folks about us integrating the Fus[1]
>> solver in order to reuse the Fus algorithm ran into a couple of bumps:
>>
>> * it would be laborous to create a programmatic Python API between Fus
>> and Pulp because we can't directly use the libsolv thingies (pools,
>> solvables and friends) in such an API because Fus is written utilizing
>> GObject, which is incompatible with Swig, which in turn is used in libsolv
>> to expose the python bindings. One would have to either re-wrap libsolv
>> code in Fus to work with pygobject or submit PRs against libsolv to support
>> GObject introspection. I dunno the details of either approach (yet) but
>> from the sad faces on the IRC and the Fus PR[1] it seemed like a lot of
>> work but it's still an option
>>
>> * we still should be able to integrate thru a pipe into Fus, that would
>> make it possible to dump modular and ursine metadata into Fus to perform
>> the dependency solving in a separate subprocess. We should probably
>> re-check the reasons behind our previous decision not to do the same with
>> DNF[2].
>>
>>
>> How is Integration with Fus via pipe (CLI) easier than with gobject?
>> Either way, you "can't directly use the libsolv thingies (pools, solvables
>> and friends)".  Right?  What am I missing?
>>
>>
> Right, a publish-like operation would be required every time, for all
> repositories involved in the copy to dump the metadata to the pipe(s);
> sample of this interface is can be found in Pungi:
> https://pagure.io/pungi/blob/master/f/pungi/wrappers/fus.py the "query"
> is passed thru command line.
> I just learnt Fedora will keep modules and their ursine deps in separate
> repos, so the source repo won't necessarily be closed on dependencies thus
> multiple source repos would be needed.
>
>
> This be done using the Fus gobject interface as well?
>

we'd just dump the XML (and YAML) metadata and run: fus --repo
source1,1,/path/to/pipe1 --repo source2,2,/path/to/pipe2 --repo
target,system,/path/to/target_pipe  "module(walrus)" "penguin:1-2.3" etc
then parse the textual output of fus such as:

# ---%>-
  - nothing provides policycoreutils-python-utils needed by
container-selinux-2:2.69-3.git452b90d.module_2040+0e96cf1b.noarch

Problem 1 / 1:
  - conflicting requests
  - nothing provides libpthread.so.0(GLIBC_2.2.5)(64bit) needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides libc.so.6(GLIBC_2.2.5)(64bit) needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides libpthread.so.0(GLIBC_2.3.2)(64bit) needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides /bin/bash needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides /usr/bin/python3 needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides python3-dateutil needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides dbus needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
# >%--
(fus:8524): fus-WARNING **: 15:13:09.350: Can't resolve all solvables
module:docker:2017.0:20180816194539:3ff668f0.x86_64@f29
module:container-tools:2017.0:20180816194450:80bd9113.x86_64@f29
*docker-devel-2:1.13.1-61.git9cb56fd.module_2109+7c83ead1.noarch@f29
*containers-common-0.1.31-14.dev.gitb0b750d.module_2040+0e96cf1b.x86_64@f29




>
>>
>> * we should be able to extend current libsolv solver in Pulp,
>> reimplementing the algorithm from Fus. This might be as laborous as the
>> first option. It would probably give us more flexibility as well as more
>> room for screwing things up but the responsibility would be ours alone.
>>
>> Please let me know what option seems more appealing to you; other option
>> suggestion are welcome  too.
>>
>> Cheers,
>> milan
>>
>> [1] https://github.com/fedora-modularity/fus/pull/46
>> [2] https://pulp.plan.io/issues/3528#note-7
>>
>>
>> ___
>> Pulp-dev mailing 
>> listPulp-dev@redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] To integrate Fus or not to....

2018-10-12 Thread Jeff Ortel 



On 10/12/2018 09:53 AM, Milan Kovacik wrote:



On Fri, Oct 12, 2018 at 3:59 PM Jeff Ortel > wrote:




On 10/10/2018 08:59 AM, Milan Kovacik wrote:

...that might be the question we should ask ourselves once again
when it comes to recursive copying of units between repositories.

I'd like to poll folks opinions about the possibilities that we
may have when it comes to integrating third party solvers in
Pulp. My yesterday's chat with the #fedora-modularity folks about
us integrating the Fus[1] solver in order to reuse the Fus
algorithm ran into a couple of bumps:

* it would be laborous to create a programmatic Python API
between Fus and Pulp because we can't directly use the libsolv
thingies (pools, solvables and friends) in such an API because
Fus is written utilizing GObject, which is incompatible with
Swig, which in turn is used in libsolv to expose the python
bindings. One would have to either re-wrap libsolv code in Fus to
work with pygobject or submit PRs against libsolv to support
GObject introspection. I dunno the details of either approach
(yet) but from the sad faces on the IRC and the Fus PR[1] it
seemed like a lot of work but it's still an option

* we still should be able to integrate thru a pipe into Fus, that
would make it possible to dump modular and ursine metadata into
Fus to perform the dependency solving in a separate subprocess.
We should probably re-check the reasons behind our previous
decision not to do the same with DNF[2].


How is Integration with Fus via pipe (CLI) easier than with
gobject?  Either way, you "can't directly use the libsolv thingies
(pools, solvables and friends)". Right?  What am I missing?


Right, a publish-like operation would be required every time, for all 
repositories involved in the copy to dump the metadata to the pipe(s); 
sample of this interface is can be found in Pungi: 
https://pagure.io/pungi/blob/master/f/pungi/wrappers/fus.py the 
"query" is passed thru command line.
I just learnt Fedora will keep modules and their ursine deps in 
separate repos, so the source repo won't necessarily be closed on 
dependencies thus multiple source repos would be needed.


This be done using the Fus gobject interface as well?



* we should be able to extend current libsolv solver in Pulp,
reimplementing the algorithm from Fus. This might be as laborous
as the first option. It would probably give us more flexibility
as well as more room for screwing things up but the
responsibility would be ours alone.

Please let me know what option seems more appealing to you; other
option suggestion are welcome  too.

Cheers,
milan

[1] https://github.com/fedora-modularity/fus/pull/46
[2] https://pulp.plan.io/issues/3528#note-7


___
Pulp-dev mailing list
Pulp-dev@redhat.com 
https://www.redhat.com/mailman/listinfo/pulp-dev


___
Pulp-dev mailing list
Pulp-dev@redhat.com 
https://www.redhat.com/mailman/listinfo/pulp-dev



___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Replace pulp_template repo with cookiecutter template

2018-10-12 Thread Dana Walker 
Cookiecutter certainly looks very thorough, an active community, BSD
license.  I'm fairly unfamiliar with our existing plugin_template for
Pulp.  I suspect we would gain hours by not having to maintain as much for
the template ourselves since our repo is fairly active.  Is there anything
we would lose through this approach?

--Dana


Dana Walker

Associate Software Engineer

Red Hat




On Fri, Oct 12, 2018 at 11:03 AM, Brian Bouterse 
wrote:

> A plugin writer suggested we do this, and I agree with them:
>
> https://pulp.plan.io/issues/4082
>
> Please comment on if this is a good idea or not.
>
> Thanks!
> Brian
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] Replace pulp_template repo with cookiecutter template

2018-10-12 Thread Brian Bouterse 
A plugin writer suggested we do this, and I agree with them:

https://pulp.plan.io/issues/4082

Please comment on if this is a good idea or not.

Thanks!
Brian
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] content guards

2018-10-12 Thread Jeff Ortel 
I'd like to get broader input on #3968 from comment: 
https://pulp.plan.io/issues/3968#note-14 on.


I don't think this falls within the classic discussion of thin/fat 
models.  Mainly because the methods included in what is considered a 
/fat/ model by django are still confined within the concerns of the data 
model.  We already have a few /fat/ models such as RepositoryVersion.  
What's being discussed for content-guards (and previously discussed for 
sync and publish) is different.



[1] 
https://django-best-practices.readthedocs.io/en/latest/applications.html#make-em-fat
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] To integrate Fus or not to....

2018-10-12 Thread Jeff Ortel 



On 10/10/2018 08:59 AM, Milan Kovacik wrote:
...that might be the question we should ask ourselves once again when 
it comes to recursive copying of units between repositories.


I'd like to poll folks opinions about the possibilities that we may 
have when it comes to integrating third party solvers in Pulp. My 
yesterday's chat with the #fedora-modularity folks about us 
integrating the Fus[1] solver in order to reuse the Fus algorithm ran 
into a couple of bumps:


* it would be laborous to create a programmatic Python API between Fus 
and Pulp because we can't directly use the libsolv thingies (pools, 
solvables and friends) in such an API because Fus is written utilizing 
GObject, which is incompatible with Swig, which in turn is used in 
libsolv to expose the python bindings. One would have to either 
re-wrap libsolv code in Fus to work with pygobject or submit PRs 
against libsolv to support GObject introspection. I dunno the details 
of either approach (yet) but from the sad faces on the IRC and the Fus 
PR[1] it seemed like a lot of work but it's still an option


* we still should be able to integrate thru a pipe into Fus, that 
would make it possible to dump modular and ursine metadata into Fus to 
perform the dependency solving in a separate subprocess. We should 
probably re-check the reasons behind our previous decision not to do 
the same with DNF[2].


How is Integration with Fus via pipe (CLI) easier than with gobject?  
Either way, you "can't directly use the libsolv thingies (pools, 
solvables and friends)".  Right?  What am I missing?




* we should be able to extend current libsolv solver in Pulp, 
reimplementing the algorithm from Fus. This might be as laborous as 
the first option. It would probably give us more flexibility as well 
as more room for screwing things up but the responsibility would be 
ours alone.


Please let me know what option seems more appealing to you; other 
option suggestion are welcome  too.


Cheers,
milan

[1] https://github.com/fedora-modularity/fus/pull/46
[2] https://pulp.plan.io/issues/3528#note-7


___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev


___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Changeset code in Pulp 3

2018-10-12 Thread David Davis 
Cool, I’ve opened an issue to remove changesets:

https://pulp.plan.io/issues/4078

David


On Thu, Oct 11, 2018 at 1:12 PM Jeff Ortel  wrote:

> Looks like we're fully invested in stages.  I don't think it makes sense
> to maintain both.  We can always resurrect it later (in some form) as
> needed.
>
> On 10/10/2018 01:17 PM, David Davis wrote:
>
> As part of the upcoming RC release, there was a question as to whether the
> Changeset code could  removed. AFAIK, there is only one plugin still using
> it (pulp_ansible) although there’s a ticket to update it to use the Stages
> code. I wanted to ask though if we were planning to keep the Changeset code
> in Pulp 3?
>
> David
>
>
> ___
> Pulp-dev mailing 
> listPulp-dev@redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev