On Fri, Oct 12, 2018 at 7:11 PM Jeff Ortel <jor...@redhat.com> wrote:
> > > On 10/12/2018 11:37 AM, Milan Kovacik wrote: > > > > On Fri, Oct 12, 2018 at 5:17 PM Jeff Ortel <jor...@redhat.com> wrote: > >> >> >> On 10/12/2018 09:53 AM, Milan Kovacik wrote: >> >> >> >> On Fri, Oct 12, 2018 at 3:59 PM Jeff Ortel <jor...@redhat.com> wrote: >> >>> >>> >>> On 10/10/2018 08:59 AM, Milan Kovacik wrote: >>> >>> ...that might be the question we should ask ourselves once again when it >>> comes to recursive copying of units between repositories. >>> >>> I'd like to poll folks opinions about the possibilities that we may have >>> when it comes to integrating third party solvers in Pulp. My yesterday's >>> chat with the #fedora-modularity folks about us integrating the Fus[1] >>> solver in order to reuse the Fus algorithm ran into a couple of bumps: >>> >>> * it would be laborous to create a programmatic Python API between Fus >>> and Pulp because we can't directly use the libsolv thingies (pools, >>> solvables and friends) in such an API because Fus is written utilizing >>> GObject, which is incompatible with Swig, which in turn is used in libsolv >>> to expose the python bindings. One would have to either re-wrap libsolv >>> code in Fus to work with pygobject or submit PRs against libsolv to support >>> GObject introspection. I dunno the details of either approach (yet) but >>> from the sad faces on the IRC and the Fus PR[1] it seemed like a lot of >>> work but it's still an option >>> >>> * we still should be able to integrate thru a pipe into Fus, that would >>> make it possible to dump modular and ursine metadata into Fus to perform >>> the dependency solving in a separate subprocess. We should probably >>> re-check the reasons behind our previous decision not to do the same with >>> DNF[2]. >>> >>> >>> How is Integration with Fus via pipe (CLI) easier than with gobject? >>> Either way, you "can't directly use the libsolv thingies (pools, solvables >>> and friends)". Right? What am I missing? >>> >>> >> Right, a publish-like operation would be required every time, for all >> repositories involved in the copy to dump the metadata to the pipe(s); >> sample of this interface is can be found in Pungi: >> https://pagure.io/pungi/blob/master/f/pungi/wrappers/fus.py the "query" >> is passed thru command line. >> I just learnt Fedora will keep modules and their ursine deps in separate >> repos, so the source repo won't necessarily be closed on dependencies thus >> multiple source repos would be needed. >> >> >> This be done using the Fus gobject interface as well? >> > > we'd just dump the XML (and YAML) metadata and run: fus --repo > source1,1,/path/to/pipe1 --repo source2,2,/path/to/pipe2 --repo > target,system,/path/to/target_pipe "module(walrus)" "penguin:1-2.3" etc > then parse the textual output of fus such as: > > > Can't this ^ be done with Fus through gobject as well and instead of > parsing textual output, inspect the objects returned? > > To avoid having to search for the resolved units (in Mongo), we have to maintain a mapping[3] between the solver namespace and Pulp to be able to refer back to the Pulp units. Optimizing just the parsing wouldn't have much benefit; we'd best benefit from an API that would allow Pulp to have a reference to the solver unit representation. Currently we do this thru the libsolv.i[4] bindings therefore the Fus (Py)GObject interface would have to expose a similar (hopefully smaller) interface. This still seemed a little bit too much work last time I checked with Fus folks on #fedora-modularity. Note please also that the GObject way won't allow for flexibility in the solving algorithm[5] itself should Pulp require some adjustments in the future. [3] https://github.com/pulp/pulp_rpm/blob/2-master/plugins/pulp_rpm/plugins/importers/yum/pulp_solv.py#L243,#L244 [4] https://github.com/openSUSE/libsolv/blob/master/bindings/solv.i [5] https://github.com/fedora-modularity/fus/blob/master/fus.c#L955 > > # ---%>--------- > - nothing provides policycoreutils-python-utils needed by > container-selinux-2:2.69-3.git452b90d.module_2040+0e96cf1b.noarch > > Problem 1 / 1: > - conflicting requests > - nothing provides libpthread.so.0(GLIBC_2.2.5)(64bit) needed by > atomic-1.22.1-2.module_1637+1872e86a.x86_64 > - nothing provides libc.so.6(GLIBC_2.2.5)(64bit) needed by > atomic-1.22.1-2.module_1637+1872e86a.x86_64 > - nothing provides libpthread.so.0(GLIBC_2.3.2)(64bit) needed by > atomic-1.22.1-2.module_1637+1872e86a.x86_64 > - nothing provides /bin/bash needed by > atomic-1.22.1-2.module_1637+1872e86a.x86_64 > - nothing provides /usr/bin/python3 needed by > atomic-1.22.1-2.module_1637+1872e86a.x86_64 > - nothing provides python3-dateutil needed by > atomic-1.22.1-2.module_1637+1872e86a.x86_64 > - nothing provides dbus needed by > atomic-1.22.1-2.module_1637+1872e86a.x86_64 > # ---->%---------- > (fus:8524): fus-WARNING **: 15:13:09.350: Can't resolve all solvables > module:docker:2017.0:20180816194539:3ff668f0.x86_64@f29 > module:container-tools:2017.0:20180816194450:80bd9113.x86_64@f29 > *docker-devel-2:1.13.1-61.git9cb56fd.module_2109+7c83ead1.noarch@f29 > *containers-common-0.1.31-14.dev.gitb0b750d.module_2040+0e96cf1b.x86_64@f29 > > > > >> >>> >>> * we should be able to extend current libsolv solver in Pulp, >>> reimplementing the algorithm from Fus. This might be as laborous as the >>> first option. It would probably give us more flexibility as well as more >>> room for screwing things up but the responsibility would be ours alone. >>> >>> Please let me know what option seems more appealing to you; other option >>> suggestion are welcome too. >>> >>> Cheers, >>> milan >>> >>> [1] https://github.com/fedora-modularity/fus/pull/46 >>> [2] https://pulp.plan.io/issues/3528#note-7 >>> >>> >>> _______________________________________________ >>> Pulp-dev mailing >>> listPulp-dev@redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev >>> >>> >>> _______________________________________________ >>> Pulp-dev mailing list >>> Pulp-dev@redhat.com >>> https://www.redhat.com/mailman/listinfo/pulp-dev >>> >> >> >
_______________________________________________ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev