That is a fantastic and extremely thorough answer, thank you! On Sun, Feb 23, 2020 at 8:32 PM Geoffrey Thomas <[email protected]> wrote:
> On Sun, 23 Feb 2020, Daniel Alley wrote: > > > I would like to package this library as a pre-built Python wheel: > https://github.com/fedora-modularity/libmodulemd > > > > This library uses PyGObject, so importing it looks like this: > > > > import gi > > gi.require_version("Modulemd", "2.0") > > from gi.repository import Modulemd > > > > I can't find any examples of this, nor any documentation, nor > discussion about it. It seems like it would be, at minimum, a bit more > complicated than libraries based on normal bindings. Is this > > possible, and are there any special requirements that are needed to do > so? > > It's been a bit since I've used gobject-introspection, but I _think_ the > way this works for a normal (OS-installed) GObject package is > > a) the package ordinarily provides no actual Python library > b) the package provides a girepository-1.0/Modulemd-2.0.typelib file in > the system lib directory > c) when you import it, PyGObject loads the C libmodulemd library and > generates Python bindings based on the typelib file > d) as a special case, a package _can_ provide a Python "override" library, > but that amends the autogenerated bindings, it's not a complete set of > bindings on its own (and it's Python code, not native code). Modulemd > appears to do this. > > So your users' code needs to be able to > - import PyGObject (and libgobject) itself, either from the system or from > their virtualenv > - import the C libmodulemd library from your wheel, which you can compile > for manylinux1 > - find the typelib file, which you can put in your wheel > - point libgobject at the typelib file and the C library > - point PyGObject at the override file (which should hopefully be > automatic if it's on sys.path) > > The GObject docs https://developer.gnome.org/gi/stable/GIRepository.html > say: > > > GIRepository will typically look for a girepository-1.0 directory under > > the library directory used when compiling gobject-introspection. > > > > It is possible to control the search paths programmatically, using > > g_irepository_prepend_search_path(). It is also possible to modify the > > search paths by using the GI_TYPELIB_PATH environment variable. The > > environment variable takes precedence over the default search path and > > the g_irepository_prepend_search_path() calls. > > You will also need to make sure the C library is importable. For "normal" > Python wheels, users import a compiled Python shared object (so that the > usual Python path is used as the search path), and that shared object has > a normal shared object dependency on the underlying C library which is > also shipped in the wheel.auditwheel sets an $ORIGIN-relative rpath in > that .so file (using patchelf) so that the Python module, having been > found inside the virtualenv, can find its C library in a relative path to > its own location. Since there is no compiled Python module in your case, > because PyGObject is dynamically generating the bindings at runtime, I > don't think there is a straightforward way of informing PyGObject of where > to find the C library. > > Personally, I'd approach this by first aiming for an 80% solution where I > expect users to set GI_TYPELIB_PATH and LD_LIBRARY_PATH so that the > typelib file and the C library can both be found, i.e., they use it by > running something to the effect of > > os.setenv("GI_TYPELIB_PATH", "myvenv/lib/girepository-1.0") > os.setenv("LD_LIBRARY_PATH", "myvenv/lib") > gi.require_version("Modulemd", "2.0") > from gi.repository import Modulemd > > That would let me confirm that I've actually gotten all the libraries > compiling properly inside the wheel and the code actually works. Then > there's a question of how to do this automatically. A 90% solution would > be to just decide that your wheel has a top-level Python module to do > this, e.g., you tell your users that if they're using the wheel they just > do "import Modulemd" and you create a Modulemd.py that does > > os.setenv("GI_TYPELIB_PATH", some relative path from __file__) > etc. > > (For bonus points, call g_irepository_prepend_search_path() / see if > PyGObject has some binding to it, instead of setting $GI_TYPELIB_PATH, and > use ctypes to load the actual C library using RTLD_GLOBAL so that it's > already loaded when PyGObject goes looking for it, instead of setting > $LD_LIBRARY_PATH.) > > In my (naive) opinion, a 100% solution here would be teaching PyGObject > how to find both typelib files and C libraries in paths based on sys.path, > and then your users could use the standard upstream import instructions > unmodified. (Actually, it's possible PyGObject does this already, but I > don't immediately see anything about it in the docs, and my assumption is > if you can't find examples of others doing this, the use case hasn't come > up.) > > One other question is whether your users are importing libgobject from the > system or from a wheel. For the average desktop Linux user, it's probably > fine to get libgobject from the system (and probably _preferable_ - you > likely want the same version as the Gtk/GNOME/etc. libraries they might > import, and if they're importing any of those, they almost certainly want > the system version of Gtk etc.) It appears that PyGObject is on PyPI as > sdists only, so if you don't want to assume your users have libgobject > installed, you may have to first fight the battle of packaging up GObject, > GLib, etc. into wheels. > > (Relatedly, I'm guessing the reason nobody has done this yet is that most > software that supports GObject introspection is GNOME-related in some > fashion and therefore most people want it from their OS package manager > and not from a wheel.) > > Again, it's been a while since I've worked with GObject introspection, so > if I got something wrong, anyone should feel free to correct me :) > > -- > Geoffrey Thomas > https://ldpreload.com > [email protected]
_______________________________________________ Wheel-builders mailing list [email protected] https://mail.python.org/mailman/listinfo/wheel-builders
