Hi,
a quick follow up on my py3-packaging patches.
I've split them into parts, because there's a lot of them, and some of
the older patches needed to be rebased on top of new stuff.
Two issues arised when working on the patches.
The first one is trying to support both py36 and py37 on Fedora 29.
We use 'automation/*packages*' files to decide what needs to be
installed in order to run the tests.
These files contain names of yum/dnf packages.
The thing is, that these packages are installed into
'/usr/lib/python3.7/site-packages'.
When trying to run py36 tests, they can't be found.
This can be fixed, by including the test dependencies in 'tox.ini' in
the 'deps' section.
This is how tox handles dependencies:
- if the package is already available in the OS, it uses that version
(py37 case)
- if the package is not available, it installs it to its local
environment with 'pip' (py36 case)
Listing the dependencies in the 'deps' section fixes the problem... with
one exception.
'selinux' python bindings are not available in PyPI - there's only the
yum/dnf package version.
A workaround would be to mark the tests that require 'selinux' with
something like 'test_on_target_python_only'.
I think we should focus on the packaging stuff right now, so I would
postpone the fix for this problem.
The first batch of patches [1] partially deals with the issue by a
workaround.
It limits the tested interpreter versions to "python2.7 python3" instead
of "python2.7 python3.6 python3.7".
The effect is, that if a developer runs Fedora 29, and has both py36 and
py37 installed, only py37 will be used by default (that's the one that
'python3' is pointing to).
CI is unaffected by this change (since there's only one py3 version anyway).
The second issue is byte-compiling our python modules.
Everything suggests that we're doing it twice (once by
'build-aux/py-compile' and the second time is done by
'brp-python-bytecompile' script [2]) as there are files like
'API.cpython-37.pyc' and 'API.cpython-37.opt-1.pyc' in the RPM build
root dir, and they are identical.
I'm still working on cleaning this mess up.
Other than that, there are patches that unify the makefile's test
targets [3] and the ones that add autogen.sh flags as described in the
design document [4], which are ready to be reviewed.
It would be nice if someone else played around a bit with the flags - I
use them all the time and they seem to work for me, but I'm already
occupied with the byte-compiling issue.
Thanks, Marcin
[1]
https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:build-system-overhaul-part1
[2]
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#manual-bytecompilation
[3]
https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:build-system-overhaul-part2-test-targets
[4]
https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:build-system-overhaul-part3-autogen-options
On 3/28/19 11:48 AM, Marcin Sobczyk wrote:
Hi,
I've made a google document, that describes the changes that come with
the patches and the rationale behind them:
https://docs.google.com/document/d/1XBf9eP0lh3IeRzkQ3X-c191rMpOwsTTkX-eaoPqWqR8/edit?usp=sharing
Please note, that the patches do not reflect 100% what's described in
the doc - while writing the doc I came to conclusions, that some
things should be improved.
Since Milan told, that Python 3 packaging is very high priority for
him (even if the packages don't actually work yet), I would like to
ask you to get involved in reviewing the document.
After we establish a common ground, I will align my patches to reflect
what's in the doc.
P.S. Previous Markdown version was a pain to use so I just rewrote the
thing as a simple doc.
Thanks, Marcin
On 3/18/19 12:19 PM, Marcin Sobczyk wrote:
Hi,
it took a while... after trying to grasp "the big picture", I came up
with these:
https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:build-system-overhaul
The old topic:
https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:py3-rpm-packages
was rebased (not yet 100%) on top of 'build-system-overhaul'.
I've tried to implement a solution, that would clean up our
configure.ac/Makefiles, make Python 3 packaging possible, make
running tests with multiple interpreter versions possible for the
developers, etc.
I know, that I will get "crazy looks" for writing m4 macros, but I
really believe having stuff like 'VDSM_PY_SUPPORTED_MAJOR_VERSION' et
al. will make developers' lives easier.
I'm currently trying to address issues and comments raised by Milan,
I would definitely like to get others opinions on these.
Regards, Marcin
On 3/4/19 3:46 PM, Marcin Sobczyk wrote:
On 3/1/19 2:39 PM, Milan Zamazal wrote:
Marcin Sobczyk <msobc...@redhat.com> writes:
I've posted a series of patches:
https://gerrit.ovirt.org/#/q/topic:py3-rpm-packages+(status:open+OR+status:merged)
They do a couple of things:
* add an '--enable-python3-build' argument to 'autogen.sh'
* modify 'Makefile.am' so it uses either 'vdsm.spec.in' or
'vdsm-py3.spec.in' file to build RPMs
* preserve the 'vdsm.spec[.in]' filename in dist archives for
both py2
and py3 packaging
* fix shebang lines in a dist-hook, so that py3 packages will
contain
only '#!/usr/bin/python3' shebangs
* prepare the CI for building py3 packages on fc28
Hi Marcin, cool, thank you!
So actually, if we replace the dummy 'vdsm-py3.spec.in' file in
https://gerrit.ovirt.org/#/c/98119/ with something useful (be it the
nasty, hacky one, or the shiny, new one), we're ready to build py3
packages.
I've made a dirty one: https://gerrit.ovirt.org/98150 (you can simply
cherry pick the to wherever you need).
However, it doesn't work, it's still built with Python 2 on my Fedora
28. I can see a lot of warning such as
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in
the future.
I can see that configure.ac in your patches still uses Python 2. Here
is what I used in my private hacks to change that. Something similar
should be done, depending on --enable-python3-build presence, and with
properly detected Python versions.
Good point - I'm currently working on resolving that. It got kinda
complicated but will keep you posted as soon as I get some results.
Would you like to make the necessary changes or should I make them
myself?
modified configure.ac
@@ -48,7 +48,7 @@ AM_INIT_AUTOMAKE([-Wno-portability])
# Checking for build tools
AC_PROG_LN_S
-AM_PATH_PYTHON([2.6])
+AM_PATH_PYTHON([3.6])
AC_ARG_ENABLE(
[hooks],
@@ -349,13 +349,13 @@ if test "$PYTHON3_SUPPORT" = "1"; then
fi
# Checking for python-devel
-AC_PATH_PROG([PYTHON_CONFIG], [python-config])
+AC_PATH_PROG([PYTHON_CONFIG], [python3-config])
if test "x$PYTHON_CONFIG" = "x"; then
- AC_MSG_ERROR([python-devel not found, please install it.])
+ AC_MSG_ERROR([python3-devel not found, please install it.])
fi
# Checking for python modules (sorted, please keep in order)
-AX_PYTHON_MODULE([six], [fatal], python2)
+AX_PYTHON_MODULE([six], [fatal], python3)
# External programs (sorted, please keep in order)
AC_PATH_PROG([CAT_PATH], [cat], [/bin/cat])
_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PLSJVMM7OQWAUU5P6KM5W3S5RQFRJSC2/
