Bug#1055687: khmer ftbfs with Python 3.12
Hi, Thank you for uploading 3.0.0~a3+dfsg-6! I was about to request a sync in Ubuntu. Sadly, I noticed that I forgot to forward one change that was done as an intermediate upload in Ubuntu. The change is required for Python 3.12 support. Sorry about that! I've just opened https://salsa.debian.org/med-team/khmer/-/merge_requests/1 with the missing bits to correct the mistake. Best regards, Olivier
Bug#1055687: khmer ftbfs with Python 3.12
Am Wed, Nov 29, 2023 at 02:36:55PM +0100 schrieb Olivier Gayot: > https://salsa.debian.org/med-team/khmer/-/tree/master/debian/patches Arghh, my clone was not in sync. Sorry for the noise Andreas. -- http://fam-tille.de
Bug#1055687: khmer ftbfs with Python 3.12
Hi Andreas, On Wed, 29 Nov 2023 14:26:01 +0100 Andreas Tille wrote: When looking at the first patch > it applies to a series file containing the patches > > refresh_cython > find_object_files_at_right_loc.patch > > at the end. I'd love to profit from all those patches. Where > can I find these? > You can find all the patches that are currently used for khmer 3.0.0~a3+dfsg-5 in the khmer tree from Debian Med: https://salsa.debian.org/med-team/khmer/-/tree/master/debian/patches Thanks, Olivier
Bug#1055687: khmer ftbfs with Python 3.12
Hi Olivier,
thanks a lot for your patches. When looking at the first patch
it applies to a series file containing the patches
refresh_cython
find_object_files_at_right_loc.patch
at the end. I'd love to profit from all those patches. Where
can I find these?
Kind regards
Andreas.
Am Sun, Nov 26, 2023 at 09:06:39PM +0100 schrieb Olivier Gayot:
> Package: khmer
> Followup-For: Bug #1055687
> User: ubuntu-de...@lists.ubuntu.com
> Usertags: origin-ubuntu noble ubuntu-patch
> Control: tags -1 patch
>
> Dear Maintainer,
>
> My previous patch was unfortunately very incomplete. I am submitting
> another patch that fixes the remaining issues when building with Python
> 3.12. Both debdiffs should be applied for the build to succeed.
>
> Additional fixes that were needed for the build to succeed:
>
> * Python 3.12 dropped the "imp" module. Updated to use importlib
> instead.
> * Python 3.12 is much less forgiving when a script opens a file for
> writing and forgets to close it. Most Python scripts in scripts/ failed
> to do so or did so inconsistently. This resulted in the test suite
> either hanging or failing. I went through all invocations of open() /
> get_file_writer() and ensured that the resources are cleaned up. The
> resulting patch is sadly difficult to read though.
>
> I submitted all changes upstream, although I doubt somebody will pick
> them up:
>
> https://github.com/dib-lab/khmer/pull/1922
>
> In Ubuntu, the attached patch was applied to achieve the following:
>
> * Fix build against Python 3.12 (LP: #2044383).
>
> Thanks for considering the patch.
>
>
> -- System Information:
> Debian Release: trixie/sid
> APT prefers mantic-updates
> APT policy: (500, 'mantic-updates'), (500, 'mantic-security'), (500,
> 'mantic')
> Architecture: amd64 (x86_64)
> Foreign Architectures: i386
>
> Kernel: Linux 6.1.0-16-generic (SMP w/8 CPU threads; PREEMPT)
> Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE
> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not
> set
> Shell: /bin/sh linked to /usr/bin/dash
> Init: systemd (via /run/systemd/system)
> LSM: AppArmor: enabled
> diff -Nru khmer-3.0.0~a3+dfsg/debian/control
> khmer-3.0.0~a3+dfsg/debian/control
> --- khmer-3.0.0~a3+dfsg/debian/control2023-11-25 17:44:28.0
> +0100
> +++ khmer-3.0.0~a3+dfsg/debian/control2023-11-26 02:28:32.0
> +0100
> @@ -1,6 +1,5 @@
> Source: khmer
> -Maintainer: Ubuntu Developers
> -XSBC-Original-Maintainer: Debian Med Packaging Team
>
> +Maintainer: Debian Med Packaging Team
>
> Uploaders: Michael R. Crusoe ,
> Kevin Murray
> Section: science
> diff -Nru khmer-3.0.0~a3+dfsg/debian/patches/close-opened-files.patch
> khmer-3.0.0~a3+dfsg/debian/patches/close-opened-files.patch
> --- khmer-3.0.0~a3+dfsg/debian/patches/close-opened-files.patch
> 1970-01-01 01:00:00.0 +0100
> +++ khmer-3.0.0~a3+dfsg/debian/patches/close-opened-files.patch
> 2023-11-26 02:28:32.0 +0100
> @@ -0,0 +1,1124 @@
> +Description: ensure that Python scripts close files that they open for
> writing
> + Python scripts under scripts/ in the source tree do not consistently close
> + files that they open for writing. While some of the scripts use context
> + managers, most of them do not (or do so inconsistently).
> + In previous releases of Ubuntu, this apparently was not much of a concern.
> + However, Python 3.12 seems to be much less forgiving when files are not
> + properly closed. When running the test suite, many of the files that are not
> + explicitly closed appear truncated. This leads to various tests failing or
> + hanging and causing FTBFS when the test suite runs at build time.
> + .
> + Furthermore, khmer defines the get_file_writer() function, but it cannot be
> + consistently used as a context manager because it sometimes closes the
> + underlying file descriptor ; and sometimes does not depending on the
> + arguments.
> + .
> + Fixed by defining a new FileWriter context manager and ensuring that
> + each call to open() / get_file_writer() frees up resources properly.
> +Author: Olivier Gayot
> +Bug-Ubuntu: https://launchpad.net/bugs/2044383
> +Bug-Debian: https://bugs.debian.org/1055687
> +Forwarded: https://github.com/dib-lab/khmer/pull/1922
> +Last-Update: 2023-11-26
> +---
> +This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
> +Index: khmer-3.0.0~a3+dfsg/scripts/abundance-dist.py
> +===
> +--- khmer-3.0.0~a3+dfsg.orig/scripts/abundance-dist.py 2023-11-26
> 20:23:39.915485717 +0100
> khmer-3.0.0~a3+dfsg/scripts/abundance-dist.py2023-11-26
> 20:23:39.911485747 +0100
> +@@ -42,6 +42,7 @@
> + Use '-h' for parameter help.
> + """
> +
> ++import contextlib
> + import sys
> + import csv
> + import khmer
> +@@ -143,26 +144,28 @@
> + sys.exit(1)
> +
> + if args.output_histogram_filename in ('-',
Bug#1055687: khmer ftbfs with Python 3.12
Package: khmer
Followup-For: Bug #1055687
User: ubuntu-de...@lists.ubuntu.com
Usertags: origin-ubuntu noble ubuntu-patch
Control: tags -1 patch
Dear Maintainer,
My previous patch was unfortunately very incomplete. I am submitting
another patch that fixes the remaining issues when building with Python
3.12. Both debdiffs should be applied for the build to succeed.
Additional fixes that were needed for the build to succeed:
* Python 3.12 dropped the "imp" module. Updated to use importlib
instead.
* Python 3.12 is much less forgiving when a script opens a file for
writing and forgets to close it. Most Python scripts in scripts/ failed
to do so or did so inconsistently. This resulted in the test suite
either hanging or failing. I went through all invocations of open() /
get_file_writer() and ensured that the resources are cleaned up. The
resulting patch is sadly difficult to read though.
I submitted all changes upstream, although I doubt somebody will pick
them up:
https://github.com/dib-lab/khmer/pull/1922
In Ubuntu, the attached patch was applied to achieve the following:
* Fix build against Python 3.12 (LP: #2044383).
Thanks for considering the patch.
-- System Information:
Debian Release: trixie/sid
APT prefers mantic-updates
APT policy: (500, 'mantic-updates'), (500, 'mantic-security'), (500, 'mantic')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 6.1.0-16-generic (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
diff -Nru khmer-3.0.0~a3+dfsg/debian/control khmer-3.0.0~a3+dfsg/debian/control
--- khmer-3.0.0~a3+dfsg/debian/control 2023-11-25 17:44:28.0 +0100
+++ khmer-3.0.0~a3+dfsg/debian/control 2023-11-26 02:28:32.0 +0100
@@ -1,6 +1,5 @@
Source: khmer
-Maintainer: Ubuntu Developers
-XSBC-Original-Maintainer: Debian Med Packaging Team
+Maintainer: Debian Med Packaging Team
Uploaders: Michael R. Crusoe ,
Kevin Murray
Section: science
diff -Nru khmer-3.0.0~a3+dfsg/debian/patches/close-opened-files.patch
khmer-3.0.0~a3+dfsg/debian/patches/close-opened-files.patch
--- khmer-3.0.0~a3+dfsg/debian/patches/close-opened-files.patch 1970-01-01
01:00:00.0 +0100
+++ khmer-3.0.0~a3+dfsg/debian/patches/close-opened-files.patch 2023-11-26
02:28:32.0 +0100
@@ -0,0 +1,1124 @@
+Description: ensure that Python scripts close files that they open for writing
+ Python scripts under scripts/ in the source tree do not consistently close
+ files that they open for writing. While some of the scripts use context
+ managers, most of them do not (or do so inconsistently).
+ In previous releases of Ubuntu, this apparently was not much of a concern.
+ However, Python 3.12 seems to be much less forgiving when files are not
+ properly closed. When running the test suite, many of the files that are not
+ explicitly closed appear truncated. This leads to various tests failing or
+ hanging and causing FTBFS when the test suite runs at build time.
+ .
+ Furthermore, khmer defines the get_file_writer() function, but it cannot be
+ consistently used as a context manager because it sometimes closes the
+ underlying file descriptor ; and sometimes does not depending on the
+ arguments.
+ .
+ Fixed by defining a new FileWriter context manager and ensuring that
+ each call to open() / get_file_writer() frees up resources properly.
+Author: Olivier Gayot
+Bug-Ubuntu: https://launchpad.net/bugs/2044383
+Bug-Debian: https://bugs.debian.org/1055687
+Forwarded: https://github.com/dib-lab/khmer/pull/1922
+Last-Update: 2023-11-26
+---
+This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
+Index: khmer-3.0.0~a3+dfsg/scripts/abundance-dist.py
+===
+--- khmer-3.0.0~a3+dfsg.orig/scripts/abundance-dist.py 2023-11-26
20:23:39.915485717 +0100
khmer-3.0.0~a3+dfsg/scripts/abundance-dist.py 2023-11-26
20:23:39.911485747 +0100
+@@ -42,6 +42,7 @@
+ Use '-h' for parameter help.
+ """
+
++import contextlib
+ import sys
+ import csv
+ import khmer
+@@ -143,26 +144,28 @@
+ sys.exit(1)
+
+ if args.output_histogram_filename in ('-', '/dev/stdout'):
+-countgraph_fp = sys.stdout
++countgraph_ctx = contextlib.nullcontext(enter_result=sys.stdout)
+ else:
+-countgraph_fp = open(args.output_histogram_filename, 'w')
+-countgraph_fp_csv = csv.writer(countgraph_fp)
+-# write headers:
+-countgraph_fp_csv.writerow(['abundance', 'count', 'cumulative',
+-'cumulative_fraction'])
+-
+-sofar = 0
+-for _, i in enumerate(abundances):
+-if i == 0 and not args.output_zero:
+-continue
++countgraph_ctx = open(args.output_histogram_filename, 'w')
+
+-
Bug#1055687: khmer ftbfs with Python 3.12
Package: khmer
Followup-For: Bug #1055687
User: ubuntu-de...@lists.ubuntu.com
Usertags: origin-ubuntu noble ubuntu-patch
Control: tags -1 patch
Dear Maintainer,
In Ubuntu, the attached patch was applied to achieve the following:
* Fix build against Python 3.12 (LP: #2044383).
Thanks for considering the patch.
-- System Information:
Debian Release: trixie/sid
APT prefers mantic-updates
APT policy: (500, 'mantic-updates'), (500, 'mantic-security'), (500, 'mantic')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 6.1.0-16-generic (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
diff -Nru khmer-3.0.0~a3+dfsg/debian/patches/python3.12-support.patch
khmer-3.0.0~a3+dfsg/debian/patches/python3.12-support.patch
--- khmer-3.0.0~a3+dfsg/debian/patches/python3.12-support.patch 1970-01-01
01:00:00.0 +0100
+++ khmer-3.0.0~a3+dfsg/debian/patches/python3.12-support.patch 2023-11-23
15:24:51.0 +0100
@@ -0,0 +1,24 @@
+Description: Add support for Python 3.12
+ Ever since Python 3.2, configparser.SafeConfigParser has been deprecated in
+ favor of configparser.ConfigParser. An alias existed for backward compability
+ but the alias was dropped from Python 3.12.
+Author: Olivier Gayot
+Bug-Ubuntu: https://launchpad.net/bugs/2044383
+Bug-Debian: https://bugs.debian.org/1055687
+Forwarded: https://github.com/dib-lab/khmer/pull/1922
+Last-Update: 2023-11-23
+---
+This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
+Index: khmer-3.0.0~a3+dfsg/versioneer.py
+===
+--- khmer-3.0.0~a3+dfsg.orig/versioneer.py 2019-03-13 14:42:12.0
+0100
khmer-3.0.0~a3+dfsg/versioneer.py 2023-11-23 15:19:50.025827413 +0100
+@@ -339,7 +339,7 @@
+ # configparser.NoOptionError (if it lacks "VCS="). See the docstring at
+ # the top of versioneer.py for instructions on writing your setup.cfg .
+ setup_cfg = os.path.join(root, "setup.cfg")
+-parser = configparser.SafeConfigParser()
++parser = configparser.ConfigParser()
+ with open(setup_cfg, "r") as f:
+ parser.readfp(f)
+ VCS = parser.get("versioneer", "VCS") # mandatory
diff -Nru khmer-3.0.0~a3+dfsg/debian/patches/series
khmer-3.0.0~a3+dfsg/debian/patches/series
--- khmer-3.0.0~a3+dfsg/debian/patches/series 2023-09-09 08:30:33.0
+0200
+++ khmer-3.0.0~a3+dfsg/debian/patches/series 2023-11-23 15:19:31.0
+0100
@@ -17,3 +17,4 @@
stringio-buffer.patch
refresh_cython
find_object_files_at_right_loc.patch
+python3.12-support.patch
Bug#1055687: khmer ftbfs with Python 3.12
Package: src:khmer
Version: 3.0.0~a3+dfsg-5
Severity: important
Tags: sid trixie
User: debian-pyt...@lists.debian.org
Usertags: python3.12
khmer ftbfs with Python 3.12
[...]
dh_auto_configure -O--buildsystem=pybuild
pybuild --configure -i python{version} -p "3.12 3.11"
I: pybuild base:310: python3.12 setup.py config
/<>/versioneer.py:421: SyntaxWarning: invalid escape
sequence '\s'
LONG_VERSION_PY['git'] = '''
*** NOTE: Found Cython, extension files will be transpiled if this is an
install invocation.
Traceback (most recent call last):
File "/<>/setup.py", line 200, in
"define_macros": [("VERSION", versioneer.get_version()), ],
File "/<>/versioneer.py", line 1480, in get_version
return get_versions()["version"]
^^
File "/<>/versioneer.py", line 1412, in get_versions
cfg = get_config_from_root(root)
^^
File "/<>/versioneer.py", line 342, in get_config_from_root
parser = configparser.SafeConfigParser()
^
AttributeError: module 'configparser' has no attribute
'SafeConfigParser'. Did you mean: 'RawConfigParser'?
E: pybuild pybuild:395: configure: plugin distutils failed with: exit
code=1: python3.12 setup.py config
dh_auto_configure: error: pybuild --configure -i python{version} -p
"3.12 3.11" returned exit code 13
make: *** [debian/rules:16: binary] Error 25
dpkg-buildpackage: error: debian/rules binary subprocess returned exit
status 2
