Bug#1101839: python3-tqdm: segmentation fault in destructor method
Hi,
Following on your idea here I can reproduce the segfault in unstable w/o
nxtomomill at all. In a mostly empty unstable Docker I see:
```
$ cat x.py
import pytest
from tqdm import tqdm
@pytest.mark.parametrize("progress", (None, tqdm(desc="hola")))
def test_poc(progress):
if progress is not None:
progress.total = 100
progress.n = 20
progress.refresh()
assert 1==1
$ pytest x.py
== test
session starts
===
platform linux -- Python 3.13.4, pytest-8.3.5, pluggy-1.5.0
rootdir: /x
collected 2 items
x.py ..
[100%]
=== 2 passed
in 0.02s
Segmentation fault (core dumped)
$ dpkg -l|grep tqdm
ii python3-tqdm 4.67.1-5 all
fast, extensible progress bar for Python 3 and CLI tool
```
So it seems this is indeed due some pytest--tqdm interaction; maybe
something to do with how the former creates parameter lists early during
module setup time, I'm not sure.
However if tqdm is needed to be tested with, but its output can be
omitted then `tqdm(..., disable=True)` makes the test work.
Alternatively one can postpone tqdm initialization:
```
...
@pytest.mark.parametrize("progress", (None, lambda: tqdm(desc="hola")))
def test_poc(progress):
if progress is not None:
progress = progress()
...
```
I can't find any details/prohibitions of tqdm use in pytest manuals and
I'm unsure if this actually is a tqdm bug, but maybe the workaround can
be filed against nxtomomill?
BR,
Gábor
Bug#1101839: python3-tqdm: segmentation fault in destructor method
Hi,
Well, I couldn't figure out why it's happening, but I managed to isolate
it.
To reproduce the bug, we can run only
nxtomomill/converter/edf/tests/test_edf2nx.py::test_edf_to_nx_converter
IMO the issue is in the pytest.mark.parametrize call
@pytest.mark.parametrize("progress", (None, tqdm(desc="conversion from edf")))
Removing that line, and defining progress in the test, the segmentation
fault doesn't occur.
def test_edf_to_nx_converter(duplicate_data, external_path_type):
+progress = tqdm(desc="conversion from edf") # or progress None
So, the first I think is that the issue is in the parametrize and/or
tqdm, but I made a little poc, and the error is not reproducible:
```
@pytest.mark.parametrize("progress", (None, tqdm(desc="hola")))
def test_poc(progress):
if progress is not None:
progress.total = 100
progress.n = 20
progress.refresh()
assert 1==1
```
So, now I suspect that the bug is in python3-nxtomomill side.
--
cheers,
Emmanuel Arias
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ [email protected]
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: 13796755BBC72BB8ABE2AEB5 FA9DEC5DE11C63F1
⠈⠳⣄
signature.asc
Description: PGP signature
Bug#1101839: python3-tqdm: segmentation fault in destructor method
Hi, Thanks. fyi I reproduced the bug. Now trying to figure out it. Thanks for the confirmation. -- cheers, Emmanuel Arias ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ [email protected] ⢿⡄⠘⠷⠚⠋⠀ OpenPGP: 13796755BBC72BB8ABE2AEB5 FA9DEC5DE11C63F1 ⠈⠳⣄ signature.asc Description: PGP signature
Bug#1101839: python3-tqdm: segmentation fault in destructor method
Hi Roland, For some reason I cannot reproduce the bug. I'm running autopktests in this way, previously I removed your Testuite deactivation. ``` autopkgtest python-nxtomomill_1.1.0-4.dsc -- unshare --release unstable ``` But I have a setuptools error instead: ``` ERROR Backend 'setuptools.build_meta:__legacy__' is not available. ``` Can you confirm please, if you are still reproducing the error, and please, could you provide how do you run autopkgtests? Thanks in advance! -- cheers, Emmanuel Arias ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ [email protected] ⢿⡄⠘⠷⠚⠋⠀ OpenPGP: 13796755BBC72BB8ABE2AEB5 FA9DEC5DE11C63F1 ⠈⠳⣄ signature.asc Description: PGP signature
Bug#1101839: python3-tqdm: segmentation fault in destructor method
Package: python3-tqdm
Version: 4.67.1-3
Severity: important
Dear Maintainer,
I'm seeing reproducible autopkgtest failures (segmentation fault) on
python3-nxtomomill. I've started debugging that using the following lines in
debian/rules:
export PYBUILD_TEST_CUSTOM=1
export PYBUILD_TEST_ARGS={interpreter} -X faulthandler -m pytest -p
no:faulthandler -v -k "not test_fluo2nx_application_"
This allowed me to get the following stacktrace of the interpreter at the
moment of the segfault:
[...]
nxtomomill/utils/tests/test_utils.py::TestChangeImageKeyControl::testUpdateToFlat
PASSED [ 98%]
nxtomomill/utils/tests/test_utils.py::TestChangeImageKeyControl::testUpdateToInvalid
PASSED [ 98%]
nxtomomill/utils/tests/test_utils.py::TestChangeImageKeyControl::testUpdateToProjections
PASSED [ 99%]
nxtomomill/utils/tests/test_utils.py::TestAddDarkAndFlatFromADifferentFolderWithVDS::test
PASSED [100%]
=== warnings summary
===
../../../../usr/lib/python3/dist-
packages/tomoscan/framereducer/framereducerbase.py:4
/usr/lib/python3/dist-packages/tomoscan/framereducer/framereducerbase.py:4:
DeprecationWarning: numpy.core.numerictypes is deprecated and has been renamed
to numpy._core.numerictypes. The numpy._core namespace contains private NumPy
internals and its use is discouraged, as NumPy internals can change without
warning in any release. In practice, most real-world usage of numpy.core is to
access functionality in the public NumPy API. If that is the case, use the
public NumPy API. If not, you are using NumPy internals. If you would still
like to access an internal attribute, use numpy._core.numerictypes.generic.
from numpy.core.numerictypes import generic as numy_generic
nxtomomill/app/tests/test_edf2nx_app.py::test_edf2nx_application
/usr/lib/python3/dist-packages/tqdm/std.py:636: TqdmWarning: clamping frac to
range [0, 1]
full_bar = Bar(frac,
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
168 passed, 3 deselected, 2 warnings in 29.38s
Fatal Python error: Segmentation fault
Current thread 0x7f2e7675d780 (most recent call first):
Garbage-collecting
File "/usr/lib/python3/dist-packages/tqdm/std.py", line 576 in format_meter
File "/usr/lib/python3/dist-packages/tqdm/std.py", line 1151 in __str__
File "/usr/lib/python3/dist-packages/tqdm/std.py", line 1495 in display
File "/usr/lib/python3/dist-packages/tqdm/std.py", line 1302 in close
File "/usr/lib/python3/dist-packages/tqdm/std.py", line 1148 in __del__
Segmentation fault
E: pybuild pybuild:389: test: plugin pyproject failed with: exit code=139: cd
/tmp/autopkgtest.4irIce/autopkgtest_tmp/build; {interpreter} -X faulthandler -m
pytest -p no:faulthandler -v -k "not test_fluo2nx_application_"
This seems to point at tqdm, so I'm reporting this to you :-) I'm marking this
bug as important since it breaks another unrelated package.
The segfault is quite reproducible in a cowbuilder chroot during autopkgtest
runs, but for some reason it doesn't seem to happen during the test phase of
the build.
Thanks,
Roland.
[Please disregard the "system information" below, I run reportbug on a
different workstation; the bug really is about unstable]
-- System Information:
Debian Release: 12.10
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 6.1.0-31-amd64 (SMP w/16 CPU threads; PREEMPT)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages python3-tqdm depends on:
ii python3 3.11.2-1+b1
python3-tqdm recommends no packages.
python3-tqdm suggests no packages.
-- no debconf information
