Your message dated Fri, 31 Jan 2020 20:36:51 +0000 with message-id <[email protected]> and subject line Bug#950289: fixed in python-internetarchive 1.9.0-3 has caused the Debian Bug report #950289, regarding python-internetarchive is leaking filedescriptors to be marked as done.
This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact [email protected] immediately.) -- 950289: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=950289 Debian Bug Tracking System Contact [email protected] with problems
--- Begin Message ---Package: internetarchive Version: 1.8.1-1 Severity: important I've tried to upload to archive.org and noticed ia crashes on large folders. $ ulimit -n 1024 $ ia upload asdf ./folder-with-more-than-1024-files/ [...] OSError: [Errno 24] Too many open files [...] $ The bug is present in src:python-internetarchive, I found a patch that resolves the issue from 2018 that was never applied. You can find a patch that cleanly applies to the current debian/sid below. The original author is github.com/Arkiver2: --- >8 --- diff --git a/internetarchive/utils.py b/internetarchive/utils.py index db8412a..7385857 100644 --- a/internetarchive/utils.py +++ b/internetarchive/utils.py @@ -235,14 +235,16 @@ def recursive_file_count(files, item=None, checksum=False): is_dir = False if is_dir: for x, _ in iter_directory(f): - lmd5 = get_md5(open(x, 'rb')) + with open(x, 'rb') as f_: + lmd5 = get_md5(f_) if lmd5 in md5s: continue else: total_files += 1 else: try: - lmd5 = get_md5(open(f, 'rb')) + with open(x, 'rb') as f_: + lmd5 = get_md5(f_) except TypeError: # Support file-like objects. lmd5 = get_md5(f) --- 8< ---
--- End Message ---
--- Begin Message ---Source: python-internetarchive Source-Version: 1.9.0-3 We believe that the bug you reported is fixed in the latest version of python-internetarchive, which is due to be installed in the Debian FTP archive. A summary of the changes between this version and the previous one is attached. Thank you for reporting the bug, which will now be closed. If you have further comments please address them to [email protected], and the maintainer will reopen the bug report if appropriate. Debian distribution maintenance software pp. Antoine Beaupré <[email protected]> (supplier of updated python-internetarchive package) (This message was generated automatically at their request; if you believe that there is a problem with it please contact the archive administrators by mailing [email protected]) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Format: 1.8 Date: Fri, 31 Jan 2020 14:56:30 -0500 Source: python-internetarchive Architecture: source Version: 1.9.0-3 Distribution: unstable Urgency: medium Maintainer: Antoine Beaupré <[email protected]> Changed-By: Antoine Beaupré <[email protected]> Closes: 950289 Changes: python-internetarchive (1.9.0-3) unstable; urgency=medium . * hotfix: close file after getting md5 (Closes: #950289) Checksums-Sha1: 195f0ac54ab4190127a00da0585868869e70927d 1995 python-internetarchive_1.9.0-3.dsc 68823c002ea2c047dc41df9749ab58d245f55fbf 14104 python-internetarchive_1.9.0-3.debian.tar.xz e8fc578335a87e28a86b440a11fe07baffa7cb53 7282 python-internetarchive_1.9.0-3_amd64.buildinfo Checksums-Sha256: ef3369e68056faa991ddffc2758770cb8ec6efdb97dcafbd9f0c1a240bf0e278 1995 python-internetarchive_1.9.0-3.dsc 5fdd342fcd3bce7bb622501a0302841f87c693530fb5f1739d3a394dd4825633 14104 python-internetarchive_1.9.0-3.debian.tar.xz 8bd8d4bb943c70180a53daf5ff5553694dc009e60ba07cdc071ef8dfe10ef750 7282 python-internetarchive_1.9.0-3_amd64.buildinfo Files: 897675fe7122e06d8fec4d38cbad1cdd 1995 python optional python-internetarchive_1.9.0-3.dsc d565bd1bce88e3213a880aaa7a2589a3 14104 python optional python-internetarchive_1.9.0-3.debian.tar.xz 716c11b01e8643207e0d3f18dca1312d 7282 python optional python-internetarchive_1.9.0-3_amd64.buildinfo -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEexZCBNCWcjsBljWrPqHd3bJh2XsFAl40hykACgkQPqHd3bJh 2Xsa3Af/botaTPWuKSr/IX1pleQWGVMcAAAQM2l8TXrFa33EGOiR4434JrKEbkir xjQyXGfriQ7zwA1JBa80mTUTU8cPv6Dh+wQn1VVD2/S7vXvfWrVnGBIIYBsfK042 2l3r7il4PKd6tHDivshhJnlSGyLYak0JHM6wy/I7xhgA3kpiqjnrd9KZwstZvvQF 1BnEmaSLnvybMC/cpLiVN3+54XKTzJKJoLsyHpz1Pwct9bt8Bed4At0novoaQo82 YyueXkVAysvon43SBY/HSoygfe11YsZ6hDoKc+fmZ5xm4zdPR31jCNvclwKZxtSf R9rXqn7ajVDzPJq7Cwh/IR9OuwGNJQ== =qPJv -----END PGP SIGNATURE-----
--- End Message ---
