Your message dated Sun, 02 Feb 2020 13:47:19 +0000 with message-id <[email protected]> and subject line Bug#950289: fixed in python-internetarchive 1.8.1-1+deb10u1 has caused the Debian Bug report #950289, regarding python-internetarchive is leaking filedescriptors to be marked as done.
This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact [email protected] immediately.) -- 950289: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=950289 Debian Bug Tracking System Contact [email protected] with problems
--- Begin Message ---Package: internetarchive Version: 1.8.1-1 Severity: important I've tried to upload to archive.org and noticed ia crashes on large folders. $ ulimit -n 1024 $ ia upload asdf ./folder-with-more-than-1024-files/ [...] OSError: [Errno 24] Too many open files [...] $ The bug is present in src:python-internetarchive, I found a patch that resolves the issue from 2018 that was never applied. You can find a patch that cleanly applies to the current debian/sid below. The original author is github.com/Arkiver2: --- >8 --- diff --git a/internetarchive/utils.py b/internetarchive/utils.py index db8412a..7385857 100644 --- a/internetarchive/utils.py +++ b/internetarchive/utils.py @@ -235,14 +235,16 @@ def recursive_file_count(files, item=None, checksum=False): is_dir = False if is_dir: for x, _ in iter_directory(f): - lmd5 = get_md5(open(x, 'rb')) + with open(x, 'rb') as f_: + lmd5 = get_md5(f_) if lmd5 in md5s: continue else: total_files += 1 else: try: - lmd5 = get_md5(open(f, 'rb')) + with open(x, 'rb') as f_: + lmd5 = get_md5(f_) except TypeError: # Support file-like objects. lmd5 = get_md5(f) --- 8< ---
--- End Message ---
--- Begin Message ---Source: python-internetarchive Source-Version: 1.8.1-1+deb10u1 We believe that the bug you reported is fixed in the latest version of python-internetarchive, which is due to be installed in the Debian FTP archive. A summary of the changes between this version and the previous one is attached. Thank you for reporting the bug, which will now be closed. If you have further comments please address them to [email protected], and the maintainer will reopen the bug report if appropriate. Debian distribution maintenance software pp. Antoine Beaupré <[email protected]> (supplier of updated python-internetarchive package) (This message was generated automatically at their request; if you believe that there is a problem with it please contact the archive administrators by mailing [email protected]) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Format: 1.8 Date: Fri, 31 Jan 2020 15:00:57 -0500 Source: python-internetarchive Architecture: source Version: 1.8.1-1+deb10u1 Distribution: buster Urgency: medium Maintainer: Antoine Beaupré <[email protected]> Changed-By: Antoine Beaupré <[email protected]> Closes: 950289 Changes: python-internetarchive (1.8.1-1+deb10u1) buster; urgency=medium . * hotfix: close file after getting md5 (Closes: #950289) Checksums-Sha1: 8f98ac9d19c46d133aae34fa99feb3797411b3fd 2022 python-internetarchive_1.8.1-1+deb10u1.dsc c9c850e20422f483f32b6e488a782ad5af666fbd 13952 python-internetarchive_1.8.1-1+deb10u1.debian.tar.xz 0a5bcaa3fe8c3dbe1928795c134aa5b7d572fef2 6807 python-internetarchive_1.8.1-1+deb10u1_amd64.buildinfo Checksums-Sha256: 98eda080120735bbf6ae2c2abc1042646bbb9cb26c2cf97234141d5d563f09bf 2022 python-internetarchive_1.8.1-1+deb10u1.dsc 1aa97c50dbf8b6355c7a7a50fd280da0e6effe18115dae975ccd9c824abd75f7 13952 python-internetarchive_1.8.1-1+deb10u1.debian.tar.xz 1e1afc5dc0e370d9b05604b35b7a148d36602906d0a474ed996ed83621752101 6807 python-internetarchive_1.8.1-1+deb10u1_amd64.buildinfo Files: b25f871408f570e51801ced9efce0bce 2022 python optional python-internetarchive_1.8.1-1+deb10u1.dsc 5db751a1250a597993b69d75c90c7308 13952 python optional python-internetarchive_1.8.1-1+deb10u1.debian.tar.xz 9861ef9b33da04aebb5e4d39096b5a80 6807 python optional python-internetarchive_1.8.1-1+deb10u1_amd64.buildinfo -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEexZCBNCWcjsBljWrPqHd3bJh2XsFAl40k+QACgkQPqHd3bJh 2XtRBggAs8bACP2UKZqu8R1jTE50kpbi9wAQEYlKTalXfFr0rjRtLHW40rxoGdJ6 cdlLKA/MjBNnuV+cr8iAUPtV74wY6cV3f4hktIklCYxwbSSwiIqM9CsvoX9SSRG8 mFql3aE7ikC85kdYt1lEZttXXxz1A5pxfkYcxdEENFxhZWJDf1plZFp0BwdA2mWM TY3H34HkqGZUw0VFurnUK1w0DGQoE+nwH50az6ZOEMMR7AdWmtONgczoJxkWcrwf cJ2ZES1pa+B1tLSs6ua1xeDXWkP1ewvkyP5mYXn7KIMRvYNEZ4BzANNHA8+zrrHi Fgb8WINCwu+3T+B6UiZXJapnvYtpug== =f80X -----END PGP SIGNATURE-----
--- End Message ---
