[issue22608] test_socket fails with sem_init: Too many open files

Serhiy Storchaka added the comment: Note that it is a FreeBSD-only issue. We need to test on FreeBSD (ulimit -n can be used for this) to check whether it was fixed in issue45212. -- status: pending -> open ___ Python tracker

[issue22608] test_socket fails with sem_init: Too many open files

Irit Katriel added the comment: I think this was fixed in issue45212. -- nosy: +iritkatriel resolution: -> duplicate status: open -> pending superseder: -> Dangling threads in skipped tests in test_socket ___ Python tracker

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

miss-islington added the comment: New changeset 216c040bb1fdf73e78d67ab82a43563d7593f874 by Miss Islington (bot) in branch '3.9': bpo-43592: Raise RLIMIT_NOFILE in test.libregrtest (GH-29127) https://github.com/python/cpython/commit/216c040bb1fdf73e78d67ab82a43563d7593f874 --

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Change by Gregory P. Smith : -- resolution: -> fixed stage: patch review -> commit review status: open -> closed ___ Python tracker ___

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

miss-islington added the comment: New changeset 8f6aa48cb2dc827a2cb76e35e91bf02d099875c5 by Miss Islington (bot) in branch '3.10': bpo-43592: Raise RLIMIT_NOFILE in test.libregrtest (GH-29127) https://github.com/python/cpython/commit/8f6aa48cb2dc827a2cb76e35e91bf02d099875c5 --

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Change by miss-islington : -- message_count: 10.0 -> 11.0 nosy: +miss-islington nosy_count: 6.0 -> 7.0 pull_requests: +27420 pull_request: https://github.com/python/cpython/pull/29144 ___ Python tracker

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Gregory P. Smith added the comment: New changeset 843b890334ca30cf6af27dffe29cecd06b49f7d9 by Gregory P. Smith in branch 'main': bpo-43592: Raise RLIMIT_NOFILE in test.libregrtest (GH-29127) https://github.com/python/cpython/commit/843b890334ca30cf6af27dffe29cecd06b49f7d9 --

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Change by miss-islington : -- pull_requests: +27421 pull_request: https://github.com/python/cpython/pull/29145 ___ Python tracker ___

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Change by Gregory P. Smith : -- keywords: +patch pull_requests: +27414 pull_request: https://github.com/python/cpython/pull/29127 ___ Python tracker ___

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Gregory P. Smith added the comment: macOS often has an unreasonably low default max number of file descriptors. This is easy to raise from user code, it's a soft limit. https://github.com/python/cpython/pull/29127 is out to try that from regrtest itself. -- assignee: ->

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

STINNER Victor added the comment: FYI the test is still failing with the same error on the billenstein-macos builder worker, x86-64 macOS 3.10: https://buildbot.python.org/all/#/builders/681/builds/370 "OSError: [Errno 24] Too many open

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Łukasz Langa added the comment: Before you do it, check the defaults with: $ launchctl limit maxfiles I'm curious. -- ___ Python tracker ___

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Łukasz Langa added the comment: Try setting the following: $ sudo sysctl -w kern.maxfiles=65536 $ sudo sysctl -w kern.maxfilesperproc=65536 $ sudo ulimit -n 65536 -- nosy: +lukasz.langa ___ Python tracker

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

STINNER Victor added the comment: > x86-64 macOS 3.10: https://buildbot.python.org/all/#/builders/681/builds/345 On this machine, RLIMIT_NOFILE soft limit is 256. test.pythoninfo: resource.RLIMIT_NOFILE: (256, 9223372036854775807) platform.platform: macOS-11.5.2-x86_64-i386-64bit

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

ldbot/buildarea/3.10.billenstein-macos/build/Lib/multiprocessing/popen_spawn_posix.py", line 32, in __init__ File "/Users/buildbot/buildarea/3.10.billenstein-macos/build/Lib/multiprocessing/popen_fork.py", line 19, in __init__ self._launch(process_obj) File "/Users/buildb

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Liel Fridman added the comment: I have the same error when running the tests on my machine (Same macOS version). -- nosy: +lielfr ___ Python tracker ___

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Ned Deily added the comment: FWIW, I don't recall ever seeing a "too many open files" failure for test_importlib on any of my macOS systems. -- nosy: +ned.deily ___ Python tracker <https://bugs.python.o

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

STINNER Victor added the comment: test_importlib only fails *randomly* on x86-64 macOS 3.x. For example, the build 970 is a success (but orange): "Tests result: FAILURE then SUCCESS". test_importlib failed with too many open files, and t

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

/build/Lib/multiprocessing/connection.py", line 532, in Pipe fd1, fd2 = os.pipe() OSError: [Errno 24] Too many open files /Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 110 leaked semapho

[issue43572] "Too many open files" on macOS buildbot

Antoine Pitrou added the comment: Thank you! -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue43572] "Too many open files" on macOS buildbot

Change by Antoine Pitrou : -- resolution: -> fixed stage: -> resolved status: open -> closed ___ Python tracker ___ ___

[issue43572] "Too many open files" on macOS buildbot

Matt Billenstein added the comment: Ah, word, was looking at stdout -- 256 is pretty low, raised it to 200k... -- ___ Python tracker ___

[issue43572] "Too many open files" on macOS buildbot

billenstein-macos/build/Lib/multiprocessing/popen_spawn_posix.py", line 42, in _launch prep_data = spawn.get_preparation_data(process_obj._name) File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/multiprocessing/spawn.py", line 176, in get_preparation_data dir=os.getcwd(),

[issue43572] "Too many open files" on macOS buildbot

Matt Billenstein added the comment: In any case - I've raised file/process limits on the buildbot. -- ___ Python tracker ___ ___

[issue43572] "Too many open files" on macOS buildbot

Matt Billenstein added the comment: I don't see that error message - where are you seeing it? -- ___ Python tracker ___ ___

[issue43572] "Too many open files" on macOS buildbot

New submission from Antoine Pitrou : See https://buildbot.python.org/all/#/builders/366/builds/960/steps/5/logs/stdio -- messages: 389184 nosy: mattbillenstein, pablogsal, pitrou, zach.ware priority: normal severity: normal status: open title: "Too many open files" on macO

[issue16981] ImportError hides real error when there too many open files during an import

Brett Cannon added the comment: Now that importlib is import and it would raise OSError I'm closing this as fixed. -- resolution: -> fixed stage: -> resolved status: open -> closed ___ Python tracker

[issue22608] test_socket fails with sem_init: Too many open files

Change by Mark Lawrence : -- nosy: -BreamoreBoy ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

Re: Fwd: socket: Too many open files

On 14Oct2018 20:24, Shakti Kumar wrote: On 13Oct2018 14:10, Shakti Kumar wrote: I’m running a script which basically does a traceroute to the list of hosts provided, and then pulls up some info by logging in to gateways in the path. I am running this script for a list of almost 40k hosts in

Fwd: socket: Too many open files

>On 13Oct2018 14:10, Shakti Kumar wrote: >>I’m running a script which basically does a traceroute to the list of >>hosts >>provided, and then pulls up some info by logging in to gateways in the path. >>I am running this script for a list of almost 40k hosts in our data centers. >>Also, I am using

Re: socket: Too many open files

On 13Oct2018 14:10, Shakti Kumar wrote: I’m running a script which basically does a traceroute to the list of hosts provided, and then pulls up some info by logging in to gateways in the path. I am running this script for a list of almost 40k hosts in our data centers. Also, I am using

Re: socket: Too many open files

Hi Shakti You wrote: > out = commands.getstatusoutput('traceroute ' + ip) The page https://docs.python.org/3/library/subprocess.html#legacy-shell-invocation-functions describes subprocess.getstatusoutput as one of the "legacy functions from the 2.x commands module. These operations

Fwd: socket: Too many open files

>Hello, >I’m running a script which basically does a traceroute to the list of hosts provided, and then pulls up some info by logging in to gateways in the path. >I am running this script for a list of almost 40k hosts in our data centers. >Also, I am using commands module to get the traceroute

socket: Too many open files

Hello, I’m running a script which basically does a traceroute to the list of hosts provided, and then pulls up some info by logging in to gateways in the path. I am running this script for a list of almost 40k hosts in our data centers. Also, I am using commands module to get the traceroute

[issue32290] bolen-dmg-3.6: compilation failed with OSError: [Errno 23] Too many open files in system

Ned Deily added the comment: Whether or not Tiger is supported, we no longer have any Tiger buildbots. We agreed to retire them earlier this year. -- ___ Python tracker

[issue32290] bolen-dmg-3.6: compilation failed with OSError: [Errno 23] Too many open files in system

STINNER Victor added the comment: > The macOS Tiger buildbot where all of these "too many files open" OS failures > occurred has been retired. macOS Tiger is still supported, since we still have x86 Tiger buildbot, it's just the dmg buildbot on Tiger which has been

[issue32290] bolen-dmg-3.6: compilation failed with OSError: [Errno 23] Too many open files in system

Ned Deily added the comment: The macOS Tiger buildbot where all of these "too many files open" OS failures occurred has been retired. -- resolution: -> out of date stage: -> resolved status: open -> closed ___ Python tracker

[issue32290] bolen-dmg-3.6: compilation failed with OSError: [Errno 23] Too many open files in system

STINNER Victor added the comment: Another different failure on "bolen-dmg-3.x". I don't understand the "RuntimeError: command failed: make" error :-( http://buildbot.python.org/all/#/builders/69/builds/69 (...) gcc-4.0 -c -fno-strict-aliasing -Wsign-compare

[issue32290] bolen-dmg-3.6: compilation failed with OSError: [Errno 23] Too many open files in system

, fd2 = os.pipe() OSError: [Errno 23] Too many open files in system == ERROR: test_timeout (test.test_multiprocessing_fork.WithThreadsTestQueue) -- Traceback (mos

[issue32290] bolen-dmg-3.6: compilation failed with OSError: [Errno 23] Too many open files in system

e 230, in getcwd OSError: [Errno 23] Too many open files in system Traceback (most recent call last): File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/build/Doc/venv/lib/python3.6/site-packages/sphinx/cmdline.py", line 306, in main File "/Users/db3l/buildarea.dmg/bolen-dmg-3.6/bu

[issue22610] test_ftplib fails with sem_init: Too many open files

STINNER Victor added the comment: I didn't see this error recently on any buildbot. While the patch seems safe, I prefer to not apply it to prevent any kind of regression. -- nosy: +haypo resolution: -> out of date stage: -> resolved status: open -> closed

[issue22608] test_socket fails with sem_init: Too many open files

Mark Lawrence added the comment: It's a simple patch so can we have a formal review please. -- nosy: +BreamoreBoy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22608 ___

[issue22610] test_ftplib fails with sem_init: Too many open files

Mark Lawrence added the comment: It's a simple patch so can we have a formal review please. -- nosy: +BreamoreBoy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22610 ___

[issue22608] test_socket fails with sem_init: Too many open files

Martin Panter added the comment: Maybe these patches work around the problem in these cases, but it sounds like the threading.Event class needs to grow a close() method or support the context manager protocol, rather than relying on the garbage collector to clean it up. -- nosy:

[issue22608] test_socket fails with sem_init: Too many open files

Serhiy Storchaka added the comment: I suggest to merge issue22608 and issue22610 and also clean up other long living event objects in tests. -- nosy: +serhiy.storchaka stage: - needs patch versions: +Python 3.4, Python 3.5 ___ Python tracker

[issue22608] test_socket fails with sem_init: Too many open files

New submission from Urs Traber: test_socke.py does not clean up its event objects used for synchronization. This may cause multiple such ERRORs (*): testLinebufferedWrite (__main__.LineBufferedFileObjectClassTestCase) ... sem_init: Too many open files ERROR Fixed by setting the allocated

[issue22610] test_ftplib fails with sem_init: Too many open files

New submission from Urs Traber: same issue as http://bugs.python.org/issue22608 e.g. test_sanitize (__main__.TestTLS_FTPClassMixin) ... sem_init: Too many open files ERROR Fixed by setting the allocated Event objects to None when not needed anymore. -- components: Tests files

[issue22608] test_socket fails with sem_init: Too many open files

R. David Murray added the comment: I think this is a consequence of issue 11798. Since Events use filesystem based semaphores on FreeBSD (I think?), it seems reasonable to fix this in test_socket specifically in 2.7. -- nosy: +r.david.murray ___

[issue16981] ImportError hides real error when there too many open files during an import

was not found because fopen() failed with Too many open files. There were not enough open files because OSX has insanely low limits and PostgreSQL was using a large chunk of that. ulimit -n 4096 resolved the errors. I spent a LOT of time trying to figure that out (see the thread at http

[issue16981] ImportError hides real error when there too many open files during an import

R. David Murray added the comment: I wonder if importlib replicates this behavior, it may need fixing as well. -- nosy: +brett.cannon, eric.snow, r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16981

[issue16981] ImportError hides real error when there too many open files during an import

Brett Cannon added the comment: A quick trace through importlib._bootstrap through hg.python.org would suggest that the OSError would propagate when accessing source (OSError is swallowed when you try and write bytecode, but that's legitimate semantics). --

Too many open files

I'm building a pipeline involving a number of shell tools. In each case, I create a temporary file using tempfile.mkstmp() and invoke a command (cmd /tmp/tmpfile) on it using subprocess.Popen. At the end of each section, I call close() on the file handles and use os.remove() to delete them. Even

Re: Too many open files

psaff...@googlemail.com wrote: I'm building a pipeline involving a number of shell tools. In each case, I create a temporary file using tempfile.mkstmp() and invoke a command (cmd /tmp/tmpfile) on it using subprocess.Popen. At the end of each section, I call close() on the file handles and

socket.error 24: too many open files

I'm new to python, but have been writing programs in other languages for about 15 years now. As part of my job I develop applications that interface with Asterisk in various ways. As a way of getting my feet wet I decided to try to rewrite an outbound call script that I have in another language

Re: socket.error 24: too many open files

In article a66896a7-3fac-40c9-9051-6ae9eb2ce...@r15g2000prh.googlegroups.com, TheDavidFactor thedavidfac...@gmail.com wrote: I have double checked that it is closing the socket. I don't know what else to check, any suggestions would be much appreciated. All of the symptoms you report point

Re: socket.error 24: too many open files

TheDavidFactor wrote: [...] It's a deamon that runs on a linux box and every 15 seconds it checks a MySQL table for new records, if there are any it creates a .call file on the Asterisk server using ssh, it also checks the Asterisk server, again via ssh, for any finished calls and if there are

Re: Too many open files

consuming. I'd rather have the files all open for the duration, do all writes and then close them all at the end. The problem I have under windows is that as soon as I get to 500 files I get the Too many open files message. I tried the same thing in Delphi and I can get to 3000 files. How can I

Re: Too many open files

En Mon, 04 Feb 2008 12:50:15 -0200, Christian Heimes [EMAIL PROTECTED] escribi�: Jeff wrote: Why don't you start around 50 threads at a time to do the file writes? Threads are effective for IO. You open the source file, start a queue, and start sending data sets to be written to the

Re: Too many open files

Steven D'Aprano [EMAIL PROTECTED] wrote: On Mon, 04 Feb 2008 13:57:39 +0100, AMD wrote: The problem I have under windows is that as soon as I get to 500 files I get the Too many open files message. I tried the same thing in Delphi and I can get to 3000 files. How can I increase the number

Re: Too many open files

rather have the files all open for the duration, do all writes and then close them all at the end. The problem I have under windows is that as soon as I get to 500 files I get the Too many open files message. I tried the same thing in Delphi and I can get to 3000 files. How can I increase

Re: Too many open files

Why don't you start around 50 threads at a time to do the file writes? Threads are effective for IO. You open the source file, start a queue, and start sending data sets to be written to the queue. Your source file processing can go on while the writes are done in other threads. --

Re: Too many open files

On Mon, 04 Feb 2008 13:57:39 +0100, AMD wrote: The problem I have under windows is that as soon as I get to 500 files I get the Too many open files message. I tried the same thing in Delphi and I can get to 3000 files. How can I increase the number of open files in Python? Windows XP has

Re: Too many open files

Jeff wrote: Why don't you start around 50 threads at a time to do the file writes? Threads are effective for IO. You open the source file, start a queue, and start sending data sets to be written to the queue. Your source file processing can go on while the writes are done in other

Re: Too many open files

rather have the files all open for the duration, do all writes and then close them all at the end. The problem I have under windows is that as soon as I get to 500 files I get the Too many open files message. I tried the same thing in Delphi and I can get to 3000 files. How can I increase

Too many open files

the files all open for the duration, do all writes and then close them all at the end. The problem I have under windows is that as soon as I get to 500 files I get the Too many open files message. I tried the same thing in Delphi and I can get to 3000 files. How can I increase the number of open files