Brad Schoening created CASSANDRA-17397:
------------------------------------------
Summary: cqlshlib unit test test_copyutil.py can fail due to too
many open files
Key: CASSANDRA-17397
URL: https://issues.apache.org/jira/browse/CASSANDRA-17397
Project: Cassandra
Issue Type: Bug
Reporter: Brad Schoening
The cqlshlib test test_copyutil.py fails on MacOS with
*E OSError: [Errno 24] Too many open files*
This happens because the test does not close the file descriptors after use.
The default ulimit on MacOS is small, 256:
% ulimit -n
256
The simple fix is to close the pipe after use. Increasing ulimit nofiles works
also, but shouldn't be necessary.
*test/test_copyutil.py*:80: in _test_get_ranges_murmur3_base
export_task = ExportTask(shell, self.ks, self.table, self.columns,
self.fname, overridden_opts, self.protocol_version, self.config_file)
*copyutil.py*:638: in __init__
CopyTask.__init__(self, shell, ks, table, columns, fname, opts,
protocol_version, config_file, 'to')
*copyutil.py*:275: in __init__
self.inmsg = ReceivingChannels(self.num_processes)
*copyutil.py*:186: in __init__
self.pipes = [OneWayPipe() for _ in range(num_channels)]
*copyutil.py*:186: in <listcomp>
self.pipes = [OneWayPipe() for _ in range(num_channels)]
*copyutil.py*:103: in __init__
self.reader, self.writer = mp.Pipe(duplex=False)
*/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py*:63:
in Pipe
return Pipe(duplex)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
duplex = False
def Pipe(duplex=True):
'''
Returns pair of connection objects at either end of a pipe
'''
if duplex:
s1, s2 = socket.socketpair()
s1.setblocking(True)
s2.setblocking(True)
c1 = Connection(s1.detach())
c2 = Connection(s2.detach())
else:
> fd1, fd2 = os.pipe()
*E OSError: [Errno 24] Too many open files*
*/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py*:532:
OSError
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
