[ https://issues.apache.org/jira/browse/CASSANDRA-17397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brad Schoening updated CASSANDRA-17397: --------------------------------------- Description: The cqlshlib unit tests have some regular failures and warnings. # test_copyutil.py fails with too many open files when ulimit is 256 # nose creates deprecated warnings # cassconnect.py:39: DeprecationWarning: Legacy execution parameters # sslhandling.py:42: DeprecationWarning: The SafeConfigParser class has been renamed to ConfigParser The cqlshlib test test_copyutil.py fails on MacOS with *E OSError: [Errno 24] Too many open files* This happens because the test does not close the file descriptors after use. The default ulimit on MacOS is small, 256: % ulimit -n 256 The simple fix is to close the pipe after use. Increasing ulimit nofiles works also, but shouldn't be necessary. {*}test/test_copyutil.py{*}:80: in _test_get_ranges_murmur3_base export_task = ExportTask(shell, self.ks, self.table, self.columns, self.fname, overridden_opts, self.protocol_version, self.config_file) {*}copyutil.py{*}:638: in _{_}init{_}_ CopyTask._{_}init{_}_(self, shell, ks, table, columns, fname, opts, protocol_version, config_file, 'to') {*}copyutil.py{*}:275: in _{_}init{_}_ self.inmsg = ReceivingChannels(self.num_processes) {*}copyutil.py{*}:186: in _{_}init{_}_ self.pipes = [OneWayPipe() for _ in range(num_channels)] {*}copyutil.py{*}:186: in <listcomp> self.pipes = [OneWayPipe() for _ in range(num_channels)] {*}copyutil.py{*}:103: in _{_}init{_}_ self.reader, self.writer = mp.Pipe(duplex=False) {*}/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py{*}:63: in Pipe return Pipe(duplex) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ duplex = False def Pipe(duplex=True): ''' Returns pair of connection objects at either end of a pipe ''' if duplex: s1, s2 = socket.socketpair() s1.setblocking(True) s2.setblocking(True) c1 = Connection(s1.detach()) c2 = Connection(s2.detach()) else: > fd1, fd2 = os.pipe() *E OSError: [Errno 24] Too many open files* {*}/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py{*}:532: OSError was: The cqlshlib test test_copyutil.py fails on MacOS with *E OSError: [Errno 24] Too many open files* This happens because the test does not close the file descriptors after use. The default ulimit on MacOS is small, 256: % ulimit -n 256 The simple fix is to close the pipe after use. Increasing ulimit nofiles works also, but shouldn't be necessary. *test/test_copyutil.py*:80: in _test_get_ranges_murmur3_base export_task = ExportTask(shell, self.ks, self.table, self.columns, self.fname, overridden_opts, self.protocol_version, self.config_file) *copyutil.py*:638: in __init__ CopyTask.__init__(self, shell, ks, table, columns, fname, opts, protocol_version, config_file, 'to') *copyutil.py*:275: in __init__ self.inmsg = ReceivingChannels(self.num_processes) *copyutil.py*:186: in __init__ self.pipes = [OneWayPipe() for _ in range(num_channels)] *copyutil.py*:186: in <listcomp> self.pipes = [OneWayPipe() for _ in range(num_channels)] *copyutil.py*:103: in __init__ self.reader, self.writer = mp.Pipe(duplex=False) */usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py*:63: in Pipe return Pipe(duplex) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ duplex = False def Pipe(duplex=True): ''' Returns pair of connection objects at either end of a pipe ''' if duplex: s1, s2 = socket.socketpair() s1.setblocking(True) s2.setblocking(True) c1 = Connection(s1.detach()) c2 = Connection(s2.detach()) else: > fd1, fd2 = os.pipe() *E OSError: [Errno 24] Too many open files* */usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py*:532: OSError > cleanup cqlshlib unit test failures and warnings > ------------------------------------------------ > > Key: CASSANDRA-17397 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17397 > Project: Cassandra > Issue Type: Bug > Reporter: Brad Schoening > Priority: Normal > > The cqlshlib unit tests have some regular failures and warnings. > # test_copyutil.py fails with too many open files when ulimit is 256 > # nose creates deprecated warnings > # cassconnect.py:39: DeprecationWarning: Legacy execution parameters > # sslhandling.py:42: DeprecationWarning: The SafeConfigParser class has been > renamed to ConfigParser > > The cqlshlib test test_copyutil.py fails on MacOS with > *E OSError: [Errno 24] Too many open files* > This happens because the test does not close the file descriptors after use. > The default ulimit on MacOS is small, 256: > % ulimit -n > 256 > The simple fix is to close the pipe after use. Increasing ulimit nofiles > works also, but shouldn't be necessary. > > > {*}test/test_copyutil.py{*}:80: in _test_get_ranges_murmur3_base > export_task = ExportTask(shell, self.ks, self.table, self.columns, > self.fname, overridden_opts, self.protocol_version, self.config_file) > {*}copyutil.py{*}:638: in _{_}init{_}_ > CopyTask._{_}init{_}_(self, shell, ks, table, columns, fname, opts, > protocol_version, config_file, 'to') > {*}copyutil.py{*}:275: in _{_}init{_}_ > self.inmsg = ReceivingChannels(self.num_processes) > {*}copyutil.py{*}:186: in _{_}init{_}_ > self.pipes = [OneWayPipe() for _ in range(num_channels)] > {*}copyutil.py{*}:186: in <listcomp> > self.pipes = [OneWayPipe() for _ in range(num_channels)] > {*}copyutil.py{*}:103: in _{_}init{_}_ > self.reader, self.writer = mp.Pipe(duplex=False) > {*}/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py{*}:63: > in Pipe > return Pipe(duplex) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > > duplex = False > > def Pipe(duplex=True): > ''' > Returns pair of connection objects at either end of a pipe > ''' > if duplex: > s1, s2 = socket.socketpair() > s1.setblocking(True) > s2.setblocking(True) > c1 = Connection(s1.detach()) > c2 = Connection(s2.detach()) > else: > > fd1, fd2 = os.pipe() > *E OSError: [Errno 24] Too many open files* > > {*}/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py{*}:532: > OSError -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org