I have been bitten by this same problem but managed to debug it to the
fact that autopkgtest-virt-qemu is not closing the sockets.
On python3.6 and python3.7.1 (likely, haven't tested this one out yet),
the socket is closed when its references are gone (this is done on
socket_finalized on Modules/socketmodule.c).
As you can see on that same code, a ResourceWarning is raised, but one
can only see that on debug builds. Both 3.6 and 3.7 raise that, but only
by using some strace I could notice that 3.7.2rc1 is not closing the
socket on a function exit. And that's only when the 'with timeout' is
used, because of calling signal.signal. I will attach my minimal case
where this happens as well.
The attached autopkgtest-virt-qemu patch also fixes the problem for me.
I don't close the socket on all failure paths. Maybe we should implement
a "With Context Manager" for that as well. Though thinking a little more
about that, the code would be pretty ugly, and we should just rely on a
working python doing the correct resource management.
Regards.
Cascardo.
import signal
import socket
import time
class Timeout(RuntimeError):
pass
def alarm_handler(*a):
raise Timeout()
def get_unix_socket():
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
signal.signal(signal.SIGALRM, alarm_handler)
return s
def test_tty():
sock = get_unix_socket()
signal.signal(signal.SIGALRM, alarm_handler)
test_tty()
time.sleep(1)
>From dcd5895b808ca377255f9cc980e9ba54bbed5f75 Mon Sep 17 00:00:00 2001
From: Thadeu Lima de Souza Cascardo <casca...@debian.org>
Date: Fri, 21 Dec 2018 17:14:11 -0200
Subject: [PATCH] qemu: close sockets after being done with them
With Python 3.7.2-rc1, the socket won't be closed after being finalized.
That causes a problem when trying to connect to the same qemu unix socket
again. It won't be able to receive anything, and that breaks
autopkgtest-virt-qemu.
Closing the sockets before exiting the functions on the success cases will
make it work around that problem.
Signed-off-by: Thadeu Lima de Souza Cascardo <casca...@debian.org>
---
virt/autopkgtest-virt-qemu | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/virt/autopkgtest-virt-qemu b/virt/autopkgtest-virt-qemu
index 573bce7..8cdcc74 100755
--- a/virt/autopkgtest-virt-qemu
+++ b/virt/autopkgtest-virt-qemu
@@ -125,6 +125,7 @@ def wait_boot():
# don't help to determine if the system is *really* booted; running
# commands too early causes the system time to be all wrong
time.sleep(3)
+ term.close()
def check_ttyS1_shell():
@@ -134,8 +135,10 @@ def check_ttyS1_shell():
term.sendall(b'echo -n o; echo k\n')
try:
VirtSubproc.expect(term, b'ok', 1)
+ term.close()
return True
except VirtSubproc.Timeout:
+ term.close()
return False
@@ -190,6 +193,7 @@ def login_tty_and_setup_shell():
VirtSubproc.expect(term, None, 10, 'accepted ttyS1 shell command')
term.sendall(b'exit\n')
VirtSubproc.expect(term, b'\nlogout', 10)
+ term.close()
def setup_baseimage():
@@ -215,6 +219,8 @@ def setup_baseimage():
term.sendall(b'udevadm settle --exit-if-exists=/dev/baseimage\n')
VirtSubproc.expect(term, b'#', 10)
+ term.close()
+ monitor.close()
def setup_shared(shared_dir):
@@ -275,6 +281,7 @@ while not os.path.exists(fexit):
EOF
''')
VirtSubproc.expect(term, b'# ', 5)
+ term.close()
def setup_config(shared_dir):
@@ -311,6 +318,8 @@ def setup_config(shared_dir):
if b'\n# ' not in out:
VirtSubproc.expect(term, b'# ', 5)
+ term.close()
+
def make_auxverb(shared_dir):
'''Create auxverb script'''
@@ -513,6 +522,7 @@ def determine_normal_user(shared_dir):
adtlog.debug('determine_normal_user: got user "%s"' % normal_user)
else:
adtlog.debug('determine_normal_user: no uid in [1000,59999] available')
+ term.close()
def hook_open():
@@ -642,6 +652,7 @@ def hook_prepare_reboot():
monitor = VirtSubproc.get_unix_socket(os.path.join(workdir, 'monitor'))
monitor.sendall(b'device_del virtio-baseimage\n')
VirtSubproc.expect(monitor, b'(qemu)', 10)
+ monitor.close()
def hook_wait_reboot():
--
2.20.1
