[issue44226] Threads shutting down in Py 2.7 but not in Py 3.69 while making SSH connection using Paramiko module
Muralidhar BN added the comment: Dear Eric Smith, Do we have any analysis for this issue for information shared ? Appreciate your quick reply. PS: Making a simple socket connection instead of paramiko or pyssh ssh connection gives similar error Thank you -- ___ Python tracker <https://bugs.python.org/issue44226> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44226] Threads shutting down in Py 2.7 but not in Py 3.69 while making SSH connection using Paramiko module
Muralidhar BN added the comment:
We have checked this issue by adapting another 3rd party SSH packages ( replace
paramiko with pyssh). Using this we have found same o/p results. Which means it
is not 3rd party SSH package ( paramiko or pyssh) issue.
Issue report in Paramiko github will be removed
(https://github.com/paramiko/paramiko/issues/1856)
To our analysis this issue is not observed when "logging" module is disabled.
Request to provide your analysis from python perspective ?
Executing code qa-test-execute.py in Py 2.7 (Ubuntu 14.04.6 LTS)
Command 1 :
sudo python ./qa-test-execute.py
Output 1 :
2021-05-25 21:23:44,889[BaseCommandLine __init__ 140392385488400][DEBUG]:
Attempt to close ssh-session within 10.0 seconds.
Exception AssertionError: 'attempt to release recursive lock not owned by
thread' in > ignored
Command 2 :
2021-05-25 21:23:44,889[BaseCommandLine __init__ 140392385488400][DEBUG]:
Attempt to close ssh-session within 10.0 seconds.
Exception AssertionError: 'attempt to release recursive lock not owned by
thread' in > ignored
Executing code qa-test-execute.py in Py 3.69 (Ubuntu 18.04.5 LTS)
Command 1 :
sudo python3 ./qa-test-execute.py
Output 1 :
2021-05-25 22:32:54,945[BaseCommandLine __init__ 140490991571128][DEBUG]:
Attempt to close ssh-session within 10.0 seconds.
2021-05-25 22:32:54,945[BaseCommandLine __init__ 140490991571128][DEBUG]:
Closing internal ssh-client.
Fatal Python error: could not acquire lock for <_io.BufferedWriter
name=''> at interpreter shutdown, possibly due to daemon threads
Thread 0x7fc69880e700 (most recent call first):
File "/usr/lib/python3.6/logging/__init__.py", line 997 in emit
File "/usr/lib/python3.6/logging/__init__.py", line 865 in handle
File "/usr/lib/python3.6/logging/__init__.py", line 1516 in callHandlers
File "/usr/lib/python3.6/logging/__init__.py", line 1454 in handle
File "/usr/lib/python3.6/logging/__init__.py", line 1444 in _log
File "/usr/lib/python3.6/logging/__init__.py", line 1296 in debug
File "./qa-test-execute_sajal.py", line 132 in __nonblocking_close1
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Current thread 0x7fc69dab5740 (most recent call first):
File "./qa-test-execute_sajal.py", line 48 in call_with_timeout1
File "./qa-test-execute_sajal.py", line 117 in close1
File "./qa-test-execute_sajal.py", line 83 in __del__
Aborted (core dumped)
Command 2 :
sudo python3 ./qa-test-execute.py 2>&1 | tee
launcher_gt20907_24052021_execute_py369_1.log
Output 2 :
2021-05-25 22:33:26,393[BaseCommandLine __init__ 140711046725760][DEBUG]:
Attempt to close ssh-session within 10.0 seconds.
2021-05-25 22:33:26,393[BaseCommandLine __init__ 140711046725760][DEBUG]:
Closing internal ssh-client.
^C
Behaviour of same code is different when executed in Py 2.7 & Py 3.69.
qa-test-execute.py
#!/usr/bin/env python3
import sys
import traceback
import logging
import logging.config
import time
import threading
import multiprocessing
import pyssh
def lock_acquire_with_timeout1(lock, timeout_seconds):
"""
Try to acquire lock without specified timeout
@param lock: threading lock to be acquired
@type lock: threading.Lock
@param timeout_seconds: maximal time to make lock acquire attempts
@type timeout_seconds: float
"""
begin_time = time.time()
while time.time() - begin_time < timeout_seconds:
if lambda: lock.acquire(False):
return True
else:
time.sleep(1.0)
return None
def call_with_timeout1(method_to_call, timeout_seconds):
"""
Good for potentially "freeze" methods calls. Executes passed method in
separate thread. Waits for control returns within timeout. If timeout
exceed - return control to callee thread. Separate execution thread will
still be active.
@param method_to_call: method te be called
@type method_to_call: function
@param timeout_seconds: maximal time to wait for method call finished
@type timeout_seconds: float
"""
stop_thread = threading.Barrier(2)
thread_name = threading._newname("{}-%d".format(__name__))
call_thread = threading.Thread(target=method_to_call, name=thread_name)
call_thread.daemon = True
call_thread.start()
print ("threading.activeCount() : %s",threading.activeCount())
print ("threading.currentThread() : %s", threading.currentThread())
print ("threading.enumerate() : %s", threading.enumerate() )
call_thread.join(timeout=time
[issue44226] Threads shutting down in Py 2.7 but not in Py 3.69 while making SSH connection using Paramiko module
New submission from Muralidhar BN :
Threads shutting down in Py 2.7 but not in Py 3.69 while making SSH connection
using Paramiko module
Executing code qa-test-execute.py in Py 2.7 (Ubuntu 14.04.6 LTS)
Command 1 :
sudo python ./qa-test-execute.py
Output 1 :
2021-05-24 23:35:59,889[BaseCommandLine __init__ 139916740505872][DEBUG]:
Attempt to close ssh-session within 10.0 seconds.
Exception AssertionError: 'attempt to release recursive lock not owned by
thread' in > ignored
Command 2 :
sudo python ./qa-test-execute.py 2>&1 | tee 24052021_py27_execute_1.log
Output 2 :
2021-05-24 23:50:16,303[BaseCommandLine __init__ 139863250567440][DEBUG]:
Attempt to close ssh-session within 10.0 seconds.
Exception AssertionError: 'attempt to release recursive lock not owned by
thread' in > ignored
Executing code qa-test-execute.py in Py 3.69 (Ubuntu 18.04.5 LTS)
Command 1 :
sudo python ./qa-test-execute.py
Output 1 :
2021-05-24 23:53:49,293[BaseCommandLine __init__ 139973197423840][DEBUG]:
Attempt to close ssh-session within 10.0 seconds.
2021-05-24 23:53:49,293[BaseCommandLine __init__ 139973197423840][DEBUG]:
Closing internal ssh-client.
Fatal Python error: could not acquire lock for <_io.BufferedWriter
name=''> at interpreter shutdown, possibly due to daemon threads
Command 2 :
sudo python3 ./qa-test-execute.py 2>&1 | tee
launcher_gt20907_24052021_execute_py369_1.log
Output 2 : Terminal hangs & CTRL C to return prompt
2021-05-24 23:56:31,646[BaseCommandLine __init__ 140516619855072][DEBUG]:
Attempt to close ssh-session within 10.0 seconds.
2021-05-24 23:56:31,646[BaseCommandLine __init__ 140516619855072][DEBUG]:
Closing internal ssh-client.
^C
Behaviour of same code is different when executed in Py 2.7 & Py 3.69.
Threads not terminating normally conflicting with paramiko / logging module
qa-test-execute.py
#!/usr/bin/env python3
import sys
import traceback
import logging
import logging.config
import time
import threading
import multiprocessing
import paramiko
def lock_acquire_with_timeout1(lock, timeout_seconds):
"""
Try to acquire lock without specified timeout
@param lock: threading lock to be acquired
@type lock: threading.Lock
@param timeout_seconds: maximal time to make lock acquire attempts
@type timeout_seconds: float
"""
begin_time = time.time()
while time.time() - begin_time < timeout_seconds:
if lambda: lock.acquire(False):
return True
else:
time.sleep(1.0)
return None
def call_with_timeout1(method_to_call, timeout_seconds):
"""
Good for potentially "freeze" methods calls. Executes passed method in
separate thread. Waits for control returns within timeout. If timeout
exceed - return control to callee thread. Separate execution thread will
still be active.
@param method_to_call: method te be called
@type method_to_call: function
@param timeout_seconds: maximal time to wait for method call finished
@type timeout_seconds: float
"""
stop_thread = threading.Barrier(2)
thread_name = threading._newname("{}-%d".format(__name__))
call_thread = threading.Thread(target=method_to_call, name=thread_name)
call_thread.daemon = True
call_thread.start()
print ("threading.activeCount() : %s",threading.activeCount())
print ("threading.currentThread() : %s", threading.currentThread())
print ("threading.enumerate() : %s", threading.enumerate() )
call_thread.join(timeout=timeout_seconds)
if call_thread.is_alive():
stop_thread.abort()
return not call_thread.is_alive()
def format_all_threads_stacks1():
"""
@return: formatted stacks for all running threads.
"""
stacktraces = []
for thread_id, stack in
list(dict(list(sys._current_frames().items())).items()):
for thread1 in threading.enumerate():
if thread1.ident == thread_id:
stacktraces.append('Thread %s (daemon=%r) stacktrace: \n%s' %
(thread1.name, thread1.daemon,
''.join(traceback.format_stack(stack
else:
thread = None
return '\n'.join(stacktraces)
class SSHClient_noauth1(paramiko.SSHClient):
def _auth(self, username, *args):
self._transport.auth_none(username)
return
class BaseCommandLine1(object):
def __init__(self, connection_timeout=None):
self._connection_timeout = connection_timeout
self._connection_lock = multiprocessing.RLock()
self._ssh_client = None
self.logger = logging.getLogger('BaseCommandLine __init__
{}'.format(id(self)))
self.reset_connection1()
def __del__(self):
self.close1()
def _wait_f
