Mag Gam wrote:
Hello,
Currently, I am using a bash script to ssh into 400 servers and get an
output, ie check if a file exists on a local filesystem. I am doing
this linearly, however I am interesting in doing this with threads and
more important using Python standard threading library.
My pseudo code would be something like this:
Have a list of the servers in a dictionary
Create the number of threads which correspond to servers. ie 400 threads
Each thread will spawn ssh and execute the file test
Once all threads are completed, I send a signal saying "all is one",
and it will create some sort of output.
Is there a better way of doing this? My goal is to learn python and
threading, I know this is a silly example but its worth a try :-)
I wouldn't use that many threads. It's better to use a relatively small
number of them (not more than 10, perhaps?) and let each thread do
multiple jobs, one at a time.
Here some example code:
# For Python 2.5 and later
from threading import Thread
from Queue import Queue
class ThreadTask(object):
def __init__(self, task_queue):
self.task_queue = task_queue
def __call__(self):
while True:
task = self.task_queue.get()
if task is None:
# A None indicates that there are no more tasks.
break
self.process_task(task)
# Put back the None so that the next thread will also see it.
self.task_queue.put(None)
def process_task(self, task):
# Put your code here
...
MAX_RUNNING = 10
# Put the tasks into a queue, ending with a None.
task_queue = Queue()
for t in task_list:
task_queue.put(t)
task_queue.put(None)
# Start the threads to do the work.
thread_list = []
for i in range(MAX_RUNNING):
t = Thread(target=ThreadTask(task_queue))
t.start()
thread_list.append(t)
# The threads will finish when they see the None in the task queue.
for t in thread_list:
t.join()
# When you get here all the task will have finished.
--
http://mail.python.org/mailman/listinfo/python-list
