Re: Multi-threaded FTP Question
Thanks so much for your help on this. The server that I'm connecting to is the culprit. They only allow five connections at a time. I assumed that it was a code issue. I think that we're conditioned to expect that the problem is on the software side of things. -Derek [EMAIL PROTECTED] wrote: > [EMAIL PROTECTED] wrote: > > I'm trying to use ftp in python in a multi-threaded way on a windows > > box - python version 2.4.3. Problem is that it appears that it's only > > possible to have five instances/threads at one point in time. Errors > > look like: > > > >File "C:\Python24\lib\ftplib.py", line 107, in __init__ > > self.connect(host) > > File "C:\Python24\lib\ftplib.py", line 132, in connect > > self.welcome = self.getresp() > > File "C:\Python24\lib\ftplib.py", line 208, in getresp > > resp = self.getmultiline() > > File "C:\Python24\lib\ftplib.py", line 194, in getmultiline > > line = self.getline() > > File "C:\Python24\lib\ftplib.py", line 184, in getline > > if not line: raise EOFError > > EOFError > > > > Is it possible to have more than five simultaneous ftp connections? > > > > Thanks. > > > > Derek > > It might be XP SP2's worm protection as well: > > http://www.speedguide.net/read_articles.php?id=1497 -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-threaded FTP Question
[EMAIL PROTECTED] wrote: > I'm trying to use ftp in python in a multi-threaded way on a windows > box - python version 2.4.3. Problem is that it appears that it's only > possible to have five instances/threads at one point in time. Errors > look like: > >File "C:\Python24\lib\ftplib.py", line 107, in __init__ > self.connect(host) > File "C:\Python24\lib\ftplib.py", line 132, in connect > self.welcome = self.getresp() > File "C:\Python24\lib\ftplib.py", line 208, in getresp > resp = self.getmultiline() > File "C:\Python24\lib\ftplib.py", line 194, in getmultiline > line = self.getline() > File "C:\Python24\lib\ftplib.py", line 184, in getline > if not line: raise EOFError > EOFError > > Is it possible to have more than five simultaneous ftp connections? > > Thanks. > > Derek It might be XP SP2's worm protection as well: http://www.speedguide.net/read_articles.php?id=1497 -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-threaded FTP Question
Dennis Lee Bieber wrote: > On 11 Jul 2006 06:45:42 -0700, [EMAIL PROTECTED] declaimed the > following in comp.lang.python: > > Could it be that the SERVER is limiting things to 5 > concurrent/parallel connections from any single IP? > > I know I've encountered sites that only allowed two FTP downloads at > a time... This is what I was starting to think as well. The only thing that looked funky with the OP's code was that it looked like he was writing everything to a filename of "" (unless he's intentionally modified his code to not show where he's setting that). - Jeremy M. Jones -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-threaded FTP Question
There are n instances of ftplib.FTP.
Funny thing is that I tried to run the code twice, concurrently, in two
separate IDLE instances. Each instance had four threads/ftp calls.
Again, only five ftp connections were allowed on my computer. The code
errored out on the sixth attempt.
I wonder if there' s some underlying software being called that has a
limit on the number of ftp connections allowed at any point in time.
I'm tempted to try this on cygwin to see if the behavior is different.
Derek
import ftplib, zipfile, datetime, os
from threading import Thread, Lock
class ftpDownload(Thread):
def __init__(self, year, quarter):
Thread.__init__(self)
self.quarter = str(quarter)
self.year = str(year)
self.filename = ""
self.ftp = ftplib.FTP('xxx')
self.ftp.login()
self.lock = Lock()
def run(self):
filehandle = open(self.filename,'wb')
self.ftp.retrbinary('RETR /x/xxx/' + self.year + '/QTR'
+ self.quarter + '/xxx.zip', filehandle.write)
filehandle.close()
self.ftp.close()
ftplist = []
for year in range(1990,2000):
for quarter in range(1,5):
current = ftpDownload(year, quarter)
ftplist.append(current)
current.start()
for job in ftplist:
job.join()
Jeremy Jones wrote:
> [EMAIL PROTECTED] wrote:
> > I'm trying to use ftp in python in a multi-threaded way on a windows
> > box - python version 2.4.3. Problem is that it appears that it's only
> > possible to have five instances/threads at one point in time. Errors
> > look like:
> >
> >File "C:\Python24\lib\ftplib.py", line 107, in __init__
> > self.connect(host)
> > File "C:\Python24\lib\ftplib.py", line 132, in connect
> > self.welcome = self.getresp()
> > File "C:\Python24\lib\ftplib.py", line 208, in getresp
> > resp = self.getmultiline()
> > File "C:\Python24\lib\ftplib.py", line 194, in getmultiline
> > line = self.getline()
> > File "C:\Python24\lib\ftplib.py", line 184, in getline
> > if not line: raise EOFError
> > EOFError
> >
> > Is it possible to have more than five simultaneous ftp connections?
> >
> > Thanks.
> >
> > Derek
>
> I replied to this about 4 hours ago from my gmail email account (not my
> google groups account associated with the same email addres), but
> haven't seen it show up, so I apologize if this is a dupe.
>
> Would you mind posting your code? Are you trying to pass the same FTP
> connection object to all 5 threads?
>
> - Jeremy M. Jones
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multi-threaded FTP Question
[EMAIL PROTECTED] wrote: > I'm trying to use ftp in python in a multi-threaded way on a windows > box - python version 2.4.3. Problem is that it appears that it's only > possible to have five instances/threads at one point in time. Errors > look like: > >File "C:\Python24\lib\ftplib.py", line 107, in __init__ > self.connect(host) > File "C:\Python24\lib\ftplib.py", line 132, in connect > self.welcome = self.getresp() > File "C:\Python24\lib\ftplib.py", line 208, in getresp > resp = self.getmultiline() > File "C:\Python24\lib\ftplib.py", line 194, in getmultiline > line = self.getline() > File "C:\Python24\lib\ftplib.py", line 184, in getline > if not line: raise EOFError > EOFError > > Is it possible to have more than five simultaneous ftp connections? > > Thanks. > > Derek I replied to this about 4 hours ago from my gmail email account (not my google groups account associated with the same email addres), but haven't seen it show up, so I apologize if this is a dupe. Would you mind posting your code? Are you trying to pass the same FTP connection object to all 5 threads? - Jeremy M. Jones -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-threaded FTP Question
On 11 Jul 2006 06:45:42 -0700, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: I'm trying to use ftp in python in a multi-threaded way on a windowsbox - python version 2.4.3. Problem is that it appears that it's onlypossible to have five instances/threads at one point in time. Errorslook like: File "C:\Python24\lib\ftplib.py", line 107, in __init__self.connect (host) File "C:\Python24\lib\ftplib.py", line 132, in connectself.welcome = self.getresp() File "C:\Python24\lib\ftplib.py", line 208, in getrespresp = self.getmultiline() File "C:\Python24\lib\ftplib.py", line 194, in getmultilineline = self.getline() File "C:\Python24\lib\ftplib.py", line 184, in getlineif not line: raise EOFErrorEOFError Is it possible to have more than five simultaneous ftp connections?Would you mind posting your code? Are you trying to pass the same FTP connection object to all 5 threads? -- Jeremy M. Joneshttp://jeremymjones.com -- http://mail.python.org/mailman/listinfo/python-list
Multi-threaded FTP Question
I'm trying to use ftp in python in a multi-threaded way on a windows box - python version 2.4.3. Problem is that it appears that it's only possible to have five instances/threads at one point in time. Errors look like: File "C:\Python24\lib\ftplib.py", line 107, in __init__ self.connect(host) File "C:\Python24\lib\ftplib.py", line 132, in connect self.welcome = self.getresp() File "C:\Python24\lib\ftplib.py", line 208, in getresp resp = self.getmultiline() File "C:\Python24\lib\ftplib.py", line 194, in getmultiline line = self.getline() File "C:\Python24\lib\ftplib.py", line 184, in getline if not line: raise EOFError EOFError Is it possible to have more than five simultaneous ftp connections? Thanks. Derek -- http://mail.python.org/mailman/listinfo/python-list
