On Sep 15, 1:32 am, MRAB <pyt...@mrabarnett.plus.com> wrote: > Bakes wrote: > > On Sep 13, 11:47 pm, MRAB <pyt...@mrabarnett.plus.com> wrote: > >> Bakes wrote: > >>> On 13 Sep, 22:41, Chris Rebert <c...@rebertia.com> wrote: > >>>> On Sun, Sep 13, 2009 at 2:34 PM, Bakes <ba...@ymail.com> wrote: > >>>>> I am using a simple python script to download my logfiles. This is on > >>>>> a while loop, the logfile grows rapidly, so it is necessary for python > >>>>> to start downloading the new script as soon as it has finished the > >>>>> old. > >>>>> It works fine (for about 20 minutes), then crashes. I have removed a > >>>>> couple of excepts, and have narrowed the error down to a 'error_perm: > >>>>> 550 logfile.log: The data is invalid.' error. > >>>>> Does anyone know what the problem might be regarding this, and what I > >>>>> might do to fix it? > >>>> Including an actual code snippet and the full error traceback would help > >>>> a lot. > >>>> According tohttp://en.wikipedia.org/wiki/List_of_FTP_server_return_codes, > >>>> error code 550 translates to: > >>>> "Requested action not taken. File unavailable (e.g., file not found, > >>>> no access)." > >>>> Does the logfile get rotated or something, thus causing it to briefly > >>>> not exist? > >>>> It might also help if you explain how your logfile system works. > >>>> Cheers, > >>>> Chris > >>>> --http://blog.rebertia.com > >>> It's a cod4 gameserver logfile, being downloaded for a python bot to > >>> parse. > >>> The logfile is downloaded using this try/except while loop. > >>> while True: > >>> try: > >>> if ftp == False: > >>> self.debug('FTP connection not active, attempting to > >>> (re)connect') > >>> ftp = self.ftpconnect() > >>> size=os.path.getsize('games_mp.log') > >>> ftp.retrbinary('RETR ' + os.path.basename(self.ftpconfig > >>> ['path']), handleDownload, rest=size) > >>> if self.console._paused: > >>> self.console.unpause() > >>> except: > >>> print error > >>> self.debug('Lost connection to server, pausing until updated > >>> properly, Sleeping 10 seconds') > >>> self.console.pause() > >>> try: > >>> ftp.close() > >>> self.debug('FTP Connection Closed') > >>> except: > >>> self.debug('FTP does not appear to be open, so not > >>> closed') > >>> ftp = False > >>> time.sleep(10) > >>> I can only assume that occasionally, the logfile is being written to > >>> by the gameserver at the same time that it's downloading. > >>> If this was the case, do you think a try: download except: sleep > >>> 900msec then download loop would work? > >> Bare excepts are almost always a bad idea because they'll catch _all_ > >> exceptions, both those you expect could happen and those you don't. > >> Catch only those you expect. > > >> For example, if the file 'games_mp.log' doesn't exist then > >> os.path.getsize('games_mp.log') will raise an exception, and if you > >> forgot to import the os module then that will raise a NameError > >> exception. > > >> Anyway, I can't see how you leave the loop; I'd expect something like a > >> 'break' statement. > > >> And as a matter of style, I'd prefer None to False to indicate when > >> there's no FTP connection (and "if not ftp" instead of "if ftp == > >> False"). > > > I removed the try/except and saw when it failed. > > > I'll change those things, games_mp.log is guaranteed to be there (file > > made in another script), os is imported correctly. > > > So, what do you think the error could be? > > How does control leave the while loop? > > If you're running on Windows and you're opening the file 'games_mp.log' > with mode 'w' then there'll be the issue of the line-endings. If the log > file on the server uses '\n' for the line endings and you're using > '\r\n' then os.path.getsize('games_mp.log') will return a larger size > then you expect. In that case, could ftp.retrbinary() be complaining > because it's given you the entire file and then you're trying to > download from an offset that's beyond the end? > > For example, suppose the file on the server contains just "foo\n" (4 > bytes): > > 1. You open a file locally in text mode ('w'). > > 2. You download the entire file, "foo\n". > > 3. You write out the data, but because you've opened in text mode it > writes "foo\r\n" to the file. The local file size is now 5 bytes. > > 4. The loop means that you then try to download from offset 5, which is > beyond the end of the file on the server. > > 5. Error?
It opens it in append mode, and if it errors with that message, the ending is different. I have come across that problem before, and it has been fixed (download in binary is the fix, mostly). -- http://mail.python.org/mailman/listinfo/python-list