Hello,
On Sep 18, 2009, at 5:16 AM, Ansuman Dash wrote:
I have written it like that. It is like press 1 and it ll download
file1 and press 2 it ll download file2 etc
But my question was I am using "time.sleep()" to make my script to
wait for the file download and then validate it in log file, so is
there any other way I can synchronize my code with the download.
I am asking this because some files are very huge (120MB) and
download also depends on N/W bandwidth so sometimes it ll be done in
30mins and some times it takes 60 mins. So I can't rely on
"time.sleep()"
I had a similar problem.
I used pyinotify on a linux box. inotify is a kernel hook
that you can use to trigger actions on events... like when
a file write is completed.
Note: I'm a total hack at this. I'm sure there's a more
elegant way to do what I'm dong, but I needed a fix fast,
and this worked for me. I'd appreciate any tips anyone has
to offer to make this cleaner and/or more pythonic. :-)
I'll be glad to try to answer any questions about this hackery.
Best,
Kurt
Here's a slightly cut-down version of my code:
http://pastebin.com/f239b0413
inotify_published_file_handler.py
#
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# encoding: utf-8
"""
inotify_published_file_handler.py
rev. 20090706-01
Created by Kurt Bendl.
Usage
-
sudo su - www-data -c 'cd /web/log/; \
nohup /web/scripts/inotify_published_file_handler.py > \
/web/log/inotify-errors.txt 2>&1 &'
Purpose
-
Monitor the $WEBROOT/aapracing/import/publish directory.
Once a file is closed, act on it:
1. Copy the PDF and XML files from source_dir to web_filebin_dir
2. If the file is an.XML, copy it to the xml_ftp_dir
3. Make the dummy file for PHP publishing process
4. Remove the source file
Requirements
* Linux kernel 2.6.13 +
* pyinotify 2.8.6 +
Installation
--
To install pyinotify on ubuntu/debian:
`sudo easy_install pyinotify`
Docs
-
Docs on pyinotify can be found here: http://trac.dbzteam.org/pyinotify/wiki
"""
import os
import re
import shutil
import pyinotify
import datetime
### production config info
source_dir = "/web/site/aapracing/publish/data/publish/"
web_filebin_dir = "/web/site/filebin/downloads/"
reference_path = '/web/site/aapracing/publish/data/published/'
xml_ftp_dir = "/home/ftp/private/xml/"
filez = '(PDF|pdf|XML|xml|txt)'
logfile_path = "/web/log/inotify.log"
event_mask = pyinotify.IN_CLOSE_WRITE
wm = pyinotify.WatchManager()
def getNow():
"""Just return the current time for timestamping logs"""
return datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
def makeReferenceFile(tfile):
open(tfile, 'w').close()
class SourceMonitor(pyinotify.ProcessEvent):
""" Watches the source dir for IN_CLOSE_WRITE event"""
def process_IN_CLOSE_WRITE(self, event):
"""when an IN_CLOSE_WRITE happens, do something"""
if re.search('(.*\.'+filez+'$)', event.pathname):
# We have a match, put a copy into the filebin dir
shutil.copy2(event.pathname, web_filebin_dir)
logfile = open(logfile_path, "a")
logfile.write("%s: %s moved to %s. \n" %
(getNow(), event.pathname, web_filebin_dir))
if re.search('(.*\.(XML|xml)$)', event.pathname):
# If it's and XML, put a copy in the FTP dir
shutil.copy2(event.pathname, xml_ftp_dir)
logfile.write("%s: %s moved to %s. \n" %
(getNow(), event.pathname, xml_ftp_dir))
# Make the dummy file marker to enable PHP
# to know the file is really published
fhandle = os.path.basename(event.pathname)
open(reference_path + fhandle, 'w').close()
# Now, whack the source file since we're done with it.
os.remove(event.pathname)
p = SourceMonitor()
notifier = pyinotify.Notifier(wm, p)
wdd = wm.add_watch(source_dir, event_mask)
print "This should have been started with:\n\n"
print " sudo su - www-data -c 'cd /web/log/; nohup /web/scripts/
inotify_published_file_handler.py > /web/log/inotify-errors.txt 2>&1
&' \n\n"
notifier.loop()
--
Kurt Bendl, Consultant
Internet Tool & Die
http://tool.net/
502-759-7104
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
