Re: File Monitoring

[JupiterHost.Net]

PerlDiscuss - Perl Newsgroups and mailing lists wrote:

Hi there,
Hello,

Can anyone recommend a good way to have a perl script that constantly
monitors for the creation of new files on UNIX and launches another
process?
Not sure if someone else recommended this but:

[untested code - for example only]
 #!/usr/bin/perl
 use strict;
 use warnings;
 while(1) {
   opendir DIRH, '/foo/bar' or die $!;
   my @curlst = ;
   closedir(DIRH);
   if(grep /watch for me/, @curlst) {
   # launch other process here...
   }
   sleep(15);
 }
Then start it running like so:
./thisscript.pl &
Thanks!
np, HTH

Lee.M - JupiterHost.Net

Cheers,
Ben

--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: File Monitoring

[Wiggins d Anconia]

> Hi there,
> 
> Can anyone recommend a good way to have a perl script that constantly
> monitors for the creation of new files on UNIX and launches another
> process?
> 
> Thanks!
> 
> Cheers,
> Ben
> 

Lee's method would be the sufficient underkill way. I have an
application that does this type of thing that was written using POE,
which made it trivial, but may make it overkill. For us we needed the
extensibility and scalability that POE offered so it wasn't overkill.

There is a directory watching component in POE or you could grow your
own, sorry I can't provide the one I have (licensing issues). 
POE::Wheel::Run is ideal for managing external processes called from
within the app.

http://poe.perl.org

Has more, be warned there is a learning curve here...

http://danconia.org


-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: File Monitoring

[Dani Pardo]

   Hi, I've done this app. What I do is put into a file the output of 
ls -lR /directory_to_watch.
 Then use diff to compare the results. This gives the advantage that, 
if a line starts with +, then a new file has been added. If it's -,  a 
file has been deleted :)

$ ls -lR > listing1.txt
$ ls -lR > listing2.txt
$ diff listing1.txt listing2.txt > diff.txt

Of course, this works on a Un*x environment.

--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

RE: File Monitoring

[NYIMI Jose (BMB)]

> -Original Message-
> From: Wiggins d Anconia [mailto:[EMAIL PROTECTED] 
> Sent: Monday, May 10, 2004 4:36 PM
> To: PerlDiscuss - Perl Newsgroups and mailing lists; 
> [EMAIL PROTECTED]
> Subject: Re: File Monitoring

[snip]

> There is a directory watching component in POE 

Where to find doc on that (link) ?

>or you could grow your own, sorry
>I can't provide the one I have (licensing issues).

Why did you re-invented the wheel ;)
Or Am I missing some basic infos ?

Thanks,

José.


 DISCLAIMER 

"This e-mail and any attachment thereto may contain information which is confidential 
and/or protected by intellectual property rights and are intended for the sole use of 
the recipient(s) named above. 
Any use of the information contained herein (including, but not limited to, total or 
partial reproduction, communication or distribution in any form) by other persons than 
the designated recipient(s) is prohibited. 
If you have received this e-mail in error, please notify the sender either by 
telephone or by e-mail and delete the material from any computer".

Thank you for your cooperation.

For further information about Proximus mobile phone services please see our website at 
http://www.proximus.be or refer to any Proximus agent.


--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>

RE: File Monitoring

[Wiggins d Anconia]

> > -Original Message-
> > From: Wiggins d Anconia [mailto:[EMAIL PROTECTED] 
> > Sent: Monday, May 10, 2004 4:36 PM
> > To: PerlDiscuss - Perl Newsgroups and mailing lists; 
> > [EMAIL PROTECTED]
> > Subject: Re: File Monitoring
> 
> [snip]
> 
> > There is a directory watching component in POE 
> 
> Where to find doc on that (link) ?
> 

http://search.cpan.org/~cholet/POE-Component-DirWatch-0.01/DirWatch.pm

> >or you could grow your own, sorry
> >I can't provide the one I have (licensing issues).
> 
> Why did you re-invented the wheel ;)
> Or Am I missing some basic infos ?
> 

Good question.  Several reasons, mostly dealing with our application
overall.

1. The component mentioned is a very good example of using POE's inline
sessions to do a polling type of activity but isn't terribly robust (I
don't believe it was designed nor needs to be as a core component).
 
2. The callback interface didn't quite suit our needs as we were going
for more of an overall OOP structure and wanted continuity across the
app components.

3. I found it difficult to manage the directory watchers, for a couple
it would be sufficient but we were talking about upwards of 40 to start,
with more coming in the future.  One of the key goals of our app was to
have the ability to pause/resume as well as start/stop a particular
watcher at any given time, the session based watcher made this more
difficult (in my mind) than an object based one.

4. As a production app in a financial organization we needed a lot of
logging, and scalability, but could sacrifice development time and
maintenance.  We also wanted the logging to be seemless across the app,
so we built the logging directly into the watcher we made.

In the end because of the simplicity of the base module I don't know
that we reinvented the wheel, more like just put a nice steel belted
tire around it :-).  The underlying session that forms the core of the
component is still intact, the same events are used, but the data
structure itself is more accessible and the HEAP usage was dropped in
favor of a standard OOP hash.  These things made it simpler to control
the session since it is just a basic Perl object and will allow us to
change the implementation in the future if need be since it is
completely encapsulated (aka no fussing with POE events outside the
object's internal implementation).

I am still attempting to get the organization to allow me to release it,
that or I may build "my own" from scratch and incorporate it into
another app that I can release :-).  But there is no timeline for all of
that...

http://danconia.org

-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>

RE: File Monitoring

[NYIMI Jose (BMB)]

> -Original Message-
> From: Dani Pardo [mailto:[EMAIL PROTECTED] 
> Sent: Monday, May 10, 2004 4:25 PM
> To: [EMAIL PROTECTED]
> Subject: Re: File Monitoring
> 
> 
> Hi, I've done this app. What I do is put into a file the 
> output of 
> ls -lR /directory_to_watch.
>   Then use diff to compare the results. This gives the 
> advantage that, 
> if a line starts with +, then a new file has been added. If 
> it's -,  a 
> file has been deleted :)

Is this a joke  ? :-)

Here's a typical diff output:

17,18d16
< (who writes under the
< pseudonym "Omniscient Trash")
45,46c43,44
< soon every Tom, Dick or Harry
< will be writing his own Perl book
---
> soon every Tom, Randal and Larry
> will be writing their own Perl book
69a68,69
> Copyright (c) 1998, The Perl Journal.
> All rights reserved.

José.


 DISCLAIMER 

"This e-mail and any attachment thereto may contain information which is confidential 
and/or protected by intellectual property rights and are intended for the sole use of 
the recipient(s) named above. 
Any use of the information contained herein (including, but not limited to, total or 
partial reproduction, communication or distribution in any form) by other persons than 
the designated recipient(s) is prohibited. 
If you have received this e-mail in error, please notify the sender either by 
telephone or by e-mail and delete the material from any computer".

Thank you for your cooperation.

For further information about Proximus mobile phone services please see our website at 
http://www.proximus.be or refer to any Proximus agent.


--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>

RE: File Monitoring

[NYIMI Jose (BMB)]

> -Original Message-
> From: Wiggins d Anconia [mailto:[EMAIL PROTECTED] 
> Sent: Monday, May 10, 2004 6:02 PM
> To: NYIMI Jose (BMB); PerlDiscuss - Perl Newsgroups and 
> mailing lists; [EMAIL PROTECTED]
> Subject: RE: File Monitoring
> 
> 
> > > -Original Message-
> > > From: Wiggins d Anconia [mailto:[EMAIL PROTECTED]
> > > Sent: Monday, May 10, 2004 4:36 PM
> > > To: PerlDiscuss - Perl Newsgroups and mailing lists; 
> > > [EMAIL PROTECTED]
> > > Subject: Re: File Monitoring
> > 
> > [snip]
> > 
> > > There is a directory watching component in POE
> > 
> > Where to find doc on that (link) ?
> > 
> 
> http://search.cpan.org/~cholet/POE-Component-DirWatch-0.01/DirWatch.pm
> 
> > >or you could grow your own, sorry
> > >I can't provide the one I have (licensing issues).
> > 
> > Why did you re-invented the wheel ;)
> > Or Am I missing some basic infos ?
> > 
> 
> Good question.  Several reasons, mostly dealing with our 
> application overall.
> 
> 1. The component mentioned is a very good example of using 
> POE's inline sessions to do a polling type of activity but 
> isn't terribly robust (I don't believe it was designed nor 
> needs to be as a core component).
>  
> 2. The callback interface didn't quite suit our needs as we 
> were going for more of an overall OOP structure and wanted 
> continuity across the app components.
> 
> 3. I found it difficult to manage the directory watchers, for 
> a couple it would be sufficient but we were talking about 
> upwards of 40 to start, with more coming in the future.  One 
> of the key goals of our app was to have the ability to 
> pause/resume as well as start/stop a particular watcher at 
> any given time, the session based watcher made this more 
> difficult (in my mind) than an object based one.
> 
> 4. As a production app in a financial organization we needed 
> a lot of logging, and scalability, but could sacrifice 
> development time and maintenance.  We also wanted the logging 
> to be seemless across the app, so we built the logging 
> directly into the watcher we made.
> 
> In the end because of the simplicity of the base module I 
> don't know that we reinvented the wheel, more like just put a 
> nice steel belted tire around it :-).  The underlying session 
> that forms the core of the component is still intact, the 
> same events are used, but the data structure itself is more 
> accessible and the HEAP usage was dropped in favor of a 
> standard OOP hash.  These things made it simpler to control 
> the session since it is just a basic Perl object and will 
> allow us to change the implementation in the future if need 
> be since it is completely encapsulated (aka no fussing with 
> POE events outside the object's internal implementation).
> 
> I am still attempting to get the organization to allow me to 
> release it, that or I may build "my own" from scratch and 
> incorporate it into another app that I can release :-).  But 
> there is no timeline for all of that...
> 

Thanks for sharing your thoughts.

Watching a directory for incoming files is very common need
In organization like Telecom Mobile Operator (my current job).
We have to deal with proprietary systems (Nokia, Siemens, Alcatel).
Sometimes the only interaction that you have with those systems is
to watch a predefined directory to see if a new file has been created by those "black 
boxes".
Then start a process which treat the file's content
(Billing or CDR (Call Details Records) in general ).
Since the treatment can take long time, you need multitask behavior.

We have developed perl scripts to do the job but without POE.
I always thought that POE could provide a better alternative.

Learning POE is in my pipe for a while
But still not have time to start :).

Thanks again,

José.


 DISCLAIMER 

"This e-mail and any attachment thereto may contain information which is confidential 
and/or protected by intellectual property rights and are intended for the sole use of 
the recipient(s) named above. 
Any use of the information contained herein (including, but not limited to, total or 
partial reproduction, communication or distribution in any form) by other persons than 
the designated recipient(s) is prohibited. 
If you have received this e-mail in error, please notify the sender either by 
telephone or by e-mail and delete the material from any computer".

Thank you for your cooperation.

For further information about Proximus mobile phone services please see our website at 
http://www.proximus.be or refer to any Proximus agent.


--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>

Re: File Monitoring

[Dani Pardo]

NYIMI Jose (BMB) wrote:

Is this a joke  ? :-)

Here's a typical diff output:

17,18d16
< (who writes under the
< pseudonym "Omniscient Trash")
45,46c43,44
< soon every Tom, Dick or Harry
< will be writing his own Perl book
---
 

soon every Tom, Randal and Larry
will be writing their own Perl book
   

69a68,69
 

Copyright (c) 1998, The Perl Journal.
All rights reserved.
   

 

 Don't use diff with the actual files!, use diff to compare the file 
listing that dives "ls". Here is my code (this script is called every 
10min from crond):

#!/usr/bin/perl -w

# Store the new file listing
system("ls -lR   /home/originals > /tmp/Originals_PDF/pdflist.2");
# If ther's an older listing,
if( -r "/tmp/Originals_PDF/pdflist.1")
{
  # Create a diff file that contains the diference between listings
  system("diff /tmp/Originals_PDF/pdflist.1 
/tmp/Originals_PDF/pdflist.2 > /tmp/Originals_PDF/pdf.diff");
  open (FD, "/tmp/Originals_PDF/pdf.diff");
  # Foreach file in the diff
  while()
   {
 if (/^\>/) # If they start with > (a new file)
   {
 push @originals, $_;
}
   }
  close(FD);
  unlink("/tmp/Originals_PDF/pdf.diff");
 }

# Update the listing
system("cp /tmp/Originals_PDF/pdflist.2 /tmp/Originals_PDF/pdflist.1");
# If we find a newly-added file, we send an e-mail
if(defined (@originals))
 {
 open (MAIL, "|mail -s '---Notificacio creacio PDF'  
[EMAIL PROTECTED]");
 print MAIL "Listing of newly added files::\n\n";
 foreach $arxiu (@originals)
  {
print MAIL "$arxiu";
  }
 close(MAIL);
 }



--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

Re: File Monitoring

[Robin Sheat]

On Mon, May 10, 2004 at 08:35:53AM -0600, Wiggins d Anconia wrote:
> There is a directory watching component in POE or you could grow your
There is also the FAM daemon, which taps into the kernel to do it. I'm 
sure that there is a perl module to talk to that.

-- 
Robin <[EMAIL PROTECTED]> JabberID: <[EMAIL PROTECTED]>

Hostes alienigeni me abduxerunt. Qui annus est?

PGP Key 0x776DB663 Fingerprint=DD10 5C62 1E29 A385 9866 0853 CD38 E07A 776D B663


pgp0.pgp
Description: PGP signature