On Mar 24, 2011, at 10:09 AM, Jonathan Coveney wrote:
> I am not sure if this is the right listserv, forgive me if it is not.
A better choice would likely be hdfs-user@, since this is really about
watching files in HDFS.
> My
> goal is this: monitor HDFS until a file is create, and then kick off a job.
> Ideally I'd want to do this continuously, but the file would be create
> hourly (with some sort of variance). I guess I could make a script that
> would ping the server every 5 minutes or something, but I was wondering if
> there might be a more elegant way?
Two ways off the top of my head:
1) Read/watch the edits stream
2) Read/watch the HDFS audit log
Given the latter is text built by log4j, that should be relatively
simple to implement.
There was a JIRA asking for this functionally to be built in recently, btw.
