On 03/24/2011 01:09 PM, Jonathan Coveney wrote:
I am not sure if this is the right listserv, forgive me if it is not. My
goal is this: monitor HDFS until a file is create, and then kick off a job.
Ideally I'd want to do this continuously, but the file would be create
hourly (with some sort of variance). I guess I could make a script that
would ping the server every 5 minutes or something, but I was wondering if
there might be a more elegant way?
Thanks
Jon
I suppose you could do this using HDFS, but this sounds to me like
Zookeeper is much better suited to this type of application. You could
just add watcher onto a particular zookeeper node and you'd get notified
about updates to it and its children.
HTH,
DR
