Hello, Needless to say I am newbie to Flume, but I've got a basic flow working in which I am importing a log file from my linux box to hdfs. I am using
a1.sources.r1.command = tail -F /var/log/xyz.log which is working like a stream of messages. This is good! Now what I want to do is copy log files from a directory on a remote machine on a regular basis. For example: username@machinename:/var/log/logdir/<multiple files> One way to do it is to simply 'scp' files from the remote directory into my box on a regular basis, but what's the best way to do this in Flume? Please let me know. Thanks for the help.
