Hi Mirko,
Thanks for your reply. It works for me as well.
Now I was able to mount the folder on the master node and
configured Flume such that it can either poll for logs in real time or even
for periodic retrieval.
Thanks,
Mahesh Balija.
Calsof Labs.
On Thu, Jan 17, 2013
Hi,
My log files are generated and saved in a windows machine.
Now I have to move those remote files to the Hadoop cluster (HDFS)
either in synchronous or asynchronous way.
I have gone through flume (Various source types) but was not helpful.
Please suggest whether there
ftp auto upload?
2013/1/17 Mahesh Balija balijamahesh@gmail.com:
the Hadoop cluster (HDFS) either in synchronous or asynchronou
Yes. It is possible. I haven't tries windows+flume+hadoop combo
personally, but it should work. You may find this
linkhttp://mapredit.blogspot.in/2012/07/run-flume-13x-on-windows.htmluseful.
Alex
has explained beautifully how to run Flume on a windows box.If I
get time i'll try to simulate your
That link talks about just installing Flume on Windows machine (NOT even
have configs to push logs to the Hadoop cluster), but what if I have to
collect logs from various clients, then I will endup installing in all
clients.
I have installed Flume successfully on Linux but I have to configure it
One approach I used in my lab was the data-gateway,
which is a small linux box which just mounts Windows Shares
and a single flume node on the gateway corresponds to the
HDFS cluster. With tail or periodic log rotation you have control
over all logfiles, depending on your use case. Either grab all
