[
https://issues.apache.org/jira/browse/FLUME-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Paiva updated FLUME-2863:
------------------------------
Description:
Hi guys,
I have to install flume on Windows environment and, have some issues..
1st attempt - I used a file flume-node0.9.4.exe from cloudera git website
(https://github.com/cloudera/flume/downloads) than, the log files on flume give
that error:
WARN agent.MultiMasterRPC: Could not connect to any master nodes (tried 1:
[localhost:35872])
INFO agent.MultiMasterRPC: No active master RPC connection
2nd attempt - I downloaded flume from apache website
(https://flume.apache.org/download.html) file: apache-flume-1.6.0-bin.tar.gz,
extract all files and configure files: flume.conf, flume-env.sh and, executed
flume-ng.cmd (command: flume-ng shaman) I don't receive any error message, just
simple shutdown of flume-ng.cmd window..
Anyone has any experience with Flume on Windows sending .log files to HDFS in
another environment?
Attach my file flume.conf to all see what I wanna do..
-- define source, channel and sink
shaman.sources = tail_source1
shaman.channels = ch1
shaman.sinks = hdfs_sink1
-- define tail source
shaman.sources.tail_source1.type = exec
shaman.sources.tail_source1.channels = ch1
shaman.sources.tail_source1.shell = /bin/bash -c
shaman.sources.tail_source1.command = tail -F F:/LogFiles/w3r54
shaman.sources.tail_source1.interceptors = ts
shaman.sources.tail_source1.interceptors.ts.type = timestamp
-- define in-memory channel
shaman.channels.ch1.type = memory
shaman.channels.ch1.capacity = 100000
shaman.channels.ch1.transactionCapacity = 1000
-- define HDFS sink properties
shaman.sinks.hdfs_sink1.type = hdfs
shaman.sinks.hdfs_sink1.hdfs.path =
hdfs://xxxx:8020/user/admin/fromflume/%y%m%d/%H%M%S
shaman.sinks.hdfs_sink1.hdfs.fileType = DataStream
shaman.sinks.hdfs_sink1.channel = ch1
Thanks!
Alan..
was:
Hi guys,
I have to install flume on Windows environment and, have some issues..
1st attempt - I used a file flume-node0.9.4.exe from cloudera git website
(https://github.com/cloudera/flume/downloads) than, the log files on flume give
that error:
WARN agent.MultiMasterRPC: Could not connect to any master nodes (tried 1:
[localhost:35872])
INFO agent.MultiMasterRPC: No active master RPC connection
2nd attempt - I downloaded flume from apache website
(https://flume.apache.org/download.html) file: apache-flume-1.6.0-bin.tar.gz,
extract all files and configure files: flume.conf, flume-env.sh and, executed
flume-ng.cmd (command: flume-ng shaman) I don't receive any error message, just
simple shutdown of flume-ng.cmd window..
Anyone has any experience with Flume on Windows sending .log files to HDFS in
another environment?
Attach my file flume.conf to all see what I wanna do..
## define source, channel and sink
shaman.sources = tail_source1
shaman.channels = ch1
shaman.sinks = hdfs_sink1
## define tail source
shaman.sources.tail_source1.type = exec
shaman.sources.tail_source1.channels = ch1
shaman.sources.tail_source1.shell = /bin/bash -c
shaman.sources.tail_source1.command = tail -F F:/LogFiles/w3r54
shaman.sources.tail_source1.interceptors = ts
shaman.sources.tail_source1.interceptors.ts.type = timestamp
## define in-memory channel
shaman.channels.ch1.type = memory
shaman.channels.ch1.capacity = 100000
shaman.channels.ch1.transactionCapacity = 1000
## define HDFS sink properties
shaman.sinks.hdfs_sink1.type = hdfs
shaman.sinks.hdfs_sink1.hdfs.path =
hdfs://xxxx:8020/user/admin/fromflume/%y%m%d/%H%M%S
shaman.sinks.hdfs_sink1.hdfs.fileType = DataStream
shaman.sinks.hdfs_sink1.channel = ch1
Thanks!
Alan..
> How to install Flume on Windows server (best practices)
> -------------------------------------------------------
>
> Key: FLUME-2863
> URL: https://issues.apache.org/jira/browse/FLUME-2863
> Project: Flume
> Issue Type: Question
> Components: Configuration
> Affects Versions: v1.6.0
> Environment: Windows Server 2008 R2 Datacenter
> Intel Xeon CPU e5 - 4Gb ram
> Flume1.6.0.
> Flume0.9.4 (exe)
> Reporter: Alan Paiva
> Priority: Trivial
> Labels: build, newbie
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> Hi guys,
> I have to install flume on Windows environment and, have some issues..
> 1st attempt - I used a file flume-node0.9.4.exe from cloudera git website
> (https://github.com/cloudera/flume/downloads) than, the log files on flume
> give that error:
> WARN agent.MultiMasterRPC: Could not connect to any master nodes (tried 1:
> [localhost:35872])
> INFO agent.MultiMasterRPC: No active master RPC connection
> 2nd attempt - I downloaded flume from apache website
> (https://flume.apache.org/download.html) file: apache-flume-1.6.0-bin.tar.gz,
> extract all files and configure files: flume.conf, flume-env.sh and, executed
> flume-ng.cmd (command: flume-ng shaman) I don't receive any error message,
> just simple shutdown of flume-ng.cmd window..
> Anyone has any experience with Flume on Windows sending .log files to HDFS in
> another environment?
> Attach my file flume.conf to all see what I wanna do..
> -- define source, channel and sink
> shaman.sources = tail_source1
> shaman.channels = ch1
> shaman.sinks = hdfs_sink1
> -- define tail source
> shaman.sources.tail_source1.type = exec
> shaman.sources.tail_source1.channels = ch1
> shaman.sources.tail_source1.shell = /bin/bash -c
> shaman.sources.tail_source1.command = tail -F F:/LogFiles/w3r54
> shaman.sources.tail_source1.interceptors = ts
> shaman.sources.tail_source1.interceptors.ts.type = timestamp
> -- define in-memory channel
> shaman.channels.ch1.type = memory
> shaman.channels.ch1.capacity = 100000
> shaman.channels.ch1.transactionCapacity = 1000
> -- define HDFS sink properties
> shaman.sinks.hdfs_sink1.type = hdfs
> shaman.sinks.hdfs_sink1.hdfs.path =
> hdfs://xxxx:8020/user/admin/fromflume/%y%m%d/%H%M%S
> shaman.sinks.hdfs_sink1.hdfs.fileType = DataStream
> shaman.sinks.hdfs_sink1.channel = ch1
> Thanks!
> Alan..
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
