Hey Chris,
I'm not clear what functionality you would want from the TailSource
could offer that's not already offered by (a) using ExecSource (b)
putting flume inside your application or (c) using the asyncronous log
spooler that I am working on.
It's impossible to correctly "watch" a file from w
Hi Chris,
A few months back I actually ported the original flumes tail source, but
it was decided(and I agree with the reasoning) not to include it for a
number of reasons, which can be seen on the original ticket at
https://issues.apache.org/jira/browse/FLUME-931 . One of the big ones is
the
Otis,
I don't know much about Flume 0.9 but I think you can fork off events into
separate flows. There are some docs here that might help you figure out how
to do it (I don't think they are hosted anywhere on Apache):
http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_flow_isolation
Reg
Hey guys,
I'm sure this is not a new question, but I haven't found an answer in my
searches. I'm curious why there is as of yet no Tail Source with NG? It
seems one of the most common use cases for Flume is to tail a log file and
dump it "somewhere". Given that, it sure would seem that a Tail S
Thanks, easy enough solution, this worked for me.
On Wed, Aug 29, 2012 at 10:42 AM, Brock Noland wrote:
> Hi,
>
> This is my fault. At this point FC is dependent on Hadoop. We want to
> remove it, but it will take some time. This is in
> https://issues.apache.org/jira/browse/FLUME-1285
>
> The s
Hi,
This is my fault. At this point FC is dependent on Hadoop. We want to
remove it, but it will take some time. This is in
https://issues.apache.org/jira/browse/FLUME-1285
The simplest way to solve this is to put the hadoop command in your
path before you start flume.
Brock
On Wed, Aug 29, 201
Good morning. I'm trying to test Flume for use for our companies log
collection. These are application transaction logs (not syslogs) so I need
the agents channel data to be fully recoverable. I have done some initial
setup testing using the memory channel for my initial tests and it's been
goin
Hi all,
I have an Exec Source running a tail -F on a log4J-generated log file that
gets rolled once a day. It seems that when log4J rolls the file to the new
date, the hdfs sink ends up with a .tmp file. I haven't figured out if
there is any data loss yet, but was curious if this is expected beh
Hello,
I was wondering if there is any sort of message routing available in pre-NG
version of Flume?
Concretely, I have a stream of messages belonging to different users and I want
to route some messages to one Flume Collector (e.g. based on user ID) and other
messages to the other Flume Coll
Worth trying out the AVRO_EVENT serializer which comes out of the box,
which writes the data in Avro container format (including all headers).
Regards,
Mike
On Wed, Aug 29, 2012 at 2:11 AM, Hari Shreedharan wrote:
> Hi,
>
> The interceptor is probably working as expected. The reason for this is
Hi,
The interceptor is probably working as expected. The reason for this is that
the serializer that the FILE_ROLL sink uses by default is the BodyText
serializer which simply serializes the body. The hostname is in the header
though, so you need to write a serializer that can write the header
Hi All,
I am a newbie to flume. I have started using it for log aggregation in my
company.
I have been desperately trying to make the hostname interceptor work for me ,
but i am not getting the hostName in the final aggregated log file.
The conf file at my AGGREGATOR machine is
# Describe/co
On Mon, Aug 27, 2012 at 7:53 AM, Bhaskar V. Karambelkar wrote:
> Hi Mike,
> There are indeed Jira issues for this.
> A flume jira issue , which was fixed in 1.2.0. FLUME-1163.
>
Thanks Bhaskar, I just marked that a dup of
https://issues.apache.org/jira/browse/FLUME-1219 which has the relevant
p
13 matches
Mail list logo
