adding timestsamp interceptor to the flume

2015-12-04 Thread DurgaPrasad Bandaru
i have query that how to intercept the exec source file.give me one
example?


Adding an interceptor

2012-11-07 Thread Rahul Ravindran
Apologies. I am new to Flume, and I am probably missing something fairly 
obvious. I am attempting to test using a timestamp interceptor and host 
interceptor but I see only a sequence of numbers in the remote end.

Below is the flume config:



agent1.channels.ch1.type = MEMORY
agent1.channels.ch1.capacity = 500

agent1.sources.seq_gen.channels = ch1
agent1.sources.seq_gen.type = SEQ

agent1.sources.seq_gen.interceptors = inter1 host1
#agent1.sources.seq_gen.interceptors.inter1.type = 
org.apache.flume.interceptor.TimestampInterceptor$Builder
agent1.sources.seq_gen.interceptors.inter1.type = TIMESTAMP
agent1.sources.seq_gen.interceptors.inter1.preserveExisting = false

#agent1.sources.seq_gen.interceptors.host1.type = 
org.apache.flume.interceptor.HostInterceptor$Builder
agent1.sources.seq_gen.interceptors.host1.type = HOST
agent1.sources.seq_gen.interceptors.host1.preserveExisting = false
agent1.sources.seq_gen.interceptors.host1.hostHeader = hostname
agent1.sources.seq_gen.interceptors.host1.useIP = false


agent1.sinks.avroSink1.type = avro
agent1.sinks.avroSink1.channel = ch1
agent1.sinks.avroSink1.hostname = remote server
agent1.sinks.avroSink1.port = 4545
agent1.sinks.avroSink1.connect-timeout = 30

agent1.channels = ch1
agent1.sources = seq_gen
agent1.sinks = avroSink1

Re: Adding an interceptor

2012-11-07 Thread Hari Shreedharan
Rahul, 

The interceptor adds headers, not content to the body. Unless you are somehow 
writing the headers out, you will not see the headers in the output. The 
sequence of numbers you see are generated by the SEQ source - which is what it 
does.


Hari 

-- 
Hari Shreedharan


On Wednesday, November 7, 2012 at 1:18 PM, Rahul Ravindran wrote:

 Apologies. I am new to Flume, and I am probably missing something fairly 
 obvious. I am attempting to test using a timestamp interceptor and host 
 interceptor but I see only a sequence of numbers in the remote end.
 
 Below is the flume config:
 
 
 
 agent1.channels.ch1.type = MEMORY
 agent1.channels.ch1.capacity = 500
 
 agent1.sources.seq_gen.channels = ch1
 agent1.sources.seq_gen.type = SEQ
 
 agent1.sources.seq_gen.interceptors = inter1 host1
 #agent1.sources.seq_gen.interceptors.inter1.type = 
 org.apache.flume.interceptor.TimestampInterceptor$Builder
 agent1.sources.seq_gen.interceptors.inter1.type = TIMESTAMP
 agent1.sources.seq_gen.interceptors.inter1.preserveExisting = false
 
 #agent1.sources.seq_gen.interceptors.host1.type = 
 org.apache.flume.interceptor.HostInterceptor$Builder
 agent1.sources.seq_gen.interceptors.host1.type = HOST
 agent1.sources.seq_gen.interceptors.host1.preserveExisting = false
 agent1.sources.seq_gen.interceptors.host1.hostHeader = hostname
 agent1.sources.seq_gen.interceptors.host1.useIP = false
 
 
 agent1.sinks.avroSink1.type = avro
 agent1.sinks.avroSink1.channel = ch1
 agent1.sinks.avroSink1.hostname = remote server
 agent1.sinks.avroSink1.port = 4545
 agent1.sinks.avroSink1.connect-timeout = 30
 
 agent1.channels = ch1
 agent1.sources = seq_gen
 agent1.sinks = avroSink1