Hello Richard, Thank you very much. I'll try this configuration. Could you please clarify below questions
a1.sources.r1.providerURL = tcp://<activemqserver>:61616 a1.sources.r1.destinationName = TESTQUEUE -----> *This is queue name right?* a1.sources.r1.destinationType = QUEUE Regards, Rajesh On Tue, Mar 4, 2014 at 8:13 PM, richard ross <[email protected]>wrote: > Hello Rajesh: > > I just worked on a POC for the work that I am doing with JMS --> HDFS > configuration using ActiveMQ. I was able to use everything right out of the > box, going with JMS Source --> Memory Channel --> HDFS Sink, based on the > guide (https://flume.apache.org/FlumeUserGuide.html). Below is my example > config that I was playing around with. Note that for ActiveMQ, you do not > want to specify the connectionFactory in the source configuration ( > https://flume.apache.org/FlumeUserGuide.html#jms-source). A documentation > defect was created for this point ( > https://issues.apache.org/jira/browse/FLUME-2339). As for questions 3 and > 4, I will let someone with more experience answer. > > # Name the components on this agent > a1.sources = r1 > a1.sinks = k1 > a1.channels = c1 > > # Describe the source > #a1.sources.r1.type = netcat > #a1.sources.r1.bind = localhost > #a1.sources.r1.port = 44444 > a1.sources.r1.type = jms > a1.sources.r1.channels = c1 > a1.sources.r1.initialContextFactory = > org.apache.activemq.jndi.ActiveMQInitialContextFactory > a1.sources.r1.providerURL = tcp://<activemqserver>:61616 > a1.sources.r1.destinationName = TESTQUEUE > a1.sources.r1.destinationType = QUEUE > > # Describe the sink > #a1.sinks.k1.type = logger > a1.sinks.k1.type = hdfs > a1.sinks.k1.hdfs.path = hdfs://<namenode>:8020/user/root/flumetest/%Y%m%d > a1.sinks.k1.hdfs.filePrefix = messages > a1.sinks.k1.hdfs.rollInterval = 30 > #a1.sinks.k1.hdfs.rollSize = 1024 > a1.sinks.k1.hdfs.rollCount = 1000 > a1.sinks.k1.hdfs.fileType = DataStream > > # Use a channel which buffers events in memory > a1.channels.c1.type = memory > a1.channels.c1.capacity = 1000 > a1.channels.c1.transactionCapacity = 100 > > # Bind the source and sink to the channel > a1.sources.r1.channels = c1 > a1.sinks.k1.channel = c1 > > Hope this helps, > Richard. > > > On Mon, Mar 3, 2014 at 10:22 PM, Madabhattula Rajesh Kumar < > [email protected]> wrote: > >> Hello Team, >> >> I'm new to flume. Could you please help me on below query >> >> 1. How to configure JMS source(TIBCO and ActiveMQ) in flume? >> 2. Do i need to write custom source for above requirement? (Or) Out >> Of the Box flume JMS source implementation is fine? >> 3. What is the best channel mechanism for above requirement? >> 4. Each message will be sink into HDFS? (or) Can we consolidate some >> messages and sink into HDFS? >> >> Thank you in advance >> >> Regards, >> Rajesh >> > >
