Is there any easy way to do that without recompiling samza? I'm trying to localize that into the 'hello-samza' and looking at http://samza.apache.org/learn/documentation/latest/jobs/logging.html leads me to believe that I have to do this in the base samza project (not hello-samza).
On Sun, Mar 22, 2015 at 11:37 AM, Ash W Matheson <ash.mathe...@gmail.com> wrote: > Sure - I'll do that in a bit and send it up to pastebin. > > On Sun, Mar 22, 2015 at 11:35 AM, Chinmay Soman <chinmay.cere...@gmail.com > > wrote: > >> Can you please enable debug level logging and paste the log? >> >> On Sun, Mar 22, 2015 at 11:28 AM, Ash W Matheson <ash.mathe...@gmail.com> >> wrote: >> >> > No, it's behind some corporate stuff - I just redacted it so I could >> share >> > it up. >> > >> > On Sun, Mar 22, 2015 at 11:17 AM, Chinmay Soman < >> chinmay.cere...@gmail.com >> > > >> > wrote: >> > >> > > Just for sanity check, is the broker host 'redacted:9092' or ' >> > > redactedec:9092'. >> > > >> > > Just wanted to rule out any typos. Are the 2 above hosts the same ? >> > > >> > > On Sun, Mar 22, 2015 at 11:08 AM, Ash W Matheson < >> ash.mathe...@gmail.com >> > > >> > > wrote: >> > > >> > > > Also, here's the producer: http://pastebin.com/qMNJabTZ >> > > > >> > > > >> > > > On Sun, Mar 22, 2015 at 10:57 AM, Ash W Matheson < >> > ash.mathe...@gmail.com >> > > > >> > > > wrote: >> > > > >> > > > > Yep, first thing I checked (got bitten by that earlier in the week >> > with >> > > > no >> > > > > data actually in the topic). >> > > > > >> > > > > On Sun, Mar 22, 2015 at 10:56 AM, Chinmay Soman < >> > > > chinmay.cere...@gmail.com >> > > > > > wrote: >> > > > > >> > > > >> Can you double check that you can read data from your Kafka >> broker ? >> > > > >> >> > > > >> > ./deploy/kafka/bin/kafka-topics.sh --describe --zookeeper >> > > > localhost:2181 >> > > > >> --topic myTopic >> > > > >> > ./deploy/kafka/bin/kafka-console-consumer.sh --zookeeper >> > > > localhost:2181 >> > > > >> --topic myTopic --from-beginning >> > > > >> >> > > > >> I've seen cases where if the Kafka broker isn't shutdown >> properly, >> > > > >> something like this happens. >> > > > >> >> > > > >> On Sun, Mar 22, 2015 at 10:35 AM, Ash W Matheson < >> > > > ash.mathe...@gmail.com> >> > > > >> wrote: >> > > > >> >> > > > >> > Hey all, >> > > > >> > >> > > > >> > Evaluating Samza currently and am running into some odd issues. >> > > > >> > >> > > > >> > I'm currently working off the 'hello-samza' repo and trying to >> > > parse a >> > > > >> > simple kafka topic that I've produced through an extenal java >> app >> > > > >> (nothing >> > > > >> > other than a series of sentences) and it's failing pretty hard >> for >> > > me. >> > > > >> The >> > > > >> > base 'hello-samza' set of apps works fine, but as soon as I >> change >> > > the >> > > > >> > configuration to look at a different Kafka/zookeeper I get the >> > > > >> following in >> > > > >> > the userlogs: >> > > > >> > >> > > > >> > 2015-03-22 17:07:09 KafkaSystemAdmin [WARN] Unable to fetch >> last >> > > > offsets >> > > > >> > for streams [myTopic] due to kafka.common.KafkaException: >> fetching >> > > > topic >> > > > >> > metadata for topics [Set(myTopic)] from broker >> > > > >> > [ArrayBuffer(id:0,host:redacted,port:9092)] failed. Retrying. >> > > > >> > >> > > > >> > >> > > > >> > The modifications are pretty straightforward. In the >> > > > >> > Wikipedia-parser.properties, I've changed the following: >> > > > >> > task.inputs=kafka.myTopic >> > > > >> > systems.kafka.consumer.zookeeper.connect=redacted:2181/ >> > > > >> > systems.kafka.consumer.auto.offset.reset=smallest >> > > > >> > systems.kafka.producer.metadata.broker.list=redacted:9092 >> > > > >> > >> > > > >> > and in the actual java file WikipediaParserStreamTask.java >> > > > >> > public void process(IncomingMessageEnvelope envelope, >> > > > MessageCollector >> > > > >> > collector, TaskCoordinator coordinator) { >> > > > >> > Map<String, Object> jsonObject = (Map<String, Object>) >> > > > >> > envelope.getMessage(); >> > > > >> > WikipediaFeedEvent event = new >> WikipediaFeedEvent(jsonObject); >> > > > >> > >> > > > >> > try { >> > > > >> > System.out.println(event.getRawEvent()); >> > > > >> > >> > > > >> > And then following the compile/extract/run process outlined in >> the >> > > > >> > hello-samza website. >> > > > >> > >> > > > >> > Any thoughts? I've looked online for any 'super simple' >> examples >> > of >> > > > >> > ingesting kafka in samza with very little success. >> > > > >> > >> > > > >> >> > > > >> >> > > > >> >> > > > >> -- >> > > > >> Thanks and regards >> > > > >> >> > > > >> Chinmay Soman >> > > > >> >> > > > > >> > > > > >> > > > >> > > >> > > >> > > >> > > -- >> > > Thanks and regards >> > > >> > > Chinmay Soman >> > > >> > >> >> >> >> -- >> Thanks and regards >> >> Chinmay Soman >> > >
