My understanding of sync is that producer waits on .send until Kafka receives the message. And async means it just dispatches the message without any gurantees that message is delivered. Did I get that part right? On Fri, Oct 17, 2014 at 1:28 PM, Gwen Shapira <[email protected]> wrote:
> Sorry if I'm confusing you :) > > Kafka 0.8.1.1 has two producers sync and async. You are using the sync > producer without waiting for acks. I hope this helps? > > Regardless, did you check if the partition got created? are you able > to produce messages? are you able to consume them? > > Gwen > > On Fri, Oct 17, 2014 at 4:13 PM, Mohit Anchlia <[email protected]> > wrote: > > Still don't understand the difference. If it's not waiting for the ack > then > > doesn't it make async? > > On Fri, Oct 17, 2014 at 12:55 PM, <[email protected]> wrote: > > > >> Its using the sync producer without waiting for any broker to > acknowledge > >> the write. This explains the lack of errors you are seeing. > >> > >> — > >> Sent from Mailbox > >> > >> On Fri, Oct 17, 2014 at 3:15 PM, Mohit Anchlia <[email protected]> > >> wrote: > >> > >> > Little confused :) From one of the examples I am using property > >> > request.required.acks=0, > >> > I thought this sets the producer to be async? > >> > On Fri, Oct 17, 2014 at 11:59 AM, Gwen Shapira <[email protected] > > > >> > wrote: > >> >> 0.8.1.1 producer is Sync by default, and you can set producer.type to > >> >> async if needed. > >> >> > >> >> On Fri, Oct 17, 2014 at 2:57 PM, Mohit Anchlia < > [email protected]> > >> >> wrote: > >> >> > Thanks! How can I tell if I am using async producer? I thought all > the > >> >> > sends are async in nature > >> >> > On Fri, Oct 17, 2014 at 11:44 AM, Gwen Shapira < > [email protected] > >> > > >> >> > wrote: > >> >> > > >> >> >> If you have "auto.create.topics.enable" set to "true" (default), > >> >> >> producing to a topic creates it. > >> >> >> > >> >> >> Its a bit tricky because the "send" that creates the topic can > fail > >> >> >> with "leader not found" or similar issue. retrying few times will > >> >> >> eventually succeed as the topic gets created and the leader gets > >> >> >> elected. > >> >> >> > >> >> >> Is it possible that you are not getting errors because you are > using > >> >> >> async producer? > >> >> >> > >> >> >> Also "no messages are delivered" can have many causes. Check if > the > >> >> >> topic exists using: > >> >> >> bin/kafka-topics.sh --list --zookeeper localhost:2181 > >> >> >> > >> >> >> Perhaps the topic was created and the issue is elsewhere (the > >> consumer > >> >> >> is a usual suspect! perhaps look in the FAQ for tips with that > issue) > >> >> >> > >> >> >> Gwen > >> >> >> > >> >> >> On Fri, Oct 17, 2014 at 12:56 PM, Mohit Anchlia < > >> [email protected] > >> >> > > >> >> >> wrote: > >> >> >> > Is Kafka supposed to throw exception if topic doesn't exist? It > >> >> appears > >> >> >> > that there is no exception thrown even though no messages are > >> >> delivered > >> >> >> and > >> >> >> > there are errors logged in Kafka logs. > >> >> >> > >> >> > >> >
