Hi Misha, If it works locally, then I would still suspect that it is due to a transient timing issue: note that the create-topic script is non-blocking, i.e. even when it returns it does not necessarily guarantee that the leader metadata information has been completed propagating to brokers.
Guozhang On Thu, Jul 28, 2016 at 1:48 AM, mishadoff <mishad...@gmail.com> wrote: > Thanks Guozhang, > > Yes, I rely on auto-create, and it works locally. Maybe I need to tweak > some timeout conf for that? > Also I identified, that even I manually create a topic, it lists but I can > not produce messages to this topic with the same exception. > Producing to other topics works well, so it seems like server problem? > > — Misha > > > On Jul 28, 2016, at 01:44, Guozhang Wang <wangg...@gmail.com> wrote: > > > > Misha, > > > > Did you pre-create the sink topic before starting your application or you > > are relying on the broker-side auto-create for that topic? > > > > If you are relying on auto-create, then there is a transient period where > > the topic is created but the metadata has not been propagated to the > > brokers so they do not know they are the leader of the created topic > > partitions yet. And I'd recommend not relying on it since it is really > > meant for debugging environment only. > > > > Guozhang > > > > > > On Wed, Jul 27, 2016 at 5:45 AM, mishadoff <mishad...@gmail.com> wrote: > > > >> Hello, > >> > >> I’ve a simplest ever kafka streams application which just reads from one > >> kafka topic A and write to another topic B. > >> > >> When I run it on my local environment (local zk, local kafka broker, > local > >> kafka streams app) everything works fine, topic B created and filled > with > >> messages from A > >> If I run it on existing kafka cluster (remote zk, remote kafka, LOCAL > >> kafka streams) my app is not working anymore. > >> > >> It succesfully read the remote topic A, succesfully process the message > >> and generate a producer record, creates a B topic in remote kafka, bud > >> during send I get an error > >> > >> ``` > >> 15:36:47.242 [kafka-producer-network-thread | > >> example-message-counter3-1-StreamThread-1-producer] ERROR > >> o.a.k.s.p.internals.RecordCollector - Error sending record: null > >> org.apache.kafka.common.errors.NotLeaderForPartitionException: This > server > >> is not the leader for that topic-partition > >> ``` > >> > >> Could you point me to direction where to start debug or what problems > >> might cause this behaviour? > >> > >> Thanks, > >> — Misha > > > > > > > > > > -- > > -- Guozhang > > -- -- Guozhang
