Hey Murilo,
feel free to file a JIRA and paste your full topology. It seems like a bug
to me.
Boyang
On Thu, Feb 6, 2020 at 8:17 AM Murilo Tavares <murilo...@gmail.com> wrote:
> Answering my own question, obviously this is a stateless application, so
> there’s no reset needed. Mu bad.
> But the NPE does seem to be linked to the fact that the topics were not
> sorted, therefore different instances could have different topologies in
> respect to the order.
> Sorting the topics seems to have fixed the NPEs.
> Thanks
> Murilo
>
>
> On Wed, Feb 5, 2020 at 1:34 PM Murilo Tavares <murilo...@gmail.com> wrote:
>
> > Hi
> > I have a KafkaStreams application that's pretty simple, and acts as a
> > repartitioner... It reads from input topics and send to output topics,
> > based on a input-to-output topics map. It has a custom Repartitioner that
> > will be responsible for assigning new partitions for the data in the
> output
> > topics.
> >
> > The topology is simple:
> > protected Topology buildTopology(Map<String, String> topics) {
> > StreamsBuilder builder = new StreamsBuilder();
> > for (Entry<String, String> entry : topics.entrySet()) {
> > builder
> > .stream(entry.getKey(), Consumed.with(Serdes.ByteArray(),
> > Serdes.ByteArray()))
> > .to(entry.getValue(), Produced.with(Serdes.ByteArray(),
> > Serdes.ByteArray()));
> > }
> > return builder.build();
> > }
> >
> > The resulting topolgy will look like this:
> > Sub-topology: 0
> > Source: KSTREAM-SOURCE-0000000000 (topics: [kafka.inputA])
> > --> KSTREAM-SINK-0000000001
> > Sink: KSTREAM-SINK-0000000001 (topic: kafka.inputA-repartitioned)
> > <-- KSTREAM-SOURCE-0000000000
> >
> > Sub-topology: 1
> > Source: KSTREAM-SOURCE-0000000002 (topics: [kafka.inputB])
> > --> KSTREAM-SINK-0000000003
> > Sink: KSTREAM-SINK-0000000003 (topic: kafka.inputB-repartitioned)
> > <-- KSTREAM-SOURCE-0000000002
> >
> > Sub-topology: 2
> > Source: KSTREAM-SOURCE-0000000004 (topics: [kafka.inputC])
> > --> KSTREAM-SINK-0000000005
> > Sink: KSTREAM-SINK-0000000005 (topic: kafka.inputC-repartitioned)
> > <-- KSTREAM-SOURCE-0000000004
> > ... (about 25 diffent sub-topologies in total)
> >
> > I am facing some inconsistent NPE in some environments, that I'll send in
> > the next e-mail not to polute this one anymore.
> >
> > But I am wondering if I can really do that, and what would happen if I:
> > 1) add a new sub-topolgy;
> > 2) adding the new sub-topology breaks the order of the iteration;
> >
> > Will any of the above operations require a reset to the KafkaStreams
> > application (or new app.id)?
> >
> > Thanks
> > Murilo
> >
>
