[
https://issues.apache.org/jira/browse/KAFKA-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238307#comment-15238307
]
Guozhang Wang commented on KAFKA-3544:
--------------------------------------
The topics that Kafka Streams reads as sources or writes as sinks, or more
explicitly,
{code}
builder.stream("topic1");
builder.table("topic2");
stream / table.through("topic3");
stream / table.to("topic4");
{code}
In these cases, currently topic1/2/3/4 are considered "external" to the Kafka
Streams, which are not auto-created by the library as other changelog /
re-partition for aggregation, and users are supposed to create these
themselves. It's just that by default, Kafka brokers will auto-create the
topics when it receives metadata requests from the clients (in this case, from
Kafka Streams) and depending on the timing it may succeed, but some times it
may now.
As I mentioned before, we plan to auto-embed "through" for repartitioning
before joining, and hence those topics will be considered as internal topics as
well, but for now they are out of the library "black box".
> Missing topics on startup
> -------------------------
>
> Key: KAFKA-3544
> URL: https://issues.apache.org/jira/browse/KAFKA-3544
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.0.0
> Reporter: Greg Fodor
> Assignee: Guozhang Wang
> Labels: semantics
>
> When running a relatively complex job with multiple tasks and state stores,
> on the first run I get errors due to some of the intermediate topics not
> existing. Subsequent runs work OK. My assumption is streams may be creating
> topics lazily, so if downstream tasks are initializing before their parents
> have had a chance to create their necessary topics then the children will
> attempt to start consuming from topics that do not exist yet.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
