Leaonard, Thanks for the reply and would look into those options. But as for the original question, could we create a topic dynamically when required .
Prasanna. On Mon, Jun 1, 2020 at 2:18 PM Leonard Xu <xbjt...@gmail.com> wrote: > Hi, kumar > > Flink support consume/produce from/to multiple kafka topics[1], in your > case you can implement KeyedSerializationSchema(legacy interface) or > KafkaSerializationSchema[2] to make one producer instance support send data > to multiple topics. There is an ITCase you can reference[3]. > > > Best, > Leonard Xu > > [1] > https://ci.apache.org/projects/flink/flink-docs-master/dev/connectors/kafka.html#kafka-producer > [2] > https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/connectors/kafka/KafkaSerializationSchema.java > > [3] > https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-0.11/src/test/java/org/apache/flink/streaming/connectors/kafka/Kafka011ITCase.java#L126 > > > 在 2020年6月1日，15:35，Prasanna kumar <prasannakumarram...@gmail.com> 写道： > > Hi, > > I have Use Case where i read events from a Single kafka Stream comprising > of JSON messages. > > Requirement is to split the stream into multiple output streams based on > some criteria say based on Type of Event or Based on Type and Customer > associated with the event. > > We could achieve the splitting of stream using Side outputs as i have seen > in the documentation. > > Our business environment is such that there could be new event types > flowing in and would the Flink Kafka producer create the topics dynamically > based on the inflowing events. I did not see any documentation saying > that it could create. > > Or should it be always pre created by running a script separately. (Not a > good scalable practice in our case) > > Thanks, > Prasanna. > > >
