Hi we are using the following version of KafkaUtils.createDirectStream() from
spark 1.5.0
createDirectStream(JavaStreamingContext jssc,
Class<K> keyClass,
Class<V> valueClass,
Class<KD> keyDecoderClass,
Class<VD> valueDecoderClass,
Class<R> recordClass,
java.util.Map<String,String> kafkaParams,
java.util.Map<kafka.common.TopicAndPartition,Long> fromOffsets,
Function<kafka.message.MessageAndMetadata<K,V>,R> messageHandler)
while the streaming app is running, the kafka topic got expanded by increasing
the partitionsfrom 10 to 20.
The problem is that the running app doesn't change to include the 10 new
partitions. We have to stopthe app and feed the fromOffsets map the new
partitions and restart.
Is there any way to get this done automatically? Curious to know if you ran
into same problem andwhats your solution/workaround?
Thanks-Vinay
