Hi,

I want to consume from both a specific topic "a_topic" as well as all topics 
that match a certain prefix "prefix.*".

When I do that using a single instance of a ConsumerConnector, I get a hang 
when creating the 2nd set of message streams.

Code:
        ConsumerConnector consumer = 
Consumer.createJavaConsumerConnector(consumerConfig);
        Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
        topicCountMap.put("a_topic", new Integer(1));
        
        Map<String, List<KafkaStream<byte[], byte[]>>> consumerMap = 
consumer.createMessageStreams(topicCountMap);

        // do stuff with resulting streams
        
        TopicFilter whitelist = new Whitelist("prefix\\.*");
        List<KafkaStream<byte[], byte[]>> wildcardStreams = 
consumer.createMessageStreamsByFilter(whitelist, 1);

It hangs when inside createMessageStreamsByFilter(), within 
createEphemeralPathExpectConflictHandleZKBug(): There is an info() message 
saying:
        info("I wrote this conflicted ephemeral node [%s] at %s a while back in 
a different session, ".format(data, path)
                  + "hence I will backoff for this node to be deleted by 
Zookeeper and retry")

Is this expected to work? Can a ConsumerConnector be used like this, or should 
I have 2 ConsumerConnectors; one for the specific topic, and another for the 
wildcarded topics?

It works when I use 2 ConsumerConnectors, but I just wanted to check if this is 
expected or not.

Thanks,
-James

Reply via email to