Fwd: [DISCUSS] KIP 130: Expose states of active tasks to KafkaStreams public API

2017-03-25 Thread Guozhang Wang
Forwarding Matthias's reply to the channel (I think he meant to do that but reply only to me). -- Forwarded message -- From: Matthias J. Sax Date: Thu, Mar 23, 2017 at 4:30 PM Subject: Re: [DISCUSS] KIP 130: Expose states of active tasks to KafkaStreams public API To: Guo

Re: [DISCUSS] KIP 130: Expose states of active tasks to KafkaStreams public API

2017-03-25 Thread Guozhang Wang
Re 1): this is a good point. May be we can move `StreamsMetadata#streamThreads` as `KafkaStreams#localThreadsMetadata`? 3): this is a minor suggestion about function name of `assignedPartitions`, to `topicPartitions` to be consistent with `StreamsMetadata`? Guozhang On Thu, Mar 23, 2017 at 4:30

Re: [DISCUSS] KIP 130: Expose states of active tasks to KafkaStreams public API

2017-03-27 Thread Florian Hussonnois
Hi Guozhang, Matthias, It's a great idea to add sub topologies descriptions. This would help developers to better understand topology concept. I agree that is not really user-friendly to check if `StreamsMetadata#streamThreads` is not returning null. The method name localThreadsMetadata looks go

Re: [DISCUSS] KIP 130: Expose states of active tasks to KafkaStreams public API

2017-03-28 Thread Florian Hussonnois
Hi all, I've updated the KIP and the PR to reflect your suggestions. https://cwiki.apache.org/confluence/display/KAFKA/KIP+130%3A+Expose+states+of+active+tasks+to+KafkaStreams+public+API https://github.com/apache/kafka/pull/2612 Also, I've exposed property StreamThread#state as a string through t

Re: [DISCUSS] KIP 130: Expose states of active tasks to KafkaStreams public API

2017-03-28 Thread Matthias J. Sax
Thanks for updating the KIP! I think it's good as is -- I would not add anything more to TaskMetadata. About subtopologies and tasks. We do have the concept of subtopologies already in KIP-120. It's only missing and ID that allow to link a subtopology to a task. IMHO, adding a simple variable to