Re: KTable aggregations send intermediate results downstream?

2016-08-24 Thread Mathieu Fenniak
Hi Guozhang, I've been working around this issue by dropping down to the Processor API, but, I was hoping you might be able to point out if there is a flaw is in this proposed change: https://github.com/apache/kafka/compare/trunk...mfenniak:suppress-duplicate-repartition-output This

Multiple Consumers From Same Group Assigned Same Partition

2016-08-24 Thread Nicholas Harezga
I am currently attempting to upgrade my software to use Kafka 0.9 from 0.8.2. I am trying to switch over to the new Consumer API to allow for rebalancing as machines are added or removed from our cluster. I am running into an issue where the same partition on a topic is being assigned to

Re: KIP-33 Opt out from Time Based indexing

2016-08-24 Thread Jan Filipiak
Hey Jun, I go and try again :), wrote the first one in quite a stressful environment. The bottom line is that I, for our use cases, see a to small use/effort ratio in this time index. We do not bootstrap new consumers for key-less logs so frequently and when we do it, they usually want

Re: Questions about Apache Kafka

2016-08-24 Thread Marko Bonaći
Hi Karin, regarding 5 (fsyncing to disk), take a look at the broker configuration parameters whose names start with log.flush. http://kafka.apache.org/documentation.html#brokerconfigs Marko Bonaći Monitoring | Alerting | Anomaly Detection | Centralized Log Management Solr & Elasticsearch Support

Re: Questions about Apache Kafka

2016-08-24 Thread Ian Wrigley
Actually, for 3 and 4 Kafka Streams is also a very good option — and comes as part of the standard Apache Kafka distribution. Ian. --- Ian Wrigley Director, Education Services Confluent, Inc > On Aug 24, 2016, at 2:37 PM, David Garcia wrote: > > Regarding 3 and 4: