Re: [DISCUSS] KIP-174 - Deprecate and remove internal converter configs in WorkerConfig

2018-01-04 Thread Ewen Cheslack-Postava
Sorry I lost track of this thread. If things are in good shape we should probably vote on this and get the deprecation commit through. It seems like a good idea as this has been confusing to users from day one. -Ewen On Wed, Aug 9, 2017 at 5:18 AM, UMESH CHAUDHARY wrote: >

Re: Streams - State store directory created automatically

2018-01-04 Thread Ted Yu
Looks like the .checkpoint file was generated from this code in ProcessorStateManager : // write the checkpoint file before closing, to indicate clean shutdown try { if (checkpoint == null) { checkpoint = new OffsetCheckpoint(new File(baseDir,

Streams - State store directory created automatically

2018-01-04 Thread Kristopher Kane
I just noticed /tmp/kafka-streams//0_[0,1]/{.checkpoint,.lock] (there are two partitions on the incoming topic) being automatically created during an integration test. My Streams app doesn't use a state store and only contains mapValues and a .to termination operation. Anyone know what this is

Re: Dynamic topic subscription without using pattern

2018-01-04 Thread Jordan Pilat
Did you want to subscribe to all topics as they are created, or to exclude some of them? If the latter, is there a pattern in the names of the topics you wish to exclude? - Jordan Pilat On 2018-01-03 23:39, chintan mavawala wrote: > Hi, > > We have a use case where

Re: Any suggestion about Kafka/Zookeeper migration.

2018-01-04 Thread Tim Visher
On Thu, Jan 4, 2018 at 11:55 AM, Tony Liu wrote: > Thanks Timmy. > > I appreciate your suggestion, but for us, it's hard to do that , the > problem are : > >- 6+ Kafka cluster. >- more than 200 micro services or applications are talking with Kafka >now. > >

Re: Any suggestion about Kafka/Zookeeper migration.

2018-01-04 Thread Tony Liu
Thanks Timmy. I appreciate your suggestion, but for us, it's hard to do that , the problem are : - 6+ Kafka cluster. - more than 200 micro services or applications are talking with Kafka now. Any other suggestion for our case ? On Thu, Jan 4, 2018 at 6:06 AM, Tim Visher

Re: Any suggestion about Kafka/Zookeeper migration.

2018-01-04 Thread Tim Visher
On Wed, Jan 3, 2018 at 11:39 PM, Tony Liu wrote: > This post here is aimed to ask experience about what did you do migration > `Kafka/zookeeper` ? :) > > I think the easiest way to migrate kafka is to do a blue/green deploy. Stand up an entire new cluster, cut over