Re: Kafka Streams Avro SerDe version/id caching

2018-01-05 Thread Kristopher Kane
Just a follow up caching example for the DSL with emphasis on the consumer (deserializer): final KafkaAvroDeserializer kafkaAvroDeserializer = new KafkaAvroDeserializer( new CachedSchemaRegistryClient(config.getProperty(AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG), 1024)); fi

Re: Kafka Streams Avro SerDe version/id caching

2017-10-19 Thread Kristopher Kane
I fixated on using the key/value deserializer classes in the consumer properties. Overloading the consumer constructor is the way to enable schema caching: CachedSchemaRegistryClient cachedSchemaRegistryClient = new CachedSchemaRegistryClient("registry_url", 1000); KafkaAvroDeserializer kafkaAvro

Re: Kafka Streams Avro SerDe version/id caching

2017-10-03 Thread Svante Karlsson
I've implemented the same logic for a c++ client - caching is the only way to go since the performance impact of not doing it would be to big. So bet on caching on all clients. 2017-10-03 18:12 GMT+02:00 Damian Guy : > If you are using the confluent schema registry then the will be cached by > th

Re: Kafka Streams Avro SerDe version/id caching

2017-10-03 Thread Damian Guy
If you are using the confluent schema registry then the will be cached by the SchemaRegistryClient. Thanks, Damian On Tue, 3 Oct 2017 at 09:00 Ted Yu wrote: > I did a quick search in the code base - there doesn't seem to be caching as > you described. > > On Tue, Oct 3, 2017 at 6:36 AM, Kristop

Re: Kafka Streams Avro SerDe version/id caching

2017-10-03 Thread Ted Yu
I did a quick search in the code base - there doesn't seem to be caching as you described. On Tue, Oct 3, 2017 at 6:36 AM, Kristopher Kane wrote: > If using a Byte SerDe and schema registry in the consumer configs of a > Kafka streams application, does it cache the Avro schemas by ID and version

Kafka Streams Avro SerDe version/id caching

2017-10-03 Thread Kristopher Kane
If using a Byte SerDe and schema registry in the consumer configs of a Kafka streams application, does it cache the Avro schemas by ID and version after fetching from the registry once? Thanks, Kris