struggling with runtime Schema in connect

i see kafka connect invented its own runtime data type system in org.apache.kafka.connect.data however i struggle to understand how this is used. the payload in kafka is bytes. kafka does not carry any "schema" metadata. so how does connect know what the schema is of a ConnectRecord? if i write

Effects of operation with log.retention.bytes and log.segment.bytes

Hello, I have been running some stress tests regarding log retention whilst writing to kafka continuously every 5 seconds. I have set the following properties (apart from other mandatory ones): log.retention.minutes=3 log.retention.hours. log.retention.bytes=26214400 log.segment.bytes=10485760

Re[2]: Kafka behavior when consuming a topic which doesn't exist?

Please ensure that you set the desired minimum partition and replication settings in the server.properties file. These are the ones which will be used for auto creation of topics. KR, -- Original Message -- From: "Ali Akhtar" To: users@kafka.apache.org Sent:

Re: Kafka behavior when consuming a topic which doesn't exist?

Oh gotcha, thanks. So a topic will be created if topic creation is enabled. On Sat, Jul 8, 2017 at 8:14 PM, M. Manna wrote: > Please check my previous email. > > On Sat, 8 Jul 2017 at 2:32 am, Ali Akhtar wrote: > > > What happens if auto creation is

Re: Kafka behavior when consuming a topic which doesn't exist?

Please check my previous email. On Sat, 8 Jul 2017 at 2:32 am, Ali Akhtar wrote: > What happens if auto creation is enabled but the topic doesn't exist. > Consumers subscribe to that topic which doesn't exist. Then messages are > posted to that topic. > > Will the consumer

Re: [DISCUSS] Streams DSL/StateStore Refactoring

Hi Matthias thanks, Exactly what I was guessing. I don't understand why custom stores in DSL? and I don't understand why we are not concidering a more generic config based appraoch? StateStores in DSL => what I really think we are looking for PAPA => DSL => PAPI back and forth switcharoo