Re: kafka stream ktable with suppress operator

I assume you're using RocksDB as your state stores... The bytes out you're seeing on the changelog topics is probably because they are restoring your state stores. If your state stores are in-memory, then on every application startup they're going to be restored from the changelog topics. If

Kafka Connect API Timing Out

Hi, I have a kafka connect cluster that some time the API is timing out. Calling to "/" endpoint is usually fine but calling to "/connectors" or "/connectors/" will time out. I tried to look into logs but there's too many log entries so couldn't find anything specific. I also tried exec into the

Re: kafka stream ktable with suppress operator

Hi, I'm using 2.3.1 now and having the same issue. During restarting, I noticed a lot logging like below: Seeking to EARLIEST offset of partition XX-KSTREAM-REDUCE-STATE-STORE-14-changelog-41 Seeking to EARLIEST offset of partition XX-KTABLE-SUPPRESS-STATE-STORE-20-changelog-41

Re: Transaction error in Kafka producer perf test

I understand that we could have used batch size as 0 here, if I am interpreting the documentation correctly.We essentially want no batching, and setting any value less than 256 in this particular case would achieve that, since the message size is 256 bytes for any message produced. Sincerely,

Re: Transaction error in Kafka producer perf test

We are consciously choosing the batch size in the experiment to be 1 because that is what our producer use case will need to use. Regarding the request timeout that you have mentioned, how does one set that? The command line is already passing two timeout values: ‘max.block.ms’, and

Re: Transaction error in Kafka producer perf test

Hi, Per test is based on a set of tuning parameters e.g. batch.size, axes, partitions, network latency etc. Your transactions are failing because your batch has expired, (or at least, that’s what it shows on the log). You have to tune your request timeout and batch.size correctly to improve on

Re: Transaction error in Kafka producer perf test

Anyone with a pointer on this? Do transactions work reliably with Kafka perf test tools? If yes, then is there a way to make it work in this scenario? Sincerely, Anindya Haldar Oracle Responsys > On Oct 25, 2019, at 2:51 PM, Anindya Haldar wrote: > > We are evaluating Kafka for some of our

Re: [EXTERNAL] SSL setup failing

@Peter I have the feeling is related to client.auth required, in the end each broker is a client for the rest in the cluster. Try with client.auth=none, and check if the connect. If so, the review SSL conf related to that. Cheers Get Outlook for Android

Re: Attempt to prove Kafka transactions work

On Sunday, October 27, 2019, Boyang Chen wrote: > Hey Edward, > > just to summarize and make sure I understood your question, you want to > implement some Chaos testing to validate Kafka EOS model, but not sure how > to start or curious about whether there are already works in the community >

Re: [EXTERNAL] SSL setup failing

@Jose >9092 is as well SSL protocol? Yes, it is. As you see in the config snippet from my initial email. > Zookeeper is connecting over SSL? Yes, at least as far as I can tell. It's set up there too but neither of those are making verifying that easy... >So then I would review all certificates

Re: [EXTERNAL] SSL setup failing

@Peter 9092 is as well SSL protocol? Zookeeper is connecting over SSL? So then I would review all certificates to check if valid. As well there is a Kafka broker property 'advertised.host.name' you could set with same hostname in the certificate. Thanks

Re: [EXTERNAL] SSL setup failing

Sorry, if I was unclear before. I'm absolutely new to Kafka and how it works. @Jose >That happening when clients trying to SSL connect? There are no clients at the moment just one Kafka broker which spews the errors in the server.log. To be specific, there is a ZooKeeper client which has no

serdeProps what does that mean

Hi all I need a custom serde for Kafka. Our serde needs a SchemaStore instance with some state, and I'm trying to figure out how to get this into the serde. I'm trying to follow along with this example

Re: [EXTERNAL] SSL setup failing

@Peter That happening when clients trying to SSL connect? Review SSL configuration related ssl.client.auth=required is right in client side. Here you have explained possible errors, and you could find what happening:

Re: [EXTERNAL] SSL setup failing

Hi, not sure what it means "Tries to communicate with itself". Are you talking about local network loopback? Also, have you tried ssl debug using openssl? What did you observe? The exception is handshake exception. This is quite common when your cert validation fails. How have you setup your

Re: [EXTERNAL] SSL setup failing

@Jose >It looks like communication problem between brokers. As I mentioned, "I can't get the first broker started". The message above is from when the broker tries to communicate with "itself": [Controller id=1001, targetBrokerId=1001]). Nevertheless, I went through the checklist and everything

Re: [EXTERNAL] SSL setup failing

@Peter It looks like communication problem between brokers. But ensure: 1) Crtificates are valid and properly signed by root CA or intermediate one in the chain 2) Clients and brokers having private key and certificate in their keystore and properly configured to point to its path 3) Clients