Re: Kafka process killed during first attempt of startup

2023-02-15 Thread Angel Motta
Hello Brian It was the most basic Azure vm instance (1 vCPU and 0.5 GB RAM) running Ubuntu Server 20 LTS. Good point about the 'dmesg' . I forgot to check that log. I tried again using a VM with 0.5GB and JVM Heap size of 256MB and then with 400MB and you were right in 'dmesg' logs *I could see th

Re: Kafka process killed during first attempt of startup

2023-02-15 Thread Brian @ Rickabaugh.Net
Angel, What OS are you running on the VM? Some Linux variant? The Linux kernel will get aggressive with processes that get really greedy with resources, especially memory. You might check your ‘dmesg’ logs to look for reports of the kernel asserting itself. It will summarily kill user proces

Remote code execution flaw patched in Apache Kafka

2023-02-15 Thread Turritopsis Dohrnii Teo En Ming
Subject: Remote code execution flaw patched in Apache Kafka Good day from Singapore, Sharing this article. Hot off the press. Article: Remote code execution flaw patched in Apache Kafka Link: https://portswigger.net/daily-swig/remote-code-execution-flaw-patched-in-apache-kafka Thank you. Reg

Re: Kafka process killed during first attempt of startup

2023-02-15 Thread Angel Motta
I tried with another virtual machine with 1GB of RAM and the jvm heap size setting to 700MB and It worked! Kafka started successfully. It looks like the default setting of 1GB heap size in the server.properties is the minimum? or near the minimum right? I wanted the minimum resources (memory) beca

Question regarding Kafka compaction and tombstone

2023-02-15 Thread Nanda Naga
Hi, I am using Kafka 3.2 (in windows) and for a topic i to send tombstone records. Everything was ok but i always see last value for the key (even i see null records present after delete.retention.ms period) Example Key1 value1 Key2 value2 Key1 - null record - tombstone record and so on I am

Re: Kafka process killed during first attempt of startup

2023-02-15 Thread Angel Motta
I deleted the content of the log directory and format again as is indicated in the quickstart but same problem. Kafka process is killed during startup in the same way. On Wed, Feb 15, 2023 at 9:11 PM sunil chaudhari wrote: > Ok sorry to misinterpret your message. > However if you are learning

Re: Kafka process killed during first attempt of startup

2023-02-15 Thread sunil chaudhari
Ok sorry to misinterpret your message. However if you are learning and its your first time, you can clear the data from the logs directory defined in server.properties. And start fresh instance and then see what happens. I hope you have followed all documents properly. On Thu, 16 Feb 2023 at 7:07

Re: Kafka process killed during first attempt of startup

2023-02-15 Thread Angel Motta
Hello Sunil Thanks for your message. Only to clarify that I never could successfully start Kafka. This my first attempt to start Kafka and I get the "killed" message as shown in the log. Knowing this fact is necessary to remove all data and logs? If this helps, could you tell me how to do this?

Re: Kafka process killed during first attempt of startup

2023-02-15 Thread sunil chaudhari
Remove all data and logs. And start it. Next time when you want to stop then dont kill the process with kill command. Stop it gracefully using kafka-server-stop under /bin Kafka needs stop signal to do some cleanup operations before it stops. So kill is not the option. On Thu, 16 Feb 2023 at 6:49

Kafka process killed during first attempt of startup

2023-02-15 Thread Angel Motta
Hello everyone! New Kafka user here. I am learning Kafka and trying to run it for the first time (part of my CS undergrad thesis) I have the minimum server in AWS EC2 with 500MB of RAM and whenever I try to start Kafka (kraft mode) I get "killed" as a last message. First I had to change the heap s

Re: Kafka Connect ClusterConfigState.inconsistentConnectors() not handled by distributed Worker?

2023-02-15 Thread Greg Harris
Frank, > I don't think forcing the API users to introduce the nonce is desirable. I agree. That is why the nonce is a workaround, and not a proper solution. It's something to alleviate the symptoms in the short-term until a bugfix & upgrade can fix it. > Have you had any ideas on how this can be

Re: Kafka Connect ClusterConfigState.inconsistentConnectors() not handled by distributed Worker?

2023-02-15 Thread Frank Grimes
So we've just hit this issue again just with the MM2 connector and trying to add a new mirrored topic.We're running MirrorMaker 2 in Strimzi. i.e. "connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector"We have 6 worker nodes.We changed the config to add a new mirror topic. i.e