Re: Few partitions stuck in under replication

2022-03-07 Thread Liam Clarke-Hutchinson
Hi Dhirendra, so after looking into your stack trace further, it shows that the AlterISRRequest is failing when trying to interact with ZooKeeper. It doesn't give us more information as to why currently. Can you please set some loggers to DEBUG level to help analyse the issue further? These

Re: Few partitions stuck in under replication

2022-03-07 Thread Dhirendra Singh
Hi Thomas, I see the IllegalStateException but as i pasted earlier it is java.lang.IllegalStateException: Failed to enqueue `AlterIsr` request with state LeaderAndIsr(leader=1, leaderEpoch=2728, isr=List(1, 2), zkVersion=4719) for partition __consumer_offsets-4 I upgraded to version 2.8.1 but

Re: Free spots available for 6 week Apache Kafka course

2022-03-07 Thread Bala vinayagam
Interested. Thanks,Bala Sent from Yahoo Mail on Android On Mon, 7 Mar 2022 at 23:50, vinay deshpande wrote: Hi Justin, I'm interested in the course. Thanks, Vinay On Mon, Mar 7, 2022 at 9:30 AM David Luu wrote: > Sorry my bad, didn't notice replies go back to the group. > > On Mon,

Re: securing sasl/scram username and password in kafka connect

2022-03-07 Thread Men Lim
Chris, here's the content of the files ## distributor file: bootstrap.servers=broker:9096 group.id=dbz-dev key.converter=org.apache.kafka.connect.json.JsonConverter value.converter=org.apache.kafka.connect.json.JsonConverter key.converter.schemas.enable=false

Re: securing sasl/scram username and password in kafka connect

2022-03-07 Thread Chris Egerton
It looks like the file config provider isn't actually set up on the Connect worker. What does your Connect worker config look like (usually a file called something like connect-distributed.properties)? Feel free to change any sensitive values to a string like "", but please don't remove them

Re: securing sasl/scram username and password in kafka connect

2022-03-07 Thread Men Lim
Thanks for the response Chris. I went thru the setup again and it appeared I might have had a typo somewhere last friday. Currently, I'm running into a file permission issue. the file has the following permissions: -rw-r--r-- 1 adm admn 88 Mar 7 21:23 connector_credentials.properties I have

Re: securing sasl/scram username and password in kafka connect

2022-03-07 Thread Chris Egerton
Hi Men, That config snippet has a small syntax error: all double quotes should be escaped. Assuming you tried something like this: "database.history.producer.sasl.jaas.config": "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"${file:/path/file.pro:user\"}

Re: securing sasl/scram username and password in kafka connect

2022-03-07 Thread Men Lim
Hi Chris, I was getting an unauthorized/authentication error message when I was trying it out last Friday. I tried looking for the exact message in the connect.log.* files but was not very successful. In my connector file, I have { "name":"blah", "config": { ... ...

Re: Free spots available for 6 week Apache Kafka course

2022-03-07 Thread vinay deshpande
Hi Justin, I'm interested in the course. Thanks, Vinay On Mon, Mar 7, 2022 at 9:30 AM David Luu wrote: > Sorry my bad, didn't notice replies go back to the group. > > On Mon, Mar 7, 2022, 9:25 AM David Luu wrote: > > > Hi Justin, > > > > I am interested in the course. > > > > Regards, > >

Re: Free spots available for 6 week Apache Kafka course

2022-03-07 Thread David Luu
Sorry my bad, didn't notice replies go back to the group. On Mon, Mar 7, 2022, 9:25 AM David Luu wrote: > Hi Justin, > > I am interested in the course. > > Regards, > David > > On Mon, Mar 7, 2022, 7:11 AM Justin Mclean wrote: > >> >> Hi, >> >> I’m from OpenSI, a not for profit open source

Re: Free spots available for 6 week Apache Kafka course

2022-03-07 Thread David Luu
Hi Justin, I am interested in the course. Regards, David On Mon, Mar 7, 2022, 7:11 AM Justin Mclean wrote: > > Hi, > > I’m from OpenSI, a not for profit open source foundation. We have a > limited number of free spots for a 6 week online Apache Kafka course > starting on the 22nd of March. It

Re: securing sasl/scram username and password in kafka connect

2022-03-07 Thread Chris Egerton
Hi Men, The config provider mechanism should work for every property in a connector config, and every property in a worker config except for the plugin.path property (see KAFKA-9845 [1]). You can also use it for only part of a single property, or even multiple parts, like in this example

Re: consumer hpa autoscaling

2022-03-07 Thread David Ballano Fernandez
Thanks, also thanks Fares for pointing me to Keda. On Sun, Mar 6, 2022 at 4:18 PM Liam Clarke-Hutchinson wrote: > > I was trying to see what the goals of enabling Hpa on the consumer would > be. Since like you say there is a partition upper limit which will limit > the consumer throughput. so

securing sasl/scram username and password in kafka connect

2022-03-07 Thread Men Lim
Hi all, recently, I found out about config.providers=file config.providers.file.class=org.apache.kafka.common.config.provider.FileConfigProvider This works great to remove our embedded database password into an external file. However, it does not work when I tried to do the same thing with

Free spots available for 6 week Apache Kafka course

2022-03-07 Thread Justin Mclean
Hi, I’m from OpenSI, a not for profit open source foundation. We have a limited number of free spots for a 6 week online Apache Kafka course starting on the 22nd of March. It will take about 8 hours a week of your time to complete it. If anyone is interested, please get in touch with me

Re: How to achieve high availability in a Kafka Streams app during deployment?

2022-03-07 Thread Ismar Slomic
Hi Matthias and thanks for replying to this thread. > "Hard to answer from a 10,000ft view." We tried hard to include a detailed explanation of our use case in the Stack Overflow thread(