Re: [DISCUSS] Apache Kafka 3.2.1 release
Thanks David :) On Thu, Jul 21, 2022 at 6:00 PM David Arthur wrote: > Viktor, seeing as it's been on trunk for a while and is a very small > change, it seems fine to include in this release. I just finished building > the RC, but haven't started a vote thread yet. Good timing on your part :) > I'll merge this PR shortly and start a new RC build. > > -David > > On Thu, Jul 21, 2022 at 11:42 AM Viktor Somogyi-Vass > wrote: > > > Hi David, > > > > Found an issue (tight loop in the consumer), fixed it on trunk and > > backported it onto the 3.2 branch. Is it possible to include this in the > > 3.2.1 release? > > https://github.com/apache/kafka/pull/12417 > > > > Thanks, > > Viktor > > > > On Tue, Jul 19, 2022 at 5:57 PM Randall Hauch wrote: > > > > > Hi, Chris S and Chris E, > > > > > > Thanks for quickly working on and reviewing the > > > https://issues.apache.org/jira/browse/KAFKA-14079 issue mentioned > > > above. The two PRs you created, one for the `trunk` branch and one for > > the > > > `3.2` branch, have both been merged, and the issue has been marked as > > > resolved. > > > > > > Best regards, > > > > > > Randall > > > > > > On Sun, Jul 17, 2022 at 5:44 PM Christopher Shannon < > > > christopher.l.shan...@gmail.com> wrote: > > > > > > > Hi Chris E. > > > > > > > > Thanks for all the feedback earlier, I updated the PR based on your > > > > comments and also pushed a second PR for trunk for 3.3.0 > > > > > > > > I agree the impact is high which is why I found the issue. I recently > > > > turned on this feature and suddenly my connect worker/task kept > > > > periodically falling over with OOM errors. Finally I took a heap dump > > and > > > > saw a ton of submitted record objects in memory and started > > investigating > > > > and that's how I figured out it was related to this new feature.. > > > > > > > > Chris > > > > > > > > On Sun, Jul 17, 2022 at 1:48 PM Chris Egerton < > fearthecel...@gmail.com > > > > > > > wrote: > > > > > > > > > Hi Chris, > > > > > > > > > > Good find, and thanks for filing a fix. I agree that we should get > > this > > > > > into 3.2.1 if possible. The risk is fairly low (the functional > parts > > of > > > > the > > > > > fix are just two lines long) and the impact of the bug is high for > > > users > > > > > who have configured source connectors with "errors.tolerance" set > to > > > > "all". > > > > > > > > > > Cheers, > > > > > > > > > > Chris > > > > > > > > > > On Sat, Jul 16, 2022 at 12:26 PM Christopher Shannon < > > > > > christopher.l.shan...@gmail.com> wrote: > > > > > > > > > > > HI, > > > > > > > > > > > > I think I found a memory leak that was introduced in 3.2.0 in a > > > > Connector > > > > > > SourceTask. I created a JIRA: > > > > > > https://issues.apache.org/jira/browse/KAFKA-14079 and small PR > > with > > > a > > > > > fix: > > > > > > https://github.com/apache/kafka/pull/12412 > > > > > > > > > > > > I think this should be included in 3.2.1. It should also go into > > > 3.3.0 > > > > > but > > > > > > there was a lot of refactoring done there with the source task > code > > > due > > > > > to > > > > > > KIP-618 so another PR needs to be done for that if this is > merged. > > > > > > > > > > > > Chris > > > > > > > > > > > > On Fri, Jul 15, 2022 at 10:06 AM David Arthur > > > > wrote: > > > > > > > > > > > > > Here is the release plan for 3.2.1 > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.2.1 > > > > > > > > > > > > > > I am working on getting clarity on the one open blocker. Once > > that > > > is > > > > > > > resolved (or rescheduled for a future release), I will build > the > > > > first > > > > > > > release candidate. > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > On Thu, Jul 14, 2022 at 3:10 AM Luke Chen > > > wrote: > > > > > > > > > > > > > > > +1, Thanks David! > > > > > > > > > > > > > > > > On Thu, Jul 14, 2022 at 1:16 PM David Jacot < > > > david.ja...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > +1. Thanks David. > > > > > > > > > > > > > > > > > > Le mer. 13 juil. 2022 à 23:43, José Armando García Sancio > > > > > > > > > a écrit : > > > > > > > > > > > > > > > > > > > +1. Thanks for volunteering David. > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > -José > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > David Arthur > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > David Arthur >
Re: [DISCUSS] Apache Kafka 3.2.1 release
Viktor, seeing as it's been on trunk for a while and is a very small change, it seems fine to include in this release. I just finished building the RC, but haven't started a vote thread yet. Good timing on your part :) I'll merge this PR shortly and start a new RC build. -David On Thu, Jul 21, 2022 at 11:42 AM Viktor Somogyi-Vass wrote: > Hi David, > > Found an issue (tight loop in the consumer), fixed it on trunk and > backported it onto the 3.2 branch. Is it possible to include this in the > 3.2.1 release? > https://github.com/apache/kafka/pull/12417 > > Thanks, > Viktor > > On Tue, Jul 19, 2022 at 5:57 PM Randall Hauch wrote: > > > Hi, Chris S and Chris E, > > > > Thanks for quickly working on and reviewing the > > https://issues.apache.org/jira/browse/KAFKA-14079 issue mentioned > > above. The two PRs you created, one for the `trunk` branch and one for > the > > `3.2` branch, have both been merged, and the issue has been marked as > > resolved. > > > > Best regards, > > > > Randall > > > > On Sun, Jul 17, 2022 at 5:44 PM Christopher Shannon < > > christopher.l.shan...@gmail.com> wrote: > > > > > Hi Chris E. > > > > > > Thanks for all the feedback earlier, I updated the PR based on your > > > comments and also pushed a second PR for trunk for 3.3.0 > > > > > > I agree the impact is high which is why I found the issue. I recently > > > turned on this feature and suddenly my connect worker/task kept > > > periodically falling over with OOM errors. Finally I took a heap dump > and > > > saw a ton of submitted record objects in memory and started > investigating > > > and that's how I figured out it was related to this new feature.. > > > > > > Chris > > > > > > On Sun, Jul 17, 2022 at 1:48 PM Chris Egerton > > > > wrote: > > > > > > > Hi Chris, > > > > > > > > Good find, and thanks for filing a fix. I agree that we should get > this > > > > into 3.2.1 if possible. The risk is fairly low (the functional parts > of > > > the > > > > fix are just two lines long) and the impact of the bug is high for > > users > > > > who have configured source connectors with "errors.tolerance" set to > > > "all". > > > > > > > > Cheers, > > > > > > > > Chris > > > > > > > > On Sat, Jul 16, 2022 at 12:26 PM Christopher Shannon < > > > > christopher.l.shan...@gmail.com> wrote: > > > > > > > > > HI, > > > > > > > > > > I think I found a memory leak that was introduced in 3.2.0 in a > > > Connector > > > > > SourceTask. I created a JIRA: > > > > > https://issues.apache.org/jira/browse/KAFKA-14079 and small PR > with > > a > > > > fix: > > > > > https://github.com/apache/kafka/pull/12412 > > > > > > > > > > I think this should be included in 3.2.1. It should also go into > > 3.3.0 > > > > but > > > > > there was a lot of refactoring done there with the source task code > > due > > > > to > > > > > KIP-618 so another PR needs to be done for that if this is merged. > > > > > > > > > > Chris > > > > > > > > > > On Fri, Jul 15, 2022 at 10:06 AM David Arthur > > > wrote: > > > > > > > > > > > Here is the release plan for 3.2.1 > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.2.1 > > > > > > > > > > > > I am working on getting clarity on the one open blocker. Once > that > > is > > > > > > resolved (or rescheduled for a future release), I will build the > > > first > > > > > > release candidate. > > > > > > > > > > > > -David > > > > > > > > > > > > On Thu, Jul 14, 2022 at 3:10 AM Luke Chen > > wrote: > > > > > > > > > > > > > +1, Thanks David! > > > > > > > > > > > > > > On Thu, Jul 14, 2022 at 1:16 PM David Jacot < > > david.ja...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > +1. Thanks David. > > > > > > > > > > > > > > > > Le mer. 13 juil. 2022 à 23:43, José Armando García Sancio > > > > > > > > a écrit : > > > > > > > > > > > > > > > > > +1. Thanks for volunteering David. > > > > > > > > > > > > > > > > > > -- > > > > > > > > > -José > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > David Arthur > > > > > > > > > > > > > > > > > > > > > -- David Arthur
Re: [DISCUSS] Apache Kafka 3.2.1 release
Hi David, Found an issue (tight loop in the consumer), fixed it on trunk and backported it onto the 3.2 branch. Is it possible to include this in the 3.2.1 release? https://github.com/apache/kafka/pull/12417 Thanks, Viktor On Tue, Jul 19, 2022 at 5:57 PM Randall Hauch wrote: > Hi, Chris S and Chris E, > > Thanks for quickly working on and reviewing the > https://issues.apache.org/jira/browse/KAFKA-14079 issue mentioned > above. The two PRs you created, one for the `trunk` branch and one for the > `3.2` branch, have both been merged, and the issue has been marked as > resolved. > > Best regards, > > Randall > > On Sun, Jul 17, 2022 at 5:44 PM Christopher Shannon < > christopher.l.shan...@gmail.com> wrote: > > > Hi Chris E. > > > > Thanks for all the feedback earlier, I updated the PR based on your > > comments and also pushed a second PR for trunk for 3.3.0 > > > > I agree the impact is high which is why I found the issue. I recently > > turned on this feature and suddenly my connect worker/task kept > > periodically falling over with OOM errors. Finally I took a heap dump and > > saw a ton of submitted record objects in memory and started investigating > > and that's how I figured out it was related to this new feature.. > > > > Chris > > > > On Sun, Jul 17, 2022 at 1:48 PM Chris Egerton > > wrote: > > > > > Hi Chris, > > > > > > Good find, and thanks for filing a fix. I agree that we should get this > > > into 3.2.1 if possible. The risk is fairly low (the functional parts of > > the > > > fix are just two lines long) and the impact of the bug is high for > users > > > who have configured source connectors with "errors.tolerance" set to > > "all". > > > > > > Cheers, > > > > > > Chris > > > > > > On Sat, Jul 16, 2022 at 12:26 PM Christopher Shannon < > > > christopher.l.shan...@gmail.com> wrote: > > > > > > > HI, > > > > > > > > I think I found a memory leak that was introduced in 3.2.0 in a > > Connector > > > > SourceTask. I created a JIRA: > > > > https://issues.apache.org/jira/browse/KAFKA-14079 and small PR with > a > > > fix: > > > > https://github.com/apache/kafka/pull/12412 > > > > > > > > I think this should be included in 3.2.1. It should also go into > 3.3.0 > > > but > > > > there was a lot of refactoring done there with the source task code > due > > > to > > > > KIP-618 so another PR needs to be done for that if this is merged. > > > > > > > > Chris > > > > > > > > On Fri, Jul 15, 2022 at 10:06 AM David Arthur > > wrote: > > > > > > > > > Here is the release plan for 3.2.1 > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.2.1 > > > > > > > > > > I am working on getting clarity on the one open blocker. Once that > is > > > > > resolved (or rescheduled for a future release), I will build the > > first > > > > > release candidate. > > > > > > > > > > -David > > > > > > > > > > On Thu, Jul 14, 2022 at 3:10 AM Luke Chen > wrote: > > > > > > > > > > > +1, Thanks David! > > > > > > > > > > > > On Thu, Jul 14, 2022 at 1:16 PM David Jacot < > david.ja...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > > +1. Thanks David. > > > > > > > > > > > > > > Le mer. 13 juil. 2022 à 23:43, José Armando García Sancio > > > > > > > a écrit : > > > > > > > > > > > > > > > +1. Thanks for volunteering David. > > > > > > > > > > > > > > > > -- > > > > > > > > -José > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > David Arthur > > > > > > > > > > > > > > >
Re: [DISCUSS] Apache Kafka 3.2.1 release
Hi, Chris S and Chris E, Thanks for quickly working on and reviewing the https://issues.apache.org/jira/browse/KAFKA-14079 issue mentioned above. The two PRs you created, one for the `trunk` branch and one for the `3.2` branch, have both been merged, and the issue has been marked as resolved. Best regards, Randall On Sun, Jul 17, 2022 at 5:44 PM Christopher Shannon < christopher.l.shan...@gmail.com> wrote: > Hi Chris E. > > Thanks for all the feedback earlier, I updated the PR based on your > comments and also pushed a second PR for trunk for 3.3.0 > > I agree the impact is high which is why I found the issue. I recently > turned on this feature and suddenly my connect worker/task kept > periodically falling over with OOM errors. Finally I took a heap dump and > saw a ton of submitted record objects in memory and started investigating > and that's how I figured out it was related to this new feature.. > > Chris > > On Sun, Jul 17, 2022 at 1:48 PM Chris Egerton > wrote: > > > Hi Chris, > > > > Good find, and thanks for filing a fix. I agree that we should get this > > into 3.2.1 if possible. The risk is fairly low (the functional parts of > the > > fix are just two lines long) and the impact of the bug is high for users > > who have configured source connectors with "errors.tolerance" set to > "all". > > > > Cheers, > > > > Chris > > > > On Sat, Jul 16, 2022 at 12:26 PM Christopher Shannon < > > christopher.l.shan...@gmail.com> wrote: > > > > > HI, > > > > > > I think I found a memory leak that was introduced in 3.2.0 in a > Connector > > > SourceTask. I created a JIRA: > > > https://issues.apache.org/jira/browse/KAFKA-14079 and small PR with a > > fix: > > > https://github.com/apache/kafka/pull/12412 > > > > > > I think this should be included in 3.2.1. It should also go into 3.3.0 > > but > > > there was a lot of refactoring done there with the source task code due > > to > > > KIP-618 so another PR needs to be done for that if this is merged. > > > > > > Chris > > > > > > On Fri, Jul 15, 2022 at 10:06 AM David Arthur > wrote: > > > > > > > Here is the release plan for 3.2.1 > > > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.2.1 > > > > > > > > I am working on getting clarity on the one open blocker. Once that is > > > > resolved (or rescheduled for a future release), I will build the > first > > > > release candidate. > > > > > > > > -David > > > > > > > > On Thu, Jul 14, 2022 at 3:10 AM Luke Chen wrote: > > > > > > > > > +1, Thanks David! > > > > > > > > > > On Thu, Jul 14, 2022 at 1:16 PM David Jacot > > > > > wrote: > > > > > > > > > > > +1. Thanks David. > > > > > > > > > > > > Le mer. 13 juil. 2022 à 23:43, José Armando García Sancio > > > > > > a écrit : > > > > > > > > > > > > > +1. Thanks for volunteering David. > > > > > > > > > > > > > > -- > > > > > > > -José > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > David Arthur > > > > > > > > > >
Re: [DISCUSS] Apache Kafka 3.2.1 release
Hi Chris E. Thanks for all the feedback earlier, I updated the PR based on your comments and also pushed a second PR for trunk for 3.3.0 I agree the impact is high which is why I found the issue. I recently turned on this feature and suddenly my connect worker/task kept periodically falling over with OOM errors. Finally I took a heap dump and saw a ton of submitted record objects in memory and started investigating and that's how I figured out it was related to this new feature.. Chris On Sun, Jul 17, 2022 at 1:48 PM Chris Egerton wrote: > Hi Chris, > > Good find, and thanks for filing a fix. I agree that we should get this > into 3.2.1 if possible. The risk is fairly low (the functional parts of the > fix are just two lines long) and the impact of the bug is high for users > who have configured source connectors with "errors.tolerance" set to "all". > > Cheers, > > Chris > > On Sat, Jul 16, 2022 at 12:26 PM Christopher Shannon < > christopher.l.shan...@gmail.com> wrote: > > > HI, > > > > I think I found a memory leak that was introduced in 3.2.0 in a Connector > > SourceTask. I created a JIRA: > > https://issues.apache.org/jira/browse/KAFKA-14079 and small PR with a > fix: > > https://github.com/apache/kafka/pull/12412 > > > > I think this should be included in 3.2.1. It should also go into 3.3.0 > but > > there was a lot of refactoring done there with the source task code due > to > > KIP-618 so another PR needs to be done for that if this is merged. > > > > Chris > > > > On Fri, Jul 15, 2022 at 10:06 AM David Arthur wrote: > > > > > Here is the release plan for 3.2.1 > > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.2.1 > > > > > > I am working on getting clarity on the one open blocker. Once that is > > > resolved (or rescheduled for a future release), I will build the first > > > release candidate. > > > > > > -David > > > > > > On Thu, Jul 14, 2022 at 3:10 AM Luke Chen wrote: > > > > > > > +1, Thanks David! > > > > > > > > On Thu, Jul 14, 2022 at 1:16 PM David Jacot > > > wrote: > > > > > > > > > +1. Thanks David. > > > > > > > > > > Le mer. 13 juil. 2022 à 23:43, José Armando García Sancio > > > > > a écrit : > > > > > > > > > > > +1. Thanks for volunteering David. > > > > > > > > > > > > -- > > > > > > -José > > > > > > > > > > > > > > > > > > > > > > > > -- > > > David Arthur > > > > > >
Re: [DISCUSS] Apache Kafka 3.2.1 release
Hi Chris, Good find, and thanks for filing a fix. I agree that we should get this into 3.2.1 if possible. The risk is fairly low (the functional parts of the fix are just two lines long) and the impact of the bug is high for users who have configured source connectors with "errors.tolerance" set to "all". Cheers, Chris On Sat, Jul 16, 2022 at 12:26 PM Christopher Shannon < christopher.l.shan...@gmail.com> wrote: > HI, > > I think I found a memory leak that was introduced in 3.2.0 in a Connector > SourceTask. I created a JIRA: > https://issues.apache.org/jira/browse/KAFKA-14079 and small PR with a fix: > https://github.com/apache/kafka/pull/12412 > > I think this should be included in 3.2.1. It should also go into 3.3.0 but > there was a lot of refactoring done there with the source task code due to > KIP-618 so another PR needs to be done for that if this is merged. > > Chris > > On Fri, Jul 15, 2022 at 10:06 AM David Arthur wrote: > > > Here is the release plan for 3.2.1 > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.2.1 > > > > I am working on getting clarity on the one open blocker. Once that is > > resolved (or rescheduled for a future release), I will build the first > > release candidate. > > > > -David > > > > On Thu, Jul 14, 2022 at 3:10 AM Luke Chen wrote: > > > > > +1, Thanks David! > > > > > > On Thu, Jul 14, 2022 at 1:16 PM David Jacot > > wrote: > > > > > > > +1. Thanks David. > > > > > > > > Le mer. 13 juil. 2022 à 23:43, José Armando García Sancio > > > > a écrit : > > > > > > > > > +1. Thanks for volunteering David. > > > > > > > > > > -- > > > > > -José > > > > > > > > > > > > > > > > > > -- > > David Arthur > > >
Re: [DISCUSS] Apache Kafka 3.2.1 release
HI, I think I found a memory leak that was introduced in 3.2.0 in a Connector SourceTask. I created a JIRA: https://issues.apache.org/jira/browse/KAFKA-14079 and small PR with a fix: https://github.com/apache/kafka/pull/12412 I think this should be included in 3.2.1. It should also go into 3.3.0 but there was a lot of refactoring done there with the source task code due to KIP-618 so another PR needs to be done for that if this is merged. Chris On Fri, Jul 15, 2022 at 10:06 AM David Arthur wrote: > Here is the release plan for 3.2.1 > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.2.1 > > I am working on getting clarity on the one open blocker. Once that is > resolved (or rescheduled for a future release), I will build the first > release candidate. > > -David > > On Thu, Jul 14, 2022 at 3:10 AM Luke Chen wrote: > > > +1, Thanks David! > > > > On Thu, Jul 14, 2022 at 1:16 PM David Jacot > wrote: > > > > > +1. Thanks David. > > > > > > Le mer. 13 juil. 2022 à 23:43, José Armando García Sancio > > > a écrit : > > > > > > > +1. Thanks for volunteering David. > > > > > > > > -- > > > > -José > > > > > > > > > > > > -- > David Arthur >
Re: [DISCUSS] Apache Kafka 3.2.1 release
Here is the release plan for 3.2.1 https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.2.1 I am working on getting clarity on the one open blocker. Once that is resolved (or rescheduled for a future release), I will build the first release candidate. -David On Thu, Jul 14, 2022 at 3:10 AM Luke Chen wrote: > +1, Thanks David! > > On Thu, Jul 14, 2022 at 1:16 PM David Jacot wrote: > > > +1. Thanks David. > > > > Le mer. 13 juil. 2022 à 23:43, José Armando García Sancio > > a écrit : > > > > > +1. Thanks for volunteering David. > > > > > > -- > > > -José > > > > > > -- David Arthur
Re: [DISCUSS] Apache Kafka 3.2.1 release
+1, Thanks David! On Thu, Jul 14, 2022 at 1:16 PM David Jacot wrote: > +1. Thanks David. > > Le mer. 13 juil. 2022 à 23:43, José Armando García Sancio > a écrit : > > > +1. Thanks for volunteering David. > > > > -- > > -José > > >
Re: [DISCUSS] Apache Kafka 3.2.1 release
+1. Thanks David. Le mer. 13 juil. 2022 à 23:43, José Armando García Sancio a écrit : > +1. Thanks for volunteering David. > > -- > -José >
Re: [DISCUSS] Apache Kafka 3.2.1 release
+1. Thanks for volunteering David. -- -José
