Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1493

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-14607) Move Scheduler/KafkaScheduler to server-common

[Ismael Juma (Jira)]

Ismael Juma created KAFKA-14607:
---

 Summary: Move Scheduler/KafkaScheduler to server-common
 Key: KAFKA-14607
 URL: https://issues.apache.org/jira/browse/KAFKA-14607
 Project: Kafka
  Issue Type: Sub-task
Reporter: Ismael Juma
Assignee: Ismael Juma
 Fix For: 3.5.0






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

[Ismael Juma]

Hi all,

I updated the KIP to make it explicit that the Java clients shipped with
Apache Kafka 4.0 would only be guaranteed to work with brokers running
Apache Kafka 2.1 or newer, since they too would no longer have access to
the removed protocol APIs. Thanks to Jason for asking about this aspect
offline.

I will leave this KIP open for discussion for another couple or weeks or
so. Please do share your thoughts given the high impact of the change.

Ismael

On Tue, Jan 3, 2023 at 8:17 AM Ismael Juma  wrote:

> Hi all,
>
> I would like to start a discussion regarding the removal of very old
> client protocol API versions in Apache Kafka 4.0 to improve maintainability
> & supportability of Kafka. Please take a look at the proposal:
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-896%3A+Remove+old+client+protocol+API+versions+in+Kafka+4.0
>
> Ismael
>

Re: [ANNOUNCE] New committer: Satish Duggana

[Satish Duggana]

Thanks again, everyone!

On Sat, 31 Dec 2022 at 01:27, Dhiraj Dwarapudi
 wrote:
>
> Congratulations Sato :-)
>
> Regards,
> Dhiraj
>
> On Thu, Dec 29, 2022 at 10:17 AM Bill Bejeck  wrote:
>
> > Congratulations Satish!
> >
> > -Bill
> >
> > On Thu, Dec 29, 2022 at 11:05 AM Satish Duggana 
> > wrote:
> >
> > > Thank you, everyone!
> > >
> > > On Wed, 28 Dec 2022 at 00:02, Matthias J. Sax  wrote:
> > > >
> > > > Congrats!
> > > >
> > > > On 12/27/22 10:20 AM, Kirk True wrote:
> > > > > Congrats, Satish!
> > > > >
> > > > > On Fri, Dec 23, 2022, at 10:07 AM, Jun Rao wrote:
> > > > >> Hi, Everyone,
> > > > >>
> > > > >> The PMC of Apache Kafka is pleased to announce a new Kafka committer
> > > Satish
> > > > >> Duggana.
> > > > >>
> > > > >> Satish has been a long time Kafka contributor since 2017. He is the
> > > main
> > > > >> driver behind KIP-405 that integrates Kafka with remote storage, a
> > > > >> significant and much anticipated feature in Kafka.
> > > > >>
> > > > >> Congratulations, Satish!
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> Jun (on behalf of the Apache Kafka PMC)
> > > > >>
> > > > >
> > >
> >

Re: [ANNOUNCE] New committer: Edoardo Comar

[Satish Duggana]

Congratulations, Edorado!

On Sun, 8 Jan 2023 at 00:15, Viktor Somogyi-Vass
 wrote:
>
> Congrats Edoardo!
>
> On Sat, Jan 7, 2023, 18:15 Bill Bejeck  wrote:
>
> > Congratulations, Edoardo!
> >
> > -Bill
> >
> > On Sat, Jan 7, 2023 at 12:11 PM John Roesler  wrote:
> >
> > > Congrats, Edoardo!
> > > -John
> > >
> > > On Fri, Jan 6, 2023, at 20:47, Matthias J. Sax wrote:
> > > > Congrats!
> > > >
> > > > On 1/6/23 5:15 PM, Luke Chen wrote:
> > > >> Congratulations, Edoardo!
> > > >>
> > > >> Luke
> > > >>
> > > >> On Sat, Jan 7, 2023 at 7:58 AM Mickael Maison <
> > mickael.mai...@gmail.com
> > > >
> > > >> wrote:
> > > >>
> > > >>> Congratulations Edo!
> > > >>>
> > > >>>
> > > >>> On Sat, Jan 7, 2023 at 12:05 AM Jun Rao 
> > > wrote:
> > > 
> > >  Hi, Everyone,
> > > 
> > >  The PMC of Apache Kafka is pleased to announce a new Kafka committer
> > > >>> Edoardo
> > >  Comar.
> > > 
> > >  Edoardo has been a long time Kafka contributor since 2016. His major
> > >  contributions are the following.
> > > 
> > >  KIP-302: Enable Kafka clients to use all DNS resolved IP addresses
> > >  KIP-277: Fine Grained ACL for CreateTopics API
> > >  KIP-136: Add Listener name to SelectorMetrics tags
> > > 
> > >  Congratulations, Edoardo!
> > > 
> > >  Thanks,
> > > 
> > >  Jun (on behalf of the Apache Kafka PMC)
> > > >>>
> > > >>
> > >
> >

Re: [DISCUSS] KIP-894: Use incrementalAlterConfigs API for syncing topic configurations

[Ismael Juma]

Hi,

Thanks for the KIP. Have we considered using incremental alter configs by
default and fallback to the legacy one if the former is unavailable?

The config could have 3 possible values: requested, required, never. The
default would be requested. In 4.0, this could would be removed and it
would effectively be required.

Ismael

On Sat, Jan 7, 2023, 10:03 AM John Roesler  wrote:

> Hi Tina,
>
> Thanks for the KIP!
>
> I hope someone with prior MM or Kafka config experience is able to chime
> in here; I have neither.
>
> I took a look at your KIP, and it makes sense to me. I also think your
> migration plan is a good one.
>
> One suggestion: I'm not sure how concerned you are about people's ability
> to migrate, but if you want to make it as smooth as possible, you could add
> one more step. In the 4.0 release, while removing
> `use.incremental.alter.configs`, you can also add
> `use.legacy.alter.configs` to give users a path to continue using the old
> behavior even after the default changes.
>
> Clearly, this will prolong the deprecation period, with implications on
> code maintenance, so there is some downside. But generally, I've found
> going above and beyond to support smooth upgrades for users to be well
> worth it in the long run.
>
> Thanks again,
> -John
>
>
> On Fri, Jan 6, 2023, at 05:49, Gantigmaa Selenge wrote:
> > Hi everyone,
> >
> > I would like to start a discussion on the MirrorMaker update that
> > proposes
> > replacing the deprecated alterConfigs API with the
> > incrementalAlterConfigs
> > API for syncing topic configurations. Please take a look at the proposal
> > here:
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-894%3A+Use+incrementalAlterConfigs+API+for+syncing+topic+configurations
> >
> >
> > Regards,
> > Tina
>

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1492

[Apache Jenkins Server]

See 

Re: [ANNOUNCE] New committer: Edoardo Comar

[Viktor Somogyi-Vass]

Congrats Edoardo!

On Sat, Jan 7, 2023, 18:15 Bill Bejeck  wrote:

> Congratulations, Edoardo!
>
> -Bill
>
> On Sat, Jan 7, 2023 at 12:11 PM John Roesler  wrote:
>
> > Congrats, Edoardo!
> > -John
> >
> > On Fri, Jan 6, 2023, at 20:47, Matthias J. Sax wrote:
> > > Congrats!
> > >
> > > On 1/6/23 5:15 PM, Luke Chen wrote:
> > >> Congratulations, Edoardo!
> > >>
> > >> Luke
> > >>
> > >> On Sat, Jan 7, 2023 at 7:58 AM Mickael Maison <
> mickael.mai...@gmail.com
> > >
> > >> wrote:
> > >>
> > >>> Congratulations Edo!
> > >>>
> > >>>
> > >>> On Sat, Jan 7, 2023 at 12:05 AM Jun Rao 
> > wrote:
> > 
> >  Hi, Everyone,
> > 
> >  The PMC of Apache Kafka is pleased to announce a new Kafka committer
> > >>> Edoardo
> >  Comar.
> > 
> >  Edoardo has been a long time Kafka contributor since 2016. His major
> >  contributions are the following.
> > 
> >  KIP-302: Enable Kafka clients to use all DNS resolved IP addresses
> >  KIP-277: Fine Grained ACL for CreateTopics API
> >  KIP-136: Add Listener name to SelectorMetrics tags
> > 
> >  Congratulations, Edoardo!
> > 
> >  Thanks,
> > 
> >  Jun (on behalf of the Apache Kafka PMC)
> > >>>
> > >>
> >
>

[jira] [Created] (KAFKA-14606) Use virtual threads to publish Kafka records

[Bart De Neuter (Jira)]

Bart De Neuter created KAFKA-14606:
--

 Summary: Use virtual threads to publish Kafka records
 Key: KAFKA-14606
 URL: https://issues.apache.org/jira/browse/KAFKA-14606
 Project: Kafka
  Issue Type: Improvement
  Components: producer 
Reporter: Bart De Neuter


Since JDK 19, virtual threads have been added to the JDK as a preview:

[https://openjdk.org/jeps/425]

Virtual threads allows you to use the hardware optimal as it is a lightweight 
thread that runs on a carrier thread (OS thread). When IO happens, the carrier 
thread is not blocked and can continue doing other work. Currently it doesn't 
seem to be possible to make 
`org.apache.kafka.clients.producer.internals.Sender` run on a virtual thread.

An instance of `org.apache.kafka.common.utils.KafkaThread` is being started. 

Is it possible to give the possibility to use virtual threads for publishing 
records?

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] KIP-894: Use incrementalAlterConfigs API for syncing topic configurations

[John Roesler]

Hi Tina,

Thanks for the KIP!

I hope someone with prior MM or Kafka config experience is able to chime in 
here; I have neither.

I took a look at your KIP, and it makes sense to me. I also think your 
migration plan is a good one.

One suggestion: I'm not sure how concerned you are about people's ability to 
migrate, but if you want to make it as smooth as possible, you could add one 
more step. In the 4.0 release, while removing `use.incremental.alter.configs`, 
you can also add `use.legacy.alter.configs` to give users a path to continue 
using the old behavior even after the default changes.

Clearly, this will prolong the deprecation period, with implications on code 
maintenance, so there is some downside. But generally, I've found going above 
and beyond to support smooth upgrades for users to be well worth it in the long 
run.

Thanks again,
-John


On Fri, Jan 6, 2023, at 05:49, Gantigmaa Selenge wrote:
> Hi everyone,
>
> I would like to start a discussion on the MirrorMaker update that 
> proposes
> replacing the deprecated alterConfigs API with the 
> incrementalAlterConfigs
> API for syncing topic configurations. Please take a look at the proposal
> here:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-894%3A+Use+incrementalAlterConfigs+API+for+syncing+topic+configurations
>
>
> Regards,
> Tina

Re: [ANNOUNCE] New committer: Edoardo Comar

[Bill Bejeck]

Congratulations, Edoardo!

-Bill

On Sat, Jan 7, 2023 at 12:11 PM John Roesler  wrote:

> Congrats, Edoardo!
> -John
>
> On Fri, Jan 6, 2023, at 20:47, Matthias J. Sax wrote:
> > Congrats!
> >
> > On 1/6/23 5:15 PM, Luke Chen wrote:
> >> Congratulations, Edoardo!
> >>
> >> Luke
> >>
> >> On Sat, Jan 7, 2023 at 7:58 AM Mickael Maison  >
> >> wrote:
> >>
> >>> Congratulations Edo!
> >>>
> >>>
> >>> On Sat, Jan 7, 2023 at 12:05 AM Jun Rao 
> wrote:
> 
>  Hi, Everyone,
> 
>  The PMC of Apache Kafka is pleased to announce a new Kafka committer
> >>> Edoardo
>  Comar.
> 
>  Edoardo has been a long time Kafka contributor since 2016. His major
>  contributions are the following.
> 
>  KIP-302: Enable Kafka clients to use all DNS resolved IP addresses
>  KIP-277: Fine Grained ACL for CreateTopics API
>  KIP-136: Add Listener name to SelectorMetrics tags
> 
>  Congratulations, Edoardo!
> 
>  Thanks,
> 
>  Jun (on behalf of the Apache Kafka PMC)
> >>>
> >>
>

Re: [ANNOUNCE] New committer: Edoardo Comar

[John Roesler]

Congrats, Edoardo!
-John

On Fri, Jan 6, 2023, at 20:47, Matthias J. Sax wrote:
> Congrats!
>
> On 1/6/23 5:15 PM, Luke Chen wrote:
>> Congratulations, Edoardo!
>> 
>> Luke
>> 
>> On Sat, Jan 7, 2023 at 7:58 AM Mickael Maison 
>> wrote:
>> 
>>> Congratulations Edo!
>>>
>>>
>>> On Sat, Jan 7, 2023 at 12:05 AM Jun Rao  wrote:

 Hi, Everyone,

 The PMC of Apache Kafka is pleased to announce a new Kafka committer
>>> Edoardo
 Comar.

 Edoardo has been a long time Kafka contributor since 2016. His major
 contributions are the following.

 KIP-302: Enable Kafka clients to use all DNS resolved IP addresses
 KIP-277: Fine Grained ACL for CreateTopics API
 KIP-136: Add Listener name to SelectorMetrics tags

 Congratulations, Edoardo!

 Thanks,

 Jun (on behalf of the Apache Kafka PMC)
>>>
>>

[jira] [Created] (KAFKA-14605) Change the log level to warn when logIfAllowed is set.

[Beibei Zhao (Jira)]

Beibei Zhao created KAFKA-14605:
---

 Summary: Change the log level to warn when logIfAllowed is set.
 Key: KAFKA-14605
 URL: https://issues.apache.org/jira/browse/KAFKA-14605
 Project: Kafka
  Issue Type: Improvement
Reporter: Beibei Zhao


StandardAuthorizer log at INFO level when logIfDenied is set(otherwise, we log 
at TRACE), but at debug level when logIfAllowed is set.
Since audit log is security log, it should be logged at default verbosity 
level, not debug or trace when logIfAllowed is set.
So I think, log at INFO when allow, and log at WARN when deny is better.

{code:java}
private void logAuditMessage(
.. ) {
switch (rule.result()) {
case ALLOWED:
if (action.logIfAllowed() && auditLog.isDebugEnabled()) {
auditLog.debug(..); // info
} else if (auditLog.isTraceEnabled()) {
auditLog.trace(buildAuditMessage(principal, requestContext, 
action, rule));
}
return;

case DENIED:
if (action.logIfDenied()) {
auditLog.info(..); // warn
} else if (auditLog.isTraceEnabled()) {
auditLog.trace(buildAuditMessage(principal, requestContext, 
action, rule));
}
}
}
{code}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-14604) SASL session expiration time will be overflowed when calculation

[Luke Chen (Jira)]

Luke Chen created KAFKA-14604:
-

 Summary: SASL session expiration time will be overflowed when 
calculation
 Key: KAFKA-14604
 URL: https://issues.apache.org/jira/browse/KAFKA-14604
 Project: Kafka
  Issue Type: Bug
Affects Versions: 3.3.1
Reporter: Luke Chen
Assignee: Luke Chen


When sasl server of client set a large expiration time, the timeout value might 
be overflowed, and cause the session timeout immediately.

 

[Here|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/security/authenticator/SaslServerAuthenticator.java#L694]'s
 the sasl server timeout's calculation

[Here|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/security/authenticator/SaslClientAuthenticator.java#L692]'s
 the sasl client timeout's calculation

 

something like this:
{code:java}
sessionExpirationTimeNanos = authenticationEndNanos + 1000 * 1000 * 
sessionLifetimeMs; {code}
So, if the configured or returned sessionLifetimeMs is a large number, after 
the calculation, it'll be a negative value, and cause the session timeout each 
check.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1491

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-14603) Move KafkaMetricsGroup to server-common module.

[Satish Duggana (Jira)]

Satish Duggana created KAFKA-14603:
--

 Summary: Move KafkaMetricsGroup to server-common module.
 Key: KAFKA-14603
 URL: https://issues.apache.org/jira/browse/KAFKA-14603
 Project: Kafka
  Issue Type: Sub-task
  Components: core
Reporter: Satish Duggana






--
This message was sent by Atlassian Jira
(v8.20.10#820010)