[jira] [Resolved] (KAFKA-13969) CVE-2022-24823 in netty 4.1.76.Final
[ https://issues.apache.org/jira/browse/KAFKA-13969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjin Lee resolved KAFKA-13969. - Resolution: Duplicate > CVE-2022-24823 in netty 4.1.76.Final > > > Key: KAFKA-13969 > URL: https://issues.apache.org/jira/browse/KAFKA-13969 > Project: Kafka > Issue Type: Bug >Affects Versions: 2.8.1 >Reporter: Dominique Mongelli >Priority: Minor > > Netty reported a new MEDIUM CVE: > [https://github.com/netty/netty/security/advisories/GHSA-269q-hmxg-m83q] > NVD: [https://nvd.nist.gov/vuln/detail/CVE-2022-24823] > It is fixed in netty 4.1.77.Final. > It should be noted that this CVE impacts applications running on Java 6 or > lower. > > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [kafka] dpcollins-google opened a new pull request, #12399: KAFKA-14063: Prevent malicious tiny payloads from causing OOMs with variably sized collections
dpcollins-google opened a new pull request, #12399: URL: https://github.com/apache/kafka/pull/12399 When parsing code receives a payload for a variable length field where the length is specified in the code as some arbitrarily large number (assume INT32_MAX for example) this will immediately try to allocate an ArrayList to hold this many elements, before checking whether this is a reasonable array size given the available data. The fix for this is to instead throw a runtime exception if the length of a variably sized container exceeds the amount of remaining data. Then, the worst a user can do is force the server to allocate 8x the size of the actual delivered data (if they claim there are N elements for a container of Objects (i.e. not a byte string) and each Object bottoms out in an 8 byte pointer in the ArrayList's backing array). This was identified by fuzzing the kafka request parsing code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (KAFKA-14063) Kafka message parsing can cause ooms with small antagonistic payloads
Daniel Collins created KAFKA-14063: -- Summary: Kafka message parsing can cause ooms with small antagonistic payloads Key: KAFKA-14063 URL: https://issues.apache.org/jira/browse/KAFKA-14063 Project: Kafka Issue Type: Bug Components: generator Affects Versions: 3.2.0 Reporter: Daniel Collins When parsing code receives a payload for a variable length field where the length is specified in the code as some arbitrarily large number (assume INT32_MAX for example) this will immediately try to allocate an ArrayList to hold this many elements, before checking whether this is a reasonable array size given the available data. The fix for this is to instead throw a runtime exception if the length of a variably sized container exceeds the amount of remaining data. Then, the worst a user can do is force the server to allocate 8x the size of the actual delivered data (if they claim there are N elements for a container of Objects (i.e. not a byte string) and each Object bottoms out in an 8 byte pointer in the ArrayList's backing array). This was identified by fuzzing the kafka request parsing code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [kafka] kirktrue commented on pull request #12398: KAFKA-14062: OAuth client token refresh fails with SASL extensions
kirktrue commented on PR #12398: URL: https://github.com/apache/kafka/pull/12398#issuecomment-1179598843 Thanks for the feedback, @dajac! > Could we add a unit test? Yes, I'd like to see that too. @emissionnebula: has the testing for this been mostly manual up to this point? I am willing to try to come up with one or more tests around this, but I can't commit to it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #12398: KAFKA-14062: OAuth client token refresh fails with SASL extensions
dajac commented on PR #12398: URL: https://github.com/apache/kafka/pull/12398#issuecomment-1179598266 Could we add a unit test? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] kirktrue commented on pull request #12398: KAFKA-14062: OAuth client token refresh fails with SASL extensions
kirktrue commented on PR #12398: URL: https://github.com/apache/kafka/pull/12398#issuecomment-1179594826 cc @omkreddy @YiDing-Duke @stanislavkozlovski @rite2nikhil @emissionnebula @ijuma -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] kirktrue opened a new pull request, #12398: KAFKA-14062: Remove SaslExtensions overridden equals() and hashCode() methods
kirktrue opened a new pull request, #12398: URL: https://github.com/apache/kafka/pull/12398 What Kafka client is adding and removing the SASL extensions alternatively at the time of token refresh. During the window when the extensions are not present in the subject. If a connection to a broker is reattempted, it fails with the error that the extensions are missing. Why In clients, a Subject object is maintained which contains two sets each for Private and Public Credentials. Public Credentials includes the extensions. These values are stored in a `SaslExtensions` object which internally maintains these in a HashMap. At the time of token refresh, a SaslExtensions object with these extensions is added to the public credentials set. As a next step, the refresh thread tries to logout the client for the older credentials. So it tries to remove the older token (private credential) and older SaslExtensions object (public credential) from the sets maintained in the Subject object. SaslExtensions Class overrides the `equals` and `hashcode` functions and directly calls the `equals` and `hashcode` functions of HashMap. So at the time refresh when a new SaslExtensions object is added, because the extension values don't change, it results in a no-op because the hashes of the existing SaslExtensions object and the new object will be equals. But in the logout step, the only SaslExtensions object present in the set gets removed. After removing the extensions in 1st refresh, the extensions will get added again at the time of 2nd refresh. So, this addition and removal keep happening alternatively. The addition and removal of private credentials (tokens) from Subject work just fine because the tokens are always different. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KAFKA-14062) OAuth client token refresh fails with SASL extensions
[
https://issues.apache.org/jira/browse/KAFKA-14062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirk True updated KAFKA-14062:
--
Summary: OAuth client token refresh fails with SASL extensions (was: OAuth
token refresh causes client authentication to fail)
> OAuth client token refresh fails with SASL extensions
> -
>
> Key: KAFKA-14062
> URL: https://issues.apache.org/jira/browse/KAFKA-14062
> Project: Kafka
> Issue Type: Bug
> Components: admin, clients, consumer, producer , security
>Affects Versions: 3.1.0, 3.2.0, 3.1.1, 3.3.0, 3.3
>Reporter: Kirk True
>Assignee: Kirk True
>Priority: Major
> Labels: OAuth
> Fix For: 3.1.2, 3.2.1
>
>
> While testing OAuth for Connect an issue surfaced where authentication that
> was successful initially fails during token refresh. This appears to be due
> to missing SASL extensions on refresh, though those extensions were present
> on initial authentication.
> During token refresh, the Kafka client adds and removes any SASL extensions.
> If a refresh is attempted during the window when the extensions are not
> present in the subject, the refresh fails with the following error:
> {code:java}
> [2022-04-11 20:33:43,250] INFO [AdminClient clientId=adminclient-8] Failed
> authentication with / (Authentication failed: 1 extensions are
> invalid! They are: xxx: Authentication failed)
> (org.apache.kafka.common.network.Selector){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-14062) OAuth token refresh causes client authentication to fail
Kirk True created KAFKA-14062:
-
Summary: OAuth token refresh causes client authentication to fail
Key: KAFKA-14062
URL: https://issues.apache.org/jira/browse/KAFKA-14062
Project: Kafka
Issue Type: Bug
Components: admin, clients, consumer, producer , security
Affects Versions: 3.1.1, 3.2.0, 3.1.0, 3.3.0, 3.3
Reporter: Kirk True
Assignee: Kirk True
Fix For: 3.1.2, 3.2.1
While testing OAuth for Connect an issue surfaced where authentication that was
successful initially fails during token refresh. This appears to be due to
missing SASL extensions on refresh, though those extensions were present on
initial authentication.
During token refresh, the Kafka client adds and removes any SASL extensions. If
a refresh is attempted during the window when the extensions are not present in
the subject, the refresh fails with the following error:
{code:java}
[2022-04-11 20:33:43,250] INFO [AdminClient clientId=adminclient-8] Failed
authentication with / (Authentication failed: 1 extensions are
invalid! They are: xxx: Authentication failed)
(org.apache.kafka.common.network.Selector){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [kafka] hachikuji commented on pull request #12380: MINOR: Get rid of agent checks in Jenkinsfile
hachikuji commented on PR #12380: URL: https://github.com/apache/kafka/pull/12380#issuecomment-1179582348 @showuon @ijuma @mumrah @dajac Thanks. I've removed these builds from the Jenkinsfile. I am not sure I know how to run them only against trunk. If one of you knows how to do that, let me know or we can consider doing it separately. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] hachikuji commented on pull request #12390: KAFKA-14055; Txn markers should not be removed by matching records in the offset map
hachikuji commented on PR #12390: URL: https://github.com/apache/kafka/pull/12390#issuecomment-1179580222 @junrao Thanks for reviewing. The ARM agent has been a bit unreliable. It looks like the ARM node has been offline for a while. There is some discussion [here](https://github.com/apache/kafka/pull/12380/files) about removing it from the PR build. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #12385: MINOR: Expose client information on RequestContext as additional public API beyond request logs (continuation of KIP 511)
dajac commented on PR #12385: URL: https://github.com/apache/kafka/pull/12385#issuecomment-1179576258 @methodmissing We also expose a metric with the number of clients per software name and version. Have you already checked this out? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KAFKA-13953) kafka Console consumer fails with CorruptRecordException
[
https://issues.apache.org/jira/browse/KAFKA-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564575#comment-17564575
]
Doguscan Namal commented on KAFKA-13953:
I put the part of the data that I was able to read. Changing the leadership of
the partition did not help, so I assume that all of them are corrupted. (data
retention is expired now, so unfortunately I could not verify it)
* It did not corrupted in the RecordBatch but rather in the record level. I
could read all of the data in that batch up to the problematic offset.
* It fails in the following line
[https://github.com/apache/kafka/blob/2.5.1/clients/src/main/java/org/apache/kafka/common/record/DefaultRecord.java#L296-L297]
. sizeOfBodyInBytes is read as -155493822, which should have been 1156.
* I overrode that value for this offset and it was able to read the record
data up to its 3rd header among its 5 headers, up to here
[https://github.com/apache/kafka/blob/2.5.1/clients/src/main/java/org/apache/kafka/common/record/DefaultRecord.java#L545]
* It failed because headerValueSize was read as 1991988702, which should have
been 13. I realized that `ByteUtils.readVarint` reading 5 bytes of data instead
of 1.
* Even overriding this headerValueSize did not help after this point. On next
read headerKeySize was read as -58 and processing failed.
This makes me think that Kafka only has the partial record for this offset. And
although there are more records in this batch after this point none of them are
accessible because this particular offset is corrupted.
Q1) Could it be possible for producer to send a corrupted batch? Due to
zero-copy maybe it just copied the received content to the data log file?
Q2) I also see `ProduceMessageConversionsPerSec` metric for this topic. Could
it be related to it i.e. message conversion?
* Topic is configured to use zstd compression
* Kafka version 2.5.1
Here is a data from my println when this record is being read:
recordStart:0 attributes: 0 timestampDelta: 391 timestamp 1656027641475 offset:
88062375700 sequence:40017233 key: java.nio.HeapByteBuffer[pos=0 lim=25
cap=1199] value: java.nio.HeapByteBuffer[pos=0 lim=961 cap=1149] numHeaders: 5
headerValueSize: 12 capacity: 147
headerValueSize: 8 capacity: 122
headerValueSize: 1991988702 capacity: 90
> kafka Console consumer fails with CorruptRecordException
> -
>
> Key: KAFKA-13953
> URL: https://issues.apache.org/jira/browse/KAFKA-13953
> Project: Kafka
> Issue Type: Bug
> Components: consumer, controller, core
>Affects Versions: 2.7.0
>Reporter: Aldan Brito
>Priority: Blocker
>
> Kafka consumer fails with corrupt record exception.
> {code:java}
> opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server *.*.*.*:
> --topic BQR-PULL-DEFAULT --from-beginning >
> /opt/nokia/kafka-zookeeper-clustering/kafka/topic-data/tmpsdh/dumptest
> [{*}2022-05-15 18:34:15,146]{*} ERROR Error processing message, terminating
> consumer process: (kafka.tools.ConsoleConsumer$)
> org.apache.kafka.common.KafkaException: Received exception when fetching the
> next record from BQR-PULL-DEFAULT-30. If needed, please seek past the record
> to continue consumption.
> at
> org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.fetchRecords(Fetcher.java:1577)
> at
> org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.access$1700(Fetcher.java:1432)
> at
> org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:684)
> at
> org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:635)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1276)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1237)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1210)
> at
> kafka.tools.ConsoleConsumer$ConsumerWrapper.receive(ConsoleConsumer.scala:438)
> at kafka.tools.ConsoleConsumer$.process(ConsoleConsumer.scala:104)
> at kafka.tools.ConsoleConsumer$.run(ConsoleConsumer.scala:78)
> at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:55)
> at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)
> Caused by: org.apache.kafka.common.errors.CorruptRecordException: Record size
> 0 is less than the minimum record overhead (14)
> Processed a total of 15765197 messages {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (KAFKA-14061) mirror-maker2: ability to store internal topics in one single Kafka cluster
[ https://issues.apache.org/jira/browse/KAFKA-14061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated KAFKA-14061: -- Description: I run mm2 in dedicated mode (not in Kafka Connect). It's possible to replicate from source to the target cluster without creating any internal topics on the source cluster. Thanks to KIP-716: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-716%3A+Allow+configuring+the+location+of+the+offset-syncs+topic+with+MirrorMaker2] But it's impossible to use mm2 for bi-directional replication in case when mm2 has no ACLs to create internal topics on one of two clusters. I want to replicate data between our Kafka cluster and many others, like a network hub. I have ACLs to create internal topics only on my own cluster, but not others. My proposal is to add configuration which allows to store all the internal topics of all clusters (heartbeats, checkpoints, configs, statuses, offsets) in one single Kafka cluster. was: It's possible to replicate from source to the target cluster without creating any internal topics on the source cluster. Thanks to KIP-716: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-716%3A+Allow+configuring+the+location+of+the+offset-syncs+topic+with+MirrorMaker2] But it's impossible to use mm2 for bi-directional replication in case when mm2 has no ACLs to create internal topics on one of two clusters. I want to replicate data between our Kafka cluster and many others, like a network hub. I have ACLs to create internal topics only on my own cluster, but not others. My proposal is to add configuration which allows to store all the internal topics of all clusters (heartbeats, checkpoints, configs, statuses, offsets) in one single Kafka cluster. > mirror-maker2: ability to store internal topics in one single Kafka cluster > --- > > Key: KAFKA-14061 > URL: https://issues.apache.org/jira/browse/KAFKA-14061 > Project: Kafka > Issue Type: Improvement >Reporter: Guram Savinov >Priority: Major > Labels: ACLs, mirror-maker > > I run mm2 in dedicated mode (not in Kafka Connect). > It's possible to replicate from source to the target cluster without creating > any internal topics on the source cluster. Thanks to KIP-716: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-716%3A+Allow+configuring+the+location+of+the+offset-syncs+topic+with+MirrorMaker2] > But it's impossible to use mm2 for bi-directional replication in case when > mm2 has no ACLs to create internal topics on one of two clusters. > I want to replicate data between our Kafka cluster and many others, like a > network hub. > I have ACLs to create internal topics only on my own cluster, but not others. > My proposal is to add configuration which allows to store all the internal > topics of all clusters (heartbeats, checkpoints, configs, statuses, offsets) > in one single Kafka cluster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-14061) mirror-maker2: ability to store internal topics in one single Kafka cluster
[ https://issues.apache.org/jira/browse/KAFKA-14061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guram Savinov updated KAFKA-14061: -- Summary: mirror-maker2: ability to store internal topics in one single Kafka cluster (was: Ability to store internal topics in one single Kafka cluster) > mirror-maker2: ability to store internal topics in one single Kafka cluster > --- > > Key: KAFKA-14061 > URL: https://issues.apache.org/jira/browse/KAFKA-14061 > Project: Kafka > Issue Type: Improvement >Reporter: Guram Savinov >Priority: Major > Labels: ACLs, mirror-maker > > It's possible to replicate from source to the target cluster without creating > any internal topics on the source cluster. Thanks to KIP-716: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-716%3A+Allow+configuring+the+location+of+the+offset-syncs+topic+with+MirrorMaker2] > But it's impossible to use mm2 for bi-directional replication in case when > mm2 has no ACLs to create internal topics on one of two clusters. > I want to replicate data between our Kafka cluster and many others, like a > network hub. > I have ACLs to create internal topics only on my own cluster, but not others. > My proposal is to add configuration which allows to store all the internal > topics of all clusters (heartbeats, checkpoints, configs, statuses, offsets) > in one single Kafka cluster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-14061) Ability to store internal topics in one single Kafka cluster
Guram Savinov created KAFKA-14061: - Summary: Ability to store internal topics in one single Kafka cluster Key: KAFKA-14061 URL: https://issues.apache.org/jira/browse/KAFKA-14061 Project: Kafka Issue Type: Improvement Reporter: Guram Savinov It's possible to replicate from source to the target cluster without creating any internal topics on the source cluster. Thanks to KIP-716: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-716%3A+Allow+configuring+the+location+of+the+offset-syncs+topic+with+MirrorMaker2] But it's impossible to use mm2 for bi-directional replication in case when mm2 has no ACLs to create internal topics on one of two clusters. I want to replicate data between our Kafka cluster and many others, like a network hub. I have ACLs to create internal topics only on my own cluster, but not others. My proposal is to add configuration which allows to store all the internal topics of all clusters (heartbeats, checkpoints, configs, statuses, offsets) in one single Kafka cluster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [kafka] showuon commented on pull request #12381: KAFKA-13474: Allow reconfiguration of SSL certs for broker to controller connection
showuon commented on PR #12381: URL: https://github.com/apache/kafka/pull/12381#issuecomment-1179518774 back port back to 3.2 branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KAFKA-13474) Regression in dynamic update of broker certificate
[
https://issues.apache.org/jira/browse/KAFKA-13474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Chen updated KAFKA-13474:
--
Fix Version/s: 3.2.1
> Regression in dynamic update of broker certificate
> --
>
> Key: KAFKA-13474
> URL: https://issues.apache.org/jira/browse/KAFKA-13474
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 2.7.0, 3.1.0, 2.7.2, 2.8.1, 3.0.0, 3.2.0, 3.1.1, 3.2.1
>Reporter: Igor Shipenkov
>Assignee: Divij Vaidya
>Priority: Critical
> Fix For: 3.3.0, 3.2.1
>
> Attachments: failed-controller-single-session-2029.pcap.gz
>
>
> h1. Problem
> It seems, after updating listener SSL certificate with dynamic broker
> configuration update, old certificate is somehow still used for broker client
> SSL factory. Because of this broker fails to create new connection to
> controller after old certificate expires.
> h1. History
> Back in KAFKA-8336 there was an issue, when client-side SSL factory wasn't
> updating certificate, when it was changed with dynamic configuration. That
> bug have been fixed in version 2.3 and I can confirm, that dynamic update
> worked for us with kafka 2.4. But now we have updated clusters to 2.7 and see
> this (or at least similar) problem again.
> h1. Affected versions
> First we've seen this on confluent 6.1.2, which (I think) based on kafka
> 2.7.0. Then I tried vanilla versions 2.7.0 and 2.7.2 and can reproduce
> problem on them just fine
> h1. How to reproduce
> * Have zookeeper somewhere (in my example it will be "10.88.0.21:2181").
> * Get vanilla version 2.7.2 (or 2.7.0) from
> [https://kafka.apache.org/downloads] .
> * Make basic broker config like this (don't forget to actually create
> log.dirs):
> {code:none}
> broker.id=1
> listeners=SSL://:9092
> advertised.listeners=SSL://localhost:9092
> log.dirs=/tmp/broker1/data
> zookeeper.connect=10.88.0.21:2181
> security.inter.broker.protocol=SSL
> ssl.protocol=TLSv1.2
> ssl.client.auth=required
> ssl.endpoint.identification.algorithm=
> ssl.keystore.type=PKCS12
> ssl.keystore.location=/tmp/broker1/secrets/broker1.keystore.p12
> ssl.keystore.password=changeme1
> ssl.key.password=changeme1
> ssl.truststore.type=PKCS12
> ssl.truststore.location=/tmp/broker1/secrets/truststore.p12
> ssl.truststore.password=changeme
> {code}
> (I use here TLS 1.2 just so I can see client certificate in TLS handshake in
> traffic dump, you will get same error with default TLS 1.3 too)
> ** Repeat this config for another 2 brokers, changing id, listener port and
> certificate accordingly.
> * Make basic client config (I use for it one of brokers' certificates):
> {code:none}
> security.protocol=SSL
> ssl.key.password=changeme1
> ssl.keystore.type=PKCS12
> ssl.keystore.location=/tmp/broker1/secrets/broker1.keystore.p12
> ssl.keystore.password=changeme1
> ssl.truststore.type=PKCS12
> ssl.truststore.location=/tmp/broker1/secrets/truststore.p12
> ssl.truststore.password=changeme
> ssl.endpoint.identification.algorithm=
> {code}
> * Create usual local self-signed PKI for test
> ** generate self-signed CA certificate and private key. Place certificate in
> truststore.
> ** create keys for broker certificates and create requests from them as
> usual (I'll use here same subject for all brokers)
> ** create 2 certificates as usual
> {code:bash}
> openssl x509 \
>-req -CAcreateserial -days 1 \
>-CA ca/ca-cert.pem -CAkey ca/ca-key.pem \
>-in broker1.csr -out broker1.crt
> {code}
> ** Use "faketime" utility to make third certificate expire soon:
> {code:bash}
> # date here is some point yesterday, so certificate will expire like 10-15
> minutes from now
> faketime "2021-11-23 10:15" openssl x509 \
>-req -CAcreateserial -days 1 \
>-CA ca/ca-cert.pem -CAkey ca/ca-key.pem \
>-in broker2.csr -out broker2.crt
> {code}
> ** create keystores from certificates and place them according to broker
> configs from earlier
> * Run 3 brokers with your configs like
> {code:bash}
> ./bin/kafka-server-start.sh server2.properties
> {code}
> (I start it here without daemon mode to see logs right on terminal - just use
> "tmux" or something to run 3 brokers simultaneously)
> ** you can check that one broker certificate will expire soon with
> {code:bash}
> openssl s_client -connect localhost:9093 -text | grep -A2 Valid
> {code}
> * Issue new certificate to replace one, which will expire soon, place it in
> keystore and replace old keystore with it.
> * Use dynamic configuration to make broker re-read keystore:
> {code:bash}
> ./bin/kafka-configs --command-config ssl.properties --bootstrap-server
> localhost:9092 --entity-type brokers --entity-name "2" --alter --add-config
>
[GitHub] [kafka] showuon merged pull request #12381: KAFKA-13474: Allow reconfiguration of SSL certs for broker to controller connection
showuon merged PR #12381: URL: https://github.com/apache/kafka/pull/12381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #12381: KAFKA-13474: Allow reconfiguration of SSL certs for broker to controller connection
divijvaidya commented on PR #12381: URL: https://github.com/apache/kafka/pull/12381#issuecomment-1179497415 @showuon please take a look again when you get a chance. The test failures are unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
