[jira] [Comment Edited] (KAFKA-15783) Unable to set batchSize in log4j2 Kafka appender
[
https://issues.apache.org/jira/browse/KAFKA-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782928#comment-17782928
]
Diego Pettisani edited comment on KAFKA-15783 at 11/4/23 9:33 PM:
--
Sorry,
only now I verified that we can modify {{ linger.ms}} and {{batch.size}} in
this way:
{code:xml}
localhost:9092
30
16384
{code}
I try to close this issue.
Thanks at all
was (Author: diepet):
Sorry,
only now I verified that we can modify{{ linger.ms}} and {{batch.size}} in this
way:
{code:xml}
localhost:9092
30
16384
{code}
I try to close this issue.
Thanks at all
> Unable to set batchSize in log4j2 Kafka appender
>
>
> Key: KAFKA-15783
> URL: https://issues.apache.org/jira/browse/KAFKA-15783
> Project: Kafka
> Issue Type: Bug
> Components: logging
>Affects Versions: 3.6.0
>Reporter: Diego Pettisani
>Priority: Minor
>
> When I try to configure the batchSize of the Kafka log4j2 appender the
> application logs the following error:
> {noformat}
> ERROR StatusConsoleListener Kafka contains an invalid element or attribute
> "batchSize"
> {noformat}
> This is an example of configuration that fails:
> {code:xml}
>
>
>
>
> pattern="%d{HH:mm:ss.SSS} [%t] %-5level
> %logger{36} - %msg%n" />
>
> batchSize="8192">
> pattern="%d{HH:mm:ss.SSS} [%t] %-5level
> %logger{36} - %msg%n" />
>name="bootstrap.servers">localhost:9092
>
>
>
>
>
>
>
>
>
>
> {code}
> Please note that other parameters like {{syncSend}} work fine.
> Could be possible that log4j2 expects this field:
> https://github.com/apache/kafka/blob/3.6.0/log4j-appender/src/main/java/org/apache/kafka/log4jappender/KafkaLog4jAppender.java#L83C11-L83C11
> as a String for working fine?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-15783) Unable to set batchSize in log4j2 Kafka appender
[
https://issues.apache.org/jira/browse/KAFKA-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782928#comment-17782928
]
Diego Pettisani edited comment on KAFKA-15783 at 11/4/23 9:33 PM:
--
Sorry,
only now I verified that we can modify {{linger.ms}} and {{batch.size}} in this
way:
{code:xml}
localhost:9092
30
16384
{code}
I try to close this issue.
Thanks at all
was (Author: diepet):
Sorry,
only now I verified that we can modify {{ linger.ms}} and {{batch.size}} in
this way:
{code:xml}
localhost:9092
30
16384
{code}
I try to close this issue.
Thanks at all
> Unable to set batchSize in log4j2 Kafka appender
>
>
> Key: KAFKA-15783
> URL: https://issues.apache.org/jira/browse/KAFKA-15783
> Project: Kafka
> Issue Type: Bug
> Components: logging
>Affects Versions: 3.6.0
>Reporter: Diego Pettisani
>Priority: Minor
>
> When I try to configure the batchSize of the Kafka log4j2 appender the
> application logs the following error:
> {noformat}
> ERROR StatusConsoleListener Kafka contains an invalid element or attribute
> "batchSize"
> {noformat}
> This is an example of configuration that fails:
> {code:xml}
>
>
>
>
> pattern="%d{HH:mm:ss.SSS} [%t] %-5level
> %logger{36} - %msg%n" />
>
> batchSize="8192">
> pattern="%d{HH:mm:ss.SSS} [%t] %-5level
> %logger{36} - %msg%n" />
>name="bootstrap.servers">localhost:9092
>
>
>
>
>
>
>
>
>
>
> {code}
> Please note that other parameters like {{syncSend}} work fine.
> Could be possible that log4j2 expects this field:
> https://github.com/apache/kafka/blob/3.6.0/log4j-appender/src/main/java/org/apache/kafka/log4jappender/KafkaLog4jAppender.java#L83C11-L83C11
> as a String for working fine?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-15783) Unable to set batchSize in log4j2 Kafka appender
[
https://issues.apache.org/jira/browse/KAFKA-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782928#comment-17782928
]
Diego Pettisani commented on KAFKA-15783:
-
Sorry,
only now I verified that we can modify linger.ms and batch.size in this way:
{code:xml}
localhost:9092
30
16384
{code}
I try to close this issue.
Thanks at all
> Unable to set batchSize in log4j2 Kafka appender
>
>
> Key: KAFKA-15783
> URL: https://issues.apache.org/jira/browse/KAFKA-15783
> Project: Kafka
> Issue Type: Bug
> Components: logging
>Affects Versions: 3.6.0
>Reporter: Diego Pettisani
>Priority: Minor
>
> When I try to configure the batchSize of the Kafka log4j2 appender the
> application logs the following error:
> {noformat}
> ERROR StatusConsoleListener Kafka contains an invalid element or attribute
> "batchSize"
> {noformat}
> This is an example of configuration that fails:
> {code:xml}
>
>
>
>
> pattern="%d{HH:mm:ss.SSS} [%t] %-5level
> %logger{36} - %msg%n" />
>
> batchSize="8192">
> pattern="%d{HH:mm:ss.SSS} [%t] %-5level
> %logger{36} - %msg%n" />
>name="bootstrap.servers">localhost:9092
>
>
>
>
>
>
>
>
>
>
> {code}
> Please note that other parameters like {{syncSend}} work fine.
> Could be possible that log4j2 expects this field:
> https://github.com/apache/kafka/blob/3.6.0/log4j-appender/src/main/java/org/apache/kafka/log4jappender/KafkaLog4jAppender.java#L83C11-L83C11
> as a String for working fine?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-15783) Unable to set batchSize in log4j2 Kafka appender
[
https://issues.apache.org/jira/browse/KAFKA-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782928#comment-17782928
]
Diego Pettisani edited comment on KAFKA-15783 at 11/4/23 9:32 PM:
--
Sorry,
only now I verified that we can modify{{ linger.ms}} and {{batch.size}} in this
way:
{code:xml}
localhost:9092
30
16384
{code}
I try to close this issue.
Thanks at all
was (Author: diepet):
Sorry,
only now I verified that we can modify linger.ms and batch.size in this way:
{code:xml}
localhost:9092
30
16384
{code}
I try to close this issue.
Thanks at all
> Unable to set batchSize in log4j2 Kafka appender
>
>
> Key: KAFKA-15783
> URL: https://issues.apache.org/jira/browse/KAFKA-15783
> Project: Kafka
> Issue Type: Bug
> Components: logging
>Affects Versions: 3.6.0
>Reporter: Diego Pettisani
>Priority: Minor
>
> When I try to configure the batchSize of the Kafka log4j2 appender the
> application logs the following error:
> {noformat}
> ERROR StatusConsoleListener Kafka contains an invalid element or attribute
> "batchSize"
> {noformat}
> This is an example of configuration that fails:
> {code:xml}
>
>
>
>
> pattern="%d{HH:mm:ss.SSS} [%t] %-5level
> %logger{36} - %msg%n" />
>
> batchSize="8192">
> pattern="%d{HH:mm:ss.SSS} [%t] %-5level
> %logger{36} - %msg%n" />
>name="bootstrap.servers">localhost:9092
>
>
>
>
>
>
>
>
>
>
> {code}
> Please note that other parameters like {{syncSend}} work fine.
> Could be possible that log4j2 expects this field:
> https://github.com/apache/kafka/blob/3.6.0/log4j-appender/src/main/java/org/apache/kafka/log4jappender/KafkaLog4jAppender.java#L83C11-L83C11
> as a String for working fine?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (KAFKA-15783) Unable to set batchSize in log4j2 Kafka appender
[
https://issues.apache.org/jira/browse/KAFKA-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Diego Pettisani resolved KAFKA-15783.
-
Resolution: Not A Problem
> Unable to set batchSize in log4j2 Kafka appender
>
>
> Key: KAFKA-15783
> URL: https://issues.apache.org/jira/browse/KAFKA-15783
> Project: Kafka
> Issue Type: Bug
> Components: logging
>Affects Versions: 3.6.0
>Reporter: Diego Pettisani
>Priority: Minor
>
> When I try to configure the batchSize of the Kafka log4j2 appender the
> application logs the following error:
> {noformat}
> ERROR StatusConsoleListener Kafka contains an invalid element or attribute
> "batchSize"
> {noformat}
> This is an example of configuration that fails:
> {code:xml}
>
>
>
>
> pattern="%d{HH:mm:ss.SSS} [%t] %-5level
> %logger{36} - %msg%n" />
>
> batchSize="8192">
> pattern="%d{HH:mm:ss.SSS} [%t] %-5level
> %logger{36} - %msg%n" />
>name="bootstrap.servers">localhost:9092
>
>
>
>
>
>
>
>
>
>
> {code}
> Please note that other parameters like {{syncSend}} work fine.
> Could be possible that log4j2 expects this field:
> https://github.com/apache/kafka/blob/3.6.0/log4j-appender/src/main/java/org/apache/kafka/log4jappender/KafkaLog4jAppender.java#L83C11-L83C11
> as a String for working fine?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]
philipnee commented on code in PR #14690:
URL: https://github.com/apache/kafka/pull/14690#discussion_r1382450219
##
clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java:
##
@@ -181,33 +262,465 @@ public void
updateState(ConsumerGroupHeartbeatResponseData response) {
public void transitionToFenced() {
resetEpoch();
transitionTo(MemberState.FENCED);
+
+// Release assignment
+CompletableFuture callbackResult =
invokeOnPartitionsRevokedOrLostToReleaseAssignment();
+callbackResult.whenComplete((result, error) -> {
+if (error != null) {
+log.debug("OnPartitionsLost callback invocation failed while
releasing assignment" +
+"after member got fenced. Member will rejoin the group
anyways.", error);
+}
+subscriptions.assignFromSubscribed(Collections.emptySet());
+transitionToJoining();
+});
+}
+
+/**
+ * {@inheritDoc}
+ */
+@Override
+public void transitionToFatal() {
+log.error("Member {} transitioned to {} state", memberId,
MemberState.FATAL);
+
+// Update epoch to indicate that the member is not in the group
anymore, so that the
+// onPartitionsLost is called to release assignment.
+memberEpoch = LEAVE_GROUP_EPOCH;
+invokeOnPartitionsRevokedOrLostToReleaseAssignment();
+
+transitionTo(MemberState.FATAL);
+}
+
+/**
+ * {@inheritDoc}
+ */
+@Override
+public void transitionToJoining() {
+resetEpoch();
+transitionTo(MemberState.JOINING);
}
/**
* {@inheritDoc}
*/
@Override
-public void transitionToFailed() {
-log.error("Member {} transitioned to {} state", memberId,
MemberState.FAILED);
-transitionTo(MemberState.FAILED);
+public CompletableFuture leaveGroup() {
+transitionTo(MemberState.LEAVING);
+
+CompletableFuture callbackResult =
invokeOnPartitionsRevokedOrLostToReleaseAssignment();
+callbackResult.whenComplete((result, error) -> {
+
+// Clear the subscription, no matter if the callback execution
failed or succeeded.
+subscriptions.assignFromSubscribed(Collections.emptySet());
+
+// Transition to ensure that a heartbeat request is sent out to
effectively leave the
+// group (even in the case where the member had no assignment to
release or when the
+// callback execution failed.)
+transitionToSendingLeaveGroup();
+
+});
+
+// Return callback future to indicate that the leave group is done
when the callbacks
+// complete, without waiting for the heartbeat to be sent out. (Best
effort to send it
+// but do not hold the leave group operation for it)
+return callbackResult;
}
+/**
+ * Release member assignment by calling the user defined callbacks for
onPartitionsRevoked or
+ * onPartitionsLost.
+ *
+ * If the member is part of the group (epoch > 0), this will
invoke onPartitionsRevoked.
+ * This will be the case when releasing assignment because the member
is intentionally
+ * leaving the group (after a call to unsubscribe)
+ *
+ * If the member is not part of the group (epoch <=0), this will
invoke onPartitionsLost.
+ * This will be the case when releasing assignment after being fenced
.
+ *
+ *
+ * @return Future that will complete when the callback execution completes.
+ */
+private CompletableFuture
invokeOnPartitionsRevokedOrLostToReleaseAssignment() {
+SortedSet droppedPartitions = new
TreeSet<>(COMPARATOR);
+droppedPartitions.addAll(subscriptions.assignedPartitions());
+
+CompletableFuture callbackResult;
+if (droppedPartitions.isEmpty()) {
+// No assignment to release
+callbackResult = CompletableFuture.completedFuture(null);
+} else {
+// Release assignment
+if (memberEpoch > 0) {
+// Member is part of the group. Invoke onPartitionsRevoked.
+callbackResult = revokePartitions(droppedPartitions);
+} else {
+// Member is not part of the group anymore. Invoke
onPartitionsLost.
+callbackResult =
invokeOnPartitionsLostCallback(droppedPartitions);
+}
+}
+return callbackResult;
+}
+
+/**
+ * Reset member epoch to the value required for the leave the group
heartbeat request, and
+ * transition to the {@link MemberState#SENDING_LEAVE_REQUEST} state so
that a heartbeat
+ * request is sent out with it.
+ */
+private void transitionToSendingLeaveGroup() {
+memberEpoch = leaveGroupEpoch();
+currentAssignment = new HashSet<>();
+targetAssignment = Optional.empty();
+
Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]
philipnee commented on PR #14690: URL: https://github.com/apache/kafka/pull/14690#issuecomment-1793538245 Hi @AndrewJSchofield - Thank you for your inputs here, your point is absolutely valid in terms of the abstraction it provides. My main concern is the rebalance callback invocation. Maybe I misunderstood the PR, but I think the state transition in the whenComplete can be completed by the main thread if the user supplies a callback, unless the callback is not supplied. I will double-check the syntax. Could you be more specific about `using CF to enqueue an event`? Do you mean by enqueuing a callback completion signal to some background thread queue, to notify the thread to perform the subsequent action? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]
AndrewJSchofield commented on PR #14690: URL: https://github.com/apache/kafka/pull/14690#issuecomment-1793496177 Hi @philipnee, thanks for you comment. It does raise an interesting point. My view is that CompletableFuture provides a really nice abstraction for asynchronous processing. Using it in this PR is entirely appropriate. However, I also take your overall point. The threading for CompletableFuture depends upon the details of how it is used. If a future is completed on the "wrong" thread, then processing which is dependent upon completion of that future will also execute on the same "wrong" thread. If one of the more complex methods such as `supplyAsync` is used, it runs on a thread from a worker pool. I don't think we're using that pattern here. If we actually need to achieve signalling back to a specific thread, we could use the CF to enqueue an event. That compromise seems pretty sensible to me, but I'm not sure whether we need this with the PR as written. I need to re-read the code before I'll be certain. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KAFKA-15764) Missing tests for transactions
[ https://issues.apache.org/jira/browse/KAFKA-15764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782908#comment-17782908 ] Justine Olshan commented on KAFKA-15764: I created an example of what I was saying above but all the tests passed with the current code. I wonder if I need to made some adjustments in the test. I have a WIP branch here: [https://github.com/apache/kafka/pull/14702] > Missing tests for transactions > -- > > Key: KAFKA-15764 > URL: https://issues.apache.org/jira/browse/KAFKA-15764 > Project: Kafka > Issue Type: Test >Affects Versions: 3.6.0 >Reporter: Divij Vaidya >Assignee: hudeqi >Priority: Major > > As part of https://issues.apache.org/jira/browse/KAFKA-15653 we found a bug > which was not caught by any of the existing tests in Apache Kafka. > However, the bug is consistently reproducible if we use the test suite shared > by [~twmb] at > https://issues.apache.org/jira/browse/KAFKA-15653?focusedCommentId=1846=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-1846 > > As part of this JIRA, we want to do add relevant test in Apache Kafka which > should have failed. We can look at the test suite that franz/go repository is > running, more specifically the test suite that is executed by "go test -run > Txn".(see repro.sh attached to the comment linked above). > With reference to code in repro.sh, we don't need to do docker compose down > or up in the while loop Hence, it's ok to simply do > ``` > while go test -run Txn > FRANZ_FAIL; do > done > ``` > The exit criteria for this Jira should be to have a test in Apache Kafka > which will fail on 3.6.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] DRAFT: KAFKA-15764: Missing Tests for Transactions [kafka]
jolshan opened a new pull request, #14702: URL: https://github.com/apache/kafka/pull/14702 I tried adding compression to the producer to see if it triggered the failure found in KAFKA-15653. My first attempt did not seem to yield any failures yet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.
[
https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782907#comment-17782907
]
Arpit Goyal commented on KAFKA-15388:
-
I just found the answer for the above query
https://github.com/apache/kafka/blob/5785796f985aa294c12e670da221d086a7fa887c/clients/src/main/java/org/apache/kafka/common/record/RecordBatch.java#L118
Last offset of record batch will always be equal to the value before
compaction.
> Handle topics that were having compaction as retention earlier are changed to
> delete only retention policy and onboarded to tiered storage.
>
>
> Key: KAFKA-15388
> URL: https://issues.apache.org/jira/browse/KAFKA-15388
> Project: Kafka
> Issue Type: Bug
>Reporter: Satish Duggana
>Assignee: Arpit Goyal
>Priority: Blocker
> Fix For: 3.7.0
>
>
> Context: [https://github.com/apache/kafka/pull/13561#discussion_r1300055517]
>
> There are 3 paths I looked at:
> * When data is moved to remote storage (1)
> * When data is read from remote storage (2)
> * When data is deleted from remote storage (3)
> (1) Does not have a problem with compacted topics. Compacted segments are
> uploaded and their metadata claims they contain offset from the baseOffset of
> the segment until the next segment's baseOffset. There are no gaps in offsets.
> (2) Does not have a problem if a customer is querying offsets which do not
> exist within a segment, but there are offset after the queried offset within
> the same segment. *However, it does have a problem when the next available
> offset is in a subsequent segment.*
> (3) For data deleted via DeleteRecords there is no problem. For data deleted
> via retention there is no problem.
>
> *I believe the proper solution to (2) is to make tiered storage continue
> looking for the next greater offset in subsequent segments.*
> Steps to reproduce the issue:
> {code:java}
> // TODO (christo)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] MINOR: add clarification about log compaction [kafka]
AyoubOm opened a new pull request, #14701: URL: https://github.com/apache/kafka/pull/14701 *Add a note about log compaction behavior. The term "old data" used may be misunderstood as old data with respect of time, instead of old data associated with a given key* ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (KAFKA-15371) MetadataShell is stuck when bootstrapping
[
https://issues.apache.org/jira/browse/KAFKA-15371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782882#comment-17782882
]
Christian Lefebvre edited comment on KAFKA-15371 at 11/4/23 2:10 PM:
-
It becomes more and more weird ...
* with stopped node, command hangs on "loading..."
* if I set {{{}log4j.rootLogger=INFO{}}}, I get the prompt
* with started server, I often get the NonWritableChannelException but
sometimes I get the prompt
When I got the prompt, exploring node tree works fine
was (Author: JIRAUSER302935):
It becomes more and more weird ...
* with stopped node, command hangs on "loading..."
* if I set {{log4j.rootLogger=INFO}}, I get the prompt
* with started server, I often get the NonWritableChannelException but I get
sometime the prompt
When I got the prompt, exploring node tree works fine
> MetadataShell is stuck when bootstrapping
> -
>
> Key: KAFKA-15371
> URL: https://issues.apache.org/jira/browse/KAFKA-15371
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 3.5.1
>Reporter: Deng Ziming
>Priority: Major
> Attachments: image-2023-08-17-10-35-01-039.png,
> image-2023-08-17-10-35-36-067.png, image-2023-10-31-09-04-53-966.png,
> image-2023-10-31-09-11-53-118.png, image-2023-10-31-09-12-19-051.png,
> image-2023-10-31-09-15-34-821.png
>
>
> I downloaded the 3.5.1 package and startup it, then use metadata shell to
> inspect the data
>
> {code:java}
> // shell
> bin/kafka-metadata-shell.sh --snapshot
> /tmp/kraft-combined-logs/__cluster_metadata-0/.log {code}
> Then process will stuck at loading.
>
>
> !image-2023-08-17-10-35-36-067.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-15371) MetadataShell is stuck when bootstrapping
[
https://issues.apache.org/jira/browse/KAFKA-15371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782882#comment-17782882
]
Christian Lefebvre commented on KAFKA-15371:
It becomes more and more weird ...
* with stopped node, command hangs on "loading..."
* if I set {{log4j.rootLogger=INFO}}, I get the prompt
* with started server, I often get the NonWritableChannelException but I get
sometime the prompt
When I got the prompt, exploring node tree works fine
> MetadataShell is stuck when bootstrapping
> -
>
> Key: KAFKA-15371
> URL: https://issues.apache.org/jira/browse/KAFKA-15371
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 3.5.1
>Reporter: Deng Ziming
>Priority: Major
> Attachments: image-2023-08-17-10-35-01-039.png,
> image-2023-08-17-10-35-36-067.png, image-2023-10-31-09-04-53-966.png,
> image-2023-10-31-09-11-53-118.png, image-2023-10-31-09-12-19-051.png,
> image-2023-10-31-09-15-34-821.png
>
>
> I downloaded the 3.5.1 package and startup it, then use metadata shell to
> inspect the data
>
> {code:java}
> // shell
> bin/kafka-metadata-shell.sh --snapshot
> /tmp/kraft-combined-logs/__cluster_metadata-0/.log {code}
> Then process will stuck at loading.
>
>
> !image-2023-08-17-10-35-36-067.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-15341) Enabling TS for a topic during rolling restart causes problems
[ https://issues.apache.org/jira/browse/KAFKA-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782881#comment-17782881 ] jinlong wang commented on KAFKA-15341: -- After check the logic. It seems that the `FeatureControlManager.reasonNotSupported` can check all broker support features and throw one exception to indicate which broker not enabled feature. My plan to implement this check by following steps: # Report in TieredStorage feature when broker register to controller # When handle AlterConfig request enable remotestorage config on topic, controller will use FeatureControlManager to check if all broker enabled TieredStorage and return the result of AlterConfig if success. The point need guide that how do we set the min and max version for the the VersionedFeature for TieredStrage. > Enabling TS for a topic during rolling restart causes problems > -- > > Key: KAFKA-15341 > URL: https://issues.apache.org/jira/browse/KAFKA-15341 > Project: Kafka > Issue Type: Bug >Reporter: Divij Vaidya >Priority: Major > Labels: KIP-405 > Fix For: 3.7.0 > > > When we are in a rolling restart to enable TS at system level, some brokers > have TS enabled on them and some don't. We send an alter config call to > enable TS for a topic, it hits a broker which has TS enabled, this broker > forwards it to the controller and controller will send the config update to > all brokers. When another broker which doesn't have TS enabled (because it > hasn't undergone the restart yet) gets this config change, it "should" fail > to apply it. But failing now is too late since alterConfig has already > succeeded since controller->broker config propagation is done async. > With this JIRA, we want to have controller check if TS is enabled on all > brokers before applying alter config to turn on TS for a topic. > Context: https://github.com/apache/kafka/pull/14176#discussion_r1291265129 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KAFKA-15313) Delete remote log segments partition asynchronously when a partition is deleted.
[ https://issues.apache.org/jira/browse/KAFKA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782877#comment-17782877 ] jinlong wang commented on KAFKA-15313: -- I can help on this, please assign to me thanks > Delete remote log segments partition asynchronously when a partition is > deleted. > - > > Key: KAFKA-15313 > URL: https://issues.apache.org/jira/browse/KAFKA-15313 > Project: Kafka > Issue Type: Improvement > Components: core >Reporter: Satish Duggana >Assignee: Abhijeet Kumar >Priority: Major > Labels: KIP-405 > > KIP-405 already covers the approach to delete remote log segments > asynchronously through controller and RLMM layers. > reference: https://github.com/apache/kafka/pull/13947#discussion_r1281675818 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.
[
https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782860#comment-17782860
]
Arpit Goyal edited comment on KAFKA-15388 at 11/4/23 11:45 AM:
---
[~divijvaidya] [~satish.duggana] [~christo_lolov] [~showuon] can anyone help me
with the above query ?
This would help me to proceed further.
*Summary*
In a given segment we are trying to find the right record batch for the
requested offset , but it may be possible the end offset of a given batch is
already compacted , for example
Lets say we ar looking to fetch data for offset 50
A segments contain record batch with start and end offset in the following
format, but 50th offset is historically compacted.
RB1[33-38]
RB2[42-49]
RB3[51-56]
Now if we try to fetch data for 50th offset it would return null. Is it
something expected or a bug ? If it is how would we identify which record
batch needs to be returned ?
https://github.com/satishd/kafka/blob/46c96f4868d51c84b43003bbb80bc07297016912/core/src/main/java/kafka/log/remote/RemoteLogManager.java#L1339
was (Author: JIRAUSER301926):
[~divijvaidya] [~satish.duggana] [~christo_lolov] [~showuon] can anyone help me
with the above query ?
This would help me to proceed further.
To summarize it
In a given segment we are trying to find the right record batch for the
requested offset , but it may be possible the end offset of a given batch is
already compacted , for example
Lets say we ar looking to fetch data for offset 50
A segments contain record batch with start and end offset in the following
format, but 50th offset is historically compacted.
RB1[33-38]
RB2[42-49]
RB3[51-56]
Now if we try to fetch data for 50th offset it would return null. Is it
something expected or a bug ? If it is how would we identify which record
batch needs to be returned ?
https://github.com/satishd/kafka/blob/46c96f4868d51c84b43003bbb80bc07297016912/core/src/main/java/kafka/log/remote/RemoteLogManager.java#L1339
> Handle topics that were having compaction as retention earlier are changed to
> delete only retention policy and onboarded to tiered storage.
>
>
> Key: KAFKA-15388
> URL: https://issues.apache.org/jira/browse/KAFKA-15388
> Project: Kafka
> Issue Type: Bug
>Reporter: Satish Duggana
>Assignee: Arpit Goyal
>Priority: Blocker
> Fix For: 3.7.0
>
>
> Context: [https://github.com/apache/kafka/pull/13561#discussion_r1300055517]
>
> There are 3 paths I looked at:
> * When data is moved to remote storage (1)
> * When data is read from remote storage (2)
> * When data is deleted from remote storage (3)
> (1) Does not have a problem with compacted topics. Compacted segments are
> uploaded and their metadata claims they contain offset from the baseOffset of
> the segment until the next segment's baseOffset. There are no gaps in offsets.
> (2) Does not have a problem if a customer is querying offsets which do not
> exist within a segment, but there are offset after the queried offset within
> the same segment. *However, it does have a problem when the next available
> offset is in a subsequent segment.*
> (3) For data deleted via DeleteRecords there is no problem. For data deleted
> via retention there is no problem.
>
> *I believe the proper solution to (2) is to make tiered storage continue
> looking for the next greater offset in subsequent segments.*
> Steps to reproduce the issue:
> {code:java}
> // TODO (christo)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.
[
https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782860#comment-17782860
]
Arpit Goyal edited comment on KAFKA-15388 at 11/4/23 11:45 AM:
---
[~divijvaidya] [~satish.duggana] [~christo_lolov] [~showuon] can anyone help me
with the above query ?
This would help me to proceed further.
To summarize it
In a given segment we are trying to find the right record batch for the
requested offset , but it may be possible the end offset of a given batch is
already compacted , for example
Lets say we ar looking to fetch data for offset 50
A segments contain record batch with start and end offset in the following
format, but 50th offset is historically compacted.
RB1[33-38]
RB2[42-49]
RB3[51-56]
Now if we try to fetch data for 50th offset it would return null. Is it
something expected or a bug ? If it is how would we identify which record
batch needs to be returned ?
https://github.com/satishd/kafka/blob/46c96f4868d51c84b43003bbb80bc07297016912/core/src/main/java/kafka/log/remote/RemoteLogManager.java#L1339
was (Author: JIRAUSER301926):
[~divijvaidya] [~satish.duggana] [~christo_lolov] [~showuon] can anyone help me
with the above query ?
This would help me to proceed further.
To summarize it
In a given segment we are trying to find the right record batch for the
requested offset , but it may be possible the end offset of a given batch is
already compacted , for example
Lets say we ar looking to fetch data for offset 50
A segments contain record batch with start and end offset in the following
format, but 50th offset is historically compacted.
RB1[33-38]
RB2[42-49]
RB3[51-56]
Now if we try to fetch data for 50th offset it would return null. Is it
something expected or a bug ? If it is how would we identify which record
batch needs to be returned ?
> Handle topics that were having compaction as retention earlier are changed to
> delete only retention policy and onboarded to tiered storage.
>
>
> Key: KAFKA-15388
> URL: https://issues.apache.org/jira/browse/KAFKA-15388
> Project: Kafka
> Issue Type: Bug
>Reporter: Satish Duggana
>Assignee: Arpit Goyal
>Priority: Blocker
> Fix For: 3.7.0
>
>
> Context: [https://github.com/apache/kafka/pull/13561#discussion_r1300055517]
>
> There are 3 paths I looked at:
> * When data is moved to remote storage (1)
> * When data is read from remote storage (2)
> * When data is deleted from remote storage (3)
> (1) Does not have a problem with compacted topics. Compacted segments are
> uploaded and their metadata claims they contain offset from the baseOffset of
> the segment until the next segment's baseOffset. There are no gaps in offsets.
> (2) Does not have a problem if a customer is querying offsets which do not
> exist within a segment, but there are offset after the queried offset within
> the same segment. *However, it does have a problem when the next available
> offset is in a subsequent segment.*
> (3) For data deleted via DeleteRecords there is no problem. For data deleted
> via retention there is no problem.
>
> *I believe the proper solution to (2) is to make tiered storage continue
> looking for the next greater offset in subsequent segments.*
> Steps to reproduce the issue:
> {code:java}
> // TODO (christo)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.
[
https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782860#comment-17782860
]
Arpit Goyal commented on KAFKA-15388:
-
[~divijvaidya] [~satish.duggana] [~christo_lolov] [~showuon] can anyone help me
with the above query ?
This would help me to proceed further.
To summarize it
In a given segment we are trying to find the right record batch for the
requested offset , but it may be possible the end offset of a given batch is
already compacted , for example
Lets say we ar looking to fetch data for offset 50
A segments contain record batch with start and end offset in the following
format, but 50th offset is historically compacted.
RB1[33-38]
RB2[42-49]
RB3[51-56]
Now if we try to fetch data for 50th offset it would return null. Is it
something expected or a bug ? If it is how would we identify which record
batch needs to be returned ?
> Handle topics that were having compaction as retention earlier are changed to
> delete only retention policy and onboarded to tiered storage.
>
>
> Key: KAFKA-15388
> URL: https://issues.apache.org/jira/browse/KAFKA-15388
> Project: Kafka
> Issue Type: Bug
>Reporter: Satish Duggana
>Assignee: Arpit Goyal
>Priority: Blocker
> Fix For: 3.7.0
>
>
> Context: [https://github.com/apache/kafka/pull/13561#discussion_r1300055517]
>
> There are 3 paths I looked at:
> * When data is moved to remote storage (1)
> * When data is read from remote storage (2)
> * When data is deleted from remote storage (3)
> (1) Does not have a problem with compacted topics. Compacted segments are
> uploaded and their metadata claims they contain offset from the baseOffset of
> the segment until the next segment's baseOffset. There are no gaps in offsets.
> (2) Does not have a problem if a customer is querying offsets which do not
> exist within a segment, but there are offset after the queried offset within
> the same segment. *However, it does have a problem when the next available
> offset is in a subsequent segment.*
> (3) For data deleted via DeleteRecords there is no problem. For data deleted
> via retention there is no problem.
>
> *I believe the proper solution to (2) is to make tiered storage continue
> looking for the next greater offset in subsequent segments.*
> Steps to reproduce the issue:
> {code:java}
> // TODO (christo)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-15341) Enabling TS for a topic during rolling restart causes problems
[ https://issues.apache.org/jira/browse/KAFKA-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782850#comment-17782850 ] jinlong wang commented on KAFKA-15341: -- maybe i can help, please assign to me. > Enabling TS for a topic during rolling restart causes problems > -- > > Key: KAFKA-15341 > URL: https://issues.apache.org/jira/browse/KAFKA-15341 > Project: Kafka > Issue Type: Bug >Reporter: Divij Vaidya >Priority: Major > Labels: KIP-405 > Fix For: 3.7.0 > > > When we are in a rolling restart to enable TS at system level, some brokers > have TS enabled on them and some don't. We send an alter config call to > enable TS for a topic, it hits a broker which has TS enabled, this broker > forwards it to the controller and controller will send the config update to > all brokers. When another broker which doesn't have TS enabled (because it > hasn't undergone the restart yet) gets this config change, it "should" fail > to apply it. But failing now is too late since alterConfig has already > succeeded since controller->broker config propagation is done async. > With this JIRA, we want to have controller check if TS is enabled on all > brokers before applying alter config to turn on TS for a topic. > Context: https://github.com/apache/kafka/pull/14176#discussion_r1291265129 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-15371) MetadataShell is stuck when bootstrapping
[
https://issues.apache.org/jira/browse/KAFKA-15371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782846#comment-17782846
]
Christian Lefebvre edited comment on KAFKA-15371 at 11/4/23 9:48 AM:
-
Weirdly, I had the same case than [~dengziming], without any output, but this
morning I've the same exception message than [~lirangazer]
The difference is perhaps because the server was running today but stopped
yesterday : maybe a race condition causes the error when the topic is written
by server during dump ?
How can we help to diagnose the problem ? Is there a debug mode ?
was (Author: JIRAUSER302935):
Weirdly, I had the same case than [~dengziming], without any output, but this
morning I've the same exception message than [~lirangazer]
How can we help to diagnose the problem ? Is there a debug mode ?
> MetadataShell is stuck when bootstrapping
> -
>
> Key: KAFKA-15371
> URL: https://issues.apache.org/jira/browse/KAFKA-15371
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 3.5.1
>Reporter: Deng Ziming
>Priority: Major
> Attachments: image-2023-08-17-10-35-01-039.png,
> image-2023-08-17-10-35-36-067.png, image-2023-10-31-09-04-53-966.png,
> image-2023-10-31-09-11-53-118.png, image-2023-10-31-09-12-19-051.png,
> image-2023-10-31-09-15-34-821.png
>
>
> I downloaded the 3.5.1 package and startup it, then use metadata shell to
> inspect the data
>
> {code:java}
> // shell
> bin/kafka-metadata-shell.sh --snapshot
> /tmp/kraft-combined-logs/__cluster_metadata-0/.log {code}
> Then process will stuck at loading.
>
>
> !image-2023-08-17-10-35-36-067.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-15371) MetadataShell is stuck when bootstrapping
[
https://issues.apache.org/jira/browse/KAFKA-15371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782846#comment-17782846
]
Christian Lefebvre commented on KAFKA-15371:
Weirdly, I had the same case than [~dengziming], without any output, but this
morning I've the same exception message than [~lirangazer]
How can we help to diagnose the problem ? Is there a debug mode ?
> MetadataShell is stuck when bootstrapping
> -
>
> Key: KAFKA-15371
> URL: https://issues.apache.org/jira/browse/KAFKA-15371
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 3.5.1
>Reporter: Deng Ziming
>Priority: Major
> Attachments: image-2023-08-17-10-35-01-039.png,
> image-2023-08-17-10-35-36-067.png, image-2023-10-31-09-04-53-966.png,
> image-2023-10-31-09-11-53-118.png, image-2023-10-31-09-12-19-051.png,
> image-2023-10-31-09-15-34-821.png
>
>
> I downloaded the 3.5.1 package and startup it, then use metadata shell to
> inspect the data
>
> {code:java}
> // shell
> bin/kafka-metadata-shell.sh --snapshot
> /tmp/kraft-combined-logs/__cluster_metadata-0/.log {code}
> Then process will stuck at loading.
>
>
> !image-2023-08-17-10-35-36-067.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-15513) KRaft cluster fails with SCRAM authentication enabled for control-plane
[
https://issues.apache.org/jira/browse/KAFKA-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782843#comment-17782843
]
Christian Lefebvre edited comment on KAFKA-15513 at 11/4/23 8:40 AM:
-
I noticed {{kafka-storage}} command with {{--add-scram}} option does nothing on
already formatted directory ({{bootstrap.checkpoint}} not modified)
This may explain why the scram installation fails.
I try to call {{kafka-storage}} command once for formatting and creating users
at once, but the controller startup continues to complain about "authentication
error"
was (Author: JIRAUSER302935):
I noticed `kafka-storage` command with `--add-scram` option does nothing on
already formatted directory (`bootstrap.checkpoint` not modified)
This may explain why the scram installation fails.
I try to call `kafka-storage` command once for formatting and creating users at
once, but the controller startup continues to complain about "authentication
error"
> KRaft cluster fails with SCRAM authentication enabled for control-plane
> ---
>
> Key: KAFKA-15513
> URL: https://issues.apache.org/jira/browse/KAFKA-15513
> Project: Kafka
> Issue Type: Bug
> Components: kraft
>Affects Versions: 3.6.0, 3.5.1
>Reporter: migruiz4
>Priority: Major
>
> We have observed a scenario where a KRaft cluster fails to bootstrap when
> using SCRAM authentication for controller-to-controller communications.
> The steps to reproduce are simple:
> * Deploy (at least) 2 Kafka servers using latest version 3.5.1.
> * Configure a KRaft cluster, where the controller listener uses
> SASL_PLAINTEXT + SCRAM-SHA-256 or SCRAM-SHA-512. In my case, I'm using the
> recommended in-line jaas config
> '{{{}listener.name..scram-sha-512.sasl.jaas.config{}}}'
> * Run 'kafka-storage.sh' in both nodes using option '--add-scram' to create
> the SCRAM user.
> When initialized, Controllers will fail to connect to each other with an
> authentication error:
>
> {code:java}
> [2023-08-01 11:12:45,295] ERROR [kafka-1-raft-outbound-request-thread]:
> Failed to send the following request due to authentication error:
> ClientRequest(expectResponse=true,
> callback=kafka.raft.KafkaNetworkChannel$$Lambda$687/0x7f27d443fc60@2aba6075,
> destination=0, correlationId=129, clientId=raft-client-1,
> createdTimeMs=1690888364960,
> requestBuilder=VoteRequestData(clusterId='abcdefghijklmnopqrstug',
> topics=[TopicData(topicName='__cluster_metadata',
> partitions=[PartitionData(partitionIndex=0, candidateEpoch=4, candidateId=1,
> lastOffsetEpoch=0, lastOffset=0)])])) (kafka.raft.RaftSendThread) {code}
> Some additional details about the scenario that we tested out:
> * Controller listener does work when configured with SASL+PLAIN
> * The issue only affects the Controller listener, SCRAM users created using
> the same method work for data-plane listeners and inter-broker listeners.
>
> Below you can find the exact configuration and command used to deploy:
> * server.properties
> {code:java}
> listeners=INTERNAL://:9092,CLIENT://:9091,CONTROLLER://:9093
> advertised.listeners=INTERNAL://kafka-0:9092,CLIENT://:9091
> listener.security.protocol.map=INTERNAL:PLAINTEXT,CLIENT:PLAINTEXT,CONTROLLER:SASL_PLAINTEXT
> num.network.threads=3
> num.io.threads=8
> socket.send.buffer.bytes=102400
> socket.receive.buffer.bytes=102400
> socket.request.max.bytes=104857600
> log.dirs=/bitnami/kafka/data
> num.partitions=1
> num.recovery.threads.per.data.dir=1
> offsets.topic.replication.factor=1
> transaction.state.log.replication.factor=1
> transaction.state.log.min.isr=1
> log.retention.hours=168
> log.retention.check.interval.ms=30
> controller.listener.names=CONTROLLER
> controller.quorum.voters=0@kafka-0:9093,1@kafka-1:9093
> inter.broker.listener.name=INTERNAL
> node.id=0
> process.roles=controller,broker
> sasl.enabled.mechanisms=PLAIN,SCRAM-SHA-256,SCRAM-SHA-512
> sasl.mechanism.controller.protocol=SCRAM-SHA-512
> listener.name.controller.sasl.enabled.mechanisms=SCRAM-SHA-512
> listener.name.controller.scram-sha-512.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule
> required username="controller_user" password="controller_password";{code}
> * kafka-storage.sh command
> {code:java}
> kafka-storage.sh format --config /path/to/server.properties
> --ignore-formatted --cluster-id abcdefghijklmnopqrstuv --add-scram
> SCRAM-SHA-512=[name=controller_user,password=controller_password] {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-15513) KRaft cluster fails with SCRAM authentication enabled for control-plane
[
https://issues.apache.org/jira/browse/KAFKA-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782843#comment-17782843
]
Christian Lefebvre commented on KAFKA-15513:
I noticed `kafka-storage` command with `--add-scram` option does nothing on
already formatted directory (`bootstrap.checkpoint` not modified)
This may explain why the scram installation fails.
I try to call `kafka-storage` command once for formatting and creating users at
once, but the controller startup continues to complain about "authentication
error"
> KRaft cluster fails with SCRAM authentication enabled for control-plane
> ---
>
> Key: KAFKA-15513
> URL: https://issues.apache.org/jira/browse/KAFKA-15513
> Project: Kafka
> Issue Type: Bug
> Components: kraft
>Affects Versions: 3.6.0, 3.5.1
>Reporter: migruiz4
>Priority: Major
>
> We have observed a scenario where a KRaft cluster fails to bootstrap when
> using SCRAM authentication for controller-to-controller communications.
> The steps to reproduce are simple:
> * Deploy (at least) 2 Kafka servers using latest version 3.5.1.
> * Configure a KRaft cluster, where the controller listener uses
> SASL_PLAINTEXT + SCRAM-SHA-256 or SCRAM-SHA-512. In my case, I'm using the
> recommended in-line jaas config
> '{{{}listener.name..scram-sha-512.sasl.jaas.config{}}}'
> * Run 'kafka-storage.sh' in both nodes using option '--add-scram' to create
> the SCRAM user.
> When initialized, Controllers will fail to connect to each other with an
> authentication error:
>
> {code:java}
> [2023-08-01 11:12:45,295] ERROR [kafka-1-raft-outbound-request-thread]:
> Failed to send the following request due to authentication error:
> ClientRequest(expectResponse=true,
> callback=kafka.raft.KafkaNetworkChannel$$Lambda$687/0x7f27d443fc60@2aba6075,
> destination=0, correlationId=129, clientId=raft-client-1,
> createdTimeMs=1690888364960,
> requestBuilder=VoteRequestData(clusterId='abcdefghijklmnopqrstug',
> topics=[TopicData(topicName='__cluster_metadata',
> partitions=[PartitionData(partitionIndex=0, candidateEpoch=4, candidateId=1,
> lastOffsetEpoch=0, lastOffset=0)])])) (kafka.raft.RaftSendThread) {code}
> Some additional details about the scenario that we tested out:
> * Controller listener does work when configured with SASL+PLAIN
> * The issue only affects the Controller listener, SCRAM users created using
> the same method work for data-plane listeners and inter-broker listeners.
>
> Below you can find the exact configuration and command used to deploy:
> * server.properties
> {code:java}
> listeners=INTERNAL://:9092,CLIENT://:9091,CONTROLLER://:9093
> advertised.listeners=INTERNAL://kafka-0:9092,CLIENT://:9091
> listener.security.protocol.map=INTERNAL:PLAINTEXT,CLIENT:PLAINTEXT,CONTROLLER:SASL_PLAINTEXT
> num.network.threads=3
> num.io.threads=8
> socket.send.buffer.bytes=102400
> socket.receive.buffer.bytes=102400
> socket.request.max.bytes=104857600
> log.dirs=/bitnami/kafka/data
> num.partitions=1
> num.recovery.threads.per.data.dir=1
> offsets.topic.replication.factor=1
> transaction.state.log.replication.factor=1
> transaction.state.log.min.isr=1
> log.retention.hours=168
> log.retention.check.interval.ms=30
> controller.listener.names=CONTROLLER
> controller.quorum.voters=0@kafka-0:9093,1@kafka-1:9093
> inter.broker.listener.name=INTERNAL
> node.id=0
> process.roles=controller,broker
> sasl.enabled.mechanisms=PLAIN,SCRAM-SHA-256,SCRAM-SHA-512
> sasl.mechanism.controller.protocol=SCRAM-SHA-512
> listener.name.controller.sasl.enabled.mechanisms=SCRAM-SHA-512
> listener.name.controller.scram-sha-512.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule
> required username="controller_user" password="controller_password";{code}
> * kafka-storage.sh command
> {code:java}
> kafka-storage.sh format --config /path/to/server.properties
> --ignore-formatted --cluster-id abcdefghijklmnopqrstuv --add-scram
> SCRAM-SHA-512=[name=controller_user,password=controller_password] {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] KAFKA-15038: Add metadatacache into RemoteLogManager, and refactor all relevant codes [kafka]
Owen-CH-Leung commented on PR #14136: URL: https://github.com/apache/kafka/pull/14136#issuecomment-1793384128 Hi all - I've rebased trunk. Can I ask for some feedback again ? Thanks @clolov @showuon @satishd @kamalcph -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-12947: Replace EasyMock and PowerMock with Mockito for StreamsMetricsImplTest [kafka]
dplavcic commented on PR #12449: URL: https://github.com/apache/kafka/pull/12449#issuecomment-1793353918 I will reopen this PR once changes are implemented. I'm currently working on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
