[GitHub] [kafka] dengziming commented on pull request #10958: MINOR: Add default sedre in stream test `TestTopicsTest`

2021-07-01 Thread GitBox
dengziming commented on pull request #10958: URL: https://github.com/apache/kafka/pull/10958#issuecomment-872727981 @mjsax @lct45 , PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] dengziming opened a new pull request #10958: MINOR: Add default sedre in stream test `TestTopicsTest`

2021-07-01 Thread GitBox
dengziming opened a new pull request #10958: URL: https://github.com/apache/kafka/pull/10958 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* We change

[GitHub] [kafka] dengziming opened a new pull request #10957: MINOR: Delete unused DeleteTopicsWithIdsResult.java

2021-07-01 Thread GitBox
dengziming opened a new pull request #10957: URL: https://github.com/apache/kafka/pull/10957 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* We can remove

[GitHub] [kafka] dengziming commented on pull request #10957: MINOR: Delete unused DeleteTopicsWithIdsResult.java

2021-07-01 Thread GitBox
dengziming commented on pull request #10957: URL: https://github.com/apache/kafka/pull/10957#issuecomment-872693132 @jolshan Should we delete this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] dengziming commented on pull request #10956: MINOR: fix ZkMetadataCache package name

2021-07-01 Thread GitBox
dengziming commented on pull request #10956: URL: https://github.com/apache/kafka/pull/10956#issuecomment-872689440 ping @cmccabe , I'm also thinking about moving `MetadataCache` into `kafka.server.metadata` , WDYT? -- This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] dengziming opened a new pull request #10956: MINOR: fix ZkMetadataCache package name

2021-07-01 Thread GitBox
dengziming opened a new pull request #10956: URL: https://github.com/apache/kafka/pull/10956 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* The package

[GitHub] [kafka] jeffkbkim edited a comment on pull request #10955: KafkaAdminClient getListOffsetsCalls benchmark

2021-07-01 Thread GitBox
jeffkbkim edited a comment on pull request #10955: URL: https://github.com/apache/kafka/pull/10955#issuecomment-872667997 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [kafka] jeffkbkim edited a comment on pull request #10955: KafkaAdminClient getListOffsetsCalls benchmark

2021-07-01 Thread GitBox
jeffkbkim edited a comment on pull request #10955: URL: https://github.com/apache/kafka/pull/10955#issuecomment-872667997 ran benchmark with warm up = 5, iteration = 5 With changes: ``` Benchmark (partitionCount) (topicCount) Mode

[GitHub] [kafka] jeffkbkim edited a comment on pull request #10955: KafkaAdminClient getListOffsetsCalls benchmark

2021-07-01 Thread GitBox
jeffkbkim edited a comment on pull request #10955: URL: https://github.com/apache/kafka/pull/10955#issuecomment-872667997 ran benchmark with warm up = 5, iteration = 5 With changes: ``` Benchmark (partitionCount) (topicCount) Mode

[GitHub] [kafka] jeffkbkim edited a comment on pull request #10955: KafkaAdminClient getListOffsetsCalls benchmark

2021-07-01 Thread GitBox
jeffkbkim edited a comment on pull request #10955: URL: https://github.com/apache/kafka/pull/10955#issuecomment-872667997 ran benchmark with warm up = 5, iteration = 5 With changes: ``` Benchmark (partitionCount) (topicCount) Mode

[GitHub] [kafka] jeffkbkim commented on pull request #10955: KafkaAdminClient getListOffsetsCalls benchmark

2021-07-01 Thread GitBox
jeffkbkim commented on pull request #10955: URL: https://github.com/apache/kafka/pull/10955#issuecomment-872667997 ran benchmark with warm up = 5, iteration = 5 With changes: ``` Benchmark (partitionCount) (topicCount) Mode Cnt

[GitHub] [kafka] jeffkbkim opened a new pull request #10955: KafkaAdminClient getListOffsetsCalls benchmark

2021-07-01 Thread GitBox
jeffkbkim opened a new pull request #10955: URL: https://github.com/apache/kafka/pull/10955 new benchmark to test KafkaAdminClient.getListOffsetsCalls originated from https://github.com/apache/kafka/pull/10940 ### Committer Checklist (excluded from commit message) - [ ] Verify

[jira] [Resolved] (KAFKA-12717) Remove internal converter config properties

2021-07-01 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-12717. --- Reviewer: Randall Hauch Resolution: Fixed Merged to the `trunk` branch for 3.0.0. >

[GitHub] [kafka] rhauch merged pull request #10854: KAFKA-12717: Remove internal Connect converter properties (KIP-738)

2021-07-01 Thread GitBox
rhauch merged pull request #10854: URL: https://github.com/apache/kafka/pull/10854 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] rhauch commented on pull request #10854: KAFKA-12717: Remove internal Connect converter properties (KIP-738)

2021-07-01 Thread GitBox
rhauch commented on pull request #10854: URL: https://github.com/apache/kafka/pull/10854#issuecomment-872657319 Only one Connect-related failure across all the tests, and it's an integration test that is known to be flaky. -- This is an automated message from the Apache Git Service. To

[GitHub] [kafka] gwenshap closed pull request #10937: KAFKA-10587 Rename kafka-mirror-maker CLI command line arguments for KIP-629

2021-07-01 Thread GitBox
gwenshap closed pull request #10937: URL: https://github.com/apache/kafka/pull/10937 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] xvrl commented on a change in pull request #10937: KAFKA-10587 Rename kafka-mirror-maker CLI command line arguments for KIP-629

2021-07-01 Thread GitBox
xvrl commented on a change in pull request #10937: URL: https://github.com/apache/kafka/pull/10937#discussion_r662679257 ## File path: core/src/main/scala/kafka/tools/MirrorMaker.scala ## @@ -455,7 +455,13 @@ object MirrorMaker extends Logging with KafkaMetricsGroup {

[GitHub] [kafka] xvrl commented on a change in pull request #10937: KAFKA-10587 Rename kafka-mirror-maker CLI command line arguments for KIP-629

2021-07-01 Thread GitBox
xvrl commented on a change in pull request #10937: URL: https://github.com/apache/kafka/pull/10937#discussion_r662677742 ## File path: core/src/main/scala/kafka/tools/MirrorMaker.scala ## @@ -455,7 +455,13 @@ object MirrorMaker extends Logging with KafkaMetricsGroup {

[GitHub] [kafka] xvrl commented on a change in pull request #10937: KAFKA-10587 Rename kafka-mirror-maker CLI command line arguments for KIP-629

2021-07-01 Thread GitBox
xvrl commented on a change in pull request #10937: URL: https://github.com/apache/kafka/pull/10937#discussion_r662677742 ## File path: core/src/main/scala/kafka/tools/MirrorMaker.scala ## @@ -455,7 +455,13 @@ object MirrorMaker extends Logging with KafkaMetricsGroup {

[GitHub] [kafka] xvrl commented on pull request #9439: KAFKA-10587 MirrorMaker CLI change for KIP-629

2021-07-01 Thread GitBox
xvrl commented on pull request #9439: URL: https://github.com/apache/kafka/pull/9439#issuecomment-872646404 closing in favor of #10937 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] xvrl closed pull request #9439: KAFKA-10587 MirrorMaker CLI change for KIP-629

2021-07-01 Thread GitBox
xvrl closed pull request #9439: URL: https://github.com/apache/kafka/pull/9439 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] skaundinya15 opened a new pull request #10954: WIP KIP-709: Implement batching for fetchOffsets

2021-07-01 Thread GitBox
skaundinya15 opened a new pull request #10954: URL: https://github.com/apache/kafka/pull/10954 This implements KIP-709: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=173084258 It updates the `OffsetFetch` request and response to support fetching offsets for

[jira] [Assigned] (KAFKA-12879) Compatibility break in Admin.listOffsets()

2021-07-01 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-12879: - Assignee: Kirk True > Compatibility break in Admin.listOffsets() >

[jira] [Commented] (KAFKA-6987) Reimplement KafkaFuture with CompletableFuture

2021-07-01 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373115#comment-17373115 ] Konstantine Karantasis commented on KAFKA-6987: --- Added a review on the 11th hour. If others

[GitHub] [kafka] kkonstantine commented on a change in pull request #9878: KAFKA-6987: Add KafkaFuture.toCompletionStage()

2021-07-01 Thread GitBox
kkonstantine commented on a change in pull request #9878: URL: https://github.com/apache/kafka/pull/9878#discussion_r662652853 ## File path: clients/src/main/java/org/apache/kafka/common/internals/KafkaFutureImpl.java ## @@ -27,217 +28,107 @@ /** * A flexible future

[GitHub] [kafka] guozhangwang commented on pull request #10953: MINOR: Default GRACE with Old API should set as 24H minus window-size / inactivity-gap

2021-07-01 Thread GitBox
guozhangwang commented on pull request #10953: URL: https://github.com/apache/kafka/pull/10953#issuecomment-872620746 Many thanks to @cadonna ! Need you to review now :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [kafka] guozhangwang opened a new pull request #10953: MINOR: Default GRACE with Old API should set as 24H minus window-size / inactivity-gap

2021-07-01 Thread GitBox
guozhangwang opened a new pull request #10953: URL: https://github.com/apache/kafka/pull/10953 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including

[GitHub] [kafka] cmccabe merged pull request #10946: KAFKA-12997: Expose the append time for batches from raft

2021-07-01 Thread GitBox
cmccabe merged pull request #10946: URL: https://github.com/apache/kafka/pull/10946 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662643922 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -312,26 +315,142 @@ final class KafkaMetadataLog private ( } } -

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662643681 ## File path: core/src/main/scala/kafka/server/KafkaConfig.scala ## @@ -1048,9 +1058,14 @@ object KafkaConfig {

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662643427 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -2089,7 +2090,7 @@ private long

[jira] [Updated] (KAFKA-13020) SnapshotReader should decode and report the append time in the header

2021-07-01 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio updated KAFKA-13020: --- Description: We want to expose control messages through 

[GitHub] [kafka] hachikuji commented on a change in pull request #10931: KAFKA-12998: Implement broker-side KRaft snapshots (WIP)

2021-07-01 Thread GitBox
hachikuji commented on a change in pull request #10931: URL: https://github.com/apache/kafka/pull/10931#discussion_r661670006 ## File path: core/src/main/scala/kafka/server/BrokerServer.scala ## @@ -264,19 +266,13 @@ class BrokerServer( /* Add all reconfigurables for

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662635099 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -2153,14 +2154,36 @@ private boolean maybeCompleteShutdown(long

[jira] [Resolved] (KAFKA-12974) Change the default for snapshot generation configuration

2021-07-01 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio resolved KAFKA-12974. Resolution: Fixed Already fixed. Default set to 20MB. > Change

[GitHub] [kafka] mjsax merged pull request #10917: KAFKA-10847: improve throughput of stream-stream join with spurious left/outer join fix

2021-07-01 Thread GitBox
mjsax merged pull request #10917: URL: https://github.com/apache/kafka/pull/10917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] mumrah commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
mumrah commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662623797 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -312,26 +315,142 @@ final class KafkaMetadataLog private ( } } -

[GitHub] [kafka] rhauch edited a comment on pull request #10854: KAFKA-12717: Remove internal Connect converter properties (KIP-738)

2021-07-01 Thread GitBox
rhauch edited a comment on pull request #10854: URL: https://github.com/apache/kafka/pull/10854#issuecomment-872586150 @C0urante, thanks for fixing the minor conflict! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] rhauch commented on pull request #10854: KAFKA-12717: Remove internal Connect converter properties (KIP-738)

2021-07-01 Thread GitBox
rhauch commented on pull request #10854: URL: https://github.com/apache/kafka/pull/10854#issuecomment-872586150 @C0urante, can you please fix the minor conflict? Looks like a trivial fix, but I'm unable to push to your PR branch. Thanks! -- This is an automated message from the Apache

[GitHub] [kafka] mumrah commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
mumrah commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662623797 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -312,26 +315,142 @@ final class KafkaMetadataLog private ( } } -

[GitHub] [kafka] mumrah commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
mumrah commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662622540 ## File path: core/src/main/scala/kafka/server/KafkaConfig.scala ## @@ -1048,9 +1058,14 @@ object KafkaConfig {

[GitHub] [kafka] jolshan commented on a change in pull request #10892: KAFKA-13011: Update deleteTopics Admin API

2021-07-01 Thread GitBox
jolshan commented on a change in pull request #10892: URL: https://github.com/apache/kafka/pull/10892#discussion_r662621155 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/Admin.java ## @@ -209,65 +209,59 @@ default CreateTopicsResult

[GitHub] [kafka] mumrah commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
mumrah commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662620593 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -2651,7 +2651,7 @@ object Log extends Logging { * @tparam T the type of object held within

[GitHub] [kafka] ijuma commented on a change in pull request #10892: KAFKA-13011: Update deleteTopics Admin API

2021-07-01 Thread GitBox
ijuma commented on a change in pull request #10892: URL: https://github.com/apache/kafka/pull/10892#discussion_r662620347 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/Admin.java ## @@ -209,65 +209,59 @@ default CreateTopicsResult createTopics(Collection

[jira] [Resolved] (KAFKA-8613) Make grace period mandatory

2021-07-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-8613. Resolution: Fixed > Make grace period mandatory > --- > >

[jira] [Updated] (KAFKA-13020) SnapshotReader should decode and report the append time in the header

2021-07-01 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe updated KAFKA-13020: - Summary: SnapshotReader should decode and report the append time in the header (was:

[jira] [Updated] (KAFKA-9559) Change the default "default serde" from ByteArraySerde to null

2021-07-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-9559: --- Description: The current default "default serde" is not particularly useful, and in almost

[jira] [Updated] (KAFKA-9559) Change the default "default serde" from ByteArraySerde to null

2021-07-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-9559: --- Labels: kip (was: needs-kip) > Change the default "default serde" from ByteArraySerde to

[GitHub] [kafka] mjsax merged pull request #10813: KAFKA-9559: Change default serde to be `null`

2021-07-01 Thread GitBox
mjsax merged pull request #10813: URL: https://github.com/apache/kafka/pull/10813 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] cmccabe commented on a change in pull request #10946: KAFKA-12997: Expose the append time for batches from raft

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10946: URL: https://github.com/apache/kafka/pull/10946#discussion_r662614408 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/BatchAccumulator.java ## @@ -495,6 +506,10 @@ public int sizeInBytes() {

[GitHub] [kafka] jsancio commented on a change in pull request #10946: KAFKA-12997: Expose the append time for batches from raft

2021-07-01 Thread GitBox
jsancio commented on a change in pull request #10946: URL: https://github.com/apache/kafka/pull/10946#discussion_r662610242 ## File path: metadata/src/test/java/org/apache/kafka/metalog/LocalLogManager.java ## @@ -198,7 +207,12 @@ synchronized void

[GitHub] [kafka] jsancio commented on a change in pull request #10946: KAFKA-12997: Expose the append time for batches from raft

2021-07-01 Thread GitBox
jsancio commented on a change in pull request #10946: URL: https://github.com/apache/kafka/pull/10946#discussion_r662608352 ## File path: raft/src/main/java/org/apache/kafka/raft/Batch.java ## @@ -99,38 +116,68 @@ public boolean equals(Object o) { Batch batch =

[GitHub] [kafka] jsancio commented on a change in pull request #10946: KAFKA-12997: Expose the append time for batches from raft

2021-07-01 Thread GitBox
jsancio commented on a change in pull request #10946: URL: https://github.com/apache/kafka/pull/10946#discussion_r662608275 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/BatchAccumulator.java ## @@ -207,6 +208,12 @@ private void completeCurrentBatch() {

[GitHub] [kafka] jsancio commented on a change in pull request #10946: KAFKA-12997: Expose the append time for batches from raft

2021-07-01 Thread GitBox
jsancio commented on a change in pull request #10946: URL: https://github.com/apache/kafka/pull/10946#discussion_r662604574 ## File path: raft/src/main/java/org/apache/kafka/raft/ReplicatedCounter.java ## @@ -106,7 +108,10 @@ public synchronized void handleCommit(BatchReader

[GitHub] [kafka] cmccabe commented on a change in pull request #10946: KAFKA-12997: Expose the append time for batches from raft

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10946: URL: https://github.com/apache/kafka/pull/10946#discussion_r662602197 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/BatchAccumulator.java ## @@ -207,6 +208,12 @@ private void completeCurrentBatch() {

[GitHub] [kafka] cmccabe commented on a change in pull request #10946: KAFKA-12997: Expose the append time for batches from raft

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10946: URL: https://github.com/apache/kafka/pull/10946#discussion_r662601954 ## File path: raft/src/main/java/org/apache/kafka/raft/ReplicatedCounter.java ## @@ -106,7 +108,10 @@ public synchronized void handleCommit(BatchReader

[jira] [Resolved] (KAFKA-12964) Corrupt segment recovery can delete new producer state snapshots

2021-07-01 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-12964. - Fix Version/s: 3.0.0 Resolution: Fixed merged the PR to trunk > Corrupt segment recovery can

[GitHub] [kafka] cmccabe commented on a change in pull request #10946: KAFKA-12997: Expose the append time for batches from raft

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10946: URL: https://github.com/apache/kafka/pull/10946#discussion_r662600721 ## File path: raft/src/main/java/org/apache/kafka/raft/Batch.java ## @@ -99,38 +116,68 @@ public boolean equals(Object o) { Batch batch =

[GitHub] [kafka] junrao merged pull request #10896: KAFKA-12964: Collect and rename snapshot files prior to async deletion.

2021-07-01 Thread GitBox
junrao merged pull request #10896: URL: https://github.com/apache/kafka/pull/10896 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] cmccabe commented on a change in pull request #10946: KAFKA-12997: Expose the append time for batches from raft

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10946: URL: https://github.com/apache/kafka/pull/10946#discussion_r662599796 ## File path: metadata/src/test/java/org/apache/kafka/metalog/LocalLogManager.java ## @@ -198,7 +207,12 @@ synchronized void

[jira] [Updated] (KAFKA-13025) Add more information to the Snapshot header and footer

2021-07-01 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio updated KAFKA-13025: --- Labels: kip-500 (was: ) > Add more information to the Snapshot

[GitHub] [kafka] hachikuji merged pull request #10940: KAFKA-13007: KafkaAdminClient getListOffsetsCalls reuse cluster snapshot

2021-07-01 Thread GitBox
hachikuji merged pull request #10940: URL: https://github.com/apache/kafka/pull/10940 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] jsancio commented on a change in pull request #10946: KAFKA-12997: Expose the append time for batches from raft

2021-07-01 Thread GitBox
jsancio commented on a change in pull request #10946: URL: https://github.com/apache/kafka/pull/10946#discussion_r662594464 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/BatchAccumulator.java ## @@ -495,6 +506,10 @@ public int sizeInBytes() {

[jira] [Created] (KAFKA-13026) Idempotent producer (KAFKA-10619) follow-up

2021-07-01 Thread Cheng Tan (Jira)
Cheng Tan created KAFKA-13026: - Summary: Idempotent producer (KAFKA-10619) follow-up Key: KAFKA-13026 URL: https://issues.apache.org/jira/browse/KAFKA-13026 Project: Kafka Issue Type:

[GitHub] [kafka] mimaison merged pull request #10743: KIP-699: Update FindCoordinator to resolve multiple Coordinators at a time

2021-07-01 Thread GitBox
mimaison merged pull request #10743: URL: https://github.com/apache/kafka/pull/10743 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] jsancio commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
jsancio commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662550101 ## File path: core/src/main/scala/kafka/server/KafkaConfig.scala ## @@ -1048,9 +1058,14 @@ object KafkaConfig {

[GitHub] [kafka] garyrussell commented on pull request #7670: KAFKA-7016: Not hide the stack trace for ApiException

2021-07-01 Thread GitBox
garyrussell commented on pull request #7670: URL: https://github.com/apache/kafka/pull/7670#issuecomment-872514693 Can we PLEASE get this merged? It is incredibly difficult to track down problems in code that calls the clients, if we can't tell where the call is made from. -- This is

[jira] [Created] (KAFKA-13025) Add more information to the Snapshot header and footer

2021-07-01 Thread Niket Goel (Jira)
Niket Goel created KAFKA-13025: -- Summary: Add more information to the Snapshot header and footer Key: KAFKA-13025 URL: https://issues.apache.org/jira/browse/KAFKA-13025 Project: Kafka Issue

[jira] [Updated] (KAFKA-12952) Metadata Snapshot File Delimiters

2021-07-01 Thread Niket Goel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niket Goel updated KAFKA-12952: --- Description: Create new Control Records that will serve as the header and footer for a Metadata

[GitHub] [kafka] jolshan opened a new pull request #10952: KAFKA-12257: Consumer mishandles topics deleted and recreated with the same name

2021-07-01 Thread GitBox
jolshan opened a new pull request #10952: URL: https://github.com/apache/kafka/pull/10952 Store topic ID info in consumer metadata. We will always take the topic ID from the latest metadata response and remove any topic IDs from the cache if the metadata response did not return a topic ID

[jira] [Commented] (KAFKA-12841) NPE from the provided metadata in client callback in case of ApiException

2021-07-01 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373002#comment-17373002 ] Kirk True commented on KAFKA-12841: --- I have submitted a [pull

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662522112 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -312,26 +315,142 @@ final class KafkaMetadataLog private ( } } -

[jira] [Assigned] (KAFKA-12841) NPE from the provided metadata in client callback in case of ApiException

2021-07-01 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-12841: - Assignee: Kirk True > NPE from the provided metadata in client callback in case of

[GitHub] [kafka] kirktrue commented on pull request #10951: KAFKA-12841: NPE from the provided metadata in client callback in case of ApiException

2021-07-01 Thread GitBox
kirktrue commented on pull request #10951: URL: https://github.com/apache/kafka/pull/10951#issuecomment-872470098 @ayoukhananov - you're absolutely correct that there are conditions wherein `tp` wasn't initialized. This should fix that case too. -- This is an automated message from the

[GitHub] [kafka] mumrah commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
mumrah commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662517391 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -16,34 +16,41 @@ */ package kafka.raft -import java.io.File -import

[GitHub] [kafka] mumrah commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
mumrah commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662517293 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -16,34 +16,41 @@ */ package kafka.raft -import java.io.File -import

[GitHub] [kafka] jeffkbkim commented on pull request #10940: KAFKA-13007: KafkaAdminClient getListOffsetsCalls reuse cluster snapshot

2021-07-01 Thread GitBox
jeffkbkim commented on pull request #10940: URL: https://github.com/apache/kafka/pull/10940#issuecomment-872467503 @showuon thanks! on the time complexity, i think you may have missed the for-loop wrapped around. -- This is an automated message from the Apache Git Service. To

[jira] [Comment Edited] (KAFKA-12841) NPE from the provided metadata in client callback in case of ApiException

2021-07-01 Thread Avi Youkhananov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372989#comment-17372989 ] Avi Youkhananov edited comment on KAFKA-12841 at 7/1/21, 6:31 PM: --

[jira] [Commented] (KAFKA-12841) NPE from the provided metadata in client callback in case of ApiException

2021-07-01 Thread Avi Youkhananov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372989#comment-17372989 ] Avi Youkhananov commented on KAFKA-12841: - [~kirktrue] This is what i did in this

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662509722 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -2153,14 +2154,36 @@ private boolean maybeCompleteShutdown(long

[GitHub] [kafka] jeffkbkim commented on pull request #10940: KAFKA-13007: KafkaAdminClient getListOffsetsCalls reuse cluster snapshot

2021-07-01 Thread GitBox
jeffkbkim commented on pull request #10940: URL: https://github.com/apache/kafka/pull/10940#issuecomment-872460573 i have renamed cluster() to buildCluster(). i will submit a separate PR for the benchmark -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] mumrah commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
mumrah commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662506190 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -312,26 +315,142 @@ final class KafkaMetadataLog private ( } } -

[GitHub] [kafka] mumrah commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
mumrah commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662505337 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -312,26 +315,142 @@ final class KafkaMetadataLog private ( } } -

[jira] [Commented] (KAFKA-12841) NPE from the provided metadata in client callback in case of ApiException

2021-07-01 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372984#comment-17372984 ] Kirk True commented on KAFKA-12841: --- There are some conditions inside the {{KafkaProducer}}'s

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662498528 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -312,26 +315,142 @@ final class KafkaMetadataLog private ( } } -

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662494186 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -312,26 +313,152 @@ final class KafkaMetadataLog private ( } } -

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662493849 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -312,26 +315,142 @@ final class KafkaMetadataLog private ( } } -

[jira] [Comment Edited] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()

2021-07-01 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372977#comment-17372977 ] Bruno Cadonna edited comment on KAFKA-13010 at 7/1/21, 6:01 PM: Some

[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()

2021-07-01 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372977#comment-17372977 ] Bruno Cadonna commented on KAFKA-13010: --- Some logs that might be interesting: {code} [2021-06-29

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662470041 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -16,34 +16,41 @@ */ package kafka.raft -import java.io.File -import

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662470041 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -16,34 +16,41 @@ */ package kafka.raft -import java.io.File -import

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662470041 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -16,34 +16,41 @@ */ package kafka.raft -import java.io.File -import

[GitHub] [kafka] cmccabe commented on a change in pull request #10864: KAFKA-12155 Metadata log and snapshot cleaning

2021-07-01 Thread GitBox
cmccabe commented on a change in pull request #10864: URL: https://github.com/apache/kafka/pull/10864#discussion_r662469119 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -16,34 +16,41 @@ */ package kafka.raft -import java.io.File -import

[GitHub] [kafka] OmniaGM commented on a change in pull request #10937: KAFKA-10587 Rename kafka-mirror-maker CLI command line arguments for KIP-629

2021-07-01 Thread GitBox
OmniaGM commented on a change in pull request #10937: URL: https://github.com/apache/kafka/pull/10937#discussion_r662449984 ## File path: core/src/main/scala/kafka/tools/MirrorMaker.scala ## @@ -454,8 +454,8 @@ object MirrorMaker extends Logging with KafkaMetricsGroup {

[jira] [Resolved] (KAFKA-12467) Implement QuorumController metadata snapshots

2021-07-01 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-12467. -- Fix Version/s: 3.0.0 Resolution: Fixed > Implement QuorumController metadata snapshots

[GitHub] [kafka] vamossagar12 commented on pull request #10798: KAFKA-9168: Adding direct byte buffer support to rocksdb state store

2021-07-01 Thread GitBox
vamossagar12 commented on pull request #10798: URL: https://github.com/apache/kafka/pull/10798#issuecomment-872399579 > You were referring to this commit [68a947c](https://github.com/apache/kafka/commit/68a947c0eb6a5cc4bdde24083c83f4638e708edb) as for the tweaks right? > > BTW it's

[GitHub] [kafka] vamossagar12 commented on pull request #10798: KAFKA-9168: Adding direct byte buffer support to rocksdb state store

2021-07-01 Thread GitBox
vamossagar12 commented on pull request #10798: URL: https://github.com/apache/kafka/pull/10798#issuecomment-872398600 @guozhangwang , thats the commit i was referring to. Setting a byte order and seems to have an impact on performance. Also, earlier i was calling put first and then

[jira] [Updated] (KAFKA-13024) Kafka Streams is dropping messages with null key during repartition

2021-07-01 Thread Damien Gasparina (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damien Gasparina updated KAFKA-13024: - Description: {{KStream.repartition}} is silently filtering messages with null keys. A

[jira] [Created] (KAFKA-13024) Kafka Streams is dropping messages with null key during repartition

2021-07-01 Thread Damien Gasparina (Jira)
Damien Gasparina created KAFKA-13024: Summary: Kafka Streams is dropping messages with null key during repartition Key: KAFKA-13024 URL: https://issues.apache.org/jira/browse/KAFKA-13024 Project:

[jira] [Assigned] (KAFKA-12646) Implement snapshot generation on brokers

2021-07-01 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio reassigned KAFKA-12646: -- Assignee: Colin McCabe (was: Jose Armando Garcia Sancio) >

  1   2   >