mimaison commented on PR #15516:
URL: https://github.com/apache/kafka/pull/15516#issuecomment-2252950296
Yeah it looks like the numbers are not accurate. To be honest it's a bit of
a strange performance test. The Produce Test benchmark should be much more
representative. However I did not
stanislavkozlovski commented on PR #15516:
URL: https://github.com/apache/kafka/pull/15516#issuecomment-2250176172
Has anybody noticed that the Linear Write test in KIP-390 is inaccurate?
1. It suggests that the write speed on a broker is 22GB/s. I wasn't able to
find an SSD on the
mimaison merged PR #15516:
URL: https://github.com/apache/kafka/pull/15516
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
mimaison commented on PR #15516:
URL: https://github.com/apache/kafka/pull/15516#issuecomment-2122934859
None of the test failures seem related, merging to trunk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1608274764
##
storage/src/main/java/org/apache/kafka/storage/internals/log/LogValidator.java:
##
@@ -331,12 +332,12 @@ public ValidationResult
mimaison commented on PR #15516:
URL: https://github.com/apache/kafka/pull/15516#issuecomment-2122361920
Thanks for the reviews!
I had to rebase again so I'll wait for the CI to complete.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log
showuon commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1607924069
##
clients/src/main/java/org/apache/kafka/common/config/TopicConfig.java:
##
@@ -173,6 +173,14 @@ public class TopicConfig {
"accepts 'uncompressed' which is
showuon commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1607919763
##
storage/src/main/java/org/apache/kafka/storage/internals/log/LogValidator.java:
##
@@ -331,12 +332,12 @@ public ValidationResult
assignOffsetsNonCompressed(LongRef
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1605420401
##
core/src/test/scala/unit/kafka/log/LogValidatorTest.scala:
##
@@ -1538,7 +1563,80 @@ class LogValidatorTest {
assertEquals(e.recordErrors.size, 3)
}
-
junrao commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1605313096
##
core/src/test/scala/unit/kafka/log/LogValidatorTest.scala:
##
@@ -1538,7 +1563,80 @@ class LogValidatorTest {
assertEquals(e.recordErrors.size, 3)
}
-
mimaison commented on PR #15516:
URL: https://github.com/apache/kafka/pull/15516#issuecomment-2115225074
I also added a couple of new tests in LogValidatorTest to check
recompression only happens if the compression codec is different between the
records from the producer and the topic
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1603325731
##
server-common/src/test/java/org/apache/kafka/server/record/BrokerCompressionTypeTest.java:
##
@@ -16,23 +16,38 @@
*/
package org.apache.kafka.server.record;
junrao commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1602012880
##
server-common/src/test/java/org/apache/kafka/server/record/BrokerCompressionTypeTest.java:
##
@@ -16,23 +16,38 @@
*/
package org.apache.kafka.server.record;
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1601337698
##
clients/src/test/java/org/apache/kafka/common/compress/GzipCompressionTest.java:
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1601283629
##
clients/src/test/java/org/apache/kafka/common/compress/GzipCompressionTest.java:
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1601240696
##
core/src/test/scala/unit/kafka/log/LogValidatorTest.scala:
##
@@ -1587,7 +1612,7 @@ class LogValidatorTest {
private def createTwoBatchedRecords(magicValue:
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1601222613
##
clients/src/test/java/org/apache/kafka/common/compress/NoCompressionTest.java:
##
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
junrao commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1600483273
##
clients/src/test/java/org/apache/kafka/common/compress/GzipCompressionTest.java:
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1600267022
##
clients/src/main/java/org/apache/kafka/common/compress/ZstdCompression.java:
##
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1600263287
##
clients/src/main/java/org/apache/kafka/common/compress/Lz4Compression.java:
##
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1600255317
##
clients/src/main/java/org/apache/kafka/common/compress/NoCompression.java:
##
@@ -14,37 +14,48 @@
* See the License for the specific language governing
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1600247977
##
clients/src/main/java/org/apache/kafka/common/config/TopicConfig.java:
##
@@ -173,6 +173,14 @@ public class TopicConfig {
"accepts 'uncompressed' which is
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1600241274
##
clients/src/main/java/org/apache/kafka/common/compress/NoCompression.java:
##
@@ -14,37 +14,48 @@
* See the License for the specific language governing
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1600235959
##
storage/src/main/java/org/apache/kafka/storage/internals/log/LogValidator.java:
##
@@ -331,12 +332,12 @@ public ValidationResult
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1598619060
##
clients/src/test/java/org/apache/kafka/common/record/DefaultRecordBatchTest.java:
##
@@ -500,27 +501,24 @@ private static Stream
mimaison commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1598609324
##
clients/src/test/java/org/apache/kafka/common/compress/GzipCompressionTest.java:
##
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
junrao commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1586920863
##
clients/src/main/java/org/apache/kafka/common/compress/NoCompression.java:
##
@@ -14,37 +14,48 @@
* See the License for the specific language governing permissions
showuon commented on code in PR #15516:
URL: https://github.com/apache/kafka/pull/15516#discussion_r1579334152
##
clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java:
##
@@ -189,7 +189,7 @@ public RecordAccumulator(LogContext logContext,
mimaison commented on PR #15516:
URL: https://github.com/apache/kafka/pull/15516#issuecomment-2048128916
@divijvaidya It seems you've done a bit of work around compression in the
past. Can you take a look? Thanks
--
This is an automated message from the Apache Git Service.
To respond to
29 matches
Mail list logo