[
https://issues.apache.org/jira/browse/KAFKA-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Johnny Hsu updated KAFKA-16419:
-------------------------------
Description:
Currently in the
[LogValidator.validateMessagesAndAssignOffsetsCompressed|https://github.com/apache/kafka/blob/51c9b0d0ad408754b1c5883a9c7fcc63a5f57eb8/storage/src/main/java/org/apache/kafka/storage/internals/log/LogValidator.java#L315],
there are lots of if-else checks based on the `magic` and `CompressionType`,
which makes the code complicated and increase the difficulties of maintaining.
The flow of the validation can be separated into 5 steps:
# IBP validation
## whether the compression type is valid for this IBP
# In-place assignment enablement check
## based on the magic value and compression type, decide whether we can do
in-place assignment
# batch level validation
## based on the batch origin (client, controller, etc) and magic version
# record level validation
## based on whether we can do in-place assignment, choose different iterator
## based on the magic and compression type, do different validation
# return validated results
## based on whether we can do in-place assignment, build the records or assign
it
This whole flow can be extracted into an interface, and the
LogValidator.validateMessagesAndAssignOffsetsCompressed can init an
implementation based on the passed-in records.
The implementation class will have the following fields:
# magic value
# source compression type
# target compression type
# origin
# records
# timestamp type
was:
Currently in the
[LogValidator.validateMessagesAndAssignOffsetsCompressed|http://example.com](https://github.com/apache/kafka/blob/51c9b0d0ad408754b1c5883a9c7fcc63a5f57eb8/storage/src/main/java/org/apache/kafka/storage/internals/log/LogValidator.java#L315),
there are lots of if-else checks based on the `magic` and `CompressionType`,
which makes the code complicated and increase the difficulties of maintaining.
The flow of the validation can be separated into 5 steps:
# IBP validation
## whether the compression type is valid for this IBP
# In-place assignment enablement check
## based on the magic value and compression type, decide whether we can do
in-place assignment
# batch level validation
## based on the batch origin (client, controller, etc) and magic version
# record level validation
## based on whether we can do in-place assignment, choose different iterator
## based on the magic and compression type, do different validation
# return validated results
## based on whether we can do in-place assignment, build the records or assign
it
This whole flow can be extracted into an interface, and the
LogValidator.validateMessagesAndAssignOffsetsCompressed can init an
implementation based on the passed-in records.
The implementation class will have the following fields:
# magic value
# source compression type
# target compression type
# origin
# records
# timestamp type
> Abstract validateMessagesAndAssignOffsetsCompressed of LogValidator to simply
> the process
> -----------------------------------------------------------------------------------------
>
> Key: KAFKA-16419
> URL: https://issues.apache.org/jira/browse/KAFKA-16419
> Project: Kafka
> Issue Type: Improvement
> Reporter: Johnny Hsu
> Assignee: Johnny Hsu
> Priority: Major
>
> Currently in the
> [LogValidator.validateMessagesAndAssignOffsetsCompressed|https://github.com/apache/kafka/blob/51c9b0d0ad408754b1c5883a9c7fcc63a5f57eb8/storage/src/main/java/org/apache/kafka/storage/internals/log/LogValidator.java#L315],
> there are lots of if-else checks based on the `magic` and `CompressionType`,
> which makes the code complicated and increase the difficulties of
> maintaining.
> The flow of the validation can be separated into 5 steps:
> # IBP validation
> ## whether the compression type is valid for this IBP
> # In-place assignment enablement check
> ## based on the magic value and compression type, decide whether we can do
> in-place assignment
> # batch level validation
> ## based on the batch origin (client, controller, etc) and magic version
> # record level validation
> ## based on whether we can do in-place assignment, choose different iterator
> ## based on the magic and compression type, do different validation
> # return validated results
> ## based on whether we can do in-place assignment, build the records or
> assign it
> This whole flow can be extracted into an interface, and the
> LogValidator.validateMessagesAndAssignOffsetsCompressed can init an
> implementation based on the passed-in records.
> The implementation class will have the following fields:
> # magic value
> # source compression type
> # target compression type
> # origin
> # records
> # timestamp type
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
