[
https://issues.apache.org/jira/browse/KAFKA-20712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Federico Valeri updated KAFKA-20712:
------------------------------------
Description:
After KAFKA-18616 refactored the TransactionLogMessageFormatter to extend
CoordinatorRecordMessageFormatter (commit b6adec48c5), the transactionStatus
field in the decoded output is rendered as its raw int8 value (e.g., 4) instead
of the human readable state name (e.g., CompleteCommit) which is useful for
debugging.
This affects both:
- kafka-dump-log.sh --transaction-log-decoder (which uses
TransactionLogMessageParser in DumpLogSegments)
- kafka-console-consumer.sh with TransactionLogMessageFormatter
The root cause is that the auto generated
TransactionLogValueJsonConverter.write() serializes the transactionStatus byte
field as new ShortNode(_object.transactionStatus). It has no knowledge of the
TransactionState enum, so the numeric ID is emitted as is.
Before the refactor, the formatter used a hand written implementation that was
also emitting the raw byte. However, the old Scala
TransactionLog.valueToString() (used prior to KAFKA-16666) did resolve the byte
to the state name. When the Java migration was done in KAFKA-16666 the state
name resolution was lost, but the problem only became visible now because the
output format changed.
Steps to reproduce:
{noformat}
bin/kafka-dump-log.sh --deep-iteration --print-data-log
--transaction-log-decoder
--files /tmp/kafka-logs/__transaction_state-0/00000000000000000000.log
{noformat}
Expected output (field excerpt):
{noformat}
"transactionStatus":"CompleteCommit"
{noformat}
Actual output:
{noformat}
"transactionStatus":4
{noformat}
was:
After KAFKA-18616 refactored the TransactionLogMessageFormatter to extend
CoordinatorRecordMessageFormatter (commit b6adec48c5), the transactionStatus
field in the decoded output is rendered as its raw int8 value (e.g., 4) instead
of the human readable state name (e.g., CompleteCommit).
This affects both:
- kafka-dump-log.sh --transaction-log-decoder (which uses
TransactionLogMessageParser in DumpLogSegments)
- kafka-console-consumer.sh with TransactionLogMessageFormatter
The root cause is that the auto generated
TransactionLogValueJsonConverter.write() serializes the transactionStatus byte
field as new ShortNode(_object.transactionStatus). It has no knowledge of the
TransactionState enum, so the numeric ID is emitted as is.
Before the refactor, the formatter used a hand written implementation that was
also emitting the raw byte. However, the old Scala
TransactionLog.valueToString() (used prior to KAFKA-16666) did resolve the byte
to the state name. When the Java migration was done in KAFKA-16666 the state
name resolution was lost, but the problem only became visible now because the
output format changed.
Steps to reproduce:
{noformat}
bin/kafka-dump-log.sh --deep-iteration --print-data-log
--transaction-log-decoder
--files /tmp/kafka-logs/__transaction_state-0/00000000000000000000.log
{noformat}
Expected output (field excerpt):
{noformat}
"transactionStatus":"CompleteCommit"
{noformat}
Actual output:
{noformat}
"transactionStatus":4
{noformat}
> TransactionLogMessageFormatter and TransactionLogMessageParser output numeric
> transactionStatus instead of state name
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-20712
> URL: https://issues.apache.org/jira/browse/KAFKA-20712
> Project: Kafka
> Issue Type: Bug
> Components: tools
> Affects Versions: 4.0.0
> Reporter: Federico Valeri
> Assignee: Federico Valeri
> Priority: Major
>
> After KAFKA-18616 refactored the TransactionLogMessageFormatter to extend
> CoordinatorRecordMessageFormatter (commit b6adec48c5), the transactionStatus
> field in the decoded output is rendered as its raw int8 value (e.g., 4)
> instead of the human readable state name (e.g., CompleteCommit) which is
> useful for debugging.
> This affects both:
> - kafka-dump-log.sh --transaction-log-decoder (which uses
> TransactionLogMessageParser in DumpLogSegments)
> - kafka-console-consumer.sh with TransactionLogMessageFormatter
> The root cause is that the auto generated
> TransactionLogValueJsonConverter.write() serializes the transactionStatus
> byte field as new ShortNode(_object.transactionStatus). It has no knowledge
> of the TransactionState enum, so the numeric ID is emitted as is.
> Before the refactor, the formatter used a hand written implementation that
> was also emitting the raw byte. However, the old Scala
> TransactionLog.valueToString() (used prior to KAFKA-16666) did resolve the
> byte to the state name. When the Java migration was done in KAFKA-16666 the
> state name resolution was lost, but the problem only became visible now
> because the output format changed.
> Steps to reproduce:
> {noformat}
> bin/kafka-dump-log.sh --deep-iteration --print-data-log
> --transaction-log-decoder
> --files /tmp/kafka-logs/__transaction_state-0/00000000000000000000.log
> {noformat}
> Expected output (field excerpt):
> {noformat}
> "transactionStatus":"CompleteCommit"
> {noformat}
> Actual output:
> {noformat}
> "transactionStatus":4
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
