[ https://issues.apache.org/jira/browse/KAFKA-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14056816#comment-14056816 ]
Julian Morrison commented on KAFKA-1532: ---------------------------------------- Streaming it to a scratch file (never sync'd, unlinked after use) avoids blocking the log - the blocking operation becomes "it's verified in the scratch file, copy it to the log." > Move CRC32 to AFTER the payload > ------------------------------- > > Key: KAFKA-1532 > URL: https://issues.apache.org/jira/browse/KAFKA-1532 > Project: Kafka > Issue Type: Improvement > Components: core, producer > Reporter: Julian Morrison > Assignee: Jun Rao > Priority: Minor > > To support streaming a message of known length but unknown content, take the > CRC32 out of the message header and make it a message trailer. Then client > libraries can calculate it after streaming the message to Kafka, without > materializing the whole message in RAM. -- This message was sent by Atlassian JIRA (v6.2#6252)