[
https://issues.apache.org/jira/browse/KAFKA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155647#comment-14155647
]
Guozhang Wang commented on KAFKA-1590:
--------------------------------------
[~jkreps] I agree that this would not be the default config.
By default: the request log would be written in text.
When this knob turns on: the request log would be written only as summary in
text (i.e. today's DEBUG level logging), and the detailed log will be recorded
as kafka-request-details.gzip/[any other compression types?].
I personally do not know what the % savings would be with the proposed binary /
compression format compared the the text format. Would be great if we can get
some numbers with a preliminary patch.
[~Abhishek Sharma] do you think it is possible that we write a simple
cross-platform tool to parse the request-details binary files?
> Binarize trace level request logging along with debug level text logging
> ------------------------------------------------------------------------
>
> Key: KAFKA-1590
> URL: https://issues.apache.org/jira/browse/KAFKA-1590
> Project: Kafka
> Issue Type: Bug
> Reporter: Guozhang Wang
> Assignee: Abhishek Sharma
> Labels: newbie
> Fix For: 0.9.0
>
> Attachments: Jira-1590-Log4j Binary Appender.patch
>
>
> With trace level logging, the request handling logs can grow very fast
> depending on the client behavior (e.g. consumer with 0 maxWait and hence keep
> sending fetch requests). Previously we have changed it to debug level which
> only provides a summary of the requests, omitting request details. However
> this does not work perfectly since summaries are not sufficient for
> trouble-shooting, and turning on trace level upon issues will be too late.
> The proposed solution here, is to default to debug level logging with trace
> level logging printed as binary format at the same time. The generated binary
> files can then be further compressed / rolled out. When needed, we will then
> decompress / parse the trace logs into texts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
