[jira] [Commented] (KAFKA-1590) Binarize trace level request logging along with debug level text logging
[ https://issues.apache.org/jira/browse/KAFKA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155647#comment-14155647 ] Guozhang Wang commented on KAFKA-1590: -- [~jkreps] I agree that this would not be the default config. By default: the request log would be written in text. When this knob turns on: the request log would be written only as summary in text (i.e. today's DEBUG level logging), and the detailed log will be recorded as kafka-request-details.gzip/[any other compression types?]. I personally do not know what the % savings would be with the proposed binary / compression format compared the the text format. Would be great if we can get some numbers with a preliminary patch. [~Abhishek Sharma] do you think it is possible that we write a simple cross-platform tool to parse the request-details binary files? > Binarize trace level request logging along with debug level text logging > > > Key: KAFKA-1590 > URL: https://issues.apache.org/jira/browse/KAFKA-1590 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Abhishek Sharma > Labels: newbie > Fix For: 0.9.0 > > Attachments: Jira-1590-Log4j Binary Appender.patch > > > With trace level logging, the request handling logs can grow very fast > depending on the client behavior (e.g. consumer with 0 maxWait and hence keep > sending fetch requests). Previously we have changed it to debug level which > only provides a summary of the requests, omitting request details. However > this does not work perfectly since summaries are not sufficient for > trouble-shooting, and turning on trace level upon issues will be too late. > The proposed solution here, is to default to debug level logging with trace > level logging printed as binary format at the same time. The generated binary > files can then be further compressed / rolled out. When needed, we will then > decompress / parse the trace logs into texts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1590) Binarize trace level request logging along with debug level text logging
[ https://issues.apache.org/jira/browse/KAFKA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151785#comment-14151785 ] Jay Kreps commented on KAFKA-1590: -- Hmm, I think this is going to be a bit confusing. People won't understand what the log is when they encounter it and won't be able to read it. I see the value in being able to cut down on I/O but my recommendation would be that we not enable this by default and that we include a command line utility to pretty print one of these binary logs. Also what is the % savings between the current trace level logging and the proposed binary format? > Binarize trace level request logging along with debug level text logging > > > Key: KAFKA-1590 > URL: https://issues.apache.org/jira/browse/KAFKA-1590 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Abhishek Sharma > Labels: newbie > Fix For: 0.9.0 > > Attachments: Jira-1590-Log4j Binary Appender.patch > > > With trace level logging, the request handling logs can grow very fast > depending on the client behavior (e.g. consumer with 0 maxWait and hence keep > sending fetch requests). Previously we have changed it to debug level which > only provides a summary of the requests, omitting request details. However > this does not work perfectly since summaries are not sufficient for > trouble-shooting, and turning on trace level upon issues will be too late. > The proposed solution here, is to default to debug level logging with trace > level logging printed as binary format at the same time. The generated binary > files can then be further compressed / rolled out. When needed, we will then > decompress / parse the trace logs into texts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1590) Binarize trace level request logging along with debug level text logging
[ https://issues.apache.org/jira/browse/KAFKA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151328#comment-14151328 ] Abhishek Sharma commented on KAFKA-1590: [~jkreps] - I am using Debian Linux and for Linux / Unix family, I surely suggest Midnight Commander tool for reading and writing binary files. IN case of MAC, I think any alternative tool of MC will work. HexEdit is another option that I googled out, haven't worked on it but I am sure it supports. In Case of Windows, I think Total Commander will work. [~guozhang] & [~nehanarkhede] - I am sure, you people have some other tool / utility programs names to suggest. Please suggest. > Binarize trace level request logging along with debug level text logging > > > Key: KAFKA-1590 > URL: https://issues.apache.org/jira/browse/KAFKA-1590 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Abhishek Sharma > Labels: newbie > Fix For: 0.9.0 > > Attachments: Jira-1590-Log4j Binary Appender.patch > > > With trace level logging, the request handling logs can grow very fast > depending on the client behavior (e.g. consumer with 0 maxWait and hence keep > sending fetch requests). Previously we have changed it to debug level which > only provides a summary of the requests, omitting request details. However > this does not work perfectly since summaries are not sufficient for > trouble-shooting, and turning on trace level upon issues will be too late. > The proposed solution here, is to default to debug level logging with trace > level logging printed as binary format at the same time. The generated binary > files can then be further compressed / rolled out. When needed, we will then > decompress / parse the trace logs into texts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1590) Binarize trace level request logging along with debug level text logging
[ https://issues.apache.org/jira/browse/KAFKA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151168#comment-14151168 ] Jay Kreps commented on KAFKA-1590: -- How will people read these binary files? > Binarize trace level request logging along with debug level text logging > > > Key: KAFKA-1590 > URL: https://issues.apache.org/jira/browse/KAFKA-1590 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Abhishek Sharma > Labels: newbie > Fix For: 0.9.0 > > Attachments: Jira-1590-Log4j Binary Appender.patch > > > With trace level logging, the request handling logs can grow very fast > depending on the client behavior (e.g. consumer with 0 maxWait and hence keep > sending fetch requests). Previously we have changed it to debug level which > only provides a summary of the requests, omitting request details. However > this does not work perfectly since summaries are not sufficient for > trouble-shooting, and turning on trace level upon issues will be too late. > The proposed solution here, is to default to debug level logging with trace > level logging printed as binary format at the same time. The generated binary > files can then be further compressed / rolled out. When needed, we will then > decompress / parse the trace logs into texts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1590) Binarize trace level request logging along with debug level text logging
[ https://issues.apache.org/jira/browse/KAFKA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14143360#comment-14143360 ] Guozhang Wang commented on KAFKA-1590: -- [~jkreps] agreed, binary log files should just be a option. What I proposed is that instead of introducing a new config for that we can just set it indirectly at the logging level. I.e. with trace level on the request logs, text log files will be used; with debug level on request logs (which will only be logging request summaries), the detailed log will also be written but as binary files. [~abhi21]] Currently we use daily / hourly rolling, which I think is good enough. Or do you have any ideas that needs size-based rolling? > Binarize trace level request logging along with debug level text logging > > > Key: KAFKA-1590 > URL: https://issues.apache.org/jira/browse/KAFKA-1590 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Abhishek Sharma > Labels: newbie > Fix For: 0.9.0 > > > With trace level logging, the request handling logs can grow very fast > depending on the client behavior (e.g. consumer with 0 maxWait and hence keep > sending fetch requests). Previously we have changed it to debug level which > only provides a summary of the requests, omitting request details. However > this does not work perfectly since summaries are not sufficient for > trouble-shooting, and turning on trace level upon issues will be too late. > The proposed solution here, is to default to debug level logging with trace > level logging printed as binary format at the same time. The generated binary > files can then be further compressed / rolled out. When needed, we will then > decompress / parse the trace logs into texts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1590) Binarize trace level request logging along with debug level text logging
[ https://issues.apache.org/jira/browse/KAFKA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14142913#comment-14142913 ] Abhishek Sharma commented on KAFKA-1590: [~guozhang] and [~nehanarkhede] - Do we need daily rolling or rolling based on size of file functionality in the appender class? Apart from rolling any other specific functionality you are looking for ?? I have used writeUTF method of DataInputStream for writing data in to binary format. WriteUTF method follows Modified-UTF-8 encoding which is slightly different from UTF-8 encoding and any editor capable of reading UTF-8 is good enough for reading it. I am using Gedit editor and it seems good to me. > Binarize trace level request logging along with debug level text logging > > > Key: KAFKA-1590 > URL: https://issues.apache.org/jira/browse/KAFKA-1590 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Abhishek Sharma > Labels: newbie > Fix For: 0.9.0 > > > With trace level logging, the request handling logs can grow very fast > depending on the client behavior (e.g. consumer with 0 maxWait and hence keep > sending fetch requests). Previously we have changed it to debug level which > only provides a summary of the requests, omitting request details. However > this does not work perfectly since summaries are not sufficient for > trouble-shooting, and turning on trace level upon issues will be too late. > The proposed solution here, is to default to debug level logging with trace > level logging printed as binary format at the same time. The generated binary > files can then be further compressed / rolled out. When needed, we will then > decompress / parse the trace logs into texts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1590) Binarize trace level request logging along with debug level text logging
[ https://issues.apache.org/jira/browse/KAFKA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131791#comment-14131791 ] Jay Kreps commented on KAFKA-1590: -- Are we saying that we would only produce a binary log or would this just be an option you could enable? I think for most people they would prefer text logs. The case of max wait = 0 does generate a lot of logging, but people generally don't use that since it generates unbounded load and doesn't improve latency (wait doesn't add latency). Personally I would prefer binary logging was optional. > Binarize trace level request logging along with debug level text logging > > > Key: KAFKA-1590 > URL: https://issues.apache.org/jira/browse/KAFKA-1590 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Abhishek Sharma > Labels: newbie > Fix For: 0.9.0 > > > With trace level logging, the request handling logs can grow very fast > depending on the client behavior (e.g. consumer with 0 maxWait and hence keep > sending fetch requests). Previously we have changed it to debug level which > only provides a summary of the requests, omitting request details. However > this does not work perfectly since summaries are not sufficient for > trouble-shooting, and turning on trace level upon issues will be too late. > The proposed solution here, is to default to debug level logging with trace > level logging printed as binary format at the same time. The generated binary > files can then be further compressed / rolled out. When needed, we will then > decompress / parse the trace logs into texts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1590) Binarize trace level request logging along with debug level text logging
[ https://issues.apache.org/jira/browse/KAFKA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131069#comment-14131069 ] Neha Narkhede commented on KAFKA-1590: -- [~abhioncbr] We'd rather not include more dependencies just to do logging. The request logger currently uses log4j and you can just override the appender and use it to write a binary log. At the same time, we would also need a tool that can parse and convert the request log to humanly readable text. > Binarize trace level request logging along with debug level text logging > > > Key: KAFKA-1590 > URL: https://issues.apache.org/jira/browse/KAFKA-1590 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Abhishek Sharma > Labels: newbie > Fix For: 0.9.0 > > > With trace level logging, the request handling logs can grow very fast > depending on the client behavior (e.g. consumer with 0 maxWait and hence keep > sending fetch requests). Previously we have changed it to debug level which > only provides a summary of the requests, omitting request details. However > this does not work perfectly since summaries are not sufficient for > trouble-shooting, and turning on trace level upon issues will be too late. > The proposed solution here, is to default to debug level logging with trace > level logging printed as binary format at the same time. The generated binary > files can then be further compressed / rolled out. When needed, we will then > decompress / parse the trace logs into texts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1590) Binarize trace level request logging along with debug level text logging
[ https://issues.apache.org/jira/browse/KAFKA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127936#comment-14127936 ] Guozhang Wang commented on KAFKA-1590: -- I think the first option 1) could work, [~jkreps] do you know if the new slf4j framework supports binary logging? > Binarize trace level request logging along with debug level text logging > > > Key: KAFKA-1590 > URL: https://issues.apache.org/jira/browse/KAFKA-1590 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang > Labels: newbie > Fix For: 0.9.0 > > > With trace level logging, the request handling logs can grow very fast > depending on the client behavior (e.g. consumer with 0 maxWait and hence keep > sending fetch requests). Previously we have changed it to debug level which > only provides a summary of the requests, omitting request details. However > this does not work perfectly since summaries are not sufficient for > trouble-shooting, and turning on trace level upon issues will be too late. > The proposed solution here, is to default to debug level logging with trace > level logging printed as binary format at the same time. The generated binary > files can then be further compressed / rolled out. When needed, we will then > decompress / parse the trace logs into texts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1590) Binarize trace level request logging along with debug level text logging
[ https://issues.apache.org/jira/browse/KAFKA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126641#comment-14126641 ] Abhishek Sharma commented on KAFKA-1590: Right now we are using log4j for logging in Kafka. Binary logging format is not supported in log4j. We can have two options - 1) We can write log4j Custom Appender that extends AppenderSkeleton for supporting binary logging format 2) Explore some alternative for binary logging like mfz-binlog (https://code.google.com/p/mfz-binlog/). [~guozhang] - Is there any other option? Any suggestion over this? > Binarize trace level request logging along with debug level text logging > > > Key: KAFKA-1590 > URL: https://issues.apache.org/jira/browse/KAFKA-1590 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang > Labels: newbie > Fix For: 0.9.0 > > > With trace level logging, the request handling logs can grow very fast > depending on the client behavior (e.g. consumer with 0 maxWait and hence keep > sending fetch requests). Previously we have changed it to debug level which > only provides a summary of the requests, omitting request details. However > this does not work perfectly since summaries are not sufficient for > trouble-shooting, and turning on trace level upon issues will be too late. > The proposed solution here, is to default to debug level logging with trace > level logging printed as binary format at the same time. The generated binary > files can then be further compressed / rolled out. When needed, we will then > decompress / parse the trace logs into texts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1590) Binarize trace level request logging along with debug level text logging
[ https://issues.apache.org/jira/browse/KAFKA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14121497#comment-14121497 ] Abhishek Sharma commented on KAFKA-1590: Please assign it to me. I want to give a try on this. > Binarize trace level request logging along with debug level text logging > > > Key: KAFKA-1590 > URL: https://issues.apache.org/jira/browse/KAFKA-1590 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang > Labels: newbie > Fix For: 0.9.0 > > > With trace level logging, the request handling logs can grow very fast > depending on the client behavior (e.g. consumer with 0 maxWait and hence keep > sending fetch requests). Previously we have changed it to debug level which > only provides a summary of the requests, omitting request details. However > this does not work perfectly since summaries are not sufficient for > trouble-shooting, and turning on trace level upon issues will be too late. > The proposed solution here, is to default to debug level logging with trace > level logging printed as binary format at the same time. The generated binary > files can then be further compressed / rolled out. When needed, we will then > decompress / parse the trace logs into texts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
