[jira] [Commented] (KAFKA-4829) Improve logging of StreamTask commits
[
https://issues.apache.org/jira/browse/KAFKA-4829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050793#comment-16050793
]
Steven Schlansker commented on KAFKA-4829:
--
This is talking about application logs; the link seems to refer to database
logs, not at all the same thing. Or am I missing something?
> Improve logging of StreamTask commits
> -
>
> Key: KAFKA-4829
> URL: https://issues.apache.org/jira/browse/KAFKA-4829
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Affects Versions: 0.10.2.0
>Reporter: Steven Schlansker
>Priority: Minor
> Labels: user-experience
>
> Currently I see this every commit interval:
> {code}
> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
> task StreamTask 1_31
> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
> task StreamTask 2_31
> {code}
> We have ~10 tasks in our topology, 4 topics, and 32 partitions per topic.
> This means every commit interval we log a few hundred lines of the above
> which is an order of magnitude chattier than anything else in the log
> during normal operations.
> To improve visibility of important messages, we should reduce the chattiness
> of normal commits and highlight abnormal commits. An example proposal:
> existing message is fine at TRACE level for diagnostics
> {{TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31}}
> normal fast case, wrap them all up into one summary line
> {{INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms}}
> some kind of threshold / messaging in case it doesn't complete quickly or
> logs an exception
> {{ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in 100ms}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KAFKA-4829) Improve logging of StreamTask commits
[
https://issues.apache.org/jira/browse/KAFKA-4829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049834#comment-16049834
]
Jeyhun Karimov commented on KAFKA-4829:
---
[~guozhang] I would suggest similar model to SQL Server's logging configs
(https://docs.microsoft.com/en-us/sql/relational-databases/backup-restore/recovery-models-sql-server)
I think current logging can be too much on high workloads and we need a config
parameter on this. I believe this will require a KIP. WDYT?
> Improve logging of StreamTask commits
> -
>
> Key: KAFKA-4829
> URL: https://issues.apache.org/jira/browse/KAFKA-4829
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Affects Versions: 0.10.2.0
>Reporter: Steven Schlansker
>Priority: Minor
> Labels: user-experience
>
> Currently I see this every commit interval:
> {code}
> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
> task StreamTask 1_31
> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
> task StreamTask 2_31
> {code}
> We have ~10 tasks in our topology, 4 topics, and 32 partitions per topic.
> This means every commit interval we log a few hundred lines of the above
> which is an order of magnitude chattier than anything else in the log
> during normal operations.
> To improve visibility of important messages, we should reduce the chattiness
> of normal commits and highlight abnormal commits. An example proposal:
> existing message is fine at TRACE level for diagnostics
> {{TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31}}
> normal fast case, wrap them all up into one summary line
> {{INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms}}
> some kind of threshold / messaging in case it doesn't complete quickly or
> logs an exception
> {{ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in 100ms}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KAFKA-4829) Improve logging of StreamTask commits
[
https://issues.apache.org/jira/browse/KAFKA-4829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15907078#comment-15907078
]
Nicolas Fouché commented on KAFKA-4829:
---
Some comments on how we felt that some logs were missing at INFO level:
http://mail-archives.apache.org/mod_mbox/kafka-users/201703.mbox/%3ccabqkjkj2ozswjaidar_sggztmpwgguo07iaqp9je5-5yzmr...@mail.gmail.com%3e
> Improve logging of StreamTask commits
> -
>
> Key: KAFKA-4829
> URL: https://issues.apache.org/jira/browse/KAFKA-4829
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Affects Versions: 0.10.2.0
>Reporter: Steven Schlansker
>Priority: Minor
> Labels: user-experience
>
> Currently I see this every commit interval:
> {code}
> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
> task StreamTask 1_31
> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
> task StreamTask 2_31
> {code}
> We have ~10 tasks in our topology, 4 topics, and 32 partitions per topic.
> This means every commit interval we log a few hundred lines of the above
> which is an order of magnitude chattier than anything else in the log
> during normal operations.
> To improve visibility of important messages, we should reduce the chattiness
> of normal commits and highlight abnormal commits. An example proposal:
> existing message is fine at TRACE level for diagnostics
> {{TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31}}
> normal fast case, wrap them all up into one summary line
> {{INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms}}
> some kind of threshold / messaging in case it doesn't complete quickly or
> logs an exception
> {{ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in 100ms}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KAFKA-4829) Improve logging of StreamTask commits
[
https://issues.apache.org/jira/browse/KAFKA-4829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890917#comment-15890917
]
Guozhang Wang commented on KAFKA-4829:
--
I'd like to expand the scope of this JIRA a bit to make a pass over all the
log4j entries in Streams and see:
1. If some INFO logging is too verbose / frequent that we can reduce.
2. If some TRACE / DEBUG logging is very vital and hence should be in DEBUG /
INFO logging as long as they are not too frequent.
> Improve logging of StreamTask commits
> -
>
> Key: KAFKA-4829
> URL: https://issues.apache.org/jira/browse/KAFKA-4829
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Affects Versions: 0.10.2.0
>Reporter: Steven Schlansker
>Priority: Minor
> Labels: user-experience
>
> Currently I see this every commit interval:
> {code}
> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
> task StreamTask 1_31
> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
> task StreamTask 2_31
> {code}
> We have ~10 tasks in our topology, 4 topics, and 32 partitions per topic.
> This means every commit interval we log a few hundred lines of the above
> which is an order of magnitude chattier than anything else in the log
> during normal operations.
> To improve visibility of important messages, we should reduce the chattiness
> of normal commits and highlight abnormal commits. An example proposal:
> existing message is fine at TRACE level for diagnostics
> {{TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31}}
> normal fast case, wrap them all up into one summary line
> {{INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms}}
> some kind of threshold / messaging in case it doesn't complete quickly or
> logs an exception
> {{ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in 100ms}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
