[ https://issues.apache.org/jira/browse/BEAM-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16044935#comment-16044935 ]
peay commented on BEAM-2326: ---------------------------- Thanks [~kenn]! > Verbose INFO logging with stateful DoFns and Dataflow > ------------------------------------------------------ > > Key: BEAM-2326 > URL: https://issues.apache.org/jira/browse/BEAM-2326 > Project: Beam > Issue Type: Bug > Components: runner-dataflow > Affects Versions: 0.6.0 > Reporter: peay > Assignee: Kenneth Knowles > Labels: log > Fix For: 2.0.0 > > > I am seeing a lot of INFO level logging: > {code} > jsonPayload: { > logger: > "com.google.cloud.dataflow.worker.runners.worker.BatchModeUngroupingParDoFn" > message: "Processing timers for key {} for stateful DoFn" > } > jsonPayload: { > message: "Processing key KV{one of my keys} for stateful DoFn" > logger: > "com.google.cloud.dataflow.worker.runners.worker.BatchModeUngroupingParDoFn" > > } > {code} > out of one of my stateful DoFn. There is one such group of logs for each key > I process, which leads to a very large amount of logs and possibly to a > significant slowdown. > Also, not sure if the {{Processing timers}} log message is missing some > string interpolation or if the empty key is on purpose. > At any rate, this seems more like something for {{DEBUG}} than {{INFO}} given > the large volume. -- This message was sent by Atlassian JIRA (v6.3.15#6346)