[
https://issues.apache.org/jira/browse/BEAM-5455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Weise updated BEAM-5455:
-------------------------------
Labels: portability-flink (was: )
> Don't info log for every bundle in the python sdk
> -------------------------------------------------
>
> Key: BEAM-5455
> URL: https://issues.apache.org/jira/browse/BEAM-5455
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-harness
> Reporter: Micah Wylde
> Assignee: Micah Wylde
> Priority: Minor
> Labels: portability-flink
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently the python sdk logs a bunch of messages at INFO level for every
> bundle. For the portable flink runner, which uses a bundle size of 1 in
> streaming mode, this produces huge log spew:
> {code:java}
> [grpc-default-executor-1] INFO sdk_worker.run - Got work 4
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - start
> <DataOutputOperation >
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - start
> <DoOperation window output_tags=['out']>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - start
> <DoOperation pair_with_one output_tags=['out']>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - start
> <DoOperation timestamp output_tags=['out']>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - start
> <DoOperation decode output_tags=['out']>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - start
> <DataInputOperation receivers=[ConsumerSet[.out0,
> coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish
> <DataInputOperation receivers=[ConsumerSet[.out0,
> coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish
> <DoOperation decode output_tags=['out'], receivers=[ConsumerSet[decode.out0,
> coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish
> <DoOperation timestamp output_tags=['out'],
> receivers=[ConsumerSet[timestamp.out0,
> coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish
> <DoOperation pair_with_one output_tags=['out'],
> receivers=[ConsumerSet[pair_with_one.out0,
> coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder,
> FastPrimitivesCoder]], len(consumers)=1]]>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish
> <DoOperation window output_tags=['out'], receivers=[ConsumerSet[window.out0,
> coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder,
> FastPrimitivesCoder]], len(consumers)=1]]>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish
> <DataOutputOperation >
> {code}
>
> These log messages should probably be at debug instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
