[ https://issues.apache.org/jira/browse/BEAM-5455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise updated BEAM-5455: ------------------------------- Labels: portability-flink (was: ) > Don't info log for every bundle in the python sdk > ------------------------------------------------- > > Key: BEAM-5455 > URL: https://issues.apache.org/jira/browse/BEAM-5455 > Project: Beam > Issue Type: Improvement > Components: sdk-py-harness > Reporter: Micah Wylde > Assignee: Micah Wylde > Priority: Minor > Labels: portability-flink > Time Spent: 20m > Remaining Estimate: 0h > > Currently the python sdk logs a bunch of messages at INFO level for every > bundle. For the portable flink runner, which uses a bundle size of 1 in > streaming mode, this produces huge log spew: > {code:java} > [grpc-default-executor-1] INFO sdk_worker.run - Got work 4 > [grpc-default-executor-1] INFO bundle_processor.process_bundle - start > <DataOutputOperation > > [grpc-default-executor-1] INFO bundle_processor.process_bundle - start > <DoOperation window output_tags=['out']> > [grpc-default-executor-1] INFO bundle_processor.process_bundle - start > <DoOperation pair_with_one output_tags=['out']> > [grpc-default-executor-1] INFO bundle_processor.process_bundle - start > <DoOperation timestamp output_tags=['out']> > [grpc-default-executor-1] INFO bundle_processor.process_bundle - start > <DoOperation decode output_tags=['out']> > [grpc-default-executor-1] INFO bundle_processor.process_bundle - start > <DataInputOperation receivers=[ConsumerSet[.out0, > coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]> > [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish > <DataInputOperation receivers=[ConsumerSet[.out0, > coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]> > [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish > <DoOperation decode output_tags=['out'], receivers=[ConsumerSet[decode.out0, > coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]> > [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish > <DoOperation timestamp output_tags=['out'], > receivers=[ConsumerSet[timestamp.out0, > coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]> > [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish > <DoOperation pair_with_one output_tags=['out'], > receivers=[ConsumerSet[pair_with_one.out0, > coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder, > FastPrimitivesCoder]], len(consumers)=1]]> > [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish > <DoOperation window output_tags=['out'], receivers=[ConsumerSet[window.out0, > coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder, > FastPrimitivesCoder]], len(consumers)=1]]> > [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish > <DataOutputOperation > > {code} > > These log messages should probably be at debug instead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)