[ https://issues.apache.org/jira/browse/BEAM-10334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Cwik reassigned BEAM-10334: -------------------------------- Assignee: Keshav > Message limit for grpc message is set to default 4 MB > ----------------------------------------------------- > > Key: BEAM-10334 > URL: https://issues.apache.org/jira/browse/BEAM-10334 > Project: Beam > Issue Type: Bug > Components: sdk-py-harness > Affects Versions: 2.18.0, 2.22.0 > Reporter: Keshav > Assignee: Keshav > Priority: P2 > Fix For: 2.23.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > The default limit of a grpc message is set low to 4 MB, and is not overridden > in the sdk_worker.py for the _control_channel. This was first discovered when > submitting a beam job via a flink operator orchestrated by kubernetes > ([https://github.com/GoogleCloudPlatform/flink-on-k8s-operator)|https://github.com/GoogleCloudPlatform/flink-on-k8s-operator]. > The below error (for apache-beam 2.18.0) was seen. > > {code:java} > 2020/06/24 18:37:28 Executing: python -m > apache_beam.runners.worker.sdk_worker_main Traceback (most recent call last): > File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main > "__main__", mod_spec) File "/usr/local/lib/python3.7/runpy.py", line 85, in > _run_code exec(code, run_globals) File > "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py", > line 221, in <module> main(sys.argv) File > "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py", > line 155, in main sdk_pipeline_options.view_as(ProfilingOptions)) File > "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", > line 117, in run for work_request in control_stub.Control(get_responses()): > File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 416, in > __next__ return self._next() File > "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 706, in _next > raise self grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous > of RPC that terminated with: status = StatusCode.RESOURCE_EXHAUSTED details = > "Received message larger than max (4847139 vs. 4194304)" debug_error_string = > "{"created":"@1593023849.325990420","description":"Received message larger > than max (4847139 vs. > 4194304)","file":"src/core/ext/filters/message_size/message_size_filter.cc","file_line":188,"grpc_status":8}" > {code} > > This can be fixed by modifying the options parameter to allow any size grpc > message to be sent or received. -- This message was sent by Atlassian Jira (v8.3.4#803005)