[ https://issues.apache.org/jira/browse/BEAM-6540?focusedWorklogId=194839&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-194839 ]
ASF GitHub Bot logged work on BEAM-6540: ---------------------------------------- Author: ASF GitHub Bot Created on: 05/Feb/19 22:24 Start Date: 05/Feb/19 22:24 Worklog Time Spent: 10m Work Description: drieber commented on pull request #7664: [BEAM-6540] Windmill quota piping URL: https://github.com/apache/beam/pull/7664#discussion_r254068470 ########## File path: runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/GrpcWindmillServer.java ########## @@ -688,6 +704,13 @@ private void onStreamFinished(@Nullable Throwable t) { errorCount.get(), t.toString()); } + // If the stream was stopped due to a resource exhausted error then we are throttled. + if (t instanceof StatusRuntimeException) { + StatusRuntimeException statusExc = (StatusRuntimeException) t; + if (statusExc.getStatus().getCode() == Status.Code.RESOURCE_EXHAUSTED) { Review comment: Are you 100% sure statusExc.getStatus() will never be null? If not then you should guard against it. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 194839) Time Spent: 5h 20m (was: 5h 10m) > Autoscaling should be aware of Streaming RPC Quota > --------------------------------------------------- > > Key: BEAM-6540 > URL: https://issues.apache.org/jira/browse/BEAM-6540 > Project: Beam > Issue Type: Improvement > Components: runner-dataflow > Affects Versions: 2.11.0 > Reporter: Dustin Rhodes > Assignee: Tyler Akidau > Priority: Major > Fix For: 2.11.0 > > Time Spent: 5h 20m > Remaining Estimate: 0h > > Streaming Windmill Service introduces quota for the shared windmill workers. > Autoscaling needs to be aware of throttling due to this quota in order to not > upscale. This PR adds in that reporting. > > It also introduces the flag --EnableStreamingEngine. -- This message was sent by Atlassian JIRA (v7.6.3#76005)