scwhittle commented on PR #38422: URL: https://github.com/apache/beam/pull/38422#issuecomment-4419318250
Just throttling the input doesn't handle the case where a single element is producing a large amount of data (for example if the input is a filename and it is being loaded and output). For that reason and since it might unnecessarily delay the work if the queue clears before the work triggers and output, I think it might be better or just let it process and block. Instead, maybe we could modify the lull tracking to detect it is blocking on output and modify the message? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
