ahmedabu98 commented on issue #32746: URL: https://github.com/apache/beam/issues/32746#issuecomment-2407079129
> I noticed, however, that I have not provided the triggering frequency. I will check if it changes something and let you know. Yep the triggering frequency was one of the recently added features. Streaming writes before 2.60.0 may work in some cases but its rather unpredictable. > I also found that the writer hangs indefinitely while trying to update the manifest file, but not timing out. Not sure why is that🤔 This is also one of the improvements -- we drastically reduced the number of manifest files we write. From my experience, the more manifest files we have, the longer it takes to update the next one Let me know how the new pipeline goes! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
