stankiewicz commented on PR #38603: URL: https://github.com/apache/beam/pull/38603#issuecomment-5295039119
> Thx again for looking into this issue. > > I did some tests with our pipelines by applying these commits to v2.73 and noticed some things which I'm not sure are intended: > > * Connected Solace clients now remain at 10 which is our configured `MaxNumConnections`. I could no longer observe an idle timeout of 1min that closes idle Solace clients > * In our integration tests using the direct runner, regardless of how `MaxNumConnections` is configured, 10 Solace clients are trying to connect to our containerized test queues. When running the pipeline in dataflow with `MaxNumConnections` set to 1, only 1 Solace client connects to our queue as expected. > * Our test runtime increased by ~2.5x. Although I haven't looked deeper into this yet, it might be related to the fact that a lot more Solace clients get created and try to connect to our test queue hi @Robbllle , sorry it took so long. So test runtime increase is expected as we ack correctly now after data is checkpoint to the backend. Previously messages were acknowledged too early which was introducing data loss. Can you validate this PR from the perspective of data loss/ missing finalization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
