NianticRyan commented on issue #24098:
URL: https://github.com/apache/beam/issues/24098#issuecomment-1341493186
One other interesting datapoint here...
So I implemented the workaround suggested and am passing in a custom
ClassPathScanningResourcesDetector in the options. This has resolved the issue
where we see file descriptor leaks whenever a new pipeline is started.
However, I am now seeing a new leak where every 2 hours, I see a jump in the
file descriptors used. It's always around the same amount (which is around how
many jar files we have on our classpath). Also interestingly, it happens on
xx:42 no matter what time I start my instances.
Going to try upgrading classgraph and reverting back to the original
implementation to see if I still observe this issue.
Here's a list of file descriptor reference open times on my instance.
```
1023 Dec 6 19:43
280 Dec 6 19:53
370 Dec 6 19:56
1 Dec 6 20:01
1 Dec 6 20:07
1 Dec 6 20:49
780 Dec 6 21:42
1 Dec 6 21:54
1 Dec 6 21:59
778 Dec 6 23:42
1 Dec 6 23:43
1 Dec 6 23:47
1 Dec 7 00:17
1 Dec 7 00:18
777 Dec 7 01:42
778 Dec 7 03:42
1 Dec 7 03:43
1 Dec 7 03:48
1 Dec 7 03:58
1 Dec 7 04:03
1 Dec 7 04:04
780 Dec 7 05:42
779 Dec 7 07:42
1 Dec 7 07:50
1 Dec 7 07:53
1 Dec 7 07:54
780 Dec 7 09:42
779 Dec 7 11:42
1 Dec 7 11:43
1 Dec 7 11:44
1 Dec 7 11:45
1 Dec 7 11:48
1 Dec 7 11:50
780 Dec 7 13:42
768 Dec 7 15:42
1 Dec 7 16:34
1 Dec 7 16:40
777 Dec 7 16:42
1 Dec 7 16:43
1 Dec 7 17:28
1 Dec 7 18:32
1 Dec 7 18:33
1 Dec 7 18:34
774 Dec 7 18:42
1 Dec 7 19:00
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
