SEZ9 commented on issue #12456: URL: https://github.com/apache/seatunnel/issues/12456#issuecomment-5923303163
@wangbing505 Thanks for the MAT path — that is the retaining edge we were missing. Reading it: the system `AppClassLoader` holds the `WritableComparator` class, its static `comparators` map holds a `Text$Comparator`, whose `conf` is a Hadoop `Configuration`, and the `classLoader` field of that `Configuration` pins the `SeaTunnelChildFirstClassLoader`. So this looks like a process-wide Hadoop static outliving `releaseClassLoader`, rather than a leak inside the SeaTunnel loader itself. It is consistent with what we saw in our 2.3.13 / Java 8u342 / `classloader-cache-mode=false` investigation with repeated MongoDB → Hive jobs. Nothing has been changed yet based on this; I don't want to clear Hadoop's static cache globally, since another concurrent job sharing Hadoop state could be using it. To turn this into a targeted fix, could you provide: 1. The full, untruncated path (the paste cuts off at `AppClassLoader`), and, since it is labelled "first", any further paths for other retained loaders. 2. Confirmation that the same path shows up on the latest code with a minimal repro: exact connector/JAR set, loader count and Metaspace before/after a forced GC, and whether the `comparators` entry still points at the released loader after running one identical second job (does it stay at one stale loader, or grow by one per job?). 3. Your `classloader-cache-mode` setting for the run that produced this dump. With that we can decide whether re-binding the `Configuration` classloader (or creating it outside the job loader) is enough, and whether any other roots need independent fixes. <!-- streview-comment:1434 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
