exmy commented on code in PR #5787:
URL: https://github.com/apache/incubator-gluten/pull/5787#discussion_r1604328001
##########
gluten-core/src/main/java/org/apache/gluten/vectorized/JniLibLoader.java:
##########
@@ -125,13 +139,10 @@ public static synchronized void unloadFromPath(String
libPath) {
return;
}
+ LOG.info("Starting unload library path: {} ", libPath);
REQUIRE_UNLOAD_LIBRARY_PATHS.remove(libPath);
try {
- while (Files.isSymbolicLink(Paths.get(libPath))) {
Review Comment:
When running on yarn cluster, `libPath` is a symbolic link existed on the
working directory of yarn container, like
`/data5/hadoop/yarn/local/usercache/xumingyong/appcache/application_1710462985812_777680/container_e57_1710462985812_777680_01_000321/./libch.so`.
It's an `APPLICATION` level resource and will be removed when container exits.
So it may have been removed here when reading real path, which causes
`Files.isSymbolicLink(Paths.get(libPath))` return false in that the library
cannot be unload. As a workaround, we can read real path when loading the
library.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
