ctubbsii commented on code in PR #58:
URL:
https://github.com/apache/accumulo-classloaders/pull/58#discussion_r2761587457
##########
modules/local-caching-classloader/src/main/java/org/apache/accumulo/classloader/lcc/util/LccUtils.java:
##########
@@ -33,18 +49,89 @@ public class LccUtils {
private static final Logger LOG = LoggerFactory.getLogger(LccUtils.class);
private static final ConcurrentHashMap<String,DigestUtils> DIGESTERS = new
ConcurrentHashMap<>();
+ private static final Cleaner CLEANER = Cleaner.create();
// keep at most one DigestUtils instance for each algorithm
public static DigestUtils getDigester(String algorithm) {
return DIGESTERS.computeIfAbsent(algorithm, DigestUtils::new);
}
+ private static String checksumForFileName(String algorithm, String checksum)
{
+ return algorithm.replace('/', '_') + "-" + checksum;
+ }
+
+ public static String checksumForFileName(ContextDefinition definition) {
+ return checksumForFileName(definition.getChecksumAlgorithm(),
definition.getChecksum());
+ }
+
+ public static String checksumForFileName(Resource definition) {
+ return checksumForFileName(definition.getAlgorithm(),
definition.getChecksum());
+ }
+
@SuppressFBWarnings(value = "DP_CREATE_CLASSLOADER_INSIDE_DO_PRIVILEGED",
justification = "doPrivileged is deprecated without replacement and
removed in newer Java")
- public static URLClassLoader createClassLoader(String name, URL[] urls) {
- final var cl = new URLClassLoader(name, urls,
+ public static URLClassLoader createClassLoader(ContextCacheKey cacheKey,
+ URLClassLoaderParams params) {
+ Path hardLinkDir = params.tempDirCreator
+ .apply("context-" +
checksumForFileName(cacheKey.getContextDefinition()));
+ URL[] hardLinksAsURLs = new URL[params.paths.size()];
+ int i = 0;
+ for (Path p : params.paths) {
+ boolean reFetched;
+ Path hardLink = null;
+ do {
+ reFetched = false;
+ try {
+ hardLink = Files.createLink(hardLinkDir.resolve(p.getFileName()), p);
+ } catch (NoSuchFileException e) {
+ LOG.warn(
+ "Missing file {} while creating a hard link in {}; attempting
re-download of context resources",
+ p, hardLinkDir, e);
+ params.redownloader.accept(cacheKey.getContextDefinition());
Review Comment:
In my latest commit a32152a, I got rid of the URLClassLoaderParams and moved
the hard-link creation into the LocalStore. Now, it either hard-links all or
none, and uses a new hard link working directory after re-download attempts.
This ensures all resources are fetched that are needed, without redundant
storage utilization, if the user cleans up resources while doing the
hard-linking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
