Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/3388#discussion_r103196755
--- Diff:
flink-yarn/src/main/java/org/apache/flink/yarn/AbstractYarnClusterDescriptor.java
---
@@ -742,6 +755,24 @@ public FileVisitResult
preVisitDirectory(java.nio.file.Path dir, BasicFileAttrib
}
}
+ for (URI originURI : archives) {
+ Path remoteParent =
Utils.getRemoteResourceRoot(appId.toString(), new Path(originURI.getPath()),
fs.getHomeDirectory());
+ String fragment = originURI.getFragment();
+ Path target = new Path(
+ FileUtils.localizeRemoteFiles(new
org.apache.flink.core.fs.Path(remoteParent.toUri()), originURI).toUri());
+ URI targetURI = target.toUri();
+ if (targetURI.getFragment() == null && fragment !=
null) {
+ targetURI = new URI(target.toUri().toString() +
"#" + fragment);
+ }
+ LocalResource archive =
Records.newRecord(LocalResource.class);
+ FileStatus state = fs.getFileStatus(target);
+ Utils.registerLocalResource(targetURI, state.getLen(),
state.getModificationTime(), archive);
+ localResources.put(fragment, archive);
+ paths.add(new Path(targetURI));
+
classPathBuilder.append(fragment).append(File.pathSeparator);
+ envShipFileList.append(targetURI).append(",");
+ }
--- End diff --
I've got the feeling that this code snippet contains a lot of path to URI
and back to path conversions. Maybe we could streamline it a little bit to
avoid unnecessary object instantiations.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
