hansva commented on code in PR #9370:
URL: https://github.com/apache/hop/pull/9370#discussion_r4537582686
##########
ui/src/main/java/org/apache/hop/ui/hopgui/vfs/explorer/VfsFileTransfer.java:
##########
@@ -368,18 +379,74 @@ private static void place(
}
return;
}
- for (FileObject child : source.getChildren()) {
- place(
- child,
- target.resolveFile(child.getName().getBaseName()),
- mode,
- resolver,
- cancelled,
- progress);
- }
- if (mode == Mode.MOVE && source.getChildren().length == 0) {
- // Skipped files stay behind, and so does the folder that holds them.
- source.delete();
+ if (mode == Mode.MOVE && localLink(source) != null) {
+ moveLinkInto(source, target, resolver, cancelled, progress, open);
+ return;
+ }
+ Path real = open(open, source);
Review Comment:
Only the source folder is registered in `open` here, not the existing
target. If the target folder is a link that resolves into the source, neither
the name-based refusal nor the loop check catches it. E.g. `dest/x -> /data/x`,
then move `/data/x` into `dest` with REPLACE: every file is replaced onto
itself, `source.delete()` then removes it, and the emptied source folder is
deleted too, so all files are gone and the link dangles. Could the target's
real path be checked against the source (e.g. `toRealPath()` of both) before
merging?
##########
ui/src/main/java/org/apache/hop/ui/hopgui/vfs/explorer/VfsFileTransfer.java:
##########
@@ -331,24 +337,29 @@ public static void transfer(
mode,
resolver,
cancelled,
- progress == null ? Progress.NONE : progress);
+ progress == null ? Progress.NONE : progress,
+ new HashSet<>());
}
+ /**
+ * @param open the real paths of the local folders this transfer is in, see
{@link #open}
+ */
private static void place(
FileObject source,
FileObject target,
Mode mode,
ConflictResolver resolver,
BooleanSupplier cancelled,
- Progress progress)
+ Progress progress,
+ Set<Path> open)
throws IOException {
checkCancelled(cancelled);
if (!target.exists()) {
if (mode == Mode.MOVE && source.canRenameTo(target)) {
source.moveTo(target);
progress.fileDone();
} else {
- copyTree(source, target, cancelled, progress);
+ copyTree(source, target, cancelled, progress, open);
Review Comment:
A MOVE handles links differently depending on the disk: a same-device rename
keeps a link inside the folder as a link, while this cross-device fallback
(`copyTree` + `delete`) copies what the link points to and removes only the
link. Moving `/home/a` with `data -> /mnt/big` to another disk copies all of
`/mnt/big`. The class doc says moves keep links; could `copyTree` recreate
local links as links in MOVE mode?
##########
ui/src/main/java/org/apache/hop/ui/hopgui/vfs/explorer/VfsFileTransfer.java:
##########
@@ -529,27 +688,42 @@ private static boolean restore(FileObject backup,
FileObject target) {
/**
* A folder this creates is removed again when the copy fails or is stopped,
with whatever was
* already copied into it.
+ *
+ * @param open the real paths of the local folders being copied or written
into, see {@link #open}
*/
private static void copyTree(
- FileObject source, FileObject target, BooleanSupplier cancelled,
Progress progress)
+ FileObject source,
+ FileObject target,
+ BooleanSupplier cancelled,
+ Progress progress,
+ Set<Path> open)
throws IOException {
checkCancelled(cancelled);
if (!source.isFolder()) {
copyFile(source, target, cancelled, progress);
return;
}
- target.createFolder();
+ Path realSource = open(open, source);
+ Path realTarget = null;
try {
- for (FileObject child : source.getChildren()) {
- copyTree(child, target.resolveFile(child.getName().getBaseName()),
cancelled, progress);
- }
- } catch (IOException | RuntimeException e) {
+ target.createFolder();
try {
- delete(target);
- } catch (Exception ignored) {
- // The failure itself is what gets reported.
+ realTarget = open(open, target);
+ for (FileObject child : source.getChildren()) {
+ copyTree(
+ child, target.resolveFile(child.getName().getBaseName()),
cancelled, progress, open);
+ }
+ } catch (IOException | RuntimeException e) {
Review Comment:
A `LinkLoop` thrown deep in the tree is caught by every enclosing `copyTree`
frame, and each deletes its own target, so one cyclic link (e.g. `src/self ->
src`) rolls back the whole copy, possibly after copying gigabytes, and the user
only gets an error. Skipping the looping link (with a warning) and continuing
would keep the rest of the copy.
##########
ui/src/main/java/org/apache/hop/ui/hopgui/vfs/explorer/VfsFileTransfer.java:
##########
@@ -368,18 +379,74 @@ private static void place(
}
return;
}
- for (FileObject child : source.getChildren()) {
- place(
- child,
- target.resolveFile(child.getName().getBaseName()),
- mode,
- resolver,
- cancelled,
- progress);
- }
- if (mode == Mode.MOVE && source.getChildren().length == 0) {
- // Skipped files stay behind, and so does the folder that holds them.
- source.delete();
+ if (mode == Mode.MOVE && localLink(source) != null) {
+ moveLinkInto(source, target, resolver, cancelled, progress, open);
+ return;
+ }
+ Path real = open(open, source);
+ try {
+ for (FileObject child : source.getChildren()) {
+ place(
+ child,
+ target.resolveFile(child.getName().getBaseName()),
+ mode,
+ resolver,
+ cancelled,
+ progress,
+ open);
+ }
+ } finally {
+ close(open, real);
+ }
+ if (mode == Mode.MOVE) {
+ source.refresh();
+ if (source.getChildren().length == 0) {
+ // Skipped files stay behind, and so does the folder that holds them.
+ source.delete();
+ }
+ }
+ }
+
+ /**
+ * Move a link to a folder into the existing folder {@code target}. Moving
the files themselves
+ * would empty the folder the link points to, so they are copied and then
the link is removed. The
+ * link stays when a file was skipped.
+ */
+ private static void moveLinkInto(
+ FileObject link,
+ FileObject target,
+ ConflictResolver resolver,
+ BooleanSupplier cancelled,
+ Progress progress,
+ Set<Path> open)
+ throws IOException {
+ boolean[] skipped = {false};
+ ConflictResolver counting =
+ (source, existing) -> {
+ Resolution resolution =
+ resolver == null ? Resolution.SKIP : resolver.resolve(source,
existing);
+ if (resolution == Resolution.SKIP) {
+ skipped[0] = true;
+ }
+ return resolution;
+ };
+ Path real = open(open, link);
+ try {
+ for (FileObject child : link.getChildren()) {
+ place(
+ child,
+ target.resolveFile(child.getName().getBaseName()),
+ Mode.COPY,
+ counting,
+ cancelled,
+ progress,
+ open);
+ }
+ } finally {
+ close(open, real);
+ }
+ if (!skipped[0]) {
Review Comment:
The link is kept only when a file was skipped. If a nested `place()` throws
instead (an `OTHER_KIND` refusal or a `LinkLoop`), the files already copied
stay in the target and the link stays too, so the content is duplicated and the
user only sees a failure. Maybe clean up the copied files, or report what was
copied?
##########
ui/src/main/java/org/apache/hop/ui/hopgui/vfs/explorer/VfsFileTransfer.java:
##########
@@ -433,17 +500,108 @@ public static void replaceFile(
}
}
+ /**
+ * Delete a file, or a folder with everything in it. A symbolic link is
removed by itself, also
+ * when it is inside the folder: the files it points to stay.
+ */
public static void delete(FileObject file) throws FileSystemException {
- if (file == null || !file.exists()) {
+ if (file == null) {
+ return;
+ }
+ Path link = localLink(file);
+ if (link != null) {
+ deleteLink(file, link);
+ return;
+ }
+ if (!file.exists()) {
return;
}
- if (file.isFolder()) {
+ if (!file.isFolder()) {
+ file.delete();
+ } else if (localPath(file) == null) {
+ // Links are only known on the local disk. Elsewhere the provider
removes the whole tree.
file.delete(Selectors.SELECT_ALL);
} else {
+ // Selectors.SELECT_ALL would also delete the files in a folder a link
points to.
+ for (FileObject child : file.getChildren()) {
Review Comment:
`getChildren()` can return a cached listing here; `file.refresh()` only
happens after the children are removed. If a file was created outside Hop since
the folder was listed, it's missed and `file.delete()` fails on a non-empty
folder, leaving it half deleted. Refreshing before `getChildren()`, as the MOVE
path does at line 402, would avoid that.
##########
ui/src/main/resources/org/apache/hop/ui/hopgui/vfs/explorer/messages/messages_en_US.properties:
##########
@@ -120,6 +120,7 @@ VfsFileExplorer.Transfer.IntoItself=Cannot copy or move
''{0}'' into itself
VfsFileExplorer.Transfer.Exists=''{0}'' already exists
VfsFileExplorer.Transfer.OtherKind=Cannot replace ''{0}'': one is a file and
the other is a folder
VfsFileExplorer.Transfer.Failed=Could not copy or move ''{0}'': {1}
+VfsFileExplorer.Transfer.LinkLoop=Stopped at ''{0}'': a link leads back to a
folder that is already being copied, so the copy would never end
Review Comment:
This text always says the folder is 'already being copied', but the same
check runs during a move (`place` / `moveLinkInto`). Something like 'already
part of this transfer' would fit both.
##########
ui/src/main/java/org/apache/hop/ui/hopgui/vfs/explorer/VfsFileTransfer.java:
##########
@@ -433,17 +500,108 @@ public static void replaceFile(
}
}
+ /**
+ * Delete a file, or a folder with everything in it. A symbolic link is
removed by itself, also
+ * when it is inside the folder: the files it points to stay.
+ */
public static void delete(FileObject file) throws FileSystemException {
- if (file == null || !file.exists()) {
+ if (file == null) {
+ return;
+ }
+ Path link = localLink(file);
+ if (link != null) {
Review Comment:
Small one: `delete()` now does `getFilename`/`Path.of`/`isSymbolicLink`,
then `exists`, `isFolder` and `localPath` again for every entry, plus a parent
refresh for each link. For a large local folder one `Files.walkFileTree`
without following links would be much cheaper, with `SELECT_ALL` kept for
non-local file systems.
##########
ui/src/main/java/org/apache/hop/ui/hopgui/vfs/explorer/VfsFileTransfer.java:
##########
@@ -433,17 +500,108 @@ public static void replaceFile(
}
}
+ /**
+ * Delete a file, or a folder with everything in it. A symbolic link is
removed by itself, also
+ * when it is inside the folder: the files it points to stay.
+ */
public static void delete(FileObject file) throws FileSystemException {
- if (file == null || !file.exists()) {
+ if (file == null) {
+ return;
+ }
+ Path link = localLink(file);
+ if (link != null) {
+ deleteLink(file, link);
+ return;
+ }
+ if (!file.exists()) {
return;
}
- if (file.isFolder()) {
+ if (!file.isFolder()) {
+ file.delete();
+ } else if (localPath(file) == null) {
+ // Links are only known on the local disk. Elsewhere the provider
removes the whole tree.
file.delete(Selectors.SELECT_ALL);
} else {
+ // Selectors.SELECT_ALL would also delete the files in a folder a link
points to.
+ for (FileObject child : file.getChildren()) {
+ delete(child);
+ }
+ file.refresh();
file.delete();
}
}
+ /**
+ * Commons VFS sees a link to a folder as that folder: it deletes what is in
it, and does not
+ * remove the link while that folder holds files. So the link is removed
from the disk directly.
+ */
+ private static void deleteLink(FileObject file, Path link) throws
FileSystemException {
+ try {
+ Files.delete(link);
+ } catch (IOException e) {
+ throw new FileSystemException("vfs.provider/delete.error", e,
file.getName());
+ }
+ file.refresh();
+ FileObject parent = file.getParent();
+ if (parent != null) {
+ parent.refresh();
+ }
+ }
+
+ /** The path of a file on the local disk, or null when it is elsewhere. */
+ private static Path localPath(FileObject file) {
+ try {
+ if (file == null || !"file".equals(file.getName().getScheme())) {
+ return null;
+ }
+ String name = HopVfs.getFilename(file);
+ // A network share keeps its URI.
+ return name.startsWith("file:") ? null : Path.of(name);
+ } catch (Exception e) {
+ return null;
+ }
+ }
+
+ /** The path of a symbolic link on the local disk, or null when the file is
not one. */
+ static Path localLink(FileObject file) {
+ Path path = localPath(file);
+ return path != null && Files.isSymbolicLink(path) ? path : null;
Review Comment:
`Files.isSymbolicLink` is false for Windows directory junctions (`mklink
/J`); Java reports them as 'other' files. So on Windows, deleting a folder that
contains a junction to `C:\real` still recurses into it and deletes the files
in `C:\real`, which is the #9315 data loss. Checking
`Files.readAttributes(path, BasicFileAttributes.class,
LinkOption.NOFOLLOW_LINKS).isOther()` (or the reparse-point attribute via
`DosFileAttributes`) as well would cover junctions. A test that creates a
junction would be good too, if the Windows build can run it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]