This is an automated email from the ASF dual-hosted git repository.
hansva pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hop.git
The following commit(s) were added to refs/heads/main by this push:
new 3a70efa9b5 Fix result issues in Compare folders, Folder is empty, DOS
to Unix and Check files locked, fixes #9117 (#9313)
3a70efa9b5 is described below
commit 3a70efa9b5fb29b610a6dbf1b45f5fb1015ffcfc
Author: Bart Maertens <[email protected]>
AuthorDate: Fri Nov 6 10:57:18 2026 +0100
Fix result issues in Compare folders, Folder is empty, DOS to Unix and
Check files locked, fixes #9117 (#9313)
Compare folders matches entries by their path under the compared folder,
resolves the wildcard and only uses it with 'Let me specify'. An invalid
wildcard is an error in Compare folders, Folder is empty and Check files locked
instead of a wrong result or a stopped workflow. DOS to Unix and Check files
locked treat missing result row fields as empty, and DOS to Unix matches the
wildcard against the file name (a whole-path match still works).
---
.../pages/workflow/actions/checkfilelocked.adoc | 6 +-
.../pages/workflow/actions/comparefolders.adoc | 10 +-
.../ROOT/pages/workflow/actions/dostounix.adoc | 6 +-
.../ROOT/pages/workflow/actions/folderisempty.adoc | 4 +-
.../checkfilelocked/ActionCheckFilesLocked.java | 59 +++++----
.../messages/messages_en_US.properties | 1 +
.../ActionCheckFilesLockedExecuteTest.java | 130 +++++++++++++++++++
.../actions/dostounix/ActionDosToUnix.java | 44 ++++---
.../dostounix/ActionDosToUnixExecuteTest.java | 135 ++++++++++++++++++++
.../actions/folderisempty/ActionFolderIsEmpty.java | 15 ++-
.../messages/messages_en_US.properties | 1 +
.../WorkflowActionFolderIsEmptyTest.java | 27 ++++
.../folderscompare/ActionFoldersCompare.java | 74 +++++++----
.../messages/messages_en_US.properties | 1 +
.../ActionFoldersCompareMatchingTest.java | 139 +++++++++++++++++++++
15 files changed, 566 insertions(+), 86 deletions(-)
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/checkfilelocked.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/checkfilelocked.adoc
index 4d3bffd26b..d09e2a176a 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/checkfilelocked.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/checkfilelocked.adoc
@@ -49,9 +49,9 @@ What this action passes on to the next action. See
xref:workflow/action-results.
|===
|Result|Value
|True|No file is locked. A file or folder that does not exist is not checked
and does not make the action false. With nothing to check, the action is true
too.
-|False|A file is locked: it cannot be renamed to its own name. The action
stops at the first locked file, and the number of errors is 1.
+|False|A file is locked: it cannot be renamed to its own name. The action
stops at the first locked file, and the number of errors is 1. The action is
also false, with 1 error, when *Wildcard* is not a valid regular expression, or
when an error stops the check.
|Result files|Passed on unchanged.
-|Result rows|Passed on unchanged. With *Copy previous results to args?*, the
file or folder comes from the first field and the wildcard from the second
field of each result row, and the list in the dialog is not used, also when
there are no result rows.
+|Result rows|Passed on unchanged. With *Copy previous results to args?*, the
file or folder comes from the first field and the wildcard from the second
field of each result row, and the list in the dialog is not used, also when
there are no result rows. A row with only one field is checked without a
wildcard.
|===
-NOTE: On Linux and macOS a file is reported as locked when its folder is
read-only, because it cannot be renamed. Checking stops at a result row with
only one field: that row and the rows after it are not checked, the error is
logged, and the action is still true. An invalid regular expression in
*Wildcard* is logged as an error for each file, no file in that folder is
checked, and the action is still true.
+NOTE: On Linux and macOS a file is reported as locked when its folder is
read-only, because it cannot be renamed.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/comparefolders.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/comparefolders.adoc
index d4424c2469..409267be81 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/comparefolders.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/comparefolders.adoc
@@ -33,7 +33,7 @@ The `Compare folders` action compares two folders to
determine if the content is
This name has to be unique in a single workflow.
|Include subfolders|Also compare the content of subfolders.
|Compare|Specify what to compare: All, Only Files, Only Folders, or Let me
specify...
-|Wildcard (RegExp)|If you chose Let me specify... in the previous option, you
can specify the regular expression of files to compare. Variables are not
resolved in it.
+|Wildcard (RegExp)|If you chose Let me specify... in the previous option, you
can specify the regular expression of files to compare, for example
`+.*\.csv+`. Variables are resolved. Only used with *Let me specify...*.
|Compare file size|Check this to also compare the size of files with the same
name, not only their names.
|Compare file content|Checks if files have the same content.
Note: this may cause slower processing speeds.
@@ -48,12 +48,8 @@ What this action passes on to the next action. See
xref:workflow/action-results.
[options="header", width="90%", cols="1,3"]
|===
|Result|Value
-|True|For two files: their contents are identical. For two folders: both hold
the same number of entries of the kind set in *Compare*, every name found in
the first folder is also in the second, with the same type, and with *Compare
file size* or *Compare file content* the files have the same size or content.
Without these two options, files with the same name count as equal.
-|False|The folders differ, or one path is a file and the other a folder: the
number of errors is 0. A file or folder does not exist: the number of errors is
1.
+|True|For two files: their contents are identical. For two folders: both hold
the same number of entries of the kind set in *Compare*, every entry found in
the first folder is also in the second at the same path (the same subfolder and
name), with the same type, and with *Compare file size* or *Compare file
content* the files have the same size or content. Without these two options,
files with the same name count as equal.
+|False|The folders differ, or one path is a file and the other a folder: the
number of errors is 0. A file or folder does not exist, or *Wildcard (RegExp)*
is not a valid regular expression: the number of errors is 1.
|Result files|Passed on unchanged.
|Result rows|Passed on unchanged.
|===
-
-NOTE: Entries are matched by their name only, not by their subfolder. With
*Include subfolders*, a folder with `s1/x.txt` and `s2/x.txt` and a folder with
`s1/x.txt` and `s2/y.txt` are reported as identical.
-
-NOTE: *Wildcard (RegExp)* is used as typed: variables in it are not resolved.
An invalid regular expression, which includes one with `+${...}+` in it, leaves
out every entry, and the folders are reported as identical. This also happens
with *Compare* set to *All* when an invalid wildcard is still stored from an
earlier choice.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/dostounix.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/dostounix.adoc
index 23ec2e4fad..b42a64f8c6 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/dostounix.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/dostounix.adoc
@@ -39,7 +39,7 @@ When the format is unknown, this action also has a guess
function and converts t
* Copy previous results to args: take the file or folder, the wildcard and the
conversion from the result rows of the previous action instead of from the list
below. See <<Result>>.
|Files / Folders a|
You can specify the list of files or folders with wildcards in this grid.
-Each line has its own conversion: Windows to Unix, Unix to Windows, or Guess,
which converts a file to the other format. For a folder, the wildcard is a
regular expression matched against the whole path of each file (see <<Result>>).
+Each line has its own conversion: Windows to Unix, Unix to Windows, or Guess,
which converts a file to the other format. For a folder, the wildcard is a
regular expression matched against the name of each file, for example
`+sales\.txt+` or `+.*\.txt+`. A regular expression that matches the whole
path, such as `+.*/sales\.txt+`, also selects the file.
|===
After specifying a folder and wildcard (regular expression), use the `Add`
button to add your file/folder selection to the table. Use the `Delete` or
`Edit` buttons to remove or edit selected lines in the table.
@@ -83,9 +83,7 @@ What this action passes on to the next action. See
xref:workflow/action-results.
|True|The *Success condition* is met: every file converted with *Success if
all files are processed*, at least *Nr files* files converted with *Success if
at least x files are processed*, or fewer files that could not be converted
than *Nr files* with *Success when number of error files lesser than*. The
number of errors is then 0, also when some files failed.
|False|The success condition is not met. A file that cannot be rewritten, for
example in a read-only folder, counts as an error file. A path that does not
exist, or that is not set, counts as an error, which only makes the action
false with *Success if all files are processed*. With *Success if all files are
processed*, a path error stops the action, and the remaining entries are not
processed; an error file does not stop it. The number of errors is the number
of path errors plus error f [...]
|Result files|*Add all filenames* (the default) adds the converted files and
the ones that could not be converted, *Add only processed filenames* only the
converted ones, *Add only error filenames* only the failed ones, and *Do not
add filenames* none. The result files of the previous action are kept.
-|Result rows|Passed on unchanged. With *Copy previous results to args*, the
file or folder, the wildcard and the conversion (`guess`, `dostounix` or
`unixtodos`) come from the first three fields of each result row, and the list
in the dialog is not used, also when there are no result rows.
+|Result rows|Passed on unchanged. With *Copy previous results to args*, the
file or folder, the wildcard and the conversion (`guess`, `dostounix` or
`unixtodos`) come from the first three fields of each result row, and the list
in the dialog is not used, also when there are no result rows. A missing field
counts as empty: no wildcard, and the conversion `guess`.
|Lines written|The number of files converted.
|Lines rejected|The number of files that could not be converted.
|===
-
-NOTE: A result row with fewer than three fields stops the workflow with an
error, and no hop is followed. The wildcard is matched against the whole path
of each file (for example `file:///data/in/sales.txt`), not its name:
`+.*\.txt+` or `+.*/sales\.txt+` match, `+sales\.txt+` matches nothing, and the
action is still true.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/folderisempty.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/folderisempty.adoc
index 6028d188c5..6edf0291d0 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/folderisempty.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/folderisempty.adoc
@@ -44,10 +44,8 @@ What this action passes on to the next action. See
xref:workflow/action-results.
|===
|Result|Value
|True|The folder holds no files. Subfolders do not count; with *Include
subfolders*, the files in them do. With *Limit search to (RegExp)*, only files
whose whole name matches *Wildcard (RegExp)* count.
-|False|The folder holds a file: the number of errors is 0. The folder does not
exist, the path is a file, or no folder name is set: the number of errors is 1.
+|False|The folder holds a file: the number of errors is 0. The folder does not
exist, the path is a file, no folder name is set, or *Wildcard (RegExp)* is not
a valid regular expression while *Limit search to (RegExp)* is selected: the
number of errors is 1.
|Result files|Passed on unchanged.
|Result rows|Passed on unchanged.
|Lines input|When the action is true, the number of subfolders found.
Otherwise the value of the previous action is passed on.
|===
-
-NOTE: An invalid regular expression in *Wildcard (RegExp)* stops the workflow
with an error, and no hop is followed, also when *Limit search to (RegExp)* is
not selected.
diff --git
a/plugins/actions/checkfilelocked/src/main/java/org/apache/hop/workflow/actions/checkfilelocked/ActionCheckFilesLocked.java
b/plugins/actions/checkfilelocked/src/main/java/org/apache/hop/workflow/actions/checkfilelocked/ActionCheckFilesLocked.java
index 035a9359fb..75e08a6eaa 100644
---
a/plugins/actions/checkfilelocked/src/main/java/org/apache/hop/workflow/actions/checkfilelocked/ActionCheckFilesLocked.java
+++
b/plugins/actions/checkfilelocked/src/main/java/org/apache/hop/workflow/actions/checkfilelocked/ActionCheckFilesLocked.java
@@ -21,6 +21,7 @@ import java.util.ArrayList;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
+import java.util.regex.PatternSyntaxException;
import org.apache.commons.vfs2.FileObject;
import org.apache.commons.vfs2.FileSelectInfo;
import org.apache.commons.vfs2.FileSelector;
@@ -29,7 +30,6 @@ import org.apache.hop.core.Result;
import org.apache.hop.core.RowMetaAndData;
import org.apache.hop.core.annotations.Action;
import org.apache.hop.core.exception.HopException;
-import org.apache.hop.core.exception.HopValueException;
import org.apache.hop.core.util.Utils;
import org.apache.hop.core.vfs.HopVfs;
import org.apache.hop.i18n.BaseMessages;
@@ -123,12 +123,15 @@ public class ActionCheckFilesLocked extends ActionBase
implements Cloneable, IAc
}
} catch (Exception e) {
logError(BaseMessages.getString(PKG,
"ActionCheckFilesLocked.ErrorRunningAction", e));
+ // Not every file could be checked, so the check did not pass
+ previousResult.setResult(false);
+ previousResult.setNrErrors(1);
}
return previousResult;
}
- private boolean isOneSpecifiedFileLocked() {
+ private boolean isOneSpecifiedFileLocked() throws HopException {
boolean oneFileLocked = false;
for (int i = 0; i < checkedFiles.size() && !parentWorkflow.isStopped() &&
!oneFileLocked; i++) {
CheckedFile checkedFile = checkedFiles.get(i);
@@ -147,7 +150,7 @@ public class ActionCheckFilesLocked extends ActionBase
implements Cloneable, IAc
return oneFileLocked;
}
- private boolean isOneArgumentsFileLocked(List<RowMetaAndData> rows) throws
HopValueException {
+ private boolean isOneArgumentsFileLocked(List<RowMetaAndData> rows) throws
HopException {
boolean oneFileLocked = false;
// Copy the input row to the (command line) arguments
for (int iteration = 0;
@@ -155,9 +158,9 @@ public class ActionCheckFilesLocked extends ActionBase
implements Cloneable, IAc
iteration++) {
RowMetaAndData resultRow = rows.get(iteration);
- // Get values from previous result
- String fileFolderPrevious = resultRow.getString(0, "");
- String fileMasksPrevious = resultRow.getString(1, "");
+ // Get values from previous result. A missing field counts as empty.
+ String fileFolderPrevious = resultRow.size() > 0 ?
resultRow.getString(0, "") : "";
+ String fileMasksPrevious = resultRow.size() > 1 ? resultRow.getString(1,
"") : "";
// ok we can process this file/folder
if (isDetailed()) {
@@ -174,10 +177,11 @@ public class ActionCheckFilesLocked extends ActionBase
implements Cloneable, IAc
return oneFileLocked;
}
- private boolean processCheckedFilesLine(String name, String wildcard) {
+ private boolean processCheckedFilesLine(String name, String wildcard) throws
HopException {
boolean locked = false;
String realFileFolderName = resolve(name);
- String realWildcard = resolve(wildcard);
+ // An invalid wildcard would leave every file of the folder unchecked:
fail the action instead
+ Pattern wildcardPattern = compileWildcard(resolve(wildcard));
try (FileObject fileFolder = HopVfs.getFileObject(realFileFolderName,
getVariables())) {
FileObject[] files = new FileObject[] {fileFolder};
@@ -191,7 +195,8 @@ public class ActionCheckFilesLocked extends ActionBase
implements Cloneable, IAc
PKG, "ActionCheckFilesLocked.ProcessingFolder",
realFileFolderName));
}
// Retrieve all files
- files = fileFolder.findFiles(new
TextFileSelector(fileFolder.toString(), realWildcard));
+ files =
+ fileFolder.findFiles(new TextFileSelector(fileFolder.toString(),
wildcardPattern));
if (isDetailed()) {
logDetailed(
@@ -243,19 +248,30 @@ public class ActionCheckFilesLocked extends ActionBase
implements Cloneable, IAc
return oneFileLocked;
}
+ private Pattern compileWildcard(String realWildcard) throws HopException {
+ if (Utils.isEmpty(realWildcard)) {
+ return null;
+ }
+ try {
+ return Pattern.compile(realWildcard);
+ } catch (PatternSyntaxException e) {
+ throw new HopException(
+ BaseMessages.getString(
+ PKG, "ActionCheckFilesLocked.Error.InvalidWildcard",
realWildcard, e.getMessage()));
+ }
+ }
+
private class TextFileSelector implements FileSelector {
- String fileWildcard = null;
+ Pattern fileWildcard = null;
String sourceFolder = null;
- public TextFileSelector(String sourcefolderin, String filewildcard) {
+ public TextFileSelector(String sourcefolderin, Pattern filewildcard) {
if (!Utils.isEmpty(sourcefolderin)) {
sourceFolder = sourcefolderin;
}
- if (!Utils.isEmpty(filewildcard)) {
- fileWildcard = filewildcard;
- }
+ fileWildcard = filewildcard;
}
@Override
@@ -307,17 +323,12 @@ public class ActionCheckFilesLocked extends ActionBase
implements Cloneable, IAc
* @param wildcard The wildcard
* @return True if the selected file matches the wildcard
*/
- private boolean getFileWildcard(String selectedFile, String wildcard) {
- boolean getIt = true;
-
- if (!Utils.isEmpty(wildcard)) {
- Pattern pattern = Pattern.compile(wildcard);
- // First see if the file matches the regular expression!
- Matcher matcher = pattern.matcher(selectedFile);
- getIt = matcher.matches();
+ private boolean getFileWildcard(String selectedFile, Pattern wildcard) {
+ if (wildcard == null) {
+ return true;
}
-
- return getIt;
+ Matcher matcher = wildcard.matcher(selectedFile);
+ return matcher.matches();
}
@Override
diff --git
a/plugins/actions/checkfilelocked/src/main/resources/org/apache/hop/workflow/actions/checkfilelocked/messages/messages_en_US.properties
b/plugins/actions/checkfilelocked/src/main/resources/org/apache/hop/workflow/actions/checkfilelocked/messages/messages_en_US.properties
index 0bd2315ed2..caf7acede8 100644
---
a/plugins/actions/checkfilelocked/src/main/resources/org/apache/hop/workflow/actions/checkfilelocked/messages/messages_en_US.properties
+++
b/plugins/actions/checkfilelocked/src/main/resources/org/apache/hop/workflow/actions/checkfilelocked/messages/messages_en_US.properties
@@ -20,6 +20,7 @@ ActionCheckFilesLocked.CheckingFile=Checking file [{0}] ...
ActionCheckFilesLocked.CouldNotProcess=Could not process [{0}], exception: {1}
ActionCheckFilesLocked.Description=Check if one or several files are locked by
another process
ActionCheckFilesLocked.Error.Exception.ProcessError=Error
+ActionCheckFilesLocked.Error.InvalidWildcard=The wildcard [{0}] is not a valid
regular expression: {1}
ActionCheckFilesLocked.ErrorRunningAction=Error running action! {0}
ActionCheckFilesLocked.Fields.Argument.Label=File/Folder
ActionCheckFilesLocked.Fields.Column=Specify here files to check
diff --git
a/plugins/actions/checkfilelocked/src/test/java/org/apache/hop/workflow/actions/checkfilelocked/ActionCheckFilesLockedExecuteTest.java
b/plugins/actions/checkfilelocked/src/test/java/org/apache/hop/workflow/actions/checkfilelocked/ActionCheckFilesLockedExecuteTest.java
new file mode 100644
index 0000000000..bcf681273e
--- /dev/null
+++
b/plugins/actions/checkfilelocked/src/test/java/org/apache/hop/workflow/actions/checkfilelocked/ActionCheckFilesLockedExecuteTest.java
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.workflow.actions.checkfilelocked;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.junit.jupiter.api.Assumptions.assumeFalse;
+import static org.mockito.Mockito.mock;
+
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.List;
+import org.apache.hop.core.Result;
+import org.apache.hop.core.RowMetaAndData;
+import org.apache.hop.core.logging.HopLogStore;
+import org.apache.hop.core.row.value.ValueMetaString;
+import org.apache.hop.workflow.WorkflowMeta;
+import org.apache.hop.workflow.action.ActionMeta;
+import org.apache.hop.workflow.engine.IWorkflowEngine;
+import org.apache.hop.workflow.engines.local.LocalWorkflowEngine;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.condition.EnabledOnOs;
+import org.junit.jupiter.api.condition.OS;
+import org.junit.jupiter.api.io.TempDir;
+
+/** See https://github.com/apache/hop/issues/9117 */
+class ActionCheckFilesLockedExecuteTest {
+
+ @TempDir Path folder;
+
+ private ActionCheckFilesLocked action;
+
+ @BeforeAll
+ static void setUpBeforeClass() {
+ HopLogStore.init();
+ }
+
+ @BeforeEach
+ void setUp() throws Exception {
+ IWorkflowEngine<WorkflowMeta> workflow = new LocalWorkflowEngine(new
WorkflowMeta());
+ action = new ActionCheckFilesLocked();
+ workflow.getWorkflowMeta().addAction(new ActionMeta(action));
+ action.setParentWorkflow(workflow);
+ action.setParentWorkflowMeta(mock(WorkflowMeta.class));
+ workflow.setStopped(false);
+
+ Files.writeString(folder.resolve("data.txt"), "data",
StandardCharsets.UTF_8);
+ }
+
+ @Test
+ void unlockedFolderPasses() {
+ action.getCheckedFiles().add(new
ActionCheckFilesLocked.CheckedFile(folder.toString(), ".*"));
+
+ Result result = action.execute(new Result(), 0);
+
+ assertTrue(result.isResult());
+ assertEquals(0, result.getNrErrors());
+ }
+
+ @Test
+ void invalidWildcardFailsTheCheck() {
+ action.getCheckedFiles().add(new
ActionCheckFilesLocked.CheckedFile(folder.toString(), "["));
+
+ Result result = action.execute(new Result(), 0);
+
+ assertFalse(result.isResult(), "nothing was checked, so the check must not
pass");
+ assertEquals(1, result.getNrErrors());
+ }
+
+ @Test
+ void resultRowWithOneFieldIsChecked() {
+ action.setArgFromPrevious(true);
+ RowMetaAndData row = new RowMetaAndData();
+ row.addValue(new ValueMetaString("filename"), folder.toString());
+ Result previous = new Result();
+ previous.setRows(List.of(row));
+
+ Result result = action.execute(previous, 0);
+
+ assertTrue(result.isResult());
+ assertEquals(0, result.getNrErrors());
+ }
+
+ /**
+ * A file that can not be renamed counts as locked. In a read-only folder
that is every file, so
+ * the one-field row must be checked and fail the action.
+ */
+ @Test
+ @EnabledOnOs({OS.LINUX, OS.MAC})
+ void resultRowWithOneFieldFindsALockedFile() throws Exception {
+ Path readOnly = Files.createDirectory(folder.resolve("read-only"));
+ Path lockedFile = Files.writeString(readOnly.resolve("locked.txt"), "x");
+ Files.setPosixFilePermissions(readOnly,
PosixFilePermissions.fromString("r-xr-xr-x"));
+ try {
+ assumeFalse(
+ Files.isWritable(readOnly), "folder is still writable, cannot run as
this user (root?)");
+ action.setArgFromPrevious(true);
+ RowMetaAndData row = new RowMetaAndData();
+ row.addValue(new ValueMetaString("filename"), lockedFile.toString());
+ Result previous = new Result();
+ previous.setRows(List.of(row));
+
+ Result result = action.execute(previous, 0);
+
+ assertFalse(result.isResult(), "the file can not be renamed, so it
counts as locked");
+ } finally {
+ Files.setPosixFilePermissions(readOnly,
PosixFilePermissions.fromString("rwx------"));
+ }
+ }
+}
diff --git
a/plugins/actions/dostounix/src/main/java/org/apache/hop/workflow/actions/dostounix/ActionDosToUnix.java
b/plugins/actions/dostounix/src/main/java/org/apache/hop/workflow/actions/dostounix/ActionDosToUnix.java
index 1bf0052868..82b8db0f15 100644
---
a/plugins/actions/dostounix/src/main/java/org/apache/hop/workflow/actions/dostounix/ActionDosToUnix.java
+++
b/plugins/actions/dostounix/src/main/java/org/apache/hop/workflow/actions/dostounix/ActionDosToUnix.java
@@ -27,7 +27,6 @@ import java.nio.file.Files;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
import java.util.List;
-import java.util.regex.Matcher;
import java.util.regex.Pattern;
import lombok.Getter;
import lombok.Setter;
@@ -41,6 +40,7 @@ import org.apache.hop.core.ResultFile;
import org.apache.hop.core.RowMetaAndData;
import org.apache.hop.core.annotations.Action;
import org.apache.hop.core.exception.HopException;
+import org.apache.hop.core.exception.HopValueException;
import org.apache.hop.core.util.Utils;
import org.apache.hop.core.vfs.HopVfs;
import org.apache.hop.i18n.BaseMessages;
@@ -230,10 +230,11 @@ public class ActionDosToUnix extends ActionBase {
resultRow = rows.get(iteration);
- // Get source and destination file names, also wildcard
- String vSourceFileFolderPrevious = resultRow.getString(0, null);
- String vWildcardPrevious = resultRow.getString(1, null);
- int conversionType =
ActionDosToUnix.getConversionTypeByCode(resultRow.getString(2, null));
+ // Get source file or folder, wildcard and conversion. A missing field
counts as empty.
+ String vSourceFileFolderPrevious = getResultRowString(resultRow, 0);
+ String vWildcardPrevious = getResultRowString(resultRow, 1);
+ int conversionType =
+
ActionDosToUnix.getConversionTypeByCode(getResultRowString(resultRow, 2));
if (isDetailed()) {
logDetailed(
@@ -285,6 +286,13 @@ public class ActionDosToUnix extends ActionBase {
return result;
}
+ private static String getResultRowString(RowMetaAndData row, int index)
throws HopValueException {
+ if (index >= row.size()) {
+ return null;
+ }
+ return row.getString(index, null);
+ }
+
private void displayResults() {
if (isDetailed()) {
logDetailed("=======================================");
@@ -501,13 +509,13 @@ public class ActionDosToUnix extends ActionBase {
if
(!currentFile.getParent().toString().equals(sourcefilefolder.toString())) {
// Not in the Base Folder..Only if include sub folders
- if (includeSubFolders &&
getFileWildcard(currentFile.toString(), realWildcard)) {
+ if (includeSubFolders && getFileWildcard(currentFile,
realWildcard)) {
convertOneFile(currentFile, convertion, result,
parentWorkflow);
}
} else {
// In the base folder
- if (getFileWildcard(currentFile.toString(), realWildcard)) {
+ if (getFileWildcard(currentFile, realWildcard)) {
convertOneFile(currentFile, convertion, result,
parentWorkflow);
}
}
@@ -646,25 +654,21 @@ public class ActionDosToUnix extends ActionBase {
}
/**********************************************************
+ *
+ * The wildcard is matched against the file name, like in the other file
actions. A match
+ * against the whole path is still accepted, so that existing path patterns
keep working.
*
* @param selectedfile
* @param wildcard
* @return True if the selectedfile matches the wildcard
**********************************************************/
- private boolean getFileWildcard(String selectedfile, String wildcard) {
- Pattern pattern = null;
- boolean getIt = true;
-
- if (!Utils.isEmpty(wildcard)) {
- pattern = Pattern.compile(wildcard);
- // First see if the file matches the regular expression!
- if (pattern != null) {
- Matcher matcher = pattern.matcher(selectedfile);
- getIt = matcher.matches();
- }
+ private boolean getFileWildcard(FileObject selectedfile, String wildcard) {
+ if (Utils.isEmpty(wildcard)) {
+ return true;
}
-
- return getIt;
+ Pattern pattern = Pattern.compile(wildcard);
+ return pattern.matcher(selectedfile.getName().getBaseName()).matches()
+ || pattern.matcher(selectedfile.toString()).matches();
}
@Override
diff --git
a/plugins/actions/dostounix/src/test/java/org/apache/hop/workflow/actions/dostounix/ActionDosToUnixExecuteTest.java
b/plugins/actions/dostounix/src/test/java/org/apache/hop/workflow/actions/dostounix/ActionDosToUnixExecuteTest.java
new file mode 100644
index 0000000000..73485c1844
--- /dev/null
+++
b/plugins/actions/dostounix/src/test/java/org/apache/hop/workflow/actions/dostounix/ActionDosToUnixExecuteTest.java
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.workflow.actions.dostounix;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.Mockito.mock;
+
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import org.apache.hop.core.Result;
+import org.apache.hop.core.RowMetaAndData;
+import org.apache.hop.core.logging.HopLogStore;
+import org.apache.hop.core.row.value.ValueMetaString;
+import org.apache.hop.core.variables.Variables;
+import org.apache.hop.workflow.WorkflowMeta;
+import org.apache.hop.workflow.action.ActionMeta;
+import org.apache.hop.workflow.engine.IWorkflowEngine;
+import org.apache.hop.workflow.engines.local.LocalWorkflowEngine;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+/** See https://github.com/apache/hop/issues/9117 */
+class ActionDosToUnixExecuteTest {
+
+ private static final String DOS_CONTENT = "line1\r\nline2\r\n";
+ private static final String UNIX_CONTENT = "line1\nline2\n";
+
+ @TempDir Path folder;
+
+ private ActionDosToUnix action;
+ private Path dosFile;
+
+ @BeforeAll
+ static void setUpBeforeClass() {
+ HopLogStore.init();
+ }
+
+ @BeforeEach
+ void setUp() throws Exception {
+ IWorkflowEngine<WorkflowMeta> workflow = new LocalWorkflowEngine(new
WorkflowMeta());
+ action = new ActionDosToUnix();
+ workflow.getWorkflowMeta().addAction(new ActionMeta(action));
+ action.setParentWorkflow(workflow);
+ action.setParentWorkflowMeta(mock(WorkflowMeta.class));
+ workflow.setStopped(false);
+ action.setResultFilenames("nothing");
+ // A real run inherits the system properties, the temporary folder among
them
+ action.initializeFrom(Variables.getADefaultVariableSpace());
+
+ dosFile = folder.resolve("dos.txt");
+ Files.writeString(dosFile, DOS_CONTENT, StandardCharsets.UTF_8);
+ }
+
+ @Test
+ void wildcardIsMatchedAgainstTheFileName() throws Exception {
+ addField(folder.toString(), "dos\\.txt");
+
+ Result result = action.execute(new Result(), 0);
+
+ assertTrue(result.isResult());
+ assertEquals(1, result.getNrLinesWritten(), "the file must be converted");
+ assertEquals(UNIX_CONTENT, Files.readString(dosFile,
StandardCharsets.UTF_8));
+ }
+
+ @Test
+ void wildcardOnTheWholePathStillMatches() throws Exception {
+ addField(folder.toString(), ".*/dos\\.txt");
+
+ Result result = action.execute(new Result(), 0);
+
+ assertTrue(result.isResult());
+ assertEquals(1, result.getNrLinesWritten(), "the file must be converted");
+ assertEquals(UNIX_CONTENT, Files.readString(dosFile,
StandardCharsets.UTF_8));
+ }
+
+ @Test
+ void resultRowWithOneFieldDoesNotStopTheWorkflow() throws Exception {
+ action.setArgFromPrevious(true);
+ RowMetaAndData row = new RowMetaAndData();
+ row.addValue(new ValueMetaString("filename"), dosFile.toString());
+ Result previous = new Result();
+ previous.setRows(List.of(row));
+
+ Result result = action.execute(previous, 0);
+
+ assertTrue(result.isResult());
+ assertEquals(0, result.getNrErrors());
+ assertEquals(1, result.getNrLinesWritten(), "the file must be converted
(guess)");
+ assertEquals(UNIX_CONTENT, Files.readString(dosFile,
StandardCharsets.UTF_8));
+ }
+
+ @Test
+ void resultRowWithTwoFieldsDoesNotStopTheWorkflow() throws Exception {
+ action.setArgFromPrevious(true);
+ RowMetaAndData row = new RowMetaAndData();
+ row.addValue(new ValueMetaString("folder"), folder.toString());
+ row.addValue(new ValueMetaString("wildcard"), "dos\\.txt");
+ Result previous = new Result();
+ previous.setRows(List.of(row));
+
+ Result result = action.execute(previous, 0);
+
+ assertTrue(result.isResult());
+ assertEquals(0, result.getNrErrors());
+ assertEquals(1, result.getNrLinesWritten(), "the file must be converted
(guess)");
+ }
+
+ private void addField(String sourceFileFolder, String wildcard) {
+ ActionDosToUnix.Fields field = new ActionDosToUnix.Fields();
+ field.setSourceFileFolder(sourceFileFolder);
+ field.setWildcard(wildcard);
+ field.setConversionTypes("dostounix");
+ action.getFields().add(field);
+ }
+}
diff --git
a/plugins/actions/folderisempty/src/main/java/org/apache/hop/workflow/actions/folderisempty/ActionFolderIsEmpty.java
b/plugins/actions/folderisempty/src/main/java/org/apache/hop/workflow/actions/folderisempty/ActionFolderIsEmpty.java
index 0473ec33cb..1ab0f98148 100644
---
a/plugins/actions/folderisempty/src/main/java/org/apache/hop/workflow/actions/folderisempty/ActionFolderIsEmpty.java
+++
b/plugins/actions/folderisempty/src/main/java/org/apache/hop/workflow/actions/folderisempty/ActionFolderIsEmpty.java
@@ -22,6 +22,7 @@ import java.io.Serial;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
+import java.util.regex.PatternSyntaxException;
import org.apache.commons.vfs2.FileObject;
import org.apache.commons.vfs2.FileSelectInfo;
import org.apache.commons.vfs2.FileSelector;
@@ -134,8 +135,18 @@ public class ActionFolderIsEmpty extends ActionBase
implements Cloneable, IActio
folderscount = 0;
pattern = null;
- if (!Utils.isEmpty(getWildcard())) {
- pattern = Pattern.compile(getRealWildcard());
+ // The wildcard only applies when the search is limited to it
+ String realWildcard = getRealWildcard();
+ if (isSpecifyWildcard() && !Utils.isEmpty(realWildcard)) {
+ try {
+ pattern = Pattern.compile(realWildcard);
+ } catch (PatternSyntaxException e) {
+ logError(
+ BaseMessages.getString(
+ PKG, "ActionFolderIsEmpty.Error.InvalidWildcard",
realWildcard, e.getMessage()));
+ result.setNrErrors(1);
+ return result;
+ }
}
if (folderName != null) {
diff --git
a/plugins/actions/folderisempty/src/main/resources/org/apache/hop/workflow/actions/folderisempty/messages/messages_en_US.properties
b/plugins/actions/folderisempty/src/main/resources/org/apache/hop/workflow/actions/folderisempty/messages/messages_en_US.properties
index d4df3c4478..870ada16a2 100644
---
a/plugins/actions/folderisempty/src/main/resources/org/apache/hop/workflow/actions/folderisempty/messages/messages_en_US.properties
+++
b/plugins/actions/folderisempty/src/main/resources/org/apache/hop/workflow/actions/folderisempty/messages/messages_en_US.properties
@@ -18,6 +18,7 @@
ActionFolderIsEmpty.Description=Check if a folder is empty
ActionFolderIsEmpty.Error=Error
ActionFolderIsEmpty.Error.Exception=Exception while getting file [{0}] : [{1}]
+ActionFolderIsEmpty.Error.InvalidWildcard=The wildcard [{0}] is not a valid
regular expression: {1}
ActionFolderIsEmpty.Foldername.Label=Folder name
ActionFolderIsEmpty.IncludeSubFolders.Label=Include subfolders
ActionFolderIsEmpty.IncludeSubFolders.Tooltip=Check this option if you want to
include sub folders in the search
diff --git
a/plugins/actions/folderisempty/src/test/java/org/apache/hop/workflow/actions/folderisempty/WorkflowActionFolderIsEmptyTest.java
b/plugins/actions/folderisempty/src/test/java/org/apache/hop/workflow/actions/folderisempty/WorkflowActionFolderIsEmptyTest.java
index 1fb6d2c2e7..31d04a91a5 100644
---
a/plugins/actions/folderisempty/src/test/java/org/apache/hop/workflow/actions/folderisempty/WorkflowActionFolderIsEmptyTest.java
+++
b/plugins/actions/folderisempty/src/test/java/org/apache/hop/workflow/actions/folderisempty/WorkflowActionFolderIsEmptyTest.java
@@ -90,4 +90,31 @@ class WorkflowActionFolderIsEmptyTest {
assertFalse(result.isResult(), "For non-empty folder result should be
false");
assertEquals(0, result.getNrErrors(), "There should be still no errors");
}
+
+ /** An invalid regular expression fails the action instead of the workflow,
see #9117. */
+ @Test
+ void invalidWildcardIsAnActionError() {
+ action.setFolderName(emptyDir);
+ action.setSpecifyWildcard(true);
+ action.setWildcard("${MASK}");
+ action.setVariable("MASK", "[");
+
+ Result result = action.execute(new Result(), 0);
+
+ assertFalse(result.isResult(), "an invalid wildcard must fail the action");
+ assertEquals(1, result.getNrErrors());
+ }
+
+ /** The wildcard is not used when the search is not limited to it, see
#9117. */
+ @Test
+ void invalidWildcardIsIgnoredWhenNotLimitingTheSearch() {
+ action.setFolderName(emptyDir);
+ action.setSpecifyWildcard(false);
+ action.setWildcard("[");
+
+ Result result = action.execute(new Result(), 0);
+
+ assertTrue(result.isResult(), "For empty folder result should be true");
+ assertEquals(0, result.getNrErrors());
+ }
}
diff --git
a/plugins/actions/folderscompare/src/main/java/org/apache/hop/workflow/actions/folderscompare/ActionFoldersCompare.java
b/plugins/actions/folderscompare/src/main/java/org/apache/hop/workflow/actions/folderscompare/ActionFoldersCompare.java
index 374e40314c..ba4ddc8833 100644
---
a/plugins/actions/folderscompare/src/main/java/org/apache/hop/workflow/actions/folderscompare/ActionFoldersCompare.java
+++
b/plugins/actions/folderscompare/src/main/java/org/apache/hop/workflow/actions/folderscompare/ActionFoldersCompare.java
@@ -25,6 +25,7 @@ import java.util.Map;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
+import java.util.regex.PatternSyntaxException;
import lombok.Getter;
import lombok.Setter;
import org.apache.commons.vfs2.FileObject;
@@ -34,6 +35,7 @@ import org.apache.commons.vfs2.FileType;
import org.apache.hop.core.ICheckResult;
import org.apache.hop.core.Result;
import org.apache.hop.core.annotations.Action;
+import org.apache.hop.core.exception.HopException;
import org.apache.hop.core.exception.HopFileException;
import org.apache.hop.core.io.CountingInputStream;
import org.apache.hop.core.util.Utils;
@@ -211,9 +213,12 @@ public class ActionFoldersCompare extends ActionBase
implements Cloneable, IActi
result.setResult(equalFileContents(folder1, folder2, result));
} else if (folder1.getType() == FileType.FOLDER) {
// We compare 2 folders ...
+ Pattern wildcardPattern = compileWildcard();
- FileObject[] list1 = folder1.findFiles(new
TextFileSelector(folder1.toString()));
- FileObject[] list2 = folder2.findFiles(new
TextFileSelector(folder2.toString()));
+ FileObject[] list1 =
+ folder1.findFiles(new TextFileSelector(folder1.toString(),
wildcardPattern));
+ FileObject[] list2 =
+ folder2.findFiles(new TextFileSelector(folder2.toString(),
wildcardPattern));
int lenList1 = list1.length;
int lenList2 = list2.length;
@@ -237,14 +242,17 @@ public class ActionFoldersCompare extends ActionBase
implements Cloneable, IActi
HashMap<String, String> collection1 = new HashMap<>();
HashMap<String, String> collection2 = new HashMap<>();
+ // Key the entries on their path relative to the compared
folder, so that
+ // s1/x.txt and s2/x.txt are different entries.
for (FileObject object : list1) {
- // Put files list1 in TreeMap collection1
- collection1.put(object.getName().getBaseName(),
object.toString());
+ collection1.put(
+ folder1.getName().getRelativeName(object.getName()),
object.toString());
}
for (FileObject fileObject : list2) {
- // Put files list2 in TreeMap collection2
- collection2.put(fileObject.getName().getBaseName(),
fileObject.toString());
+ collection2.put(
+ folder2.getName().getRelativeName(fileObject.getName()),
+ fileObject.toString());
}
// Let's now fetch Folder1
@@ -403,7 +411,7 @@ public class ActionFoldersCompare extends ActionBase
implements Cloneable, IActi
BaseMessages.getString(
PKG,
"ActionFoldersCompare.Log.ErrorComparing",
- realFilename2,
+ realFilename1,
realFilename2,
e.getMessage()));
} finally {
@@ -434,11 +442,13 @@ public class ActionFoldersCompare extends ActionBase
implements Cloneable, IActi
private class TextFileSelector implements FileSelector {
String sourceFolder = null;
+ private final Pattern wildcardPattern;
- public TextFileSelector(String sourcefolderin) {
+ public TextFileSelector(String sourcefolderin, Pattern wildcardPattern) {
if (!Utils.isEmpty(sourcefolderin)) {
sourceFolder = sourcefolderin;
}
+ this.wildcardPattern = wildcardPattern;
}
@Override
@@ -456,7 +466,7 @@ public class ActionFoldersCompare extends ActionBase
implements Cloneable, IActi
// In the Base Folder...
if ((info.getFile().getType() == FileType.FILE &&
compareOnly.equals("only_files"))
|| (info.getFile().getType() == FileType.FOLDER &&
compareOnly.equals("only_folders"))
- || (getFileWildcard(shortFilename) &&
compareOnly.equals("specify"))
+ || (compareOnly.equals("specify") &&
matchesWildcard(shortFilename))
|| (compareOnly.equals("all"))) {
returncode = true;
}
@@ -466,7 +476,7 @@ public class ActionFoldersCompare extends ActionBase
implements Cloneable, IActi
if ((includeSubFolders
&& (info.getFile().getType() == FileType.FILE &&
compareOnly.equals("only_files"))
|| (info.getFile().getType() == FileType.FOLDER &&
compareOnly.equals("only_folders"))
- || (getFileWildcard(shortFilename) &&
compareOnly.equals("specify"))
+ || (compareOnly.equals("specify") &&
matchesWildcard(shortFilename))
|| (compareOnly.equals("all")))) {
returncode = true;
}
@@ -485,24 +495,42 @@ public class ActionFoldersCompare extends ActionBase
implements Cloneable, IActi
public boolean traverseDescendents(FileSelectInfo info) {
return info.getDepth() == 0 || includeSubFolders;
}
+
+ /**
+ * @param selectedFile The selected file
+ * @return True if the selected file matches the wildcard, or if there is
no wildcard
+ */
+ private boolean matchesWildcard(String selectedFile) {
+ if (wildcardPattern == null) {
+ return true;
+ }
+ Matcher matcher = wildcardPattern.matcher(selectedFile);
+ return matcher.matches();
+ }
}
/**
- * @param selectedFile The selected file
- * @return True if the selected file matches the wildcard
+ * Compile the resolved wildcard. It is only used when the files to compare
are specified with a
+ * wildcard.
+ *
+ * @return the compiled wildcard, or null if there is no wildcard to apply
+ * @throws HopException if the wildcard is not a valid regular expression
*/
- private boolean getFileWildcard(String selectedFile) {
- Pattern pattern;
- boolean getIt = true;
-
- if (!Utils.isEmpty(wildcard)) {
- pattern = Pattern.compile(wildcard);
- // First see if the file matches the regular expression!
- Matcher matcher = pattern.matcher(selectedFile);
- getIt = matcher.matches();
+ private Pattern compileWildcard() throws HopException {
+ if (!"specify".equals(compareOnly)) {
+ return null;
+ }
+ String realWildcard = getRealWildcard();
+ if (Utils.isEmpty(realWildcard)) {
+ return null;
+ }
+ try {
+ return Pattern.compile(realWildcard);
+ } catch (PatternSyntaxException e) {
+ throw new HopException(
+ BaseMessages.getString(
+ PKG, "ActionFoldersCompare.Log.InvalidWildcard", realWildcard,
e.getMessage()));
}
-
- return getIt;
}
@Override
diff --git
a/plugins/actions/folderscompare/src/main/resources/org/apache/hop/workflow/actions/folderscompare/messages/messages_en_US.properties
b/plugins/actions/folderscompare/src/main/resources/org/apache/hop/workflow/actions/folderscompare/messages/messages_en_US.properties
index 306da33061..1b25f9475f 100644
---
a/plugins/actions/folderscompare/src/main/resources/org/apache/hop/workflow/actions/folderscompare/messages/messages_en_US.properties
+++
b/plugins/actions/folderscompare/src/main/resources/org/apache/hop/workflow/actions/folderscompare/messages/messages_en_US.properties
@@ -43,6 +43,7 @@ ActionFoldersCompare.Log.FilesNotSameSize=Files [{0}] and
[{1}] have not the sa
ActionFoldersCompare.Log.FilesNotSameType=Files [{0}] and [{1}] have not the
same type!
ActionFoldersCompare.Log.FolderContains=Folder [{0}] contains [{1}]
folder(s)/file(s)
ActionFoldersCompare.Log.FoldersDifferentFiles=Folders [{0}] and [{1}] have
not the same files count!
+ActionFoldersCompare.Log.InvalidWildcard=The wildcard [{0}] is not a valid
regular expression: {1}
ActionFoldersCompare.Log.IsAFile=[{0}] is a file.
ActionFoldersCompare.Log.IsAFolder=[{0}] is a folder.
ActionFoldersCompare.Log.IsUnknownFileType=Can not find [{0}] filetype.
diff --git
a/plugins/actions/folderscompare/src/test/java/org/apache/hop/workflow/actions/folderscompare/ActionFoldersCompareMatchingTest.java
b/plugins/actions/folderscompare/src/test/java/org/apache/hop/workflow/actions/folderscompare/ActionFoldersCompareMatchingTest.java
new file mode 100644
index 0000000000..c2d4f63dd9
--- /dev/null
+++
b/plugins/actions/folderscompare/src/test/java/org/apache/hop/workflow/actions/folderscompare/ActionFoldersCompareMatchingTest.java
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.workflow.actions.folderscompare;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.Mockito.mock;
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import org.apache.hop.core.Result;
+import org.apache.hop.core.logging.HopLogStore;
+import org.apache.hop.workflow.WorkflowMeta;
+import org.apache.hop.workflow.action.ActionMeta;
+import org.apache.hop.workflow.engine.IWorkflowEngine;
+import org.apache.hop.workflow.engines.local.LocalWorkflowEngine;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+/** How entries and wildcards are matched, see
https://github.com/apache/hop/issues/9117 */
+class ActionFoldersCompareMatchingTest {
+
+ @TempDir Path testFolder;
+
+ private ActionFoldersCompare action;
+ private Path left;
+ private Path right;
+
+ @BeforeAll
+ static void setUpBeforeClass() {
+ HopLogStore.init();
+ }
+
+ @BeforeEach
+ void setUp() throws IOException {
+ IWorkflowEngine<WorkflowMeta> workflow = new LocalWorkflowEngine(new
WorkflowMeta());
+ action = new ActionFoldersCompare();
+ workflow.getWorkflowMeta().addAction(new ActionMeta(action));
+ action.setParentWorkflow(workflow);
+ action.setParentWorkflowMeta(mock(WorkflowMeta.class));
+ workflow.setStopped(false);
+
+ left = Files.createDirectories(testFolder.resolve("left"));
+ right = Files.createDirectories(testFolder.resolve("right"));
+ action.setFilename1(left.toString());
+ action.setFilename2(right.toString());
+ action.setCompareOnly("all");
+ }
+
+ @Test
+ void sameFileNameInDifferentSubfoldersIsADifference() throws Exception {
+ write(left, "s1/x.txt");
+ write(left, "s2/x.txt");
+ write(right, "s1/x.txt");
+ write(right, "s2/y.txt");
+ action.setIncludeSubFolders(true);
+
+ assertFalse(action.execute(new Result(), 0).isResult());
+ }
+
+ @Test
+ void identicalSubfoldersAreEqual() throws Exception {
+ write(left, "s1/x.txt");
+ write(left, "s2/x.txt");
+ write(right, "s1/x.txt");
+ write(right, "s2/x.txt");
+ action.setIncludeSubFolders(true);
+
+ assertTrue(action.execute(new Result(), 0).isResult());
+ }
+
+ @Test
+ void wildcardIsResolved() throws Exception {
+ write(left, "a.csv");
+ write(left, "other.txt");
+ write(right, "a.csv");
+ write(right, "different.txt");
+ action.setCompareOnly("specify");
+ action.setWildcard("${MASK}");
+ action.setVariable("MASK", ".*\\.csv");
+
+ assertTrue(action.execute(new Result(), 0).isResult(), "only the csv files
are compared");
+
+ write(right, "b.csv");
+ assertFalse(action.execute(new Result(), 0).isResult(), "b.csv is only on
the right");
+ }
+
+ @Test
+ void invalidWildcardIsAnError() throws Exception {
+ write(left, "a.csv");
+ write(right, "b.csv");
+ action.setCompareOnly("specify");
+ action.setWildcard("[");
+
+ Result result = action.execute(new Result(), 0);
+
+ assertFalse(result.isResult());
+ assertEquals(1, result.getNrErrors());
+ }
+
+ @Test
+ void storedWildcardIsIgnoredWhenComparingAll() throws Exception {
+ write(left, "a.csv");
+ write(right, "b.csv");
+ action.setCompareOnly("all");
+ action.setWildcard("[");
+
+ Result result = action.execute(new Result(), 0);
+
+ assertFalse(result.isResult(), "a.csv and b.csv are different folders");
+ assertEquals(0, result.getNrErrors());
+ }
+
+ private static void write(Path root, String relativePath) throws IOException
{
+ Path file = root.resolve(relativePath);
+ Files.createDirectories(file.getParent());
+ Files.writeString(file, "same", StandardCharsets.UTF_8);
+ }
+}