This is an automated email from the ASF dual-hosted git repository.
hansva pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hop.git
The following commit(s) were added to refs/heads/main by this push:
new 0210484030 Put a file with FTP deletes local files it didn't upload,
HTTP emptie… (#9362)
0210484030 is described below
commit 0210484030d62c65309bcbe9c42dad400b80cc34
Author: Bart Maertens <[email protected]>
AuthorDate: Thu Nov 12 10:23:23 2026 +0100
Put a file with FTP deletes local files it didn't upload, HTTP emptie…
(#9362)
* Put a file with FTP deletes local files it didn't upload, HTTP empties
the target on an error, FTP timeout is in milliseconds, fixes #9242
* Address review on #9362: write HTTP replies to a part file, report FTP
skips at Basic level, fix timeout tooltips and translations
---
.../modules/ROOT/pages/workflow/actions/ftp.adoc | 2 +-
.../ROOT/pages/workflow/actions/ftpdelete.adoc | 2 +-
.../ROOT/pages/workflow/actions/ftpput.adoc | 8 +-
.../modules/ROOT/pages/workflow/actions/http.adoc | 4 +-
.../hop/workflow/actions/http/ActionHttp.java | 65 +++++++-
.../http/messages/messages_en_US.properties | 2 +
.../actions/http/ActionHttpExecutionTest.java | 184 +++++++++++++++++++++
.../hop/workflow/actions/ftpput/ActionFtpPut.java | 42 +++--
.../actions/ftp/messages/messages_de_DE.properties | 1 -
.../actions/ftp/messages/messages_en_US.properties | 4 +-
.../actions/ftp/messages/messages_es_AR.properties | 2 -
.../actions/ftp/messages/messages_fr_FR.properties | 2 -
.../actions/ftp/messages/messages_it_IT.properties | 2 -
.../actions/ftp/messages/messages_ja_JP.properties | 2 -
.../actions/ftp/messages/messages_ko_KR.properties | 2 -
.../actions/ftp/messages/messages_nl_NL.properties | 2 -
.../actions/ftp/messages/messages_pt_BR.properties | 2 -
.../actions/ftp/messages/messages_zh_CN.properties | 2 -
.../ftpdelete/messages/messages_en_US.properties | 4 +-
.../ftpdelete/messages/messages_es_AR.properties | 2 -
.../ftpdelete/messages/messages_fr_FR.properties | 2 -
.../ftpdelete/messages/messages_it_IT.properties | 2 -
.../ftpdelete/messages/messages_ja_JP.properties | 2 -
.../ftpdelete/messages/messages_ko_KR.properties | 2 -
.../ftpdelete/messages/messages_pt_BR.properties | 2 -
.../ftpdelete/messages/messages_zh_CN.properties | 2 -
.../ftpput/messages/messages_en_US.properties | 5 +-
.../ftpput/messages/messages_es_AR.properties | 2 -
.../ftpput/messages/messages_fr_FR.properties | 2 -
.../ftpput/messages/messages_it_IT.properties | 2 -
.../ftpput/messages/messages_ja_JP.properties | 2 -
.../ftpput/messages/messages_ko_KR.properties | 2 -
.../ftpput/messages/messages_pt_BR.properties | 2 -
.../ftpput/messages/messages_zh_CN.properties | 2 -
.../actions/ftpput/ActionFtpPutExecutionTest.java | 23 +++
35 files changed, 315 insertions(+), 75 deletions(-)
diff --git a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftp.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftp.adoc
index ff1c294643..e51ec80e87 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftp.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftp.adoc
@@ -50,7 +50,7 @@ This workflow retrieves files from one directory exclusively.
|Proxy password|Proxy server account password.
2+|Advanced
|Binary mode|Enable if files must be transferred in binary mode.
-|Timeout|The timeout in milliseconds, for example 30000 for 30 seconds. 0 (the
default): no timeout.
+|Timeout (ms)|The connect and reply timeout in milliseconds, for example 30000
for 30 seconds. 0 (the default) keeps the defaults of the FTP library: 60
seconds to connect, no limit on waiting for a reply.
|Use active FTP connection|Enable if you are connecting to the FTP server
using Active mode; you must set your firewall to accept connections to the port
that your FTP client will open.
The default is Passive mode.
|Control Encoding|Encoding matters when retrieving file names that contain
special characters.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpdelete.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpdelete.adoc
index 0e60a2448c..7e41fe926d 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpdelete.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpdelete.adoc
@@ -68,7 +68,7 @@ Existing workflows keep working and log a warning; new ones
should use Delete fi
|===
|Option|Description
2+|Advanced
-|Timeout|The timeout in milliseconds, for example 30000 for 30 seconds. 0 (the
default): no timeout.
+|Timeout (ms)|The connect and reply timeout in milliseconds, for example 30000
for 30 seconds. 0 (the default) keeps the defaults of the FTP library: 60
seconds to connect, no limit on waiting for a reply.
|Use active FTP connection|Enable if you are connecting to the FTP server
using Active mode; you must set your firewall to accept connections to the port
that your FTP client will open.
The default is Passive mode.
2+|Remote
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpput.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpput.adoc
index a47d6dd173..88d9c02ed1 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpput.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpput.adoc
@@ -45,7 +45,7 @@ The Put A File With FTP action uploads a file to an FTP
server.
|Test connection|Tests the configured connection.
2+|Advanced settings
|Binary mode|Enable if files must be transferred in binary mode.
-|Timeout|The timeout in milliseconds, for example 30000 for 30 seconds. 0 (the
default): no timeout.
+|Timeout (ms)|The connect and reply timeout in milliseconds, for example 30000
for 30 seconds. 0 (the default) keeps the defaults of the FTP library: 60
seconds to connect, no limit on waiting for a reply.
|Use active FTP connection|Enable if you are connecting to the FTP server
using Active mode; you must set your firewall to accept connections to the port
that your FTP client will open.
The default is Passive mode.
|Control encoding|Select the character encoding to use for determining special
characters in file names.
@@ -63,7 +63,7 @@ Select encoding that is valid for your server.
|Browse...|Browse the filesystem for a local directory.
|Wildcard (regular expression)|If you are uploading multiple files, specify a
regular expression that defines the set of files you want to upload.
|Remove files after transferal|Enable to remove files from the local directory
after they've been uploaded.
-|Don't overwrite files|Enable to skip files that already exist in the remote
directory. See the WARNING under <<Result>>.
+|Don't overwrite files|Enable to skip files that already exist in the remote
directory. A skipped file is not an error, and *Remove files after transferal*
leaves it in the local directory.
|Remote directory|The directory on the FTP server to upload files to.
|Test folder|Test if the remote directory exists. This connects to the FTP
server and attempts to access the specified directory. If this directory
doesn't exist, the upload fails.
|===
@@ -86,10 +86,8 @@ What this action passes on to the next action. See
xref:workflow/action-results.
[options="header", width="90%", cols="1,3"]
|===
|Result|Value
-|True|Every matching file was uploaded, with 0 errors. No file matching the
wildcard is also true.
+|True|Every matching file was uploaded, or skipped by *Don't overwrite files*,
with 0 errors. No file matching the wildcard is also true.
|False|A file can't be uploaded, the server can't be reached or refuses the
login, the remote directory doesn't exist, *Local directory* is empty or
doesn't exist, or the wildcard is not a valid regular expression. The number of
errors is the number of files that failed, or 1 when the action fails before
the upload.
|Result files|Passed on unchanged: the uploaded files are not added.
|Result rows|Passed on unchanged.
|===
-
-WARNING: With *Don't overwrite files* and *Remove files after transferal* both
checked, a local file that already exists on the server is not uploaded, but it
is still removed from the local directory.
diff --git a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/http.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/http.adoc
index 8b82383d0a..7d93fd8154 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/http.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/http.adoc
@@ -133,6 +133,6 @@ What this action passes on to the next action. See
xref:workflow/action-results.
|Result rows|Passed on unchanged.
|===
-WARNING: The target file is opened before the request is sent. A request that
fails, for example with a 404, leaves an empty target file: an existing file is
emptied, unless *Append to specified target file* is checked.
+The reply is first written to a part file next to the target (the target file
name followed by a random id and `.part`). Only when all of it arrived does the
part file replace the target, or get added to the end of it with *Append to
specified target file*. A request that fails, for example with a 404, or a
reply that breaks off halfway leaves an existing target file as it was and
doesn't create a new one. Because the part file is created before the request
is sent, a target folder that [...]
-NOTE: With *Run for every result row*, the rows are processed one after the
other and the action stops at the first row that fails. The result is true as
soon as one row succeeded: a first row that succeeds and a second one that
fails give true with 1 error.
+NOTE: With *Run for every result row*, the rows are processed one after the
other and the action stops at the first row that fails. One failed row makes
the action false with 1 error, even when the rows before it succeeded; their
target files are kept.
diff --git
a/plugins/actions/http/src/main/java/org/apache/hop/workflow/actions/http/ActionHttp.java
b/plugins/actions/http/src/main/java/org/apache/hop/workflow/actions/http/ActionHttp.java
index 8564150d40..47927b6b8a 100644
---
a/plugins/actions/http/src/main/java/org/apache/hop/workflow/actions/http/ActionHttp.java
+++
b/plugins/actions/http/src/main/java/org/apache/hop/workflow/actions/http/ActionHttp.java
@@ -36,6 +36,7 @@ import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
+import java.util.UUID;
import lombok.Getter;
import lombok.Setter;
import org.apache.commons.vfs2.FileObject;
@@ -295,10 +296,13 @@ public class ActionHttp extends ActionBase {
return result;
}
+ int rowsSucceeded = 0;
for (int i = 0; i < resultRows.size() && result.getNrErrors() == 0; i++) {
RowMetaAndData row = resultRows.get(i);
OutputStream outputFile = null;
+ FileObject targetFile = null;
+ FileObject partFile = null;
InputStream input = null;
long bytesReadThisRow = 0L;
long bytesWrittenThisRow = 0L;
@@ -337,12 +341,22 @@ public class ActionHttp extends ActionBase {
}
}
- // Create the output File...
- outputFile = new
CountingOutputStream(HopVfs.getOutputStream(realTargetFile, fileAppended));
-
URI uri = toUri(urlToUse);
HttpHost target = HttpClientManager.createHttpHost(uri);
+ // The reply goes to a part file next to the target, which only takes
the target's place
+ // once all of it arrived: a request that fails, or a reply that
breaks off halfway, leaves
+ // an existing target as it was. Creating the part file before the
request is sent also
+ // reports a target folder that is missing or read-only before the
server acted on it.
+ targetFile = HopVfs.getFileObject(realTargetFile);
+ if (fileAppended && targetFile.exists() && !targetFile.isWriteable()) {
+ throw new IOException(
+ BaseMessages.getString(
+ PKG, "ActionHTTP.Error.TargetNotWriteable",
HopVfs.getFriendlyURI(targetFile)));
+ }
+ partFile = HopVfs.getFileObject(realTargetFile + "." +
UUID.randomUUID() + ".part");
+ outputFile = new CountingOutputStream(HopVfs.getOutputStream(partFile,
false));
+
ClassicHttpRequest request =
Utils.isEmpty(realUploadFile) ? new HttpGet(uri) : new
HttpPost(uri);
addRequestHeaders(request);
@@ -427,6 +441,11 @@ public class ActionHttp extends ActionBase {
bytesWrittenThisRow += ((CountingOutputStream)
outputFile).getCount();
httpLineageResponseBytes = ((CountingInputStream)
input).getCount();
+ outputFile.close();
+ outputFile = null;
+ commitPartFile(partFile, targetFile, fileAppended);
+ partFile = null;
+
if (replyBuffer != null) {
storeReplyInVariable(resolvedReplyVariable,
replyBuffer.toByteArray(), contentType);
}
@@ -451,7 +470,7 @@ public class ActionHttp extends ActionBase {
result.getResultFiles().put(resultFile.getFile().toString(),
resultFile);
}
- result.setResult(true);
+ rowsSucceeded++;
if (parentWorkflow != null) {
LineageHttpIoEmitter.emitWorkflowActionHttpIo(
@@ -515,15 +534,53 @@ public class ActionHttp extends ActionBase {
BaseMessages.getString(PKG,
"ActionHTTP.Error.CanNotCloseStream", e.getMessage()));
result.setNrErrors(1);
}
+ deletePartFile(partFile);
}
result.setBytesReadThisAction(result.getBytesReadThisAction() +
bytesReadThisRow);
result.setBytesWrittenThisAction(result.getBytesWrittenThisAction() +
bytesWrittenThisRow);
}
+ // Decided once, here: one failed row fails the action, also after rows
that succeeded, and no
+ // rows at all (an empty list of result rows) is not a success either.
+ result.setResult(rowsSucceeded > 0 && result.getNrErrors() == 0);
return result;
}
+ /**
+ * Puts a reply that arrived completely in the target's place, or at the end
of the target with
+ * "Append to specified target file".
+ */
+ private static void commitPartFile(FileObject partFile, FileObject
targetFile, boolean append)
+ throws IOException {
+ if (append && targetFile.exists()) {
+ try (InputStream in = HopVfs.getInputStream(partFile);
+ OutputStream out = HopVfs.getOutputStream(targetFile, true)) {
+ in.transferTo(out);
+ }
+ partFile.delete();
+ } else {
+ HopVfs.moveFile(partFile, targetFile);
+ }
+ }
+
+ /** Removes the part file of a reply that never made it to the target. */
+ private void deletePartFile(FileObject partFile) {
+ if (partFile == null) {
+ return;
+ }
+ try {
+ partFile.delete();
+ } catch (Exception e) {
+ logError(
+ BaseMessages.getString(
+ PKG,
+ "ActionHTTP.Error.CanNotDeletePartFile",
+ HopVfs.getFriendlyURI(partFile),
+ e.getMessage()));
+ }
+ }
+
/**
* The URL to call, as a URI HttpClient can route. Anything without a scheme
and a host is
* rejected here rather than further down, where it would surface as a less
obvious error.
diff --git
a/plugins/actions/http/src/main/resources/org/apache/hop/workflow/actions/http/messages/messages_en_US.properties
b/plugins/actions/http/src/main/resources/org/apache/hop/workflow/actions/http/messages/messages_en_US.properties
index f329be5787..6b85547b87 100644
---
a/plugins/actions/http/src/main/resources/org/apache/hop/workflow/actions/http/messages/messages_en_US.properties
+++
b/plugins/actions/http/src/main/resources/org/apache/hop/workflow/actions/http/messages/messages_en_US.properties
@@ -24,6 +24,7 @@ ActionHTTP.Description=Gets or uploads a file using HTTP
(HyperText Transfer Pro
ActionHTTP.Connection.Label=REST connection
ActionHTTP.Connection.Tooltip=Optional. A REST connection supplies the whole
client: proxy, credentials, TLS and timeouts. Leave it empty to use the fields
of this action.
ActionHTTP.Error.CanNotCloseStream=Unable to close streams : {0}
+ActionHTTP.Error.CanNotDeletePartFile=Unable to remove the incomplete download
[{0}]: {1}
ActionHTTP.Error.CanNotSaveHTTPResult=I was unable to save the HTTP result to
file because of a I/O error: {0}
ActionHTTP.Error.ConnectionNotFound=REST connection [{0}] could not be found
ActionHTTP.Error.ConnectionNotLoaded=REST connection [{0}] could not be loaded
@@ -32,6 +33,7 @@ ActionHTTP.Error.ListingConnections=Error getting the list of
REST connections
ActionHTTP.Error.HttpStatus=The server returned HTTP status code {0} for URL
[{1}]
ActionHTTP.Error.NotValidURL=The specified URL [{0}] is not valid : {1}
ActionHTTP.Error.ProxyAuthenticationRequired=The proxy [{0}] requires
authentication. Fill in the proxy user name and password on the Proxy tab: the
credentials of the target server are not sent to the proxy.
+ActionHTTP.Error.TargetNotWriteable=The target file [{0}] is read-only
ActionHTTP.Error.UnableGetResultPrevious=Unable to get result from previous
action : can't continue.
ActionHTTP.Filetype.All=All files
ActionHTTP.IgnoreSsl.Label=Ignore SSL certificate check
diff --git
a/plugins/actions/http/src/test/java/org/apache/hop/workflow/actions/http/ActionHttpExecutionTest.java
b/plugins/actions/http/src/test/java/org/apache/hop/workflow/actions/http/ActionHttpExecutionTest.java
index fde34bea42..06e9027073 100644
---
a/plugins/actions/http/src/test/java/org/apache/hop/workflow/actions/http/ActionHttpExecutionTest.java
+++
b/plugins/actions/http/src/test/java/org/apache/hop/workflow/actions/http/ActionHttpExecutionTest.java
@@ -36,18 +36,25 @@ import java.nio.file.Files;
import java.nio.file.Path;
import java.security.KeyStore;
import java.security.SecureRandom;
+import java.util.ArrayList;
import java.util.List;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.stream.Stream;
import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import org.apache.commons.io.FileUtils;
import org.apache.hop.core.HopClientEnvironment;
import org.apache.hop.core.Result;
+import org.apache.hop.core.RowMetaAndData;
+import org.apache.hop.core.row.RowMeta;
+import org.apache.hop.core.row.value.ValueMetaString;
import org.apache.hop.junit.rules.RestoreHopEngineEnvironmentExtension;
import org.apache.hop.workflow.engines.local.LocalWorkflowEngine;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
+import org.junit.jupiter.api.io.TempDir;
class ActionHttpExecutionTest {
private static final String HTTP_HOST = "localhost";
@@ -62,6 +69,7 @@ class ActionHttpExecutionTest {
private static HttpServer httpServer;
private static HttpsServer httpsServer;
private static Path httpsKeyStorePath;
+ private static final AtomicInteger countedRequests = new AtomicInteger();
@BeforeAll
static void setupBeforeClass() throws Exception {
@@ -211,6 +219,156 @@ class ActionHttpExecutionTest {
assertNull(workflow.getVariable("HTTP_REPLY"));
}
+ @Test
+ void testHttpErrorStatusLeavesAnExistingTargetFileAlone(@TempDir Path dir)
throws Exception {
+ Path target = dir.resolve("notFound.txt");
+ Files.writeString(target, "previous content");
+
+ ActionHttp http = new ActionHttp();
+ http.setParentWorkflow(new LocalWorkflowEngine());
+ http.setRunForEveryRow(false);
+ http.setUrl(httpBaseUrl() + "/notFound");
+ http.setTargetFilename(target.toString());
+
+ Result result = http.execute(new Result(), 0);
+
+ assertFalse(result.getResult());
+ assertEquals(1, result.getNrErrors());
+ assertEquals("previous content", Files.readString(target));
+ assertEquals(List.of(target), filesIn(dir), "no part file is left behind");
+ }
+
+ @Test
+ void testHttpReplyThatBreaksOffLeavesAnExistingTargetFileAlone(@TempDir Path
dir)
+ throws Exception {
+ Path target = dir.resolve("truncated.txt");
+ Files.writeString(target, "previous content");
+
+ ActionHttp http = new ActionHttp();
+ http.setParentWorkflow(new LocalWorkflowEngine());
+ http.setRunForEveryRow(false);
+ http.setUrl(httpBaseUrl() + "/truncated");
+ http.setTargetFilename(target.toString());
+
+ Result result = http.execute(new Result(), 0);
+
+ assertFalse(result.getResult());
+ assertEquals(1, result.getNrErrors());
+ assertEquals("previous content", Files.readString(target));
+ assertEquals(List.of(target), filesIn(dir), "no part file is left behind");
+ }
+
+ @Test
+ void testHttpAppendAddsTheReplyToTheTargetFile(@TempDir Path dir) throws
Exception {
+ Path target = dir.resolve("append.txt");
+ Files.writeString(target, "previous content|");
+
+ ActionHttp http = new ActionHttp();
+ http.setParentWorkflow(new LocalWorkflowEngine());
+ http.setRunForEveryRow(false);
+ http.setAddFilenameToResult(false);
+ http.setFileAppended(true);
+ http.setUrl(httpBaseUrl() + "/downloadFile");
+ http.setTargetFilename(target.toString());
+
+ Result result = http.execute(new Result(), 0);
+
+ assertTrue(result.getResult());
+ assertEquals(0, result.getNrErrors());
+ assertEquals("previous content|download only payload",
Files.readString(target));
+ assertEquals(List.of(target), filesIn(dir), "no part file is left behind");
+ }
+
+ @Test
+ void testHttpAppendReplyThatBreaksOffLeavesTheTargetFileAlone(@TempDir Path
dir)
+ throws Exception {
+ Path target = dir.resolve("appendTruncated.txt");
+ Files.writeString(target, "previous content");
+
+ ActionHttp http = new ActionHttp();
+ http.setParentWorkflow(new LocalWorkflowEngine());
+ http.setRunForEveryRow(false);
+ http.setFileAppended(true);
+ http.setUrl(httpBaseUrl() + "/truncated");
+ http.setTargetFilename(target.toString());
+
+ Result result = http.execute(new Result(), 0);
+
+ assertFalse(result.getResult());
+ assertEquals("previous content", Files.readString(target));
+ assertEquals(List.of(target), filesIn(dir), "no part file is left behind");
+ }
+
+ @Test
+ void testHttpMissingTargetFolderFailsBeforeTheRequestIsSent(@TempDir Path
dir) throws Exception {
+ int requestsBefore = countedRequests.get();
+
+ ActionHttp http = new ActionHttp();
+ http.setParentWorkflow(new LocalWorkflowEngine());
+ http.setRunForEveryRow(false);
+ http.setUrl(httpBaseUrl() + "/countRequests");
+
http.setTargetFilename(dir.resolve("missing").resolve("reply.txt").toString());
+
+ Result result = http.execute(new Result(), 0);
+
+ assertFalse(result.getResult());
+ assertEquals(1, result.getNrErrors());
+ assertEquals(requestsBefore, countedRequests.get(), "the request must not
have been sent");
+ }
+
+ @Test
+ void testHttpForEveryRowFailsWhenALaterRowFails(@TempDir Path dir) throws
Exception {
+ Path first = dir.resolve("first.txt");
+ Path second = dir.resolve("second.txt");
+
+ RowMeta rowMeta = new RowMeta();
+ rowMeta.addValueMeta(new ValueMetaString("URL"));
+ rowMeta.addValueMeta(new ValueMetaString("UPLOAD"));
+ rowMeta.addValueMeta(new ValueMetaString("DESTINATION"));
+ List<RowMetaAndData> rows = new ArrayList<>();
+ rows.add(
+ new RowMetaAndData(
+ rowMeta, new Object[] {httpBaseUrl() + "/downloadFile", null,
first.toString()}));
+ rows.add(
+ new RowMetaAndData(
+ rowMeta, new Object[] {httpBaseUrl() + "/notFound", null,
second.toString()}));
+ Result previousResult = new Result();
+ previousResult.setRows(rows);
+
+ ActionHttp http = new ActionHttp();
+ http.setParentWorkflow(new LocalWorkflowEngine());
+ http.setRunForEveryRow(true);
+ http.setAddFilenameToResult(false);
+
+ Result result = http.execute(previousResult, 0);
+
+ assertFalse(result.getResult(), "a failed row must fail the action");
+ assertEquals(1, result.getNrErrors());
+ assertEquals("download only payload", Files.readString(first));
+ assertEquals(List.of(first), filesIn(dir), "a failed row must not create a
target file");
+ }
+
+ @Test
+ void testHttpForEveryRowWithoutRowsIsFalseWithoutErrors() {
+ Result previousResult = new Result();
+ previousResult.setRows(new ArrayList<>());
+
+ ActionHttp http = new ActionHttp();
+ http.setParentWorkflow(new LocalWorkflowEngine());
+ http.setRunForEveryRow(true);
+
+ Result result = http.execute(previousResult, 0);
+
+ assertFalse(result.getResult());
+ assertEquals(0, result.getNrErrors());
+ }
+
+ private static List<Path> filesIn(Path dir) throws IOException {
+ try (Stream<Path> files = Files.list(dir)) {
+ return files.sorted().toList();
+ }
+ }
+
private static void startHttpServer() throws IOException {
httpServer = HttpServer.create(new
InetSocketAddress(ActionHttpExecutionTest.HTTP_HOST, 0), 10);
httpServer.createContext(
@@ -242,6 +400,32 @@ class ActionHttpExecutionTest {
}
httpExchange.close();
});
+ httpServer.createContext(
+ "/notFound",
+ httpExchange -> {
+ httpExchange.sendResponseHeaders(404, -1);
+ httpExchange.close();
+ });
+ httpServer.createContext(
+ "/truncated",
+ httpExchange -> {
+ // Promise 1000 bytes, send a few and hang up: the client sees the
reply break off.
+ httpExchange.sendResponseHeaders(200, 1000);
+ try {
+ OutputStream os = httpExchange.getResponseBody();
+ os.write("partial".getBytes(UTF_8));
+ os.flush();
+ } finally {
+ httpExchange.close();
+ }
+ });
+ httpServer.createContext(
+ "/countRequests",
+ httpExchange -> {
+ countedRequests.incrementAndGet();
+ httpExchange.sendResponseHeaders(200, -1);
+ httpExchange.close();
+ });
httpServer.start();
}
diff --git
a/plugins/tech/ftp/src/main/java/org/apache/hop/workflow/actions/ftpput/ActionFtpPut.java
b/plugins/tech/ftp/src/main/java/org/apache/hop/workflow/actions/ftpput/ActionFtpPut.java
index 44c96de2d5..6c05fac6ad 100644
---
a/plugins/tech/ftp/src/main/java/org/apache/hop/workflow/actions/ftpput/ActionFtpPut.java
+++
b/plugins/tech/ftp/src/main/java/org/apache/hop/workflow/actions/ftpput/ActionFtpPut.java
@@ -207,6 +207,7 @@ public class ActionFtpPut extends ActionBase implements
Cloneable, IAction, IFtp
public Result execute(Result prevResult, int nr) throws HopException {
prevResult.setResult(false);
long filesPut = 0;
+ long filesSkipped = 0;
long errors = 0;
if (isDetailed()) {
@@ -229,17 +230,28 @@ public class ActionFtpPut extends ActionBase implements
Cloneable, IAction, IFtp
if (!shouldProcessFile(file, pattern)) {
continue;
}
- if (uploadFile(ftpClient, realLocalDirectory, file)) {
- filesPut++;
- deleteLocalFileIfNeeded(childUri(realLocalDirectory, file));
- } else {
- errors++;
+ switch (uploadFile(ftpClient, realLocalDirectory, file)) {
+ case UPLOADED -> {
+ filesPut++;
+ deleteLocalFileIfNeeded(childUri(realLocalDirectory, file));
+ }
+ // A file left alone because the server already has one is not an
error, but it never
+ // reached the server: don't count it as put and don't remove the
local copy.
+ case SKIPPED -> filesSkipped++;
+ case FAILED -> errors++;
}
}
if (isBasic()) {
logBasic(
BaseMessages.getString(PKG, "ActionFtpPut.Log.WeHavePut",
String.valueOf(filesPut)));
+ // Without this a run where every file was already there reads like a
wildcard that
+ // matched nothing, with the files still sitting in the local folder.
+ if (filesSkipped > 0) {
+ logBasic(
+ BaseMessages.getString(
+ PKG, "ActionFtpPut.Log.SkippedExistingCount",
String.valueOf(filesSkipped)));
+ }
}
} catch (Exception e) {
errors++;
@@ -384,9 +396,10 @@ public class ActionFtpPut extends ActionBase implements
Cloneable, IAction, IFtp
* @param ftpClient the {@link FTPClient} instance
* @param localDir the directory containing the file
* @param file the file name
- * @return true if the file was uploaded, or deliberately skipped because it
was already there
+ * @return whether the file was uploaded, deliberately skipped because it
was already there, or
+ * failed
*/
- private boolean uploadFile(FTPClient ftpClient, String localDir, String
file) {
+ private UploadOutcome uploadFile(FTPClient ftpClient, String localDir,
String file) {
String localFilename = childUri(localDir, file);
try {
if (FtpClientFactory.fileExists(ftpClient, file)) {
@@ -396,7 +409,7 @@ public class ActionFtpPut extends ActionBase implements
Cloneable, IAction, IFtp
if (isDetailed()) {
logDetailed(BaseMessages.getString(PKG,
"ActionFtpPut.Log.SkippedExisting", file));
}
- return true;
+ return UploadOutcome.SKIPPED;
}
ftpClient.deleteFile(file);
}
@@ -409,16 +422,23 @@ public class ActionFtpPut extends ActionBase implements
Cloneable, IAction, IFtp
"ActionFtpPut.Log.UploadFailed",
localFilename,
Const.NVL(ftpClient.getReplyString(), "").trim()));
- return false;
+ return UploadOutcome.FAILED;
}
}
- return true;
+ return UploadOutcome.UPLOADED;
} catch (Exception e) {
logError(BaseMessages.getString(PKG, "ActionFtpPut.Log.UploadFailed",
localFilename, ""), e);
- return false;
+ return UploadOutcome.FAILED;
}
}
+ /** What happened to a single file in {@link #uploadFile(FTPClient, String,
String)}. */
+ enum UploadOutcome {
+ UPLOADED,
+ SKIPPED,
+ FAILED
+ }
+
/**
* Deletes the source file if the "remove" option is enabled. Goes through
VFS, like everything
* else this action reads.
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_de_DE.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_de_DE.properties
index 20f2b17f2a..90decd8a2a 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_de_DE.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_de_DE.properties
@@ -76,5 +76,4 @@ ActionFtp.Server.Label=FTP-Server / IP-Adresse:
ActionFtp.Server.Tooltip=Der FTP-Server oder die IP-Adresse
ActionFtp.TargetDir.Label=Zielverzeichnis:
ActionFtp.TargetDir.Tooltip=Das lokale Verzeichnis
-ActionFtp.Timeout.Label=Timeout:
ActionFtp.User.Label=Nutzer:
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_en_US.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_en_US.properties
index a3b9d32cc5..7fde06b68a 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_en_US.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_en_US.properties
@@ -133,8 +133,8 @@ ActionFtp.TargetDir.Tooltip=The target directory on the
local server
ActionFtp.TestConnection.Label=Test connection
ActionFtp.TestConnection.Tooltip=Test connection
ActionFtp.TestFolderExists.Label=Check folder
-ActionFtp.Timeout.Label=Timeout
-ActionFtp.Timeout.Tooltip=The timeout when transferring in seconds (0 for no
timeout)
+ActionFtp.Timeout.Label=Timeout (ms)
+ActionFtp.Timeout.Tooltip=The connect and reply timeout in milliseconds. 0
keeps the defaults of the FTP library: 60 seconds to connect, no limit on
waiting for a reply
ActionFtp.Title=Get a file with FTP
ActionFtp.UnexpectedError=Unexpected error : {0}
ActionFtp.User.Label=Username
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_es_AR.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_es_AR.properties
index 19567c3b14..4131d1481c 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_es_AR.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_es_AR.properties
@@ -127,8 +127,6 @@ ActionFtp.TargetDir.Tooltip=El directorio destino en el
servidor local
ActionFtp.TestConnection.Label=Probar conexión
ActionFtp.TestConnection.Tooltip=Probar conexión
ActionFtp.TestFolderExists.Label=Verificar directorio
-ActionFtp.Timeout.Label=Límite de tiempo:
-ActionFtp.Timeout.Tooltip=El límite de tiempo para la transferencia, en
segundos (0 para sin límite)
ActionFtp.Title=Obtener un archivo con FTP
ActionFtp.UnexpectedError=Error inesperado: {0}
ActionFtp.User.Label=Nombre de usuario:
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_fr_FR.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_fr_FR.properties
index 4902a01469..6c602affed 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_fr_FR.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_fr_FR.properties
@@ -129,8 +129,6 @@ ActionFtp.TargetDir.Tooltip=Le réperoire cible sur le
serveur distant
ActionFtp.TestConnection.Label=Tester connexion
ActionFtp.TestConnection.Tooltip=Tester la connexion FTP
ActionFtp.TestFolderExists.Label=Vérifier répertoire
-ActionFtp.Timeout.Label=Latence
-ActionFtp.Timeout.Tooltip=Latence en secondes (0 pas de latence)
ActionFtp.Title=Récupération fichiers via FTP
ActionFtp.UnexpectedError=Erreur inattendue : {0}
ActionFtp.User.Label=Compte utilisateur
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_it_IT.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_it_IT.properties
index b5d41ddb3f..4c0f0d530d 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_it_IT.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_it_IT.properties
@@ -127,8 +127,6 @@ ActionFtp.TargetDir.Tooltip=La cartella di destinazione sul
server locale
ActionFtp.TestConnection.Label=Test di connessione
ActionFtp.TestConnection.Tooltip=Test di connessione
ActionFtp.TestFolderExists.Label=Controlla cartella
-ActionFtp.Timeout.Label=Timeout:
-ActionFtp.Timeout.Tooltip=Il timeout di trasferimento in secondi (0 per nessun
timeout)
ActionFtp.Title=Preleva file con FTP
ActionFtp.UnexpectedError=Errore inatteso: {0}
ActionFtp.User.Label=Username:
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_ja_JP.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_ja_JP.properties
index 303d47224e..08fb867daa 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_ja_JP.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_ja_JP.properties
@@ -129,8 +129,6 @@ ActionFtp.TargetDir.Tooltip=移動先のローカルディレクトリを入力
ActionFtp.TestConnection.Label=テスト接続
ActionFtp.TestConnection.Tooltip=テスト接続
ActionFtp.TestFolderExists.Label=ディレクトリの確認
-ActionFtp.Timeout.Label=タイムアウト:
-ActionFtp.Timeout.Tooltip=タイムアウトを秒単位で入力してください。(0の場合はタイムアウトなし)
ActionFtp.Title=FTPファイル取得
ActionFtp.UnexpectedError=Unexpected error : {0}
ActionFtp.User.Label=ユーザ名:
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_ko_KR.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_ko_KR.properties
index dfebfbf0b2..ca15c54589 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_ko_KR.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_ko_KR.properties
@@ -126,8 +126,6 @@ ActionFtp.TargetDir.Tooltip=로컬 서버의 대상 디렉토리
ActionFtp.TestConnection.Label=연결 테스트
ActionFtp.TestConnection.Tooltip=연결 테스트
ActionFtp.TestFolderExists.Label=폴더 체크
-ActionFtp.Timeout.Label=시간초과:
-ActionFtp.Timeout.Tooltip=초단위 전송 타임아웃 (0은 타임아웃 없음)
ActionFtp.Title=Get a file with FTP
ActionFtp.UnexpectedError=알 수 없는 오류 : {0}
ActionFtp.User.Label=사용자:
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_nl_NL.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_nl_NL.properties
index d34a0cc426..994c5e438a 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_nl_NL.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_nl_NL.properties
@@ -48,8 +48,6 @@ ActionFTP.Start=Dit is het begin van de FTP action
ActionFTP.Started=De FTP workflow naar {0} werd gestart.
ActionFtp.TargetDir.Label=De lokale doel folder:
ActionFtp.TargetDir.Tooltip=Dit is de folder op de lokale machine waar de
bestanden in zullen terecht komen.
-ActionFtp.Timeout.Label=Time out:
-ActionFtp.Timeout.Tooltip=Hier kan U de time-out opgeven in seconden. (0
betekent: geen time-out)\nNa deze periode wordt een overdracht afgebroken
indien er geen verdere aktiviteit is.
ActionFtp.Title=Overdracht van bestanden via FTP...
ActionFtp.User.Label=Gebruikersnaam:
ActionFtp.User.Tooltip=Geef hier de naam van de gebruiker op de FTP server
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_pt_BR.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_pt_BR.properties
index 1eb921fa49..8fdcce6251 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_pt_BR.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_pt_BR.properties
@@ -89,8 +89,6 @@ ActionFtp.TargetDir.Label=diretório alvo:
ActionFtp.TestConnection.Label=Testar conexão
ActionFtp.TestConnection.Tooltip=Testar conexão
ActionFtp.TestFolderExists.Label=checar pasta
-ActionFtp.Timeout.Label=Limite de tempo:
-ActionFtp.Timeout.Tooltip=Limite de tempo para a transferência em segundos ( 0
para sem limite )
ActionFtp.Title=Obter arquivo via FTP
ActionFtp.UnexpectedError=Erro inesperado : {0}
ActionFtp.User.Label=Nome de usuário:
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_zh_CN.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_zh_CN.properties
index 4d5673bd12..2e74ed78f2 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_zh_CN.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_zh_CN.properties
@@ -130,8 +130,6 @@ ActionFtp.TargetDir.Tooltip=本地目录
ActionFtp.TestConnection.Label=测试连接
ActionFtp.TestConnection.Tooltip=测试连接
ActionFtp.TestFolderExists.Label=检查文件夹
-ActionFtp.Timeout.Label=超时:
-ActionFtp.Timeout.Tooltip=传输超时限制,以秒为单位 (0为没有超时)
ActionFtp.Title=FTP 下载
ActionFtp.UnexpectedError=未知错误:{0}
ActionFtp.User.Label=用户名:
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_en_US.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_en_US.properties
index e4c4543801..a567ea0227 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_en_US.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_en_US.properties
@@ -94,8 +94,8 @@ ActionFtpDelete.Tab.Socks.Label=Socks Proxy
ActionFtpDelete.TestConnection.Label=Test connection
ActionFtpDelete.TestConnection.Tooltip=Test connection
ActionFtpDelete.TestFolderExists.Label=Check folder
-ActionFtpDelete.Timeout.Label=Timeout
-ActionFtpDelete.Timeout.Tooltip=The timeout when transferring in seconds (0
for no timeout)
+ActionFtpDelete.Timeout.Label=Timeout (ms)
+ActionFtpDelete.Timeout.Tooltip=The connect and reply timeout in milliseconds.
0 keeps the defaults of the FTP library: 60 seconds to connect, no limit on
waiting for a reply
ActionFtpDelete.Title=FTP delete
ActionFtpDelete.useProxy.Label=Use proxy
ActionFtpDelete.useProxy.Tooltip=Use proxy
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_es_AR.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_es_AR.properties
index fcae450877..93a38d5a24 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_es_AR.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_es_AR.properties
@@ -88,8 +88,6 @@ ActionFtpDelete.Tab.Socks.Label=Proxy Socks
ActionFtpDelete.TestConnection.Label=Verificar conexión
ActionFtpDelete.TestConnection.Tooltip=Verificar conexión
ActionFtpDelete.TestFolderExists.Label=Verificar directorio
-ActionFtpDelete.Timeout.Label=Límite de tiempo:
-ActionFtpDelete.Timeout.Tooltip=El límite de tiempo durante la transferencia,
en segundos (0 para ilimitado)
ActionFtpDelete.Title=Eliminar archivos vía FTP
ActionFtpDelete.useProxy.Label=Utilizar proxy
ActionFtpDelete.useProxy.Tooltip=Utilizar proxy
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_fr_FR.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_fr_FR.properties
index 40df4363f7..d9b200c372 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_fr_FR.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_fr_FR.properties
@@ -92,8 +92,6 @@ ActionFtpDelete.Tab.Socks.Label=Proxy
ActionFtpDelete.TestConnection.Label=Tester connexion
ActionFtpDelete.TestConnection.Tooltip=Tester la connexion
ActionFtpDelete.TestFolderExists.Label=Vérifier le répertoire
-ActionFtpDelete.Timeout.Label=Temps réponse
-ActionFtpDelete.Timeout.Tooltip=Latence
ActionFtpDelete.Title=Suppression fichiers via FTP
ActionFtpDelete.useProxy.Label=Utiliser proxy
ActionFtpDelete.useProxy.Tooltip=Utiliser un proxy
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_it_IT.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_it_IT.properties
index 08ece1e755..17d50cdcf1 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_it_IT.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_it_IT.properties
@@ -89,8 +89,6 @@ ActionFtpDelete.Tab.Socks.Label=Proxy Socks
ActionFtpDelete.TestConnection.Label=Prova la connessione
ActionFtpDelete.TestConnection.Tooltip=Testa la connessione
ActionFtpDelete.TestFolderExists.Label=Controlla cartella
-ActionFtpDelete.Timeout.Label=Timeout:
-ActionFtpDelete.Timeout.Tooltip=Timeout in secondi durante il trasferimento (0
per nessun timeout)
ActionFtpDelete.Title=Cancella file da FTP
ActionFtpDelete.useProxy.Label=Utilizza proxy
ActionFtpDelete.useProxy.Tooltip=Utilizza proxy
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_ja_JP.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_ja_JP.properties
index 59bc03d4f2..211b491a17 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_ja_JP.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_ja_JP.properties
@@ -92,8 +92,6 @@ ActionFtpDelete.Tab.Socks.Label=Socks プロキシ
ActionFtpDelete.TestConnection.Label=テスト接続
ActionFtpDelete.TestConnection.Tooltip=リモートサーバへテスト接続をします。
ActionFtpDelete.TestFolderExists.Label=フォルダ確認
-ActionFtpDelete.Timeout.Label=タイムアウト
-ActionFtpDelete.Timeout.Tooltip=秒単位の転送タイムアウト (タイムアウトなしは0)
ActionFtpDelete.Title=FTPを通してファイルを削除
ActionFtpDelete.useProxy.Label=プロキシを使用する
ActionFtpDelete.useProxy.Tooltip=プロキシを使用する場合はチェックボックスを有効にしてください。
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_ko_KR.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_ko_KR.properties
index d5980c30cd..b9da7b1abf 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_ko_KR.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_ko_KR.properties
@@ -88,8 +88,6 @@ ActionFtpDelete.Tab.Socks.Label=Socks 프록시
ActionFtpDelete.TestConnection.Label=연결 테스트
ActionFtpDelete.TestConnection.Tooltip=연결 테스트
ActionFtpDelete.TestFolderExists.Label=폴더 체크
-ActionFtpDelete.Timeout.Label=타임아웃:
-ActionFtpDelete.Timeout.Tooltip=초단위의 전송 타임아웃 (0은 타임아웃 없음)
ActionFtpDelete.Title=FTP delete
ActionFtpDelete.useProxy.Label=프록시 사용
ActionFtpDelete.useProxy.Tooltip=프록시 사용
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_pt_BR.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_pt_BR.properties
index 51dd3e0c65..9d0e57c4d1 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_pt_BR.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_pt_BR.properties
@@ -71,8 +71,6 @@ ActionFtpDelete.Tab.Socks.Label=Proxy para socks
ActionFtpDelete.TestConnection.Label=Testar conexão
ActionFtpDelete.TestConnection.Tooltip=Testar conexão
ActionFtpDelete.TestFolderExists.Label=checar pasta
-ActionFtpDelete.Timeout.Label=Limite de tempo:
-ActionFtpDelete.Timeout.Tooltip=Limite de tempo para a transferência em
segundos ( 0 para sem limite )
ActionFtpDelete.Title=deleção FTP
ActionFtpDelete.useProxy.Label=Usar proxy
ActionFtpDelete.useProxy.Tooltip=Usar proxy
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_zh_CN.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_zh_CN.properties
index d0f0911788..5b88271ed5 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_zh_CN.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpdelete/messages/messages_zh_CN.properties
@@ -94,8 +94,6 @@ ActionFtpDelete.Tab.Socks.Label=socks代理
ActionFtpDelete.TestConnection.Label=测试连接
ActionFtpDelete.TestConnection.Tooltip=测试连接
ActionFtpDelete.TestFolderExists.Label=检查文件夹
-ActionFtpDelete.Timeout.Label=超时:
-ActionFtpDelete.Timeout.Tooltip=超时时间, 以秒为单位, 0表示无限制
ActionFtpDelete.Title=FTP删除
ActionFtpDelete.useProxy.Label=使用代理
ActionFtpDelete.useProxy.Tooltip=使用代理
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_en_US.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_en_US.properties
index 54205a90a5..1e8d1c8d26 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_en_US.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_en_US.properties
@@ -84,8 +84,8 @@ ActionFtpPut.TargetSettings.Group.Label=Target (remote) files
ActionFtpPut.TestConnection.Label=Test connection
ActionFtpPut.TestConnection.Tooltip=Test FTP connection
ActionFtpPut.TestFolderExists.Label=Test folder
-ActionFtpPut.Timeout.Label=Timeout
-ActionFtpPut.Timeout.Tooltip=The timeout when transferring in seconds (0 for
no timeout)
+ActionFtpPut.Timeout.Label=Timeout (ms)
+ActionFtpPut.Timeout.Tooltip=The connect and reply timeout in milliseconds. 0
keeps the defaults of the FTP library: 60 seconds to connect, no limit on
waiting for a reply
ActionFtpPut.Title=Put a file with FTP
ActionFtpPut.Username.Label=Username
ActionFtpPut.Wildcard.Label=Wildcard (regular expression)
@@ -94,6 +94,7 @@ ActionFtpPut.Connection.Label=FTP connection
ActionFtpPut.Connection.Tooltip=An FTP connection from the metadata. Pick one
and the server settings below come from that connection; leave it empty to
configure the server here.
ActionFtpPut.Log.RemoteDirectoryNotFound=The remote directory [{0}] does not
exist on the FTP server
ActionFtpPut.Log.SkippedExisting=Skipped [{0}]: it is already on the server
and only new files are uploaded
+ActionFtpPut.Log.SkippedExistingCount=Skipped [{0}] file(s) that are already
on the FTP server: only new files are uploaded, and the skipped files stay in
the local folder
ActionFtpPut.Log.UploadFailed=Unable to upload the file [{0}]. {1}
ActionFtpPut.Log.DeleteFailed=Unable to delete the source file [{0}]
ActionFtpPut.LocalDir.NotFound=The source directory [{0}] does not exist
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_es_AR.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_es_AR.properties
index 7e5fca6012..8f42ccd163 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_es_AR.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_es_AR.properties
@@ -82,8 +82,6 @@ ActionFtpPut.TargetSettings.Group.Label=Archivos destino
(remotos)
ActionFtpPut.TestConnection.Label=Verificar conexión
ActionFtpPut.TestConnection.Tooltip=Verificar la conexión FTP
ActionFtpPut.TestFolderExists.Label=Verificar la carpeta
-ActionFtpPut.Timeout.Label=Límite de tiempo:
-ActionFtpPut.Timeout.Tooltip=El límite de tiempo para la transferencia, en
segundos (0 para no limitar)
ActionFtpPut.Title=Enviar archivos vía FTP...
ActionFtpPut.Username.Label=Nombre de usuario:
ActionFtpPut.Wildcard.Label=Comodín (expresión regular):
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_fr_FR.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_fr_FR.properties
index b98dfcc3b9..f93800653d 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_fr_FR.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_fr_FR.properties
@@ -86,8 +86,6 @@ ActionFtpPut.TargetSettings.Group.Label=Cible (distant)
fichiers
ActionFtpPut.TestConnection.Label=Tester connexion
ActionFtpPut.TestConnection.Tooltip=Tester la connexion FTP
ActionFtpPut.TestFolderExists.Label=Tester dossier
-ActionFtpPut.Timeout.Label=Délai d''expiration
-ActionFtpPut.Timeout.Tooltip=La latence en secondes (0 pas de latence)
ActionFtpPut.Title=Envoi fichiers via FTP ...
ActionFtpPut.Username.Label=Compte utilisateur
ActionFtpPut.Wildcard.Label=Caractères joker (RegExp)
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_it_IT.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_it_IT.properties
index 1eef84ab8b..498d30e723 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_it_IT.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_it_IT.properties
@@ -82,8 +82,6 @@ ActionFtpPut.TargetSettings.Group.Label=File di destinazione
(remoti)
ActionFtpPut.TestConnection.Label=Prova la connessione
ActionFtpPut.TestConnection.Tooltip=Testa la connessione FTP
ActionFtpPut.TestFolderExists.Label=Testa la cartella
-ActionFtpPut.Timeout.Label=Timeout:
-ActionFtpPut.Timeout.Tooltip=Timeout durante il trasferimento in secondi (0
per nessun timeout)
ActionFtpPut.Title=Carica file via FTP...
ActionFtpPut.Username.Label=Username:
ActionFtpPut.Wildcard.Label=Wildcard (espressione regolare):
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_ja_JP.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_ja_JP.properties
index 9cfab4c9a2..12d730cb50 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_ja_JP.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_ja_JP.properties
@@ -83,8 +83,6 @@ ActionFtpPut.TargetSettings.Group.Label=リモート設定
ActionFtpPut.TestConnection.Label=テスト接続
ActionFtpPut.TestConnection.Tooltip=リモートサーバへテスト接続をします。
ActionFtpPut.TestFolderExists.Label=ディレクトリの確認
-ActionFtpPut.Timeout.Label=タイムアウト
-ActionFtpPut.Timeout.Tooltip=タイムアウトを秒単位で入力してください。(0の場合はタイムアウトなし)
ActionFtpPut.Title=FTP 経由でファイルを転送 ...
ActionFtpPut.Username.Label=ユーザー名
ActionFtpPut.Wildcard.Label=検索文字列
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_ko_KR.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_ko_KR.properties
index 1e7d3af6ee..2679e73efe 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_ko_KR.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_ko_KR.properties
@@ -81,8 +81,6 @@ ActionFtpPut.TargetSettings.Group.Label=대상 (원격) 파일
ActionFtpPut.TestConnection.Label=연결 테스트
ActionFtpPut.TestConnection.Tooltip=FTP 연결 테스트
ActionFtpPut.TestFolderExists.Label=테스트 폴더
-ActionFtpPut.Timeout.Label=타임아웃:
-ActionFtpPut.Timeout.Tooltip=전송 중 초단위 타임아웃 (0은 타임아웃 없음)
ActionFtpPut.Title=Put a file with FTP
ActionFtpPut.Username.Label=사용자:
ActionFtpPut.Wildcard.Label=와일드카드 (정규표현식):
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_pt_BR.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_pt_BR.properties
index f0926cacc6..a29784e168 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_pt_BR.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_pt_BR.properties
@@ -58,8 +58,6 @@ ActionFtpPut.Tab.General.Label=Geral
ActionFtpPut.Tab.Socks.Label=proxy para socks
ActionFtpPut.TestConnection.Label=Testar conexão
ActionFtpPut.TestFolderExists.Label=pasta para teste
-ActionFtpPut.Timeout.Label=Limite de tempo:
-ActionFtpPut.Timeout.Tooltip=Limite de tempo para a transferência em segundos
( 0 para sem limite )
ActionFtpPut.Title=enviar arquivo por FTP
ActionFtpPut.Username.Label=Nome de usuário:
ActionFtpPut.Wildcard.Label=Máscara ( expressão regular ):
diff --git
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_zh_CN.properties
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_zh_CN.properties
index 50fb7bbf23..111a6e4165 100644
---
a/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_zh_CN.properties
+++
b/plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_zh_CN.properties
@@ -86,8 +86,6 @@ ActionFtpPut.TargetSettings.Group.Label=目标(远程)文件
ActionFtpPut.TestConnection.Label=测试连接
ActionFtpPut.TestConnection.Tooltip=测试FTP连接
ActionFtpPut.TestFolderExists.Label=测试目录
-ActionFtpPut.Timeout.Label=超时:
-ActionFtpPut.Timeout.Tooltip=传输超时时间, 以秒为单位, 0表示无限制
ActionFtpPut.Title=FTP 上传
ActionFtpPut.Username.Label=用户名:
ActionFtpPut.Wildcard.Label=通配符(正则表达式)
diff --git
a/plugins/tech/ftp/src/test/java/org/apache/hop/workflow/actions/ftpput/ActionFtpPutExecutionTest.java
b/plugins/tech/ftp/src/test/java/org/apache/hop/workflow/actions/ftpput/ActionFtpPutExecutionTest.java
index 21cd506104..e7a05c6bb1 100644
---
a/plugins/tech/ftp/src/test/java/org/apache/hop/workflow/actions/ftpput/ActionFtpPutExecutionTest.java
+++
b/plugins/tech/ftp/src/test/java/org/apache/hop/workflow/actions/ftpput/ActionFtpPutExecutionTest.java
@@ -197,6 +197,29 @@ class ActionFtpPutExecutionTest {
"only new files means the file on the server is left as it is");
}
+ @Test
+ @DisplayName("\"Remove\" keeps a local file that \"only put new files\"
didn't upload")
+ void removeKeepsAFileThatWasSkipped(@TempDir Path localDir) throws Exception
{
+ server.writeFile("p1.txt", "the-server-copy");
+ Files.writeString(localDir.resolve("p1.txt"), "the-local-copy");
+ Files.writeString(localDir.resolve("p2.txt"), "new");
+
+ ActionFtpPut put = configure(localDir, true);
+ put.setWildcard(".*\\.txt");
+ put.setRemove(true);
+ Result result = put.execute(new Result(), 0);
+
+ assertTrue(result.isResult());
+ assertEquals(0, result.getNrErrors());
+ assertEquals("the-server-copy", server.readFile("p1.txt"));
+ assertEquals("new", server.readFile("p2.txt"));
+ assertEquals(
+ "the-local-copy",
+ Files.readString(localDir.resolve("p1.txt")),
+ "a file that never reached the server must not be removed locally");
+ assertFalse(Files.exists(localDir.resolve("p2.txt")));
+ }
+
@Test
@DisplayName("Without \"only put new files\" an existing file is replaced")
void withoutOnlyNewFilesTheFileIsReplaced(@TempDir Path localDir) throws
Exception {