This is an automated email from the ASF dual-hosted git repository.
bamaer pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hop.git
The following commit(s) were added to refs/heads/main by this push:
new 9890528b32 Batch 10: Result sections for the FTP, SFTP put and HTTP
actions. #8970 (#9149)
9890528b32 is described below
commit 9890528b3243c80837aac46d3c32ec7c1f51e892
Author: Adalennis <[email protected]>
AuthorDate: Fri Oct 30 10:55:02 2026 +0100
Batch 10: Result sections for the FTP, SFTP put and HTTP actions. #8970
(#9149)
* Batch 10: Result sections for Get a file with FTP, FTP delete, Put a file
with FTP, Put a file with SFTP and HTTP. #8970
* Nr errors less than is true when Get a file with FTP or FTP delete fails
before the transfer. #8970
---
.../modules/ROOT/pages/workflow/actions/ftp.adoc | 47 ++++++++++++++++------
.../ROOT/pages/workflow/actions/ftpdelete.adoc | 44 ++++++++++++--------
.../ROOT/pages/workflow/actions/ftpput.adoc | 29 +++++++++----
.../modules/ROOT/pages/workflow/actions/http.adoc | 31 ++++++++++----
.../ROOT/pages/workflow/actions/sftpput.adoc | 26 +++++++++---
5 files changed, 127 insertions(+), 50 deletions(-)
diff --git a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftp.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftp.adoc
index 4aa6e916e6..7833c952d0 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftp.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftp.adoc
@@ -40,8 +40,8 @@ This workflow retrieves files from one directory exclusively.
|Action name|The name of the workflow action.
2+|Server
|FTP connection|An optional xref:metadata-types/ftp-connection.adoc[FTP
Connection] from the metadata. Pick one and the server, credentials, transfer
mode, timeouts and proxy settings below come from that connection; leave it
empty to configure the server on this action.
-|FTP server name/IP address|The name of the server or the IP address.
-|Server Port|Port number of the FTP server.
+|FTP server name / IP address|The name of the server or the IP address.
+|Server port|Port number of the FTP server.
|Username|The user name associated with FTP server account.
|Password|The password associated the FTP server account.
|Proxy host|Proxy server host name.
@@ -50,8 +50,8 @@ This workflow retrieves files from one directory exclusively.
|Proxy password|Proxy server account password.
2+|Advanced
|Binary mode|Enable if files must be transferred in binary mode.
-|Timeout|The FTP server timeout in seconds.
-|Use Active FTP connection|Enable if you are connecting to the FTP server
using Active mode; you must set your firewall to accept connections to the port
that your FTP client will open.
+|Timeout|The timeout in milliseconds, for example 30000 for 30 seconds. 0 (the
default): no timeout.
+|Use active FTP connection|Enable if you are connecting to the FTP server
using Active mode; you must set your firewall to accept connections to the port
that your FTP client will open.
The default is Passive mode.
|Control Encoding|Encoding matters when retrieving file names that contain
special characters.
For Western Europe and the USA, ISO-8859-1 usually suffices.
@@ -67,20 +67,20 @@ Select encoding that is valid for your server.
|Remote directory|The remote directory on FTP server from which files are
taken.
|Wildcard (regular expression)|Regular expression when you want to select
multiple files.
For example: .txt$ : get all text files A.ENG:0-9.txt : files starting with A,
ending with a number and .txt
-|Remove files after retrieval|Remove the files on the FTP server, but only
after all selected files have been successfully transferred.
-|Move files after retrieval|Move the files on the FTP server to the folder
specified below, but only after all selected files have been successfully
transferred.
-|Move to Folder|Moves files to specified folder on the server.
-|Create Folder|Creates folder that will contain files.
+|Remove files after retrieval|Removes each file from the FTP server right
after it is downloaded. See the NOTE under <<Result>>.
+|Move files after retrieval|Moves each file on the FTP server to the folder
below, right after it is downloaded. See the NOTE under <<Result>>.
+|Move to folder|The folder on the server to move the files to.
+|Create folder|Creates the move-to folder when it doesn't exist.
2+|Local
-|Target Directory|The directory where you want to place the retrieved files.
+|Target directory|The directory where you want to place the retrieved files.
|Include date in filename|Adds the system date to the filename (_20101231).
|Include time in filename|Adds the system time to the filename (_235959).
-|Specify date time format|Enable to provide your own date/time format; the
default is yyyyMMdd'_'HHmmss
+|Specify Date time format|Enable to provide your own date/time format in *Date
time format*.
|Date time format|Select date time format.
|Add date before extension|Adds date to the file name before the extension.
|Don't overwrite files|Enable to skip, rename, or fail if a file with an
identical name already exists in the target directory.
-|If file exists|Action to take if a file with an identical name already exists
in the target directory.
-|Add filenames to result|Enable to add the file name(s) read to the result of
this workflow.
+|If file exists|Used with *Don't overwrite files*: Skip (default) leaves the
local file and doesn't download the remote one, Give unique name to local file
adds the date and time to the new file's name, Fail counts the file as an error.
+|Add filename to result|Adds the downloaded files to the result files (checked
by default).
|===
=== Advanced tab
@@ -106,4 +106,25 @@ For example: .txt$ : get all text files A.ENG:0-9.txt :
files starting with A, e
|Port|Socks Proxy port number.
|Username|User name associated with the Socks Proxy account.
|Password|Password associated with the Socks Proxy account.
-|===
\ No newline at end of file
+|===
+
+== Result
+
+What this action passes on to the next action. See
xref:workflow/action-results.adoc[Action results] for how results travel
through a workflow.
+
+[options="header", width="90%", cols="1,3"]
+|===
+|Result|Value
+|True a|Depends on *Success on* (Advanced tab):
+
+* All works fine (default): no file failed. No file matching the wildcard is
also true.
+* At least we got x files: at least *Limit files* files were downloaded,
whatever the number of errors.
+* Nr errors less than: the number of errors is at most *Limit files*. The
download stops as soon as the number of errors reaches *Limit files*, and the
action is still true then. Some failures before the download also count as 1
error: see False.
+|False|The success condition is not met. Without *Nr errors less than*, the
download stops at the first file that fails. The action is also false, before
any download, when the server can't be reached or refuses the login, the remote
directory doesn't exist, the wildcard is not a valid regular expression, or
*Move files after retrieval* is checked while the move-to folder is empty, or
doesn't exist and *Create folder* is unchecked. With *Nr errors less than*, an
unreachable server, a ref [...]
+|Errors|The number of files that failed, or 1 when the action fails before the
download.
+|Result files|The downloaded files are added when *Add filename to result* is
checked (the default). The previous result files are kept.
+|Result rows|Passed on unchanged.
+|Files retrieved|The number of files downloaded.
+|===
+
+NOTE: *Remove files after retrieval* and *Move files after retrieval* act on
each file right after it is downloaded, not after all files are: when a later
file fails, the files before it are already removed or moved on the server.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpdelete.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpdelete.adoc
index 641c62fa99..6520ca0e2f 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpdelete.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpdelete.adoc
@@ -41,25 +41,15 @@ Existing workflows keep working and log a warning; new ones
should use Delete fi
|Protocol a|
* FTP
-* SFTP (deprecated, see the note below)
+* SFTP (deprecated)
-|FTP connection|An optional xref:metadata-types/ftp-connection.adoc[FTP
Connection] from the metadata. Pick one and the server, credentials, transfer
mode, timeouts and proxy settings below come from that connection; leave it
empty to configure the server on this action.
+|FTP connection|An optional xref:metadata-types/ftp-connection.adoc[FTP
Connection] from the metadata. Pick one and the server, credentials, transfer
mode, timeouts and proxy settings below come from that connection; leave it
empty to configure the server on this action. To connect with FTPS, use an FTP
connection and its *Security* option.
|SFTP connection|An optional xref:metadata-types/sftp-connection.adoc[SFTP
Connection] from the metadata, used when the protocol is SFTP.
-|FTP server name/IP address|The name of the server or the IP address.
-|Server Port|Port number of the FTP server.
+|FTP server name / IP address|The name of the server or the IP address.
+|Server port|Port number of the FTP server.
|Username|The user name associated with FTP server account.
|Password|The password associated the FTP server account.
-|Connection type a| FTPS connection type.
-
-* FTP
-* Implicit SSL
-* Authentication SSL
-* Implicit SSL with encrypted FTP connection
-* Authentication TLS
-* Implicit TLS
-* Implicit TLS with encrypted FTP connection
-
|Use proxy|Whether or not to use a proxy.
|Proxy host|Proxy server host name.
|Proxy port|Proxy server port number.
@@ -78,14 +68,14 @@ Existing workflows keep working and log a warning; new ones
should use Delete fi
|===
|Option|Description
2+|Advanced
-|Timeout|The timeout period.
+|Timeout|The timeout in milliseconds, for example 30000 for 30 seconds. 0 (the
default): no timeout.
|Use active FTP connection|Enable if you are connecting to the FTP server
using Active mode; you must set your firewall to accept connections to the port
that your FTP client will open.
The default is Passive mode.
2+|Remote
-|Copy previous results to args?
+|Copy previous results to args|Deletes the files named in the first field of
the previous result rows, instead of the files matching *Wildcard*.
|Remote directory|The directory on the remote FTP server.
|Check folder|Checks if the directory exists.
-|Wildcard (regular expression)
+|Wildcard (regular expression)|The regular expression the file names must
match. Required unless *Copy previous results to args* is checked.
2+|Success condition
|Success on a| The success condition
@@ -106,3 +96,23 @@ The default is Passive mode.
|Username|User name associated with the Socks Proxy account.
|Password|Password associated with the Socks Proxy account.
|===
+
+== Result
+
+What this action passes on to the next action. See
xref:workflow/action-results.adoc[Action results] for how results travel
through a workflow.
+
+[options="header", width="90%", cols="1,3"]
+|===
+|Result|Value
+|True a|Depends on *Success on*:
+
+* All works fine (default): no file failed. No file matching the wildcard is
also true, and so is *Copy previous results to args* without previous result
rows.
+* At least we delete x files: at least *Limit files* files were deleted.
+* Nr errors less than: the number of errors is at most *Limit files*. A
failure before deleting counts as 1 error, so the cases listed under False give
true with 1 error when *Limit files* is 1 or more.
+|False|The success condition is not met, for example because the server can't
be reached or refuses the login, the remote directory doesn't exist, the
wildcard is not a valid regular expression, or *Wildcard* is empty while *Copy
previous results to args* is unchecked. The number of errors is the number of
files that failed, or 1 when the action fails before deleting. With *Nr errors
less than*, these failures count as 1 error, so the action is true when *Limit
files* is 1 or more.
+|Result files|Passed on unchanged.
+|Result rows|Passed on unchanged.
+|Files retrieved|The number of files deleted. When the remote directory has no
files, or *Copy previous results to args* is checked without previous result
rows, the value of the previous action is passed on unchanged.
+|===
+
+NOTE: With *Copy previous results to args*, the first field of each previous
result row is the name of a file to delete in the remote directory, and
*Wildcard* is not used.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpput.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpput.adoc
index e3a8a2744c..a47d6dd173 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpput.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/ftpput.adoc
@@ -32,9 +32,9 @@ The Put A File With FTP action uploads a file to an FTP
server.
|===
|Option|Description
|Action name|The name of the workflow action.
-2+|General tab
+2+|Server settings
|FTP connection|An optional xref:metadata-types/ftp-connection.adoc[FTP
Connection] from the metadata. Pick one and the server, credentials, transfer
mode, timeouts and proxy settings below come from that connection; leave it
empty to configure the server on this action.
-|FTP server name/IP address|The name of the server or the IP address.
+|FTP server name / IP address|The name of the server or the IP address.
|Port|The port number the FTP server connects on.
|Username|The user name associated with the FTP server account.
|Password|The password associated the FTP server account.
@@ -45,10 +45,10 @@ The Put A File With FTP action uploads a file to an FTP
server.
|Test connection|Tests the configured connection.
2+|Advanced settings
|Binary mode|Enable if files must be transferred in binary mode.
-|Timeout|The FTP server timeout in seconds.
-|Use Active FTP connection|Enable if you are connecting to the FTP server
using Active mode; you must set your firewall to accept connections to the port
that your FTP client will open.
+|Timeout|The timeout in milliseconds, for example 30000 for 30 seconds. 0 (the
default): no timeout.
+|Use active FTP connection|Enable if you are connecting to the FTP server
using Active mode; you must set your firewall to accept connections to the port
that your FTP client will open.
The default is Passive mode.
-|Control Encoding|Select the character encoding to use for determining special
characters in file names.
+|Control encoding|Select the character encoding to use for determining special
characters in file names.
For Western Europe and the USA, ISO-8859-1 usually suffices.
Select encoding that is valid for your server.
|===
@@ -62,8 +62,8 @@ Select encoding that is valid for your server.
|Local directory|The local directory from which to upload.
|Browse...|Browse the filesystem for a local directory.
|Wildcard (regular expression)|If you are uploading multiple files, specify a
regular expression that defines the set of files you want to upload.
-|Remove files after transferal?|Enable to remove files from the local
directory after they've been uploaded.
-|Don't overwrite files|Enable to skip files that already exist in the remote
directory.
+|Remove files after transferal|Enable to remove files from the local directory
after they've been uploaded.
+|Don't overwrite files|Enable to skip files that already exist in the remote
directory. See the WARNING under <<Result>>.
|Remote directory|The directory on the FTP server to upload files to.
|Test folder|Test if the remote directory exists. This connects to the FTP
server and attempts to access the specified directory. If this directory
doesn't exist, the upload fails.
|===
@@ -78,3 +78,18 @@ Select encoding that is valid for your server.
|Username|The user name associated with the Socks proxy server account.
|Password|The password associated with the Socks proxy server account.
|===
+
+== Result
+
+What this action passes on to the next action. See
xref:workflow/action-results.adoc[Action results] for how results travel
through a workflow.
+
+[options="header", width="90%", cols="1,3"]
+|===
+|Result|Value
+|True|Every matching file was uploaded, with 0 errors. No file matching the
wildcard is also true.
+|False|A file can't be uploaded, the server can't be reached or refuses the
login, the remote directory doesn't exist, *Local directory* is empty or
doesn't exist, or the wildcard is not a valid regular expression. The number of
errors is the number of files that failed, or 1 when the action fails before
the upload.
+|Result files|Passed on unchanged: the uploaded files are not added.
+|Result rows|Passed on unchanged.
+|===
+
+WARNING: With *Don't overwrite files* and *Remove files after transferal* both
checked, a local file that already exists on the server is not uploaded, but it
is still removed from the local directory.
diff --git a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/http.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/http.adoc
index 622ba7a2c5..8b82383d0a 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/http.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/http.adoc
@@ -42,16 +42,16 @@ The proxy and the web server each keep their own
credentials, so the ones you fi
=== General Tab
-[options="header", width="90%, cols="1,3"]
+[options="header", width="90%", cols="1,3"]
|===
|Option|Description
-|Action name|The name of the workflow action.
+|Name of action|The name of the workflow action.
|REST connection|Optional.
The name of a xref:metadata-types/rest-connection.adoc[REST connection] that
supplies the client for this action: the proxy, the credentials, the TLS
settings and the timeouts.
When one is selected, the fields on the Authentication and Proxy tabs are not
used, and the URL below may be a path that is resolved against the connection's
base URL.
|URL|The HTTP URL of the file to retrieve, or the directory name to store an
uploaded file to.
|Ignore SSL certificate check|If selected, all SSL certificate checks are
ignored.
-|Run for every result row?|Check this if you want to run this action for every
row that was generated by a previous pipeline.
+|Run for every result row|Check this if you want to run this action for every
row that was generated by a previous pipeline.
Use the "Copy rows to result".
If selected, an HTTP request will be made for each result.
Otherwise, the file is only retrieved once
@@ -102,10 +102,10 @@ This may be useful when working on an intranet or VPN
|===
|Option|Description
|Target file|If you are downloading a file, this is its name on your local
filesystem
-|Append to specified target file?|If selected, and if the target file already
exists, Hop will append all new data to the end of the file
-|Add date and time to file name?|If selected, the date and time of the HTTP
request (in yyyMMdd_HHmmss format) will be added to the target filename
+|Append to specified target file|If selected, and if the target file already
exists, Hop will append all new data to the end of the file
+|Add date and time to file name|If selected, the date and time of the HTTP
request (in yyyMMdd_HHmmss format) will be added to the target filename
|Target file extension|If the previous option is selected, this field
specifies the extension (letters after the dot) of the target filename
-|Add filename to result filename|Any files that are copied will appear as a
result from this action; shows a list of files that were copied in this action.
+|Add filename to result filenames|Any files that are copied will appear as a
result from this action; shows a list of files that were copied in this action.
|Result variable|If specified, the HTTP reply body is stored in this workflow
variable so later actions can use it.
When running for every result row, the last reply is stored.
|===
@@ -118,4 +118,21 @@ When running for every result row, the last reply is
stored.
|#|Order that the header should be processed
|Name|The name of this HTTP header
|Value|The actual header value to pass
-|===
\ No newline at end of file
+|===
+
+== Result
+
+What this action passes on to the next action. See
xref:workflow/action-results.adoc[Action results] for how results travel
through a workflow.
+
+[options="header", width="90%", cols="1,3"]
+|===
+|Result|Value
+|True|The server answered with a status below 400 and the reply was written to
the target file, with 0 errors. A redirect is followed.
+|False|The server answered with a status of 400 or more (for example 404, or
401 when the credentials are missing or wrong), can't be reached, the URL is
not valid, or the target file can't be written. The number of errors is 1. With
*Run for every result row* and no previous result rows, the action is false
with 0 errors.
+|Result files|The target file is added when *Add filename to result filenames*
is checked (the default) and the request succeeded. The previous result files
are kept.
+|Result rows|Passed on unchanged.
+|===
+
+WARNING: The target file is opened before the request is sent. A request that
fails, for example with a 404, leaves an empty target file: an existing file is
emptied, unless *Append to specified target file* is checked.
+
+NOTE: With *Run for every result row*, the rows are processed one after the
other and the action stops at the first row that fails. The result is true as
soon as one row succeeded: a first row that succeeds and a second one that
fails give true with 1 error.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/sftpput.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/sftpput.adoc
index d74f2e7844..79d79625f9 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/sftpput.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions/sftpput.adoc
@@ -35,14 +35,12 @@ TIP: If you run into a `Algorithm Negotiation Fail` error
when connecting to you
[options="header", width="90%", cols="1,3"]
|===
|Option|Description
-|Workflow action name|The name of the workflow action.
-*Note*: This name has to be unique in a single workflow.
-A workflow action can be placed several times on the canvas, however it will
be the same workflow action.
+|Action name|The name of the workflow action.
|SFTP connection|An xref:metadata-types/sftp-connection.adoc[SFTP Connection]
in the project metadata.
When one is selected, the server, credentials, compression and proxy settings
below come from that connection and are disabled here.
Leave it empty to configure the server in this action.
-|SFTP-server name (IP)|The name of the SFTP server or the IP address.
-|SFTP port|The TCP port to use.
+|SFTP server name / IP|The name of the SFTP server or the IP address.
+|Port|The TCP port to use.
This is usually 22.
|User name|The user name to log into the SFTP server.
|Password|The password to log into the SFTP server.
@@ -89,7 +87,7 @@ Examples of regular expressions:
.*txt$ : get all text files
A.*[ENG:0-9].txt : files starting with A, ending with a number and .txt
----
-|Success when no file|Check this option if the workflow action has to succeed
when there are no files to process, otherwise the workflow will fail.
+|Success when no file|Check this option if the action has to succeed when
there are no files to send. Otherwise the action is false, with 1 error.
|After SFTP Put a|Action to take after the file is transferred. The available
options are:
* Do nothing (default)
@@ -120,3 +118,19 @@ The better and more secure option is to upgrade your keys
to more secure types.
`-Djsch.client_pubkey=ssh-rsa,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256
-Djsch.server_host_key=ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256,ssh-rsa`
+
+== Result
+
+What this action passes on to the next action. See
xref:workflow/action-results.adoc[Action results] for how results travel
through a workflow.
+
+[options="header", width="90%", cols="1,3"]
+|===
+|Result|Value
+|True|Every file was sent, with 0 errors. When no file matches, the action is
true only if *Success when no file* is checked. *Copy previous results to args*
or *Copy previous result files to args* without anything in the previous result
is also true.
+|False|No file to send (with *Success when no file* unchecked), the server
can't be reached or refuses the login, the remote directory doesn't exist and
*Create folder* is unchecked, the local directory doesn't exist, a file can't
be sent, or *Move file to* is selected while *Destination folder* is empty, or
doesn't exist and *Create destination folder* is unchecked. The number of
errors is 1.
+|Result files|The sent files are added when *Add filename to result* is
checked and *After SFTP Put* is *Do nothing*. The previous result files are
kept.
+|Result rows|Passed on unchanged.
+|Files retrieved|The number of files sent.
+|===
+
+NOTE: With *Copy previous results to args*, a file in the previous result rows
that doesn't exist is logged as an error, but it doesn't count as one and the
other files are still sent. When none of them exists, there is no file to send.
