hansva commented on code in PR #9362:
URL: https://github.com/apache/hop/pull/9362#discussion_r4537303102
##########
plugins/actions/http/src/main/java/org/apache/hop/workflow/actions/http/ActionHttp.java:
##########
@@ -521,6 +523,13 @@ public Result execute(Result previousResult, int nr) {
result.setBytesWrittenThisAction(result.getBytesWrittenThisAction() +
bytesWrittenThisRow);
}
+ // With "Run for every result row" an earlier row that succeeded has
already set the result to
+ // true. A later row that failed makes the action fail, or the workflow
would follow the
+ // success hop with errors.
+ if (result.getNrErrors() > 0) {
Review Comment:
Small one: the result is set to true per row and patched back to false here.
Setting it once before the return, e.g. `result.setResult(result.getNrErrors()
== 0)` (plus whatever 'at least one row' rule applies), and dropping the
per-row `setResult(true)`, would keep a future early `return` from bringing
back the 'true with errors' case.
##########
plugins/actions/http/src/main/java/org/apache/hop/workflow/actions/http/ActionHttp.java:
##########
@@ -412,6 +409,11 @@ public Result execute(Result previousResult, int nr) {
replyBuffer = new ByteArrayOutputStream();
}
+ // Open the target only now: opening it truncates it, and a
request that failed or
+ // came back with an error status must leave an existing target
file as it was.
+ outputFile =
Review Comment:
Opening the target after the status check helps, but it's still opened (and
truncated) before the body is read. If the server answers 200 and the
connection breaks halfway, the previous file is gone and a partial one is left,
while the new docs say a failed request leaves an existing target as it was.
Writing to a temporary file next to the target and renaming it on success would
cover that case too.
A second effect of moving this: a bad target path (e.g. `${DIR}/reply.txt`
with a missing or read-only DIR) is now only found after the request was sent.
For an upload/POST the server has already acted by then, and re-running the
workflow submits it again. Maybe check that the target's parent folder exists
and is writable before sending the request?
##########
plugins/actions/http/src/test/java/org/apache/hop/workflow/actions/http/ActionHttpExecutionTest.java:
##########
@@ -211,6 +215,60 @@ void
testHttpConfiguredSingleRunDoesNotStoreReplyWhenVariableUnset() throws Exce
assertNull(workflow.getVariable("HTTP_REPLY"));
}
+ @Test
+ void testHttpErrorStatusLeavesAnExistingTargetFileAlone() throws Exception {
+ File target = File.createTempFile("downloadedNotFound", ".tmp");
+ target.deleteOnExit();
+ FileUtils.writeStringToFile(target, "previous content", UTF_8);
+
+ ActionHttp http = new ActionHttp();
+ http.setParentWorkflow(new LocalWorkflowEngine());
+ http.setRunForEveryRow(false);
+ http.setUrl(httpBaseUrl() + "/notFound");
+ http.setTargetFilename(target.getCanonicalPath());
+
+ Result result = http.execute(new Result(), 0);
+
+ assertFalse(result.getResult());
+ assertEquals(1, result.getNrErrors());
+ assertEquals("previous content", FileUtils.readFileToString(target,
UTF_8));
+ }
+
+ @Test
+ void testHttpForEveryRowFailsWhenALaterRowFails() throws Exception {
+ File first = File.createTempFile("downloadedFirstRow", ".tmp");
+ first.deleteOnExit();
+ Path second =
Files.createTempDirectory("httpSecondRow").resolve("second.tmp");
Review Comment:
`deleteOnExit()` is registered on a file the test asserts is never created,
and the temp directory from `Files.createTempDirectory("httpSecondRow")` isn't
removed, so each run leaves an empty directory behind. JUnit's `@TempDir` would
take care of it.
##########
plugins/tech/ftp/src/main/java/org/apache/hop/workflow/actions/ftpput/ActionFtpPut.java:
##########
@@ -229,11 +229,15 @@ public Result execute(Result prevResult, int nr) throws
HopException {
if (!shouldProcessFile(file, pattern)) {
continue;
}
- if (uploadFile(ftpClient, realLocalDirectory, file)) {
- filesPut++;
- deleteLocalFileIfNeeded(childUri(realLocalDirectory, file));
- } else {
- errors++;
+ switch (uploadFile(ftpClient, realLocalDirectory, file)) {
+ case UPLOADED -> {
+ filesPut++;
+ deleteLocalFileIfNeeded(childUri(realLocalDirectory, file));
+ }
+ // A file left alone because the server already has one is not an
error, but it never
+ // reached the server: don't count it as put and don't remove the
local copy.
+ case SKIPPED -> {}
Review Comment:
Not counting skipped files as put is the right call. But the skip is only
logged at Detailed level, so at Basic level a run where every file already
existed shows 'We have put 0 files' with result true, and the files stay in the
local folder without explanation, which looks like a wildcard that matched
nothing. Could the skip (or a skipped count) be logged at Basic level?
##########
plugins/actions/http/src/test/java/org/apache/hop/workflow/actions/http/ActionHttpExecutionTest.java:
##########
@@ -211,6 +215,60 @@ void
testHttpConfiguredSingleRunDoesNotStoreReplyWhenVariableUnset() throws Exce
assertNull(workflow.getVariable("HTTP_REPLY"));
}
+ @Test
+ void testHttpErrorStatusLeavesAnExistingTargetFileAlone() throws Exception {
+ File target = File.createTempFile("downloadedNotFound", ".tmp");
Review Comment:
There's no test for a 2xx response whose body fails while streaming, which
is the case where the target file is still destroyed (see the comment on
`ActionHttp` line 414).
##########
plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftpput/messages/messages_en_US.properties:
##########
@@ -84,8 +84,8 @@ ActionFtpPut.TargetSettings.Group.Label=Target (remote) files
ActionFtpPut.TestConnection.Label=Test connection
ActionFtpPut.TestConnection.Tooltip=Test FTP connection
ActionFtpPut.TestFolderExists.Label=Test folder
-ActionFtpPut.Timeout.Label=Timeout
-ActionFtpPut.Timeout.Tooltip=The timeout when transferring in seconds (0 for
no timeout)
+ActionFtpPut.Timeout.Label=Timeout (ms)
+ActionFtpPut.Timeout.Tooltip=The connect and reply timeout in milliseconds (0
for the default)
Review Comment:
The tooltip says '0 for the default', while the manual for this field says
'0 (the default): no timeout'. Could they say the same thing about 0?
##########
plugins/tech/ftp/src/main/resources/org/apache/hop/workflow/actions/ftp/messages/messages_en_US.properties:
##########
@@ -133,8 +133,8 @@ ActionFtp.TargetDir.Tooltip=The target directory on the
local server
ActionFtp.TestConnection.Label=Test connection
ActionFtp.TestConnection.Tooltip=Test connection
ActionFtp.TestFolderExists.Label=Check folder
-ActionFtp.Timeout.Label=Timeout
-ActionFtp.Timeout.Tooltip=The timeout when transferring in seconds (0 for no
timeout)
+ActionFtp.Timeout.Label=Timeout (ms)
Review Comment:
The unit is fixed to milliseconds in en_US, but the fr_FR and es_AR tooltips
of FTP Get and FTP Put (and es_AR of FTP Delete) still say seconds. A French
user entering 30 gets a 30 ms timeout and every transfer fails at once. Could
those be corrected (or removed so they fall back to English) in this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]