hansva commented on code in PR #9359:
URL: https://github.com/apache/hop/pull/9359#discussion_r4537159542


##########
plugins/tech/mongodb/src/main/java/org/apache/hop/workflow/actions/mql/ActionMql.java:
##########
@@ -201,6 +193,72 @@ public Result execute(Result result, int nr) {
     return result;
   }
 
+  /**
+   * Parse the commands to run. A JSON array holds one command per element. 
Anything else is read as
+   * a list of comma-separated documents, so that text after the first 
document is either run or
+   * reported as invalid JSON, never silently ignored.
+   *
+   * @param mqlToExecute the JSON text
+   * @return the commands, in the order they are to be run
+   * @throws HopException when there is no command
+   */
+  static List<Document> parseCommands(String mqlToExecute) throws HopException 
{
+    String trimmed = mqlToExecute == null ? "" : mqlToExecute.trim();
+    if (trimmed.isEmpty()) {
+      throw new HopException(BaseMessages.getString(PKG, 
"ActionMQL.NoCommandFound"));
+    }
+    String array = trimmed.startsWith("[") ? trimmed : "[" + trimmed + "]";

Review Comment:
   Wrapping any non-array input in `[...]` changes what existing workflows do: 
text after the first document used to be ignored, and is now either run as 
extra commands or rejected. `{"drop":"tmp"}, {"drop":"prod"}` used to run only 
the first drop and now runs both; a script ending in `;` (e.g. `{ ping: 1 };`) 
used to work and now fails with a JSON parse error. I agree silently ignoring 
text is bad, but running extra commands that were ignored before is risky; I'd 
rather reject trailing text than execute it, and mention the change in the 
upgrade notes.



##########
plugins/tech/mongodb/src/main/java/org/apache/hop/workflow/actions/mql/ActionMql.java:
##########
@@ -201,6 +193,72 @@ public Result execute(Result result, int nr) {
     return result;
   }
 
+  /**
+   * Parse the commands to run. A JSON array holds one command per element. 
Anything else is read as
+   * a list of comma-separated documents, so that text after the first 
document is either run or
+   * reported as invalid JSON, never silently ignored.
+   *
+   * @param mqlToExecute the JSON text
+   * @return the commands, in the order they are to be run
+   * @throws HopException when there is no command
+   */
+  static List<Document> parseCommands(String mqlToExecute) throws HopException 
{
+    String trimmed = mqlToExecute == null ? "" : mqlToExecute.trim();
+    if (trimmed.isEmpty()) {
+      throw new HopException(BaseMessages.getString(PKG, 
"ActionMQL.NoCommandFound"));
+    }
+    String array = trimmed.startsWith("[") ? trimmed : "[" + trimmed + "]";
+    Document wrapper = Document.parse("{ \"array\": " + array + " }");

Review Comment:
   Because the input is embedded in `{ "array": ... }`, a JSON parse error now 
reports a position in the generated text (about 12-13 characters later than 
what the user typed), which no longer matches the line/column indicator in the 
dialog. Could the position be corrected, or the error message mention the 
original input?



##########
plugins/tech/mongodb/src/main/java/org/apache/hop/workflow/actions/mql/ActionMql.java:
##########
@@ -201,6 +193,72 @@ public Result execute(Result result, int nr) {
     return result;
   }
 
+  /**
+   * Parse the commands to run. A JSON array holds one command per element. 
Anything else is read as
+   * a list of comma-separated documents, so that text after the first 
document is either run or
+   * reported as invalid JSON, never silently ignored.
+   *
+   * @param mqlToExecute the JSON text
+   * @return the commands, in the order they are to be run
+   * @throws HopException when there is no command
+   */
+  static List<Document> parseCommands(String mqlToExecute) throws HopException 
{
+    String trimmed = mqlToExecute == null ? "" : mqlToExecute.trim();
+    if (trimmed.isEmpty()) {
+      throw new HopException(BaseMessages.getString(PKG, 
"ActionMQL.NoCommandFound"));
+    }
+    String array = trimmed.startsWith("[") ? trimmed : "[" + trimmed + "]";
+    Document wrapper = Document.parse("{ \"array\": " + array + " }");
+    if (wrapper.size() != 1) {

Review Comment:
   This guard misses trailing text that repeats the `array` key, since 
`Document` is a map and the second value replaces the first: `[ {"drop":"a"} ], 
"array": [ {"ping":1} ]` ends up with one key, passes the check, and only 
`ping` runs, so the first array is dropped silently, which the Javadoc says 
can't happen. Parsing with a `JsonReader` and checking that the input ends 
after the first value would avoid this.



##########
plugins/tech/mongodb/src/main/java/org/apache/hop/workflow/actions/mql/ActionMql.java:
##########
@@ -201,6 +193,72 @@ public Result execute(Result result, int nr) {
     return result;
   }
 
+  /**
+   * Parse the commands to run. A JSON array holds one command per element. 
Anything else is read as
+   * a list of comma-separated documents, so that text after the first 
document is either run or
+   * reported as invalid JSON, never silently ignored.
+   *
+   * @param mqlToExecute the JSON text
+   * @return the commands, in the order they are to be run
+   * @throws HopException when there is no command
+   */
+  static List<Document> parseCommands(String mqlToExecute) throws HopException 
{
+    String trimmed = mqlToExecute == null ? "" : mqlToExecute.trim();
+    if (trimmed.isEmpty()) {
+      throw new HopException(BaseMessages.getString(PKG, 
"ActionMQL.NoCommandFound"));
+    }
+    String array = trimmed.startsWith("[") ? trimmed : "[" + trimmed + "]";
+    Document wrapper = Document.parse("{ \"array\": " + array + " }");
+    if (wrapper.size() != 1) {
+      // For example: [ {...} ], "x": 1
+      throw new HopException(BaseMessages.getString(PKG, 
"ActionMQL.TextAfterCommands"));
+    }
+    List<Document> commands = wrapper.getList("array", Document.class);
+    if (commands.isEmpty()) {
+      throw new HopException(BaseMessages.getString(PKG, 
"ActionMQL.NoCommandFound"));
+    }
+    return commands;
+  }
+
+  /**
+   * A command that MongoDB accepts can still fail to write documents. Those 
errors are in the reply
+   * (writeErrors, writeConcernError) while ok is 1, so runCommand() does not 
throw.
+   *
+   * @param reply the reply of the command
+   * @param commandNr the number of the command (1-based)
+   * @param nrCommands the number of commands
+   * @throws MongoDbException when the reply holds write errors or a write 
concern error
+   */
+  static void checkReply(Document reply, int commandNr, int nrCommands) throws 
MongoDbException {
+    if (reply == null) {
+      return;
+    }
+    List<Document> writeErrors = reply.getList("writeErrors", Document.class);
+    if (writeErrors != null && !writeErrors.isEmpty()) {

Review Comment:
   Failing on any `writeErrors` breaks workflows that rely on a partial write 
on purpose, e.g. `{insert:'c', documents:[...], ordered:false}` to skip 
duplicate keys. Before this change the action was true and took the success 
hop; now it's false, and in an array of instructions the later ones don't run 
anymore. This might need an option (or at least an upgrade note).



##########
plugins/tech/mongodb/src/main/java/org/apache/hop/workflow/actions/mql/ActionMql.java:
##########
@@ -201,6 +193,72 @@ public Result execute(Result result, int nr) {
     return result;
   }
 
+  /**
+   * Parse the commands to run. A JSON array holds one command per element. 
Anything else is read as
+   * a list of comma-separated documents, so that text after the first 
document is either run or
+   * reported as invalid JSON, never silently ignored.
+   *
+   * @param mqlToExecute the JSON text
+   * @return the commands, in the order they are to be run
+   * @throws HopException when there is no command
+   */
+  static List<Document> parseCommands(String mqlToExecute) throws HopException 
{
+    String trimmed = mqlToExecute == null ? "" : mqlToExecute.trim();
+    if (trimmed.isEmpty()) {
+      throw new HopException(BaseMessages.getString(PKG, 
"ActionMQL.NoCommandFound"));
+    }
+    String array = trimmed.startsWith("[") ? trimmed : "[" + trimmed + "]";
+    Document wrapper = Document.parse("{ \"array\": " + array + " }");
+    if (wrapper.size() != 1) {
+      // For example: [ {...} ], "x": 1
+      throw new HopException(BaseMessages.getString(PKG, 
"ActionMQL.TextAfterCommands"));
+    }
+    List<Document> commands = wrapper.getList("array", Document.class);

Review Comment:
   For input that mixes a document and an array without brackets (`{a:1}, 
[{b:1}]`) or an array of non-documents (`[1, 2]`), `getList("array", 
Document.class)` throws a ClassCastException, and the log says `class 
java.util.ArrayList cannot be cast to org.bson.Document`, which doesn't tell 
the user what's wrong with the JSON. Could this be caught and turned into a 
HopException with a clear message?



##########
plugins/databases/snowflake/src/main/java/org/apache/hop/workflow/actions/snowflake/WarehouseManager.java:
##########
@@ -389,7 +391,7 @@ private String getCreateSQL() {
     if (!failIfExists && !replace) {
       sql.append("IF NOT EXISTS ");
     }
-    sql.append(warehouseName).append(" WITH ");
+    sql.append(resolve(warehouseName)).append(" WITH ");

Review Comment:
   The warehouse name is now resolved and placed into the CREATE (and ALTER, 
line 437) statement without quoting or validation. A variable value such as `X 
WITH WAREHOUSE_SIZE='XSMALL'; DROP DATABASE prod; --` runs extra statements; 
before this change CREATE and ALTER used the literal text. Could the name be 
validated as an identifier (or quoted) after resolving?



##########
plugins/tech/databricks/src/main/java/org/apache/hop/workflow/actions/databricks/ActionDatabricksJobWait.java:
##########
@@ -184,11 +184,13 @@ public void logError(String message) {
             && "CANCELED".equalsIgnoreCase(status.getResultState())) {
           result.setNrErrors(1);
           result.setResult(false);
-          logError(
+          String msg =
               BaseMessages.getString(
                   PKG,
                   "ActionDatabricksJobWait.Error.Canceled",
-                  Const.NVL(status.getStateMessage(), "")));
+                  Const.NVL(status.getStateMessage(), ""));
+          setResultVariable(resultVariableError, msg);

Review Comment:
   The error variable is now filled from 
`ActionDatabricksJobWait.Error.Canceled`, but the pt_BR text of that key 
(`messages_pt_BR.properties` line 115: `Databricks foi cancelado.`) has no 
`{0}` placeholder, so in a pt_BR locale the cancellation reason from Databricks 
is lost. The new test checks for the reason and only passes in English. Adding 
`{0}` to the pt_BR text would fix it.



##########
plugins/databases/snowflake/src/main/java/org/apache/hop/workflow/actions/snowflake/WarehouseManager.java:
##########
@@ -305,6 +305,7 @@ public Result execute(Result previousResult, int nr) throws 
HopException {
     Result result = previousResult;
     result.setResult(validate());
     if (!result.isResult()) {

Review Comment:
   Now that the name is resolved, `validate()` could also check that the 
resolved warehouse name isn't empty. With an empty name, or `${WAREHOUSE}` 
resolving to empty, the action sends `CREATE WAREHOUSE IF NOT EXISTS  WITH ...` 
and fails with a confusing Snowflake syntax error instead of a clear validation 
message.



##########
plugins/misc/documentation/src/main/java/org/apache/hop/documentation/ActionDoc.java:
##########
@@ -143,6 +143,10 @@ public Result execute(Result prevResult, int nr) {
             includingMetadata,
             generateHtml,
             removeMarkdown);
+    // The result of this action is its own: true when the documentation is 
written.
+    // DocBuilder sets it back to false (and 1 error) when that fails.
+    //
+    prevResult.setResult(true);

Review Comment:
   This sets the result to true but doesn't reset the error count, so if 
`prevResult` arrives with `nrErrors > 0` the action reports `result=true` with 
`nrErrors=1`, which contradicts the new docs ('true with 0 errors'). 
`prevResult.setNrErrors(0)` next to this would fix it. The success path in 
`WarehouseManager.execute` has the same gap.



##########
plugins/tech/mongodb/src/main/java/org/apache/hop/workflow/actions/mql/ActionMql.java:
##########
@@ -201,6 +193,72 @@ public Result execute(Result result, int nr) {
     return result;
   }
 
+  /**
+   * Parse the commands to run. A JSON array holds one command per element. 
Anything else is read as
+   * a list of comma-separated documents, so that text after the first 
document is either run or
+   * reported as invalid JSON, never silently ignored.
+   *
+   * @param mqlToExecute the JSON text
+   * @return the commands, in the order they are to be run
+   * @throws HopException when there is no command
+   */
+  static List<Document> parseCommands(String mqlToExecute) throws HopException 
{
+    String trimmed = mqlToExecute == null ? "" : mqlToExecute.trim();
+    if (trimmed.isEmpty()) {
+      throw new HopException(BaseMessages.getString(PKG, 
"ActionMQL.NoCommandFound"));
+    }
+    String array = trimmed.startsWith("[") ? trimmed : "[" + trimmed + "]";
+    Document wrapper = Document.parse("{ \"array\": " + array + " }");
+    if (wrapper.size() != 1) {
+      // For example: [ {...} ], "x": 1
+      throw new HopException(BaseMessages.getString(PKG, 
"ActionMQL.TextAfterCommands"));
+    }
+    List<Document> commands = wrapper.getList("array", Document.class);
+    if (commands.isEmpty()) {
+      throw new HopException(BaseMessages.getString(PKG, 
"ActionMQL.NoCommandFound"));
+    }
+    return commands;
+  }
+
+  /**
+   * A command that MongoDB accepts can still fail to write documents. Those 
errors are in the reply
+   * (writeErrors, writeConcernError) while ok is 1, so runCommand() does not 
throw.
+   *
+   * @param reply the reply of the command
+   * @param commandNr the number of the command (1-based)
+   * @param nrCommands the number of commands
+   * @throws MongoDbException when the reply holds write errors or a write 
concern error
+   */
+  static void checkReply(Document reply, int commandNr, int nrCommands) throws 
MongoDbException {

Review Comment:
   `checkReply` only checks top-level `writeErrors` and `writeConcernError`. 
Commands that report errors differently still succeed, e.g. on MongoDB 8 the 
`bulkWrite` command with a duplicate key returns `ok:1, nErrors:1` with the 
per-operation errors in its cursor, so the action is true while the updated 
docs say write errors make it false. Checking `nErrors` too would cover that.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to