hansva commented on code in PR #9359:
URL: https://github.com/apache/hop/pull/9359#discussion_r4537159542
##########
plugins/tech/mongodb/src/main/java/org/apache/hop/workflow/actions/mql/ActionMql.java:
##########
@@ -201,6 +193,72 @@ public Result execute(Result result, int nr) {
return result;
}
+ /**
+ * Parse the commands to run. A JSON array holds one command per element.
Anything else is read as
+ * a list of comma-separated documents, so that text after the first
document is either run or
+ * reported as invalid JSON, never silently ignored.
+ *
+ * @param mqlToExecute the JSON text
+ * @return the commands, in the order they are to be run
+ * @throws HopException when there is no command
+ */
+ static List<Document> parseCommands(String mqlToExecute) throws HopException
{
+ String trimmed = mqlToExecute == null ? "" : mqlToExecute.trim();
+ if (trimmed.isEmpty()) {
+ throw new HopException(BaseMessages.getString(PKG,
"ActionMQL.NoCommandFound"));
+ }
+ String array = trimmed.startsWith("[") ? trimmed : "[" + trimmed + "]";
Review Comment:
Wrapping any non-array input in `[...]` changes what existing workflows do:
text after the first document used to be ignored, and is now either run as
extra commands or rejected. `{"drop":"tmp"}, {"drop":"prod"}` used to run only
the first drop and now runs both; a script ending in `;` (e.g. `{ ping: 1 };`)
used to work and now fails with a JSON parse error. I agree silently ignoring
text is bad, but running extra commands that were ignored before is risky; I'd
rather reject trailing text than execute it, and mention the change in the
upgrade notes.
##########
plugins/tech/mongodb/src/main/java/org/apache/hop/workflow/actions/mql/ActionMql.java:
##########
@@ -201,6 +193,72 @@ public Result execute(Result result, int nr) {
return result;
}
+ /**
+ * Parse the commands to run. A JSON array holds one command per element.
Anything else is read as
+ * a list of comma-separated documents, so that text after the first
document is either run or
+ * reported as invalid JSON, never silently ignored.
+ *
+ * @param mqlToExecute the JSON text
+ * @return the commands, in the order they are to be run
+ * @throws HopException when there is no command
+ */
+ static List<Document> parseCommands(String mqlToExecute) throws HopException
{
+ String trimmed = mqlToExecute == null ? "" : mqlToExecute.trim();
+ if (trimmed.isEmpty()) {
+ throw new HopException(BaseMessages.getString(PKG,
"ActionMQL.NoCommandFound"));
+ }
+ String array = trimmed.startsWith("[") ? trimmed : "[" + trimmed + "]";
+ Document wrapper = Document.parse("{ \"array\": " + array + " }");
Review Comment:
Because the input is embedded in `{ "array": ... }`, a JSON parse error now
reports a position in the generated text (about 12-13 characters later than
what the user typed), which no longer matches the line/column indicator in the
dialog. Could the position be corrected, or the error message mention the
original input?
##########
plugins/tech/mongodb/src/main/java/org/apache/hop/workflow/actions/mql/ActionMql.java:
##########
@@ -201,6 +193,72 @@ public Result execute(Result result, int nr) {
return result;
}
+ /**
+ * Parse the commands to run. A JSON array holds one command per element.
Anything else is read as
+ * a list of comma-separated documents, so that text after the first
document is either run or
+ * reported as invalid JSON, never silently ignored.
+ *
+ * @param mqlToExecute the JSON text
+ * @return the commands, in the order they are to be run
+ * @throws HopException when there is no command
+ */
+ static List<Document> parseCommands(String mqlToExecute) throws HopException
{
+ String trimmed = mqlToExecute == null ? "" : mqlToExecute.trim();
+ if (trimmed.isEmpty()) {
+ throw new HopException(BaseMessages.getString(PKG,
"ActionMQL.NoCommandFound"));
+ }
+ String array = trimmed.startsWith("[") ? trimmed : "[" + trimmed + "]";
+ Document wrapper = Document.parse("{ \"array\": " + array + " }");
+ if (wrapper.size() != 1) {
Review Comment:
This guard misses trailing text that repeats the `array` key, since
`Document` is a map and the second value replaces the first: `[ {"drop":"a"} ],
"array": [ {"ping":1} ]` ends up with one key, passes the check, and only
`ping` runs, so the first array is dropped silently, which the Javadoc says
can't happen. Parsing with a `JsonReader` and checking that the input ends
after the first value would avoid this.
##########
plugins/tech/mongodb/src/main/java/org/apache/hop/workflow/actions/mql/ActionMql.java:
##########
@@ -201,6 +193,72 @@ public Result execute(Result result, int nr) {
return result;
}
+ /**
+ * Parse the commands to run. A JSON array holds one command per element.
Anything else is read as
+ * a list of comma-separated documents, so that text after the first
document is either run or
+ * reported as invalid JSON, never silently ignored.
+ *
+ * @param mqlToExecute the JSON text
+ * @return the commands, in the order they are to be run
+ * @throws HopException when there is no command
+ */
+ static List<Document> parseCommands(String mqlToExecute) throws HopException
{
+ String trimmed = mqlToExecute == null ? "" : mqlToExecute.trim();
+ if (trimmed.isEmpty()) {
+ throw new HopException(BaseMessages.getString(PKG,
"ActionMQL.NoCommandFound"));
+ }
+ String array = trimmed.startsWith("[") ? trimmed : "[" + trimmed + "]";
+ Document wrapper = Document.parse("{ \"array\": " + array + " }");
+ if (wrapper.size() != 1) {
+ // For example: [ {...} ], "x": 1
+ throw new HopException(BaseMessages.getString(PKG,
"ActionMQL.TextAfterCommands"));
+ }
+ List<Document> commands = wrapper.getList("array", Document.class);
+ if (commands.isEmpty()) {
+ throw new HopException(BaseMessages.getString(PKG,
"ActionMQL.NoCommandFound"));
+ }
+ return commands;
+ }
+
+ /**
+ * A command that MongoDB accepts can still fail to write documents. Those
errors are in the reply
+ * (writeErrors, writeConcernError) while ok is 1, so runCommand() does not
throw.
+ *
+ * @param reply the reply of the command
+ * @param commandNr the number of the command (1-based)
+ * @param nrCommands the number of commands
+ * @throws MongoDbException when the reply holds write errors or a write
concern error
+ */
+ static void checkReply(Document reply, int commandNr, int nrCommands) throws
MongoDbException {
+ if (reply == null) {
+ return;
+ }
+ List<Document> writeErrors = reply.getList("writeErrors", Document.class);
+ if (writeErrors != null && !writeErrors.isEmpty()) {
Review Comment:
Failing on any `writeErrors` breaks workflows that rely on a partial write
on purpose, e.g. `{insert:'c', documents:[...], ordered:false}` to skip
duplicate keys. Before this change the action was true and took the success
hop; now it's false, and in an array of instructions the later ones don't run
anymore. This might need an option (or at least an upgrade note).
##########
plugins/tech/mongodb/src/main/java/org/apache/hop/workflow/actions/mql/ActionMql.java:
##########
@@ -201,6 +193,72 @@ public Result execute(Result result, int nr) {
return result;
}
+ /**
+ * Parse the commands to run. A JSON array holds one command per element.
Anything else is read as
+ * a list of comma-separated documents, so that text after the first
document is either run or
+ * reported as invalid JSON, never silently ignored.
+ *
+ * @param mqlToExecute the JSON text
+ * @return the commands, in the order they are to be run
+ * @throws HopException when there is no command
+ */
+ static List<Document> parseCommands(String mqlToExecute) throws HopException
{
+ String trimmed = mqlToExecute == null ? "" : mqlToExecute.trim();
+ if (trimmed.isEmpty()) {
+ throw new HopException(BaseMessages.getString(PKG,
"ActionMQL.NoCommandFound"));
+ }
+ String array = trimmed.startsWith("[") ? trimmed : "[" + trimmed + "]";
+ Document wrapper = Document.parse("{ \"array\": " + array + " }");
+ if (wrapper.size() != 1) {
+ // For example: [ {...} ], "x": 1
+ throw new HopException(BaseMessages.getString(PKG,
"ActionMQL.TextAfterCommands"));
+ }
+ List<Document> commands = wrapper.getList("array", Document.class);
Review Comment:
For input that mixes a document and an array without brackets (`{a:1},
[{b:1}]`) or an array of non-documents (`[1, 2]`), `getList("array",
Document.class)` throws a ClassCastException, and the log says `class
java.util.ArrayList cannot be cast to org.bson.Document`, which doesn't tell
the user what's wrong with the JSON. Could this be caught and turned into a
HopException with a clear message?
##########
plugins/databases/snowflake/src/main/java/org/apache/hop/workflow/actions/snowflake/WarehouseManager.java:
##########
@@ -389,7 +391,7 @@ private String getCreateSQL() {
if (!failIfExists && !replace) {
sql.append("IF NOT EXISTS ");
}
- sql.append(warehouseName).append(" WITH ");
+ sql.append(resolve(warehouseName)).append(" WITH ");
Review Comment:
The warehouse name is now resolved and placed into the CREATE (and ALTER,
line 437) statement without quoting or validation. A variable value such as `X
WITH WAREHOUSE_SIZE='XSMALL'; DROP DATABASE prod; --` runs extra statements;
before this change CREATE and ALTER used the literal text. Could the name be
validated as an identifier (or quoted) after resolving?
##########
plugins/tech/databricks/src/main/java/org/apache/hop/workflow/actions/databricks/ActionDatabricksJobWait.java:
##########
@@ -184,11 +184,13 @@ public void logError(String message) {
&& "CANCELED".equalsIgnoreCase(status.getResultState())) {
result.setNrErrors(1);
result.setResult(false);
- logError(
+ String msg =
BaseMessages.getString(
PKG,
"ActionDatabricksJobWait.Error.Canceled",
- Const.NVL(status.getStateMessage(), "")));
+ Const.NVL(status.getStateMessage(), ""));
+ setResultVariable(resultVariableError, msg);
Review Comment:
The error variable is now filled from
`ActionDatabricksJobWait.Error.Canceled`, but the pt_BR text of that key
(`messages_pt_BR.properties` line 115: `Databricks foi cancelado.`) has no
`{0}` placeholder, so in a pt_BR locale the cancellation reason from Databricks
is lost. The new test checks for the reason and only passes in English. Adding
`{0}` to the pt_BR text would fix it.
##########
plugins/databases/snowflake/src/main/java/org/apache/hop/workflow/actions/snowflake/WarehouseManager.java:
##########
@@ -305,6 +305,7 @@ public Result execute(Result previousResult, int nr) throws
HopException {
Result result = previousResult;
result.setResult(validate());
if (!result.isResult()) {
Review Comment:
Now that the name is resolved, `validate()` could also check that the
resolved warehouse name isn't empty. With an empty name, or `${WAREHOUSE}`
resolving to empty, the action sends `CREATE WAREHOUSE IF NOT EXISTS WITH ...`
and fails with a confusing Snowflake syntax error instead of a clear validation
message.
##########
plugins/misc/documentation/src/main/java/org/apache/hop/documentation/ActionDoc.java:
##########
@@ -143,6 +143,10 @@ public Result execute(Result prevResult, int nr) {
includingMetadata,
generateHtml,
removeMarkdown);
+ // The result of this action is its own: true when the documentation is
written.
+ // DocBuilder sets it back to false (and 1 error) when that fails.
+ //
+ prevResult.setResult(true);
Review Comment:
This sets the result to true but doesn't reset the error count, so if
`prevResult` arrives with `nrErrors > 0` the action reports `result=true` with
`nrErrors=1`, which contradicts the new docs ('true with 0 errors').
`prevResult.setNrErrors(0)` next to this would fix it. The success path in
`WarehouseManager.execute` has the same gap.
##########
plugins/tech/mongodb/src/main/java/org/apache/hop/workflow/actions/mql/ActionMql.java:
##########
@@ -201,6 +193,72 @@ public Result execute(Result result, int nr) {
return result;
}
+ /**
+ * Parse the commands to run. A JSON array holds one command per element.
Anything else is read as
+ * a list of comma-separated documents, so that text after the first
document is either run or
+ * reported as invalid JSON, never silently ignored.
+ *
+ * @param mqlToExecute the JSON text
+ * @return the commands, in the order they are to be run
+ * @throws HopException when there is no command
+ */
+ static List<Document> parseCommands(String mqlToExecute) throws HopException
{
+ String trimmed = mqlToExecute == null ? "" : mqlToExecute.trim();
+ if (trimmed.isEmpty()) {
+ throw new HopException(BaseMessages.getString(PKG,
"ActionMQL.NoCommandFound"));
+ }
+ String array = trimmed.startsWith("[") ? trimmed : "[" + trimmed + "]";
+ Document wrapper = Document.parse("{ \"array\": " + array + " }");
+ if (wrapper.size() != 1) {
+ // For example: [ {...} ], "x": 1
+ throw new HopException(BaseMessages.getString(PKG,
"ActionMQL.TextAfterCommands"));
+ }
+ List<Document> commands = wrapper.getList("array", Document.class);
+ if (commands.isEmpty()) {
+ throw new HopException(BaseMessages.getString(PKG,
"ActionMQL.NoCommandFound"));
+ }
+ return commands;
+ }
+
+ /**
+ * A command that MongoDB accepts can still fail to write documents. Those
errors are in the reply
+ * (writeErrors, writeConcernError) while ok is 1, so runCommand() does not
throw.
+ *
+ * @param reply the reply of the command
+ * @param commandNr the number of the command (1-based)
+ * @param nrCommands the number of commands
+ * @throws MongoDbException when the reply holds write errors or a write
concern error
+ */
+ static void checkReply(Document reply, int commandNr, int nrCommands) throws
MongoDbException {
Review Comment:
`checkReply` only checks top-level `writeErrors` and `writeConcernError`.
Commands that report errors differently still succeed, e.g. on MongoDB 8 the
`bulkWrite` command with a duplicate key returns `ok:1, nErrors:1` with the
per-operation errors in its cursor, so the action is true while the updated
docs say write errors make it false. Checking `nErrors` too would cover that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]