bamaer commented on code in PR #9360:
URL: https://github.com/apache/hop/pull/9360#discussion_r4539257618


##########
plugins/tech/neo4j/src/main/java/org/apache/hop/neo4j/actions/propertygraph/ActionCreatePropertyGraph.java:
##########
@@ -225,8 +224,67 @@ public Result execute(Result result, int nr) throws 
HopException {
     return result;
   }
 
+  /**
+   * Run the statements one by one. When one fails, drop the tables this run 
created, newest first,
+   * so a failing CREATE PROPERTY GRAPH doesn't leave them behind. Each 
statement commits on its
+   * own, so a rollback can't undo them.
+   */
+  private void executeStatements(Database database, List<PlannedStatement> 
statements)
+      throws HopException {
+    List<String> createdTables = new ArrayList<>();
+    for (PlannedStatement statement : statements) {
+      if (isDetailed()) {
+        logDetailed("Executing: " + statement.sql());
+      }
+      try {
+        database.execStatement(statement.sql());
+      } catch (HopException e) {
+        String dropError = dropCreatedTables(database, createdTables);
+        if (dropError != null) {
+          throw new HopException(
+              "The property graph can't be created and the tables it created 
can't be dropped: "
+                  + dropError,
+              e);
+        }
+        throw e;
+      }
+      if (statement.createdTable() != null) {
+        createdTables.add(statement.createdTable());

Review Comment:
   Fixed in abcad5df39: a table used by two nodes, or by a node and a 
relationship, is now refused before any table is checked or created. The CREATE 
PROPERTY GRAPH would fail on it anyway, since each element is named after its 
table. A CREATE TABLE that fails, for example because another process just 
created the table, is never added to the drop list.



##########
plugins/tech/neo4j/src/main/java/org/apache/hop/neo4j/actions/propertygraph/ActionCreatePropertyGraph.java:
##########
@@ -225,8 +224,67 @@ public Result execute(Result result, int nr) throws 
HopException {
     return result;
   }
 
+  /**
+   * Run the statements one by one. When one fails, drop the tables this run 
created, newest first,
+   * so a failing CREATE PROPERTY GRAPH doesn't leave them behind. Each 
statement commits on its
+   * own, so a rollback can't undo them.
+   */
+  private void executeStatements(Database database, List<PlannedStatement> 
statements)

Review Comment:
   Added ActionCreatePropertyGraphTest with a mocked Database: drop order, 
ALTER TABLE not dropped, failed CREATE TABLE not dropped, the 'stay in the 
database' message, and no SQL for a duplicate table.



##########
plugins/tech/neo4j/src/test/java/org/apache/hop/neo4j/actions/GraphActionFailuresTest.java:
##########
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.neo4j.actions;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+
+import java.util.List;
+import org.apache.hop.core.HopClientEnvironment;
+import org.apache.hop.core.Result;
+import org.apache.hop.core.exception.HopException;
+import org.apache.hop.core.graph.BaseGraphDatabase;
+import org.apache.hop.core.graph.CypherGraphDialect;
+import org.apache.hop.core.graph.GraphConstraintType;
+import org.apache.hop.core.graph.GraphDatabaseMeta;
+import org.apache.hop.core.graph.IGraphConnection;
+import org.apache.hop.core.graph.IGraphDialect;
+import org.apache.hop.core.logging.ILogChannel;
+import org.apache.hop.core.variables.IVariables;
+import org.apache.hop.metadata.serializer.memory.MemoryMetadataProvider;
+import org.apache.hop.neo4j.actions.constraint.ConstraintUpdate;
+import org.apache.hop.neo4j.actions.constraint.Neo4jConstraint;
+import org.apache.hop.neo4j.actions.cypherscript.CypherScript;
+import org.apache.hop.neo4j.actions.index.IndexUpdate;
+import org.apache.hop.neo4j.actions.index.Neo4jIndex;
+import org.apache.hop.neo4j.actions.index.ObjectType;
+import org.apache.hop.neo4j.actions.index.UpdateType;
+import org.apache.hop.workflow.action.ActionBase;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+
+/**
+ * Graph index, Graph constraint and Graph script are false with 1 error when 
they fail, so a
+ * failure hop can handle it, instead of stopping the workflow (#9219).
+ */
+class GraphActionFailuresTest {
+
+  private static final String REFUSING = "refusing";
+
+  @BeforeAll
+  static void beforeAll() throws HopException {
+    HopClientEnvironment.init();
+  }
+
+  @Test
+  void indexFailures() throws Exception {
+    assertFalseWithOneError(index("does-not-exist", rangeIndex()));
+    assertFalseWithOneError(index("", rangeIndex()));
+    assertFalseWithOneError(index(REFUSING, rangeIndex()));
+
+    IndexUpdate noType = rangeIndex();
+    noType.setType(null);
+    assertFalseWithOneError(index(REFUSING, noType));
+
+    IndexUpdate noProperties = rangeIndex();
+    noProperties.setObjectProperties("");
+    assertFalseWithOneError(index(REFUSING, noProperties));

Review Comment:
   Good catch. These now run against a fake database that accepts the 
connection and records statements, and check that nothing ran. Another test 
confirms that database runs valid updates.



##########
plugins/tech/neo4j/src/main/java/org/apache/hop/neo4j/actions/constraint/Neo4jConstraint.java:
##########
@@ -73,40 +73,37 @@ public Result execute(Result result, int nr) throws 
HopException {
     // Success unless something goes wrong, whatever the result of the 
previous action
     result.setResult(true);
 
-    connection =
-        NeoConnectionUtils.findGraphConnection(getMetadataProvider(), 
resolve(connectionName));
-
-    if (connection == null) {
-      result.setResult(false);
-      result.increaseErrors(1L);
-      throw new HopException("Please specify a Neo4j connection to use");
-    }
-
-    // Loop over the constraint updates to see which need deleting...
-    //
-    for (ConstraintUpdate constraintUpdate : constraintUpdates) {
-      if (constraintUpdate.getUpdateType() == null) {
-        throw new HopException("Please make sure to always specify a 
constraint update type");
+    try {
+      connection =
+          NeoConnectionUtils.getGraphConnection(getMetadataProvider(), 
resolve(connectionName));
+
+      for (ConstraintUpdate constraintUpdate : constraintUpdates) {
+        if (constraintUpdate.getUpdateType() == null) {
+          throw new HopException("Please make sure to always specify a 
constraint update type");
+        }
       }
-      switch (constraintUpdate.getUpdateType()) {
-        case DROP:
+
+      // Drop the constraints first...
+      //
+      for (ConstraintUpdate constraintUpdate : constraintUpdates) {
+        if (constraintUpdate.getUpdateType() == UpdateType.DROP) {
           dropConstraint(constraintUpdate);
-          break;
-        default:
-          break;
+        }
       }
-    }
 
-    // Create the constraints if needed
-    //
-    for (ConstraintUpdate constraintUpdate : constraintUpdates) {
-      switch (constraintUpdate.getUpdateType()) {
-        case CREATE:
+      // ... then create the constraints
+      //
+      for (ConstraintUpdate constraintUpdate : constraintUpdates) {
+        if (constraintUpdate.getUpdateType() == UpdateType.CREATE) {
           createConstraint(constraintUpdate);
-          break;
-        default:
-          break;
+        }
       }
+    } catch (Exception e) {
+      // The updates before the failing one stay done, the ones after it are 
not run
+      //
+      logError("Error updating the graph database constraints", e);
+      result.setResult(false);
+      result.setNrErrors(1);

Review Comment:
   Fixed in all four actions, with a test that 2 incoming errors become 3. In a 
workflow the engine resets the count before these actions, so nothing changes 
there, but this is the correct way to count.



##########
plugins/tech/neo4j/src/main/java/org/apache/hop/neo4j/actions/constraint/Neo4jConstraint.java:
##########
@@ -73,40 +73,37 @@ public Result execute(Result result, int nr) throws 
HopException {
     // Success unless something goes wrong, whatever the result of the 
previous action
     result.setResult(true);
 
-    connection =
-        NeoConnectionUtils.findGraphConnection(getMetadataProvider(), 
resolve(connectionName));
-
-    if (connection == null) {
-      result.setResult(false);
-      result.increaseErrors(1L);
-      throw new HopException("Please specify a Neo4j connection to use");
-    }
-
-    // Loop over the constraint updates to see which need deleting...
-    //
-    for (ConstraintUpdate constraintUpdate : constraintUpdates) {
-      if (constraintUpdate.getUpdateType() == null) {
-        throw new HopException("Please make sure to always specify a 
constraint update type");
+    try {

Review Comment:
   Done: Graph index and Graph constraint now share GraphSchemaUpdates.execute.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to