bamaer commented on code in PR #9527:
URL: https://github.com/apache/hop/pull/9527#discussion_r4597942384


##########
engine/src/main/java/org/apache/hop/pipeline/SingleThreadedPipelineExecutor.java:
##########
@@ -369,6 +382,21 @@ public boolean oneIteration() throws HopException {
                 return false;
               }
             }
+
+            // All input arrived and was read, but the transform can still 
have work: a reader
+            // going through the file a row named, a mapping, the rows a Table 
output still holds.
+            // Like a transform without input, it is called until it says it 
is done. Otherwise it
+            // never sees the end of its input, and the single-threaded 
engine, which runs one
+            // iteration, disposes it with that work undone.
+            //
+            if (!transformDone && 
isAllInputRead(combi.transform.getInputRowSets())) {
+              while (!transformDone && !pipeline.isStopped()) {
+                transformDone = !combi.transform.processRow();
+                if (combi.transform.getErrors() > 0) {
+                  return false;
+                }
+              }
+            }

Review Comment:
   Fixed in 2904354b42: `batchComplete()` is skipped once the transform is 
done. The grouping repro gives `a 3`, `b 7` again, and the new 
`main-0007-group-by-end-of-input` fails on c97dfb73 and passes on the new head.



##########
engine/src/main/java/org/apache/hop/pipeline/SingleThreadedPipelineExecutor.java:
##########
@@ -299,6 +299,19 @@ public boolean init() {
    *
    * @return true if more iterations can be performed. False if this is not 
the case.
    */
+  /**
+   * Whether every input row set is done and has no rows left: the transform 
has read all its input.
+   * A transform removes a row set it read to the end, so no row sets left 
means the same.
+   */
+  private static boolean isAllInputRead(List<IRowSet> rowSets) {
+    for (IRowSet rowSet : rowSets) {
+      if (!rowSet.isDone() || rowSet.size() > 0) {

Review Comment:
   Fixed in 2904354b42 with `isOriginDone()`. The batch run gives 3 / 6 / 9 / 
12 rows again, and `SortRowsBatchesTest` fails against the old executor (`[3, 
3, 3, 3]`) and passes now.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to