Givo29 opened a new issue, #8701:
URL: https://github.com/apache/hop/issues/8701
### Apache Hop version?
2.18.1
### Java version?
21.X.X
### Operating system
Linux
### What happened?
When two or more parallel branches finish at the same time, the Join action
can be
started once per branch instead of once in total. Everything downstream of
the Join then runs
multiple times concurrently.
**Impact:**
In our nightly BI syncs, two short pipelines are launched in
parallel and then joined. Once or twice a week, the Join fires twice, which
runs all of the downstream steps twice. Among other things, two MySQL Bulk
Loader instances then shared the same
FIFO file and bulk-loaded the same table at the same time, which spliced
rows together and
corrupted data. The failure is intermittent and depends only on thread
timing.
**Steps to reproduce**
The attached workflow (attached as xml file due to github limitations)
reproduces the issue. It contains two stages:
1. **Working case:** three parallel Wait actions with staggered durations
(1s, 2s and 3s) lead
into a Join, followed by a Write to log action. Because the branches
finish at different
times, the message is logged once, as expected.
2. **Broken case:** four parallel Wait actions, all set to 1s, lead into a
Join, followed by a
Write to log action. Because the branches finish at the same time, the
message is logged
multiple times. This shows that the Join, and everything after it, was
executed more than
once.
As the issue is timing dependent, the number of duplicate messages can vary
between runs.
<img width="1788" height="651" alt="Image"
src="https://github.com/user-attachments/assets/653c91a2-3d2b-4705-b1d2-629f949168e2";
/>
<img width="1283" height="642" alt="Image"
src="https://github.com/user-attachments/assets/f9d3b169-ddc6-45e8-9a01-743a170561c0";
/>
[parallel-join-test.xml](https://github.com/user-attachments/files/32836615/parallel-join-test.xml)
### Issue Priority
Priority: 2
### Issue Component
Component: Workflows
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
