vbhanuchander-lang opened a new pull request, #8029:
URL: https://github.com/apache/hop/pull/8029
Addresses #8000.
The "Get data from XML" transform aborts the pipeline when a file cannot be
parsed, instead of
routing the failure to the transform connected to its error hop. Reading the
same document from a
field, or reading the file name from a field, behaves correctly — which is
what makes the file path
look like an oversight rather than a design choice.
### Cause
`openNextFile()` catches the parse failure and then unconditionally does
this:
```java
logError(...);
stopAll();
setErrors(1);
return false;
```
It never asks whether the transform has error handling configured, so
`stopAll()` takes the whole
pipeline down. That produces the "Pipeline is killing the other transforms!"
line in the report. The
in-fields path does ask — `processPutRow` checks
`getTransformMeta().isDoingErrorHandling()` and
calls `putError` — which is exactly why the two paths behave differently.
### Change
When error handling is enabled, the file is sent to the error stream and
reading continues with the
next file. When it is not, the previous behaviour is untouched.
One wrinkle worth flagging for review. `data.filenr` is normally incremented
before the document is
parsed, so recursing into `openNextFile()` moves on to the next file. But a
failure raised *earlier*
— resolving the file, reading its size, its attributes — leaves the pointer
untouched, and recursing
would retry the same file forever. The change records the pointer on entry
and steps over the file
explicitly if it has not advanced.
`openNextFile()` now declares `HopTransformException`, because `putError`
throws it. Every caller
already propagates `HopException`, so this does not ripple.
### Tests
`testErrorHandlingContinuesOnBadXmlFile` is the file-based counterpart to
the existing
`testErrorHandlingContinuesOnBadXml`: a temporary folder holding a good
file, an unparseable one and
another good file, with an error hop attached.
- both readable files are still read — 2 rows from the first, 1 from the
third
- the unreadable file yields exactly one error row
- the pipeline reports no errors
Reverting the production change fails that test with `transform should not
report errors ==>
expected: <0> but was: <1>`, which is the behaviour described in the issue.
### Verification
- `mvn -pl plugins/transforms/xml -Pskip-uitest test` — 154 tests, 0
failures, 0 errors
- `mvn spotless:check` and `mvn apache-rat:check` pass on the module
------------------------
- [x] I hereby declare this contribution to be licensed under the [Apache
License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
