mattcasters opened a new pull request, #8850:
URL: https://github.com/apache/hop/pull/8850
Fixes #8670: Fix execution details and parent links in execution information
### Description
1. **Row Sampling Persistence**:
- `FirstRowsExecutionDataSamplerStore`,
`LastRowsExecutionDataSamplerStore`, and `RandomRowsExecutionDataSamplerStore`
wrapped their internal `rows` list directly in `new RowBuffer(rowMeta, rows)`.
- When `LocalPipelineEngine.saveFinalExecutionInfoAndClose()` ran
`releasePublishedSamples()`, `store.clearSamples()` cleared the underlying
`rows` list in-place, which wiped out the sampled rows before they could be
persisted to disk/database.
- Fixed by making defensive copies in `RowBuffer` constructor and in
sampler stores under synchronization in `getSamples()`.
- In `LocalPipelineEngine`, ensured the location is closed and persisted
prior to releasing published samples.
2. **Parent-Child Link & Execution Resolution**:
- In the `Workflow` -> `Action` -> `Pipeline` hierarchy, action
executions are stored as child executions within the workflow's cache entry,
and pipeline executions point to action IDs.
- `BaseCachingExecutionInfoLocation` could not resolve child actions when
looking up `findParentId(pipelineId)`, `findExecutions(actionId)`, or
`getExecution(actionId)` when entries were not already in memory.
- Added `loadCacheEntryWithChild(childId)` to search files / database
records for parent entries containing the given child ID in
`BaseCachingExecutionInfoLocation`, `CachingFileExecutionInfoLocation`, and
`CachingDatabaseExecutionInfoLocation`.
- Fixed `FileExecutionInfoLocation.findParentId()` to search child
executions in parent execution files.
- Updated `WorkflowExecutionViewer.drillDown()` to resolve the action's
child execution ID when `selectedExecutionData` is null.
3. **Integration & Unit Tests**:
- Added new integration test suite `002` in `integration-tests/exec-info`
(`002-pipeline-with-mapping.hpl`, `002-mapping-sub.hpl`,
`002-sub-workflow.hwf`, `002-validation.hpl`,
`main-002-workflow-pipeline-mapping.hwf`) testing mapping output row capture
and parent-child link resolution across workflows, actions, and pipelines.
- Added unit test `testUncachedParentChildLinkResolution` in
`CachingFileExecutionInfoLocationTest`.
------------------------
- [x] Run `mvn clean install apache-rat:check` to make sure basic checks
pass. A more thorough check will be performed on your pull request
automatically.
- [x] If you have a group of commits related to the same change, please
squash your commits into one and force push your branch using `git rebase -i`.
- [x] Mention the appropriate issue in your description (for example:
`addresses #123`), if applicable.
To make clear that you license your contribution under the [Apache License
Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
you have to acknowledge this by using the following check-box.
- [x] I hereby declare this contribution to be licensed under the [Apache
License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
- [ ] In any other case, please file an [Apache Individual Contributor
License Agreement](https://www.apache.org/licenses/icla.pdf).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
