mattcasters opened a new pull request, #8645:
URL: https://github.com/apache/hop/pull/8645
Fixes #8640 : Explain pipelines and workflows with interactive infographics
and detailed conceptual guides
### Summary of Changes
This PR modernizes the Apache Hop documentation around **Pipelines** and
**Workflows**, replacing aging conceptual pages and outdated static screenshots
with clear, end-user-focused explanations and responsive animated infographics:
1. **Concepts Page (`getting-started/hop-concepts.adoc` & `concepts.adoc`)**:
- Replaced static PNGs with an interactive architectural overview SVG
diagram (`hop-architecture-overview.svg`).
- Cleanly separated Hop concepts for business and technical users without
jargon or internal Java classes.
- Updated snippet embeds (`item-types.adoc`, `pipeline.adoc`,
`workflow.adoc`, `transform.adoc`, `action.adoc`).
2. **Pipelines Page (`pipeline/pipelines.adoc`)**:
- Added animated SVG infographic (`pipeline-parallel-flow.svg`)
illustrating concurrent execution, in-memory row buffers, and push-based
streaming with flowing data particles.
- Clarified row buffer mechanics, throughput limits, and backpressure.
- Detailed scaling techniques: multiple copies, partitioning, and error
handling.
- Added comprehensive reference table for all 9 Pipeline Execution
Engines (Native Local, Native Remote, Native Load-balancing, Beam Flink, Beam
Spark, Beam Dataflow, Beam Direct, Native Spark, Single Threaded) explaining
what each engine is best for and how it operates.
- Added real-world use cases (batch ingestion, Kafka streaming,
interactive debugging, CI/CD testing, embedded Java, and PyHop data science).
3. **Workflows Page (`workflow/workflows.adoc`)**:
- Added animated SVG infographic (`workflow-backtracking-flow.svg`)
demonstrating sequential orchestration, conditional evaluation
(True/False/Unconditional), failure isolation, and path backtracking with
animated pulsing warning indicators.
- Clarified transaction boundaries, execution isolation, and result
passing between actions.
- Added comprehensive reference table of Workflow Execution Engines
(Local, Remote, Load-balancing).
- Added real-world orchestration scenarios (nightly warehouse ETL, API
polling & notification loops, distributed data processing, automated regression
suites, PyHop).
4. **SVG Compatibility & AsciiDoc Sizing**:
- Configured SVGs with explicit fixed integer dimensions and matching
viewBox aspect ratios to ensure proper rendering across standard browsers and
IDE previewers (such as IntelliJ AsciiDoc preview flexbox containers).
- Used `opts=inline` image macros to preserve responsive scaling and CSS
animations.
- Implemented light/dark theme styling with `@media
(prefers-color-scheme: dark)`.
5. **Verification**:
- Built documentation site via Antora: 0 conversion errors.
- Verified link integrity: 0 broken xrefs, includes, or image paths.
------------------------
- [x] Run `mvn clean install apache-rat:check` to make sure basic checks
pass. A more thorough check will be performed on your pull request
automatically.
- [x] If you have a group of commits related to the same change, please
squash your commits into one and force push your branch using `git rebase -i`.
- [x] Mention the appropriate issue in your description (for example:
`addresses #123`), if applicable.
To make clear that you license your contribution under the [Apache License
Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
you have to acknowledge this by using the following check-box.
- [x] I hereby declare this contribution to be licensed under the [Apache
License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
