This is an automated email from the ASF dual-hosted git repository.
hansva pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-hop.git
The following commit(s) were added to refs/heads/master by this push:
new 5232529 HOP-3225 document error handling
new 5db8bf1 Merge pull request #1025 from bamaer/HOP-3225
5232529 is described below
commit 523252950ab79dfc0d0f487487c5a1c2d0cd45db
Author: Bart Maertens <[email protected]>
AuthorDate: Sat Aug 28 11:15:03 2021 +0200
HOP-3225 document error handling
---
.../images/error-handling-context-dialog.png | Bin 0 -> 19585 bytes
.../ROOT/assets/images/error-handling-dialog.png | Bin 0 -> 95815 bytes
.../assets/images/error-handling-hop-option.png | Bin 0 -> 18055 bytes
.../ROOT/assets/images/error-handling-hop.png | Bin 0 -> 11924 bytes
.../ROOT/assets/images/error-handling-output.png | Bin 0 -> 69916 bytes
docs/hop-user-manual/modules/ROOT/nav.adoc | 1 +
.../modules/ROOT/pages/pipeline/errorhandling.adoc | 63 +++++++++++++++++++++
7 files changed, 64 insertions(+)
diff --git
a/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-context-dialog.png
b/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-context-dialog.png
new file mode 100644
index 0000000..d8d5f98
Binary files /dev/null and
b/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-context-dialog.png
differ
diff --git
a/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-dialog.png
b/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-dialog.png
new file mode 100644
index 0000000..70a8ae3
Binary files /dev/null and
b/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-dialog.png
differ
diff --git
a/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-hop-option.png
b/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-hop-option.png
new file mode 100644
index 0000000..f039915
Binary files /dev/null and
b/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-hop-option.png
differ
diff --git
a/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-hop.png
b/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-hop.png
new file mode 100644
index 0000000..543773c
Binary files /dev/null and
b/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-hop.png differ
diff --git
a/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-output.png
b/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-output.png
new file mode 100644
index 0000000..0e4fa11
Binary files /dev/null and
b/docs/hop-user-manual/modules/ROOT/assets/images/error-handling-output.png
differ
diff --git a/docs/hop-user-manual/modules/ROOT/nav.adoc
b/docs/hop-user-manual/modules/ROOT/nav.adoc
index 13865c5..1a2e30f 100644
--- a/docs/hop-user-manual/modules/ROOT/nav.adoc
+++ b/docs/hop-user-manual/modules/ROOT/nav.adoc
@@ -38,6 +38,7 @@ under the License.
** xref:pipeline/hop-pipeline-editor.adoc[Pipeline Editor]
** xref:pipeline/create-pipeline.adoc[Create a Pipeline]
** xref:pipeline/run-preview-debug-pipeline.adoc[Run, Preview and Debug a
Pipeline]
+** xref:pipeline/errorhandling.adoc[Error Handling]
** xref:pipeline/beam/getting-started-with-beam.adoc[Getting started with
Apache Beam]
**
xref:pipeline/pipeline-run-configurations/pipeline-run-configurations.adoc[Pipeline
Run Configurations]
***
xref:pipeline/pipeline-run-configurations/beam-dataflow-pipeline-engine.adoc[Beam
Google DataFlow]
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/errorhandling.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/errorhandling.adoc
new file mode 100644
index 0000000..607f063
--- /dev/null
+++ b/docs/hop-user-manual/modules/ROOT/pages/pipeline/errorhandling.adoc
@@ -0,0 +1,63 @@
+////
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+////
+[[PipelineErrorHandling]]
+:imagesdir: ../assets/images
+
+= Pipeline Error Handling
+
+When a major failure occurs in one of the transforms, the pipeline is notified
and halts all active operations. This is fine in most cases, pipeline failures
are typically handled in the parent workflow (check the
xref:/workflow/create-workflow.adoc[create workflow] page to learn about error
hops in workflows).
+In some cases however, there are errors you'll want to handle gracefully
without halting the entire pipeline.
+
+In these cases where you don't want your pipeline to fail when an error
occurs, Hop pipelines support error handling on transforms and hops.
+
+When you create a hop from a transform that supports error handling to another
transform, the Hop pipeline editor will ask you if you want to create a hop for
the main output or for the error handling of your transform.
+
+image:error-handling-hop-option.png[Error handling hop option, width="40%"]
+
+If you choose to create an error handling transform, the hop will be shown in
red instead of the default black (or white, if you're in dark mode).
+
+image:error-handling-hop.png[Error handling hop, width="40%"]
+
+For each transform that supports error handling, there are a number of options
you can configure.
+Click on the transform icon to open the context dialog and selected the 'Error
handling' icon.
+
+image:error-handling-context-dialog.png[Error handling context dialog,
width="60%"]
+
+In the error handling dialog, you can specify additional fields that will be
added to your pipeline stream.
+
+image:error-handling-dialog.png[Error handling dialog, width="75%"]
+
+The available options are:
+
+[options="header", width="90%"]
+|===
+|option|description
+|target transform|the transform that will receive the error information
+|enable the error handling|enable error handling from this transform
+|nr of errors fieldname|the nummer of errors that occurred in the pipeline
+|error description fieldname|fieldname to contain the error description
+|error fields fieldname|the pipeline field where an error occurred
+|error codes fieldname|the error code for the error that occurred
+|max nr errors allowed|max number of errors allowed before the pipeline fails.
+|max % errors allowed (empty = 100%)|the percentage of errors that is allowed
before the pipeline fails
+|min nr of rows to read before doing % evaluation|number of rows to read
before doing the percentage evaluation. These rows will be taken into account
in the evaluation, but the evaluation will only be performed once the specified
number of rows has been processed.
+|===
+
+An example output when trying to cast an invalid date string to a date is
shown below.
+
+image:error-handling-output.png[Error handling output, width="90%"]
+
