This is an automated email from the ASF dual-hosted git repository.
asf-gitbox-commits pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hop-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 9f3dc766c3 Website updated to 481ea60
9f3dc766c3 is described below
commit 9f3dc766c378ec02b5a5e7d6a193d49e1fa56b66
Author: jenkins <[email protected]>
AuthorDate: Fri Oct 16 14:19:40 2026 +0000
Website updated to 481ea60
---
dev-manual/next/integration-testing.html | 2 +-
manual/next/database/databases/derby.html | 2 +-
manual/next/hop-gui/file-dialog.html | 4 +-
manual/next/hop-gui/hop-web.html | 2 +-
manual/next/hop-gui/perspective-configuration.html | 5 +
manual/next/hop-run/index.html | 6 +-
manual/next/hop-tools/hop-encrypt.html | 2 +-
.../next/hop-vs-kettle/import-kettle-projects.html | 2 +-
manual/next/how-to-guides/loops-in-apache-hop.html | 2 +-
manual/next/logging/logging-basics.html | 2 +-
.../next/metadata-types/azure-authentication.html | 10 +
manual/next/metadata-types/hop-server.html | 2 +-
manual/next/password/passwords.html | 2 +-
.../next/pipeline/beam/beam-samples-dataflow.html | 2 +-
.../dataflowPipeline/google-dataflow-pipeline.html | 2 +-
manual/next/pipeline/create-pipeline.html | 2 +-
.../beam-spark-pipeline-engine.html | 6 +-
.../next/pipeline/run-preview-debug-pipeline.html | 2 +-
.../next/pipeline/transforms/aws-sns-notify.html | 2 +-
.../next/pipeline/transforms/aws-sqs-reader.html | 6 +-
manual/next/pipeline/transforms/concatfields.html | 2 +-
.../next/pipeline/transforms/dimensionlookup.html | 8 +-
manual/next/pipeline/transforms/formula.html | 4 +-
manual/next/pipeline/transforms/getfilenames.html | 2 +-
manual/next/pipeline/transforms/jdbcmetadata.html | 2 +-
manual/next/pipeline/transforms/jsoninput.html | 2 +-
manual/next/pipeline/transforms/neo4j-import.html | 2 +-
.../pipeline/transforms/redshift-bulkloader.html | 2 +-
manual/next/pipeline/transforms/replacestring.html | 2 +-
.../next/pipeline/transforms/rowdenormaliser.html | 2 +-
manual/next/pipeline/transforms/tablecompare.html | 2 +-
manual/next/pipeline/transforms/textfileinput.html | 343 ++++++++++++++++++++-
.../next/pipeline/transforms/tokenreplacement.html | 2 +-
.../transforms/userdefinedjavaexpression.html | 2 +-
manual/next/pipeline/transforms/xmljoin.html | 4 +-
manual/next/pipeline/transforms/yamlinput.html | 2 +-
manual/next/vfs/aws-s3-vfs.html | 24 ++
manual/next/vfs/google-cloud-storage-vfs.html | 14 +
.../next/workflow/actions/cassandra-exec-cql.html | 2 +-
manual/next/workflow/actions/ftpdelete.html | 4 +-
manual/next/workflow/actions/ftpput.html | 2 +-
manual/next/workflow/actions/mail.html | 2 +-
manual/next/workflow/actions/xslt.html | 2 +-
43 files changed, 440 insertions(+), 58 deletions(-)
diff --git a/dev-manual/next/integration-testing.html
b/dev-manual/next/integration-testing.html
index 2f67053a6e..542598ec4d 100644
--- a/dev-manual/next/integration-testing.html
+++ b/dev-manual/next/integration-testing.html
@@ -792,7 +792,7 @@ This script is also used inside the automated test suite
but can be used in a st
<div class="paragraph">
<p>This script can be found at following location
<code>hop/integration-tests/scripts/run-tests.sh</code>.
Be sure to export the needed system variables for the script to find the
correct paths to run the tests.
-You can aslo run a single project by adding the project name to the command
<code>./run-tests.sh projectname</code></p>
+You can also run a single project by adding the project name to the command
<code>./run-tests.sh projectname</code></p>
</div>
</div>
</div>
diff --git a/manual/next/database/databases/derby.html
b/manual/next/database/databases/derby.html
index f9175b8083..321afd657c 100644
--- a/manual/next/database/databases/derby.html
+++ b/manual/next/database/databases/derby.html
@@ -2977,7 +2977,7 @@
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
-Derby JDBC driver has two different modes, if you want to use Derby Embbeded,
replace the <em>derbyclient.jar</em> file with <em>derby.jar</em>
+Derby JDBC driver has two different modes, if you want to use Derby Embedded,
replace the <em>derbyclient.jar</em> file with <em>derby.jar</em>
</td>
</tr>
</table>
diff --git a/manual/next/hop-gui/file-dialog.html
b/manual/next/hop-gui/file-dialog.html
index a75e689fa0..2a63b64e12 100644
--- a/manual/next/hop-gui/file-dialog.html
+++ b/manual/next/hop-gui/file-dialog.html
@@ -2933,7 +2933,7 @@
</div>
<div class="paragraph">
<p>This dialog offers a lot more than basically selecting and saving files.
-Among other things, it enables you to work with VFS file sytems directly,
provides file/folder information, lets you bookmark files and folders.</p>
+Among other things, it enables you to work with VFS file systems directly,
provides file/folder information, lets you bookmark files and folders.</p>
</div>
<div class="paragraph">
<p>The list below walks you through the options that are at your disposal in
the Hop File Dialog.</p>
@@ -3030,7 +3030,7 @@ By default, this list includes:</p>
</li>
<li>
<p><strong>Bookmarks</strong>: Bookmark your favorite files and folders for
easy and fast access.
-Tools → Options contains an option to specify whether you want or
don’t want global bookmarks in the file dialog (global by defaul).</p>
+Tools → Options contains an option to specify whether you want or
don’t want global bookmarks in the file dialog (global by default).</p>
<div class="olist loweralpha">
<ol class="loweralpha" type="a">
<li>
diff --git a/manual/next/hop-gui/hop-web.html b/manual/next/hop-gui/hop-web.html
index f2bda42cb0..224cd654a6 100644
--- a/manual/next/hop-gui/hop-web.html
+++ b/manual/next/hop-gui/hop-web.html
@@ -2943,7 +2943,7 @@
With each build, an update is pushed to <a
href="https://hub.docker.com/r/apache/hop-web"; rel="noopener"
target="_blank">Docker Hub</a>.</p>
</div>
<div class="paragraph">
-<p>This continously updated docker image is by far the easiest way to try out
Hop Web:</p>
+<p>This continuously updated docker image is by far the easiest way to try out
Hop Web:</p>
</div>
<div class="paragraph">
<p>Pull the latest build with: <code>docker pull apache/hop-web</code>.</p>
diff --git a/manual/next/hop-gui/perspective-configuration.html
b/manual/next/hop-gui/perspective-configuration.html
index e396a6bf7e..bff6354c40 100644
--- a/manual/next/hop-gui/perspective-configuration.html
+++ b/manual/next/hop-gui/perspective-configuration.html
@@ -4141,6 +4141,11 @@ Check the <a
href="https://hop.apache.org/community/contribution-guides/translat
<td class="tableblock halign-left valign-top"><p class="tableblock">How long
to cache folder listing results (in seconds, default 5). If not set, falls back
to 10 seconds. Accepts variables.</p></td>
<td class="tableblock halign-left valign-top"><p
class="tableblock"><code>5</code></p></td>
</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Read block
size</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">How many
bytes one download request fetches when a file is read, like 8MB. Leave empty
for 8MB. Transforms that fetch ahead, such as Text File Input, download blocks
of this size, several at a time, each with one request. Accepts
variables.</p></td>
+<td class="tableblock halign-left valign-top"></td>
+</tr>
</tbody>
</table>
</div>
diff --git a/manual/next/hop-run/index.html b/manual/next/hop-run/index.html
index 4184e2d46f..ba154140e7 100644
--- a/manual/next/hop-run/index.html
+++ b/manual/next/hop-run/index.html
@@ -4,14 +4,14 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>Hop Run | Apache Hop</title>
-<meta name="description" content="Hop Run is a command line tool to run your
workflows and pipelines. Hop Run can be launched through the script with the
corresponding name in your Hop installation directory, either
`hop-run.sh` on Mac and Linux or `hop-run.bat` on Windows.
Except for the laucher script, all options are platform agnostic; there is no
difference in the way hop-run works on any operating system.">
+<meta name="description" content="Hop Run is a command line tool to run your
workflows and pipelines. Hop Run can be launched through the script with the
corresponding name in your Hop installation directory, either
`hop-run.sh` on Mac and Linux or `hop-run.bat` on Windows.
Except for the launcher script, all options are platform agnostic; there is no
difference in the way hop-run works on any operating system.">
<link rel="canonical"
href="https://hop.apache.org/manual/latest/hop-run/index.html";>
<meta property="og:type" content="website">
<meta property="og:site_name" content="Apache Hop">
<meta property="og:title" content="Hop Run">
-<meta property="og:description" content="Hop Run is a command line tool to run
your workflows and pipelines. Hop Run can be launched through the script with
the corresponding name in your Hop installation directory, either
`hop-run.sh` on Mac and Linux or `hop-run.bat` on Windows.
Except for the laucher script, all options are platform agnostic; there is no
difference in the way hop-run works on any operating system.">
+<meta property="og:description" content="Hop Run is a command line tool to run
your workflows and pipelines. Hop Run can be launched through the script with
the corresponding name in your Hop installation directory, either
`hop-run.sh` on Mac and Linux or `hop-run.bat` on Windows.
Except for the launcher script, all options are platform agnostic; there is no
difference in the way hop-run works on any operating system.">
<meta property="og:url"
content="https://hop.apache.org/manual/latest/hop-run/index.html";>
<meta property="og:image" content="https://hop.apache.org/img/social-card.png";>
<meta name="twitter:card" content="summary_large_image">
@@ -2937,7 +2937,7 @@
<p>Hop Run can be launched through the script with the corresponding name in
your Hop installation directory, either <code>hop-run.sh</code> on Mac and
Linux or <code>hop-run.bat</code> on Windows.</p>
</div>
<div class="paragraph">
-<p>Except for the laucher script, all options are platform agnostic; there is
no difference in the way hop-run works on any operating system.</p>
+<p>Except for the launcher script, all options are platform agnostic; there is
no difference in the way hop-run works on any operating system.</p>
</div>
</div>
</div>
diff --git a/manual/next/hop-tools/hop-encrypt.html
b/manual/next/hop-tools/hop-encrypt.html
index e32a9f6c65..28850fa8b5 100644
--- a/manual/next/hop-tools/hop-encrypt.html
+++ b/manual/next/hop-tools/hop-encrypt.html
@@ -3118,7 +3118,7 @@ OBF:1s3g1s3i1s3k1s3m</code></pre>
<h2 id="_notes_about_using_hop_encrypt_on_windows"><a class="link"
href="#_notes_about_using_hop_encrypt_on_windows">Notes about using hop-encrypt
on Windows</a></h2>
<div class="sectionbody">
<div class="paragraph">
-<p>When using the <code>hop-encrypt</code> tool on Windows, you may need to be
aware about some characters that for Windows command line are considered as
reserved characters and must be managed appropriately becuase otherwise the
tool fails in encrypting your password.</p>
+<p>When using the <code>hop-encrypt</code> tool on Windows, you may need to be
aware about some characters that for Windows command line are considered as
reserved characters and must be managed appropriately because otherwise the
tool fails in encrypting your password.</p>
</div>
<div class="paragraph">
<p>As documented at <a
href="https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/cmd";
rel="noopener" target="_blank">this page</a>, on Windows, <em>the ampersand
&, pipe |, and parentheses ( ) are special characters that must be preceded
by the escape character ^ or quotation marks when you pass them as
arguments</em>. Fort this reason when you run your hop-encrypt.bat command you
should use the escape character ^ before the any of the special characters [...]
diff --git a/manual/next/hop-vs-kettle/import-kettle-projects.html
b/manual/next/hop-vs-kettle/import-kettle-projects.html
index ce13ea233e..5ab1ba5770 100644
--- a/manual/next/hop-vs-kettle/import-kettle-projects.html
+++ b/manual/next/hop-vs-kettle/import-kettle-projects.html
@@ -3193,7 +3193,7 @@ Skipped files are left unchanged.
</table>
</div>
<div class="paragraph">
-<p>When multiple database connections with the same name but different
configurations were found (see 'Known limitations'), a
<code>connnections.csv</code> file will be created in the project folder.
+<p>When multiple database connections with the same name but different
configurations were found (see 'Known limitations'), a
<code>connections.csv</code> file will be created in the project folder.
This file contains a list of all jobs and transformations, with the
connections they use.</p>
</div>
</div>
diff --git a/manual/next/how-to-guides/loops-in-apache-hop.html
b/manual/next/how-to-guides/loops-in-apache-hop.html
index a58249c4dc..8157f657b1 100644
--- a/manual/next/how-to-guides/loops-in-apache-hop.html
+++ b/manual/next/how-to-guides/loops-in-apache-hop.html
@@ -3011,7 +3011,7 @@
2023/04/24 11:25:08 - log ${PRM_COUNTER}.0 -
2023/04/24 11:25:08 - log ${PRM_COUNTER}.0 - ------------> Linenr
1------------------------------
2023/04/24 11:25:08 - log ${PRM_COUNTER}.0 - #################################
-2023/04/24 11:25:08 - log ${PRM_COUNTER}.0 - the vaule for PRM_COUNTER is now
10
+2023/04/24 11:25:08 - log ${PRM_COUNTER}.0 - the value for PRM_COUNTER is now
10
2023/04/24 11:25:08 - log ${PRM_COUNTER}.0 - #################################
2023/04/24 11:25:08 - log ${PRM_COUNTER}.0 -
2023/04/24 11:25:08 - log ${PRM_COUNTER}.0 -
diff --git a/manual/next/logging/logging-basics.html
b/manual/next/logging/logging-basics.html
index 59cdf7af6e..f2fc055f01 100644
--- a/manual/next/logging/logging-basics.html
+++ b/manual/next/logging/logging-basics.html
@@ -3042,7 +3042,7 @@ It is also possible to use the debug plugin to set custom
logging levels on a tr
<div class="sect3">
<h4 id="_write_to_log"><a class="link" href="#_write_to_log">Write to
log</a></h4>
<div class="paragraph">
-<p>The Write to log acction writes a specific string to the Hop logging
system.</p>
+<p>The Write to log action writes a specific string to the Hop logging
system.</p>
</div>
<div class="paragraph">
<p>Check the <a href="../workflow/actions/writetolog.html" class="xref
page">Write To Log</a> page for more details.</p>
diff --git a/manual/next/metadata-types/azure-authentication.html
b/manual/next/metadata-types/azure-authentication.html
index 3a12740810..68e076463a 100644
--- a/manual/next/metadata-types/azure-authentication.html
+++ b/manual/next/metadata-types/azure-authentication.html
@@ -2972,6 +2972,16 @@ Using an Azure connection to connect to a blob store
requires you to specify the
<td class="tableblock halign-left valign-top"><p class="tableblock">Storage
Endpoint</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Leave
empty for default Azure endpoints or fill in when using a compatible
service</p></td>
</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Cache TTL
(seconds)</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">How long
folder listings are cached, in seconds. Leave empty for 5.</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Read block
size</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">How many
bytes one download request fetches when a file is read, for example 8MB or
16MB. Leave empty for 8MB.
+Larger blocks make fewer requests per file and take more memory.
+Transforms that fetch ahead, such as Text File Input, download blocks of this
size, several at a time, each with one request.</p></td>
+</tr>
</tbody>
</table>
<div class="paragraph">
diff --git a/manual/next/metadata-types/hop-server.html
b/manual/next/metadata-types/hop-server.html
index 3b850f1b28..0d1fb41e50 100644
--- a/manual/next/metadata-types/hop-server.html
+++ b/manual/next/metadata-types/hop-server.html
@@ -2990,7 +2990,7 @@
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Use https
protocal</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Use https
protocol</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">default:
false</p></td>
</tr>
</tbody>
diff --git a/manual/next/password/passwords.html
b/manual/next/password/passwords.html
index f69b598eb8..8346aec312 100644
--- a/manual/next/password/passwords.html
+++ b/manual/next/password/passwords.html
@@ -2980,7 +2980,7 @@ The example below shows a database connection with an
obfuscated password 'abcd.
<p>Obfuscated passwords for Hop Server can be stored in the server’s
.pwd file.</p>
</div>
<div class="paragraph">
-<p>In the example below, Hop’s default server hop.pwd contains the
username 'cluster' and the default obfuscated passowrd 'cluster' , with the
'OBF:' prefix.</p>
+<p>In the example below, Hop’s default server hop.pwd contains the
username 'cluster' and the default obfuscated password 'cluster' , with the
'OBF:' prefix.</p>
</div>
<div class="listingblock">
<div class="content">
diff --git a/manual/next/pipeline/beam/beam-samples-dataflow.html
b/manual/next/pipeline/beam/beam-samples-dataflow.html
index 8ebfc6b002..c721213c79 100644
--- a/manual/next/pipeline/beam/beam-samples-dataflow.html
+++ b/manual/next/pipeline/beam/beam-samples-dataflow.html
@@ -3030,7 +3030,7 @@
<p>For the sake of simplicity, check the "Use public IPs?" option. Check the
Google Cloud docs to learn more about configuring your project to run with
private IP addresses.</p>
</div>
<div class="paragraph">
-<p>In the Dataflow pipeline run configuration’s variables tab, change
the values DATA_INPUT, STATE_INPUT and DATA_OUTPUT variables to the bucket name
you just created. Als change the filename customers-noheader-1M.txt to
customers-noheader-1k.txt.</p>
+<p>In the Dataflow pipeline run configuration’s variables tab, change
the values DATA_INPUT, STATE_INPUT and DATA_OUTPUT variables to the bucket name
you just created. Also change the filename customers-noheader-1M.txt to
customers-noheader-1k.txt.</p>
</div>
<div class="paragraph">
<p><span class="image"><img
src="../../_images/beam/run-samples/hop-dataflow-run-config-variables.png"
alt="Dataflow Run Configuration - variables" width="45%"></span></p>
diff --git
a/manual/next/pipeline/beam/dataflowPipeline/google-dataflow-pipeline.html
b/manual/next/pipeline/beam/dataflowPipeline/google-dataflow-pipeline.html
index 0a4fd8d5ca..3a22873a7e 100644
--- a/manual/next/pipeline/beam/dataflowPipeline/google-dataflow-pipeline.html
+++ b/manual/next/pipeline/beam/dataflowPipeline/google-dataflow-pipeline.html
@@ -3034,7 +3034,7 @@ This file can be generated from the GUI under Tools
→ Export metadata to J
</div>
</div>
<div class="paragraph">
-<p>When selecting the Beam Flex template metadata file you will notice
required parameters showing up. You can then add the path yo yhe Hop metadata
and Hop pipeline stored in cloud storage.</p>
+<p>When selecting the Beam Flex template metadata file you will notice
required parameters showing up. You can then add the path to the Hop metadata
and Hop pipeline stored in cloud storage.</p>
</div>
</div>
</div>
diff --git a/manual/next/pipeline/create-pipeline.html
b/manual/next/pipeline/create-pipeline.html
index bfa7d08ce3..afd0b5547e 100644
--- a/manual/next/pipeline/create-pipeline.html
+++ b/manual/next/pipeline/create-pipeline.html
@@ -3245,7 +3245,7 @@ Click the 'Create hop'
image::getting-started/icons/HOP.svg[Create hop, 25px, al
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Result is
FALSE</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">pecifies
that the transform will be executed only when the result from the previous
transform is false</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies
that the transform will be executed only when the result from the previous
transform is false</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Main
output of transform</p></td>
diff --git
a/manual/next/pipeline/pipeline-run-configurations/beam-spark-pipeline-engine.html
b/manual/next/pipeline/pipeline-run-configurations/beam-spark-pipeline-engine.html
index eedbf366d9..6ebc57ba62 100644
---
a/manual/next/pipeline/pipeline-run-configurations/beam-spark-pipeline-engine.html
+++
b/manual/next/pipeline/pipeline-run-configurations/beam-spark-pipeline-engine.html
@@ -3062,7 +3062,7 @@ For durability, a reliable filesystem such as HDFS/S3/GS
is necessary.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Streaming:
minimum read time (ms)</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Mimimum
elapsed read time.</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Minimum
elapsed read time.</p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
<tr>
@@ -3129,7 +3129,7 @@ For durability, a reliable filesystem such as HDFS/S3/GS
is necessary.</p></td>
<div class="paragraph">
<p>Since execution of a pipeline on Spark is only possible from the Spark
Master it is possible to start a Hop server on the master.
Then you can remotely execute from anywhere on your Spark master of choice.
-Make sure that any referencable artifacts like the fat-jar you want to use is
available to the Hop server and that you have configured your installation as
described in the previous section.</p>
+Make sure that any referenceable artifacts like the fat-jar you want to use is
available to the Hop server and that you have configured your installation as
described in the previous section.</p>
</div>
</div>
</div>
@@ -3252,7 +3252,7 @@ You can also specify <code>*</code> to automatically
figure that out for your sy
<h2 id="_possible_errors"><a class="link" href="#_possible_errors">Possible
errors</a></h2>
<div class="sectionbody">
<div class="paragraph">
-<p>When you receive a stack trace looking lik this, it usually means it can
not find the spark master.</p>
+<p>When you receive a stack trace looking like this, it usually means it can
not find the spark master.</p>
</div>
<div class="listingblock">
<div class="content">
diff --git a/manual/next/pipeline/run-preview-debug-pipeline.html
b/manual/next/pipeline/run-preview-debug-pipeline.html
index 1f767896e3..3c20b3607c 100644
--- a/manual/next/pipeline/run-preview-debug-pipeline.html
+++ b/manual/next/pipeline/run-preview-debug-pipeline.html
@@ -3156,7 +3156,7 @@ The <a
href="pipeline-run-configurations/native-local-pipeline-engine.html" clas
</tbody>
</table>
<div class="paragraph">
-<p>The logging tab displays the log of the pipeline according to the log level
choosen on execution.</p>
+<p>The logging tab displays the log of the pipeline according to the log level
chosen on execution.</p>
</div>
<div class="imageblock text-left">
<div class="content">
diff --git a/manual/next/pipeline/transforms/aws-sns-notify.html
b/manual/next/pipeline/transforms/aws-sns-notify.html
index ceeb8e7c65..609c91e3b4 100644
--- a/manual/next/pipeline/transforms/aws-sns-notify.html
+++ b/manual/next/pipeline/transforms/aws-sns-notify.html
@@ -3006,7 +3006,7 @@
<p>Before the first execution you need to create an IAM-Role (e.g. for usage
on EC2/ECS) or an IAM-User with an AWS Key and Secret and attach the required
policies for pushing notifications via SNS.</p>
</div>
<div class="paragraph">
-<p>You also need to create one ore more subscription topics you want to push
the messages to.</p>
+<p>You also need to create one or more subscription topics you want to push
the messages to.</p>
</div>
</div>
</div>
diff --git a/manual/next/pipeline/transforms/aws-sqs-reader.html
b/manual/next/pipeline/transforms/aws-sqs-reader.html
index e81d3081c8..32a5d5709c 100644
--- a/manual/next/pipeline/transforms/aws-sqs-reader.html
+++ b/manual/next/pipeline/transforms/aws-sqs-reader.html
@@ -3006,7 +3006,7 @@
<p>Before the first execution you need to create an IAM-Role (e.g. for usage
on EC2/ECS) or an IAM-User with an AWS Key and Secret and attach the required
policies for pushing notifications via SNS.</p>
</div>
<div class="paragraph">
-<p>You also need to create one ore more subscription topics you want to push
the messages to.</p>
+<p>You also need to create one or more subscription topics you want to push
the messages to.</p>
</div>
</div>
</div>
@@ -3044,7 +3044,7 @@
<td class="tableblock halign-left valign-top"><p class="tableblock">the
AWS-Region the service is running in.</p></td>
</tr>
<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">SQS Queueu
URL</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">SQS Queue
URL</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">the URL of
the SQS Queue (starting with https:// - not the ARN!)</p></td>
</tr>
</tbody>
@@ -3053,7 +3053,7 @@
<div class="sect2">
<h3 id="_output_definition"><a class="link" href="#_output_definition">Output
definition</a></h3>
<div class="paragraph">
-<p>On the Output Definition Tab you can define the output fields for the
informations read from the SQS message as well as some initial settings for
receiving the messages.</p>
+<p>On the Output Definition Tab you can define the output fields for the
information read from the SQS message as well as some initial settings for
receiving the messages.</p>
</div>
<div class="sect3">
<h4 id="_output_settings"><a class="link" href="#_output_settings">Output
Settings</a></h4>
diff --git a/manual/next/pipeline/transforms/concatfields.html
b/manual/next/pipeline/transforms/concatfields.html
index 4d39c5c5a4..1b51e4289e 100644
--- a/manual/next/pipeline/transforms/concatfields.html
+++ b/manual/next/pipeline/transforms/concatfields.html
@@ -3045,7 +3045,7 @@ The enclosure string is optional.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Remove
concatenated fields from output?</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">If
checked, the fields that were used to create the concatenated field (specified
in <code>Target Field Name</code>) will be removed from the output and wil not
be available in later transforms.</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">If
checked, the fields that were used to create the concatenated field (specified
in <code>Target Field Name</code>) will be removed from the output and will not
be available in later transforms.</p></td>
</tr>
</tbody>
</table>
diff --git a/manual/next/pipeline/transforms/dimensionlookup.html
b/manual/next/pipeline/transforms/dimensionlookup.html
index 44da4519cc..51bb357b6a 100644
--- a/manual/next/pipeline/transforms/dimensionlookup.html
+++ b/manual/next/pipeline/transforms/dimensionlookup.html
@@ -3269,16 +3269,16 @@ These are the optional fields:</p>
<div class="ulist">
<ul>
<li>
-<p>Date of last insert or update (without stream field as source) : adds and
manges a Date field</p>
+<p>Date of last insert or update (without stream field as source) : adds and
manages a Date field</p>
</li>
<li>
-<p>Date of last insert (without stream field as source) : adds and manges a
Date field</p>
+<p>Date of last insert (without stream field as source) : adds and manages a
Date field</p>
</li>
<li>
-<p>Date of last update (without stream field as source) : adds and manges a
Date field</p>
+<p>Date of last update (without stream field as source) : adds and manages a
Date field</p>
</li>
<li>
-<p>Last version (without stream field as source) : adds and manges a Boolean
field. (converted into Char(1) or boolean database data type depending on your
database connection settings and availability of such data type)</p>
+<p>Last version (without stream field as source) : adds and manages a Boolean
field. (converted into Char(1) or boolean database data type depending on your
database connection settings and availability of such data type)</p>
</li>
<li>
<p>This acts as a current valid dimension entry entry indicator for the last
version: So when a type II attribute changes and a new version is created (to
keep track of the history) the 'Last version' attribute in the previous version
is set to 'False/N' and the new record with the latest version is set to
'True/Y'.</p>
diff --git a/manual/next/pipeline/transforms/formula.html
b/manual/next/pipeline/transforms/formula.html
index 77c81fda8f..7b70457e1d 100644
--- a/manual/next/pipeline/transforms/formula.html
+++ b/manual/next/pipeline/transforms/formula.html
@@ -3305,7 +3305,7 @@ Check the Apache POI docs page for a <a
href="https://poi.apache.org/components/
<h2 id="_notes"><a class="link" href="#_notes">Notes</a></h2>
<div class="sectionbody">
<div class="paragraph">
-<p>The kettle import process is able to migrate the existing Formula steps
contained in kettle processes in the new Hop Formula transform. Even if the
migration will go without any problem, the migration process is unable to enter
in the inner details of the Formula’s transform macros and perform all
the necessary checks to guarantee the correct behavior and the expected correct
results. Therefore, even if the majority of the macros are the same as the
existing macros you can find [...]
+<p>The kettle import process is able to migrate the existing Formula steps
contained in kettle processes in the new Hop Formula transform. Even if the
migration will go without any problem, the migration process is unable to enter
in the inner details of the Formula’s transform macros and perform all
the necessary checks to guarantee the correct behavior and the expected correct
results. Therefore, even if the majority of the macros are the same as the
existing macros you can find [...]
</div>
</div>
</div>
@@ -12645,7 +12645,7 @@ n!! = n*(n-2)*(n-4)…​(3)(1)</p></td>
<tr>
<td class="tableblock halign-left valign-top"><p
class="tableblock"><strong>constraints</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The
FormatCode is a sequence of characters with an application-defined meaning.
-Portable Contraints: The result of this function may change across locales. If
separators such as decimal or group separator are involved, conversion may give
unexpected results if the separators don’t match that of the current
locale. Across applications the result may change to the extend to which number
format codes and their subtleties are supported. Portable documents should not
use this function.</p></td>
+Portable Constraints: The result of this function may change across locales.
If separators such as decimal or group separator are involved, conversion may
give unexpected results if the separators don’t match that of the current
locale. Across applications the result may change to the extend to which number
format codes and their subtleties are supported. Portable documents should not
use this function.</p></td>
</tr>
</tbody>
</table>
diff --git a/manual/next/pipeline/transforms/getfilenames.html
b/manual/next/pipeline/transforms/getfilenames.html
index 54918559d1..1c0f19d940 100644
--- a/manual/next/pipeline/transforms/getfilenames.html
+++ b/manual/next/pipeline/transforms/getfilenames.html
@@ -3220,7 +3220,7 @@ The limit parameter will act on the total number of rows
returned and not only o
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p
class="tableblock">Limit</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">The number
of files and folders inluded in the output.</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">The number
of files and folders included in the output.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Add
filename to result?</p></td>
diff --git a/manual/next/pipeline/transforms/jdbcmetadata.html
b/manual/next/pipeline/transforms/jdbcmetadata.html
index 13487b2c5f..da2de04704 100644
--- a/manual/next/pipeline/transforms/jdbcmetadata.html
+++ b/manual/next/pipeline/transforms/jdbcmetadata.html
@@ -3075,7 +3075,7 @@
<p>This tab allows you to control how the output metadata is added to the
outputstream.</p>
</div>
<div class="paragraph">
-<p><strong>Output Fields:</strong> This grid is automatically filled with the
appropriate fields as you select a particular metadata method on the Method and
Aruments tab. Use this gridview to remove or rename output fields.</p>
+<p><strong>Output Fields:</strong> This grid is automatically filled with the
appropriate fields as you select a particular metadata method on the Method and
Arguments tab. Use this gridview to remove or rename output fields.</p>
</div>
<div class="paragraph">
<p><strong>Get fields:</strong> use this button to refill the grid. This will
re-add any removed fields, but preserve any renamed fields.</p>
diff --git a/manual/next/pipeline/transforms/jsoninput.html
b/manual/next/pipeline/transforms/jsoninput.html
index d0f6ab8b0a..a916389686 100644
--- a/manual/next/pipeline/transforms/jsoninput.html
+++ b/manual/next/pipeline/transforms/jsoninput.html
@@ -3716,7 +3716,7 @@ Name 4</code></pre>
</div>
</div>
<div class="paragraph">
-<p>You will notice that you only have 3 rowsets returned in this case ( the
null line is omited from the result)</p>
+<p>You will notice that you only have 3 rowsets returned in this case ( the
null line is omitted from the result)</p>
</div>
<div class="paragraph">
<p>To change Hop’s behavior regarding null values in JSON files, so that
null values will not be considered in JSON output, you change the
<code>HOP_JSON_INCLUDE_NULLS</code> configuration variable and set it’s
value to N</p>
diff --git a/manual/next/pipeline/transforms/neo4j-import.html
b/manual/next/pipeline/transforms/neo4j-import.html
index 9a83c5b0de..62b47fde4e 100644
--- a/manual/next/pipeline/transforms/neo4j-import.html
+++ b/manual/next/pipeline/transforms/neo4j-import.html
@@ -3057,7 +3057,7 @@ when configured appropriately.</p>
<td class="tableblock halign-left valign-top"><p class="tableblock">the field
to get the file name to import from</p></td>
</tr>
<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Fiel type
field</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">File type
field</p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">the field
to get the file type to import from</p></td>
</tr>
diff --git a/manual/next/pipeline/transforms/redshift-bulkloader.html
b/manual/next/pipeline/transforms/redshift-bulkloader.html
index 972e3bbd43..5441cf04c3 100644
--- a/manual/next/pipeline/transforms/redshift-bulkloader.html
+++ b/manual/next/pipeline/transforms/redshift-bulkloader.html
@@ -3141,7 +3141,7 @@ The Redshift Bulk Loader is linked to the database type.
It will fetch the JDBC
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Load from
existing file</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">do not
stream the contents of the current pipeline, but perform the <code>COPY</code>
load from a pre-existing file in S3. Suppoorted formats are <code>CSV</code>
(comma delimited) and <code>Parquet</code>.</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">do not
stream the contents of the current pipeline, but perform the <code>COPY</code>
load from a pre-existing file in S3. Supported formats are <code>CSV</code>
(comma delimited) and <code>Parquet</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Copy into
Redshift from existing file</p></td>
diff --git a/manual/next/pipeline/transforms/replacestring.html
b/manual/next/pipeline/transforms/replacestring.html
index 6d49f6fd2a..5d8a6867be 100644
--- a/manual/next/pipeline/transforms/replacestring.html
+++ b/manual/next/pipeline/transforms/replacestring.html
@@ -3050,7 +3050,7 @@
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Replace
with field</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Replace
the search pattern with the vlaue from the field you specify.</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Replace
the search pattern with the value from the field you specify.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whole
word</p></td>
diff --git a/manual/next/pipeline/transforms/rowdenormaliser.html
b/manual/next/pipeline/transforms/rowdenormaliser.html
index fbd2669665..3da1f6dd62 100644
--- a/manual/next/pipeline/transforms/rowdenormaliser.html
+++ b/manual/next/pipeline/transforms/rowdenormaliser.html
@@ -3078,7 +3078,7 @@ All fields can be injected, the values used for the
aggregation field are the fo
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p
class="tableblock">TYPE_AGGR_CONCAT_COMMA</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Aggragate
values separated by comma</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Aggregate
values separated by comma</p></td>
</tr>
</tbody>
</table>
diff --git a/manual/next/pipeline/transforms/tablecompare.html
b/manual/next/pipeline/transforms/tablecompare.html
index 6c1b00ef65..d549fbd79f 100644
--- a/manual/next/pipeline/transforms/tablecompare.html
+++ b/manual/next/pipeline/transforms/tablecompare.html
@@ -3048,7 +3048,7 @@ This means that you could compare two tables with a
different name, as long as t
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Reference
CTE field / Compare CTE field</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specify
the Common Table Expression to use with the table queries to compare values.
-This CTE clause will be pre-pended to the generated SQL.
+This CTE clause will be prepended to the generated SQL.
You can specify multiple CTEs but the one being read from must be called
<code>_<table name></code>
(with an underscore in front of the table name in other words).</p></td>
</tr>
diff --git a/manual/next/pipeline/transforms/textfileinput.html
b/manual/next/pipeline/transforms/textfileinput.html
index ad78f0c6fd..8093bedf8f 100644
--- a/manual/next/pipeline/transforms/textfileinput.html
+++ b/manual/next/pipeline/transforms/textfileinput.html
@@ -3170,6 +3170,9 @@ This behaves like a join option.</p></td>
</tr>
</tbody>
</table>
+<div class="paragraph">
+<p>Running this transform in several copies with filenames from a previous
transform spreads the filenames over the copies, and each copy reads the files
it receives. With <strong>Running in parallel?</strong> enabled on the
Performance tab, the copies also share the large files in chunks. See <a
href="#running-in-parallel">Running in parallel</a>.</p>
+</div>
</div>
<div class="sect2">
<h3 id="_content_tab"><a class="link" href="#_content_tab">Content Tab</a></h3>
@@ -3307,7 +3310,7 @@ To use Unicode, specify UTF-8 or UTF-16. On first use,
Hop searches your system
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p
class="tableblock">Limit</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Sets the
number of lines that are read from the file; 0 means read all lines.</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Sets the
number of lines that are read from the file; 0 means read all lines. When
running in parallel, every copy counts its own lines.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Be lenient
when parsing dates?</p></td>
@@ -3439,7 +3442,7 @@ The table below describes the available options for
defining filters:</p>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Filter
string</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">The string
to search for in the data values. Each data value is treated as aline of text
regardless of how long it is.</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">The string
to search for in the data values. Each data value is treated as a line of text
regardless of how long it is.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Filter
position</p></td>
@@ -3813,11 +3816,8 @@ See <a href="../formatting-values.html#string-to-date"
class="xref page">String
</div>
</div>
</div>
-</div>
-</div>
-<div class="sect1">
-<h2 id="_additional_output_fields_tab"><a class="link"
href="#_additional_output_fields_tab">Additional Output Fields Tab</a></h2>
-<div class="sectionbody">
+<div class="sect2">
+<h3 id="_additional_output_fields_tab"><a class="link"
href="#_additional_output_fields_tab">Additional Output Fields Tab</a></h3>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
@@ -3861,6 +3861,335 @@ See <a href="../formatting-values.html#string-to-date"
class="xref page">String
</tbody>
</table>
</div>
+<div class="sect2">
+<h3 id="_performance_tab"><a class="link" href="#_performance_tab">Performance
Tab</a></h3>
+<div class="paragraph">
+<p>The performance tab sets how the copies of this transform share the work,
and how files on remote storage are downloaded.</p>
+</div>
+<div class="admonitionblock note">
+<table>
+<tr>
+<td class="icon">
+<i class="fa icon-note" title="Note"></i>
+</td>
+<td class="content">
+<div class="paragraph">
+<p>Normally, each copy of this transform does all of its work on a single
thread.
+These settings can start additional threads:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p><strong>Read-ahead blocks</strong> above 0: blocks of the file are
downloaded on background threads while the rows are being read. See <a
href="#read-ahead">Reading from remote storage</a>.</p>
+</li>
+<li>
+<p><strong>Running in parallel?</strong> combined with <strong>Allow breaks in
enclosed fields?</strong>: before any rows are read, the large files are
scanned using one extra thread per copy.</p>
+</li>
+</ul>
+</div>
+</td>
+</tr>
+</table>
+</div>
+<table class="tableblock frame-all grid-all stretch">
+<colgroup>
+<col style="width: 50%;">
+<col style="width: 50%;">
+</colgroup>
+<thead>
+<tr>
+<th class="tableblock halign-left valign-top">Option</th>
+<th class="tableblock halign-left valign-top">Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Running in
parallel?</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Share the
files between the copies of this transform instead of every copy reading every
file. Large files are cut into chunks and each copy reads the next chunk not
yet taken. See <a href="#running-in-parallel">Running in parallel</a>.</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Chunk
size</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">The
largest piece of a file one copy reads when running in parallel, for example
64MB or 1GB. Leave it empty for 128MB. With little data the pieces are smaller,
so that every copy gets work.</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Read-ahead
blocks</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">For files
on remote storage such as S3, Azure or Google Cloud Storage: how many blocks of
a file are downloaded at the same time, each over its own connection, while its
rows are read. The block size comes from the storage connection. 0, the
default, reads on the copy’s own thread without fetching ahead. See <a
href="#read-ahead">Reading from remote storage</a>.</p></td>
+</tr>
+</tbody>
+</table>
+<div class="sect3">
+<h4 id="running-in-parallel"><a class="link"
href="#running-in-parallel">Running in parallel</a></h4>
+<div class="paragraph">
+<p>Running this transform in several copies with <strong>Running in
parallel?</strong> enabled spreads the work over the copies: every line is read
by exactly one copy.
+Set the number of copies on the transform in the pipeline (right-click the
transform, <strong>Number of copies</strong>).
+With one copy, <strong>Running in parallel?</strong> has no effect.</p>
+</div>
+<div class="sect4">
+<h5 id="_how_the_work_is_shared"><a class="link"
href="#_how_the_work_is_shared">How the work is shared</a></h5>
+<div class="paragraph">
+<p>Before the copies read anything, the files are divided into pieces called
chunks: a small file is one chunk, a large file is cut into several.
+The copies share one list of chunks. Each copy takes the next chunk nobody has
taken yet, reads it, and takes the next, until none are left.
+A fast copy therefore reads more chunks than a slow one, and a copy can read
pieces of several files.</p>
+</div>
+<div class="paragraph">
+<p>A line belongs to the chunk that holds its first byte. The copy reading
that chunk reads the whole line, even when it runs past the end of the chunk,
and the copy reading the next chunk skips it.
+That way no line is lost or read twice where a file is cut.</p>
+</div>
+</div>
+<div class="sect4">
+<h5 id="_when_a_file_is_cut_into_chunks"><a class="link"
href="#_when_a_file_is_cut_into_chunks">When a file is cut into chunks</a></h5>
+<div class="paragraph">
+<p>All files are compared to one <strong>split size</strong>, worked out from
the total size of all files to read:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>split size = smallest of ( chunk size , largest of ( 4 MB , total size /
number of copies ) )</pre>
+</div>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>A file no larger than the split size is read whole, by one copy.</p>
+</li>
+<li>
+<p>A larger file is cut into chunks of the split size.</p>
+</li>
+<li>
+<p>The chunk size is 128 MB when left empty.</p>
+</li>
+</ul>
+</div>
+<div class="paragraph">
+<p>So a file is never cut just because there are several copies:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>With the default chunk size, files of 4 MB or less are always read whole:
cutting them would cost more than it saves.</p>
+</li>
+<li>
+<p>When there is enough data to give every copy a full share without cutting,
no file is cut either. Many medium-sized files are simply divided over the
copies.</p>
+</li>
+<li>
+<p>With little data the split size is smaller than the chunk size, so that
every copy still gets work.</p>
+</li>
+<li>
+<p>A chunk size below 4 MB, such as <code>1KB</code>, is used as it is,
without the 4 MB minimum. That is mostly useful for testing.</p>
+</li>
+<li>
+<p>On storage that reads in blocks of its own, a chunk is never smaller than
that read block size, such as the <strong>Read block size</strong> of an Azure
or S3 connection: a smaller chunk would still download whole blocks. A chunk
size set below it is not an error: the transform logs a warning and uses the
block size.</p>
+</li>
+</ul>
+</div>
+<div class="paragraph">
+<p>Some examples, with the default chunk size of 128 MB:</p>
+</div>
+<table class="tableblock frame-all grid-all stretch">
+<colgroup>
+<col style="width: 25%;">
+<col style="width: 25%;">
+<col style="width: 25%;">
+<col style="width: 25%;">
+</colgroup>
+<thead>
+<tr>
+<th class="tableblock halign-left valign-top">Files</th>
+<th class="tableblock halign-left valign-top">Copies</th>
+<th class="tableblock halign-left valign-top">Split size</th>
+<th class="tableblock halign-left valign-top">Result</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">1 file of
3 MB</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">4</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">4
MB</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">The file
is read whole by one copy.</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">1 file of
10 MB</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">4</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">4
MB</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">3 chunks:
4 MB, 4 MB and 2 MB.</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">10 files
of 10 MB</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">4</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">25
MB</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">No file is
cut; the copies share the 10 files.</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">1 file of
1 GB</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">8</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">128
MB</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">8 chunks
of 128 MB.</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">1 file of
1 GB and 100 files of 1 MB</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">4</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">128
MB</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">The large
file is cut into 8 chunks; the small files are read whole.</p></td>
+</tr>
+</tbody>
+</table>
+<div class="paragraph">
+<p>Even when it is larger than the split size, a file is read whole by one
copy, with a line in the log saying why, when:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>its storage can’t start reading in the middle of a file. Local files,
S3, Google Cloud Storage, Azure, SFTP, SMB and HTTP can be read in chunks.</p>
+</li>
+<li>
+<p>its encoding has no line feed that can be found by its bytes, such as
UTF-32 or EBCDIC.</p>
+</li>
+<li>
+<p>its header lines take up more than the first 1 MB of the file.</p>
+</li>
+</ul>
+</div>
+</div>
+<div class="sect4">
+<h5 id="read-ahead"><a class="link" href="#read-ahead">Reading from remote
storage</a></h5>
+<div class="paragraph">
+<p>On object storage such as S3, Azure and Google Cloud Storage one connection
is often limited to a few MB/s, however fast the network is.
+A plain read also downloads the next part of a file only once the rows before
it are processed, so the network waits for the processing and the other way
around.</p>
+</div>
+<div class="paragraph">
+<p>With <strong>Read-ahead blocks</strong> set to 1 or more, this transform
downloads a file from remote storage in blocks, several at a time, each over
its own connection, while it processes the rows of the blocks already there.
+<strong>Read-ahead blocks</strong> sets how many blocks are downloaded at the
same time for each file or chunk being read.
+It is 0 by default: every copy of this transform then does all its work,
reading included, on its own thread, as it always did.
+With 1 or more blocks the downloads run on extra threads besides the
copy’s: up to that many per file or chunk being read, so copies × blocks
in total. Try 4 for a large file on remote storage.
+The size of a block belongs to the storage, not to this transform: it is the
<strong>Read block size</strong> of the <a
href="../../metadata-types/azure-authentication.html" class="xref page">Azure
connection</a>, the <a href="../../vfs/aws-s3-vfs.html#_read_block_size"
class="xref page">S3 connection</a> or the <a
href="../../vfs/google-cloud-storage-vfs.html#_read_block_size" class="xref
page">Google Cloud Storage options</a>, and 8 MB for other storage.
+Where the storage supports it, every block is downloaded with one request for
exactly its bytes.</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>This works for every file system that can start reading in the middle of a
file, also without <strong>Running in parallel?</strong>. Local files are
always read directly: the operating system reads ahead itself.</p>
+</li>
+<li>
+<p>Running in parallel, a copy fetches ahead up to the end of its chunk only.
The rest of the last line, which runs past the end, is fetched when it is
reached.</p>
+</li>
+<li>
+<p>With 10 copies and 4 read-ahead blocks, up to 40 downloads run at the same
time.</p>
+</li>
+<li>
+<p>Every copy holds up to <strong>Read-ahead blocks</strong> × the block size
in memory: 32 MB with the defaults, so 320 MB for 10 copies. Lower the number
of blocks, or the read block size of the connection, when memory is short.</p>
+</li>
+<li>
+<p>More blocks help as long as the network has room for more connections. When
the processing can’t keep up any more, more copies help instead.</p>
+</li>
+<li>
+<p>A block whose download fails is downloaded again, up to 3 times in all,
after waiting 1 and then 2 seconds; every retry is logged. A block that still
fails stops the read with an error, and the transform fails: no rows are made
from the part of a line read before the error, and the file is never read on
past the failed block.</p>
+</li>
+</ul>
+</div>
+</div>
+<div class="sect4">
+<h5 id="_headers_rows_and_limits"><a class="link"
href="#_headers_rows_and_limits">Headers, rows and limits</a></h5>
+<div class="ulist">
+<ul>
+<li>
+<p>The header lines are skipped once per file, by the copy that reads its
start. Chunks further on in the file start after the header.</p>
+</li>
+<li>
+<p>The rows come out in no particular order. If the order matters, sort the
rows afterwards or run one copy.</p>
+</li>
+<li>
+<p>The row number and the <strong>Limit</strong> count per copy.</p>
+</li>
+<li>
+<p>A file is added to the result, and reported to lineage, once, not once per
chunk.</p>
+</li>
+<li>
+<p>With <strong>Allow breaks in enclosed fields?</strong> enabled, a line feed
inside an enclosed field doesn’t start a new row, so a chunk can’t
simply start after any line feed. The large files are then first read once
more, on one extra thread per copy while the copies wait, to find where each
chunk’s first row starts, and the chunks are cut exactly there. This
works in UTF-8, ISO-8859, windows-125x and UTF-16, with an enclosure and escape
character in plain ASCII; o [...]
+</li>
+</ul>
+</div>
+</div>
+<div class="sect4">
+<h5 id="_filenames_from_a_previous_transform"><a class="link"
href="#_filenames_from_a_previous_transform">Filenames from a previous
transform</a></h5>
+<div class="paragraph">
+<p>With <strong>Accept filenames from previous transforms</strong> enabled,
the previous transform spreads its filename rows over the copies, so each copy
receives only part of the filenames.
+With <strong>Running in parallel?</strong> enabled the copies first collect
all filenames, from all copies, and then divide all these files into chunks
together, following the rules above. The files are read the same way as files
from the file list.</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>Every copy first reads all its filename rows, as it does without parallel
reading, so collecting them doesn’t delay reading.</p>
+</li>
+<li>
+<p>The previous transform must have finished sending filenames before the
copies start reading.</p>
+</li>
+<li>
+<p>With <strong>Pass through fields from previous transform</strong> enabled,
every row read from a file carries the fields of the row that named that file.
This is also true for a chunk read by another copy, and for a copy that
received no filename rows itself.</p>
+</li>
+<li>
+<p>A file named in two rows is read twice, once for each row, as it is without
parallel reading.</p>
+</li>
+<li>
+<p>If the previous transform copies its rows to every copy instead of
distributing them, every copy receives every filename. Each file is then still
read once, not once per copy.</p>
+</li>
+<li>
+<p>A file that doesn’t exist is handled by the copy that tries to open
it, following the error handling settings, as without parallel reading.</p>
+</li>
+</ul>
+</div>
+</div>
+<div class="sect4">
+<h5 id="_settings_that_cant_be_combined"><a class="link"
href="#_settings_that_cant_be_combined">Settings that can’t be
combined</a></h5>
+<div class="paragraph">
+<p>Reading in parallel can’t be combined with settings that need each
file read from its first line by one copy:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>wrapped lines</p>
+</li>
+<li>
+<p>paged layout</p>
+</li>
+<li>
+<p>a footer</p>
+</li>
+<li>
+<p>row numbers by file</p>
+</li>
+<li>
+<p>compression</p>
+</li>
+<li>
+<p><strong>Stop on filter</strong></p>
+</li>
+<li>
+<p>repeated fields</p>
+</li>
+</ul>
+</div>
+<div class="paragraph">
+<p>The dialog doesn’t let you switch on both. A pipeline that combines
them anyway, for example through metadata injection, doesn’t start.</p>
+</div>
+</div>
+<div class="sect4">
+<h5 id="_several_copies_without_running_in_parallel"><a class="link"
href="#_several_copies_without_running_in_parallel">Several copies without
running in parallel</a></h5>
+<div class="paragraph">
+<p>Without <strong>Running in parallel?</strong> every copy reads all the
files of its own file list, whole. Give each copy its own files, for example
with <code>${Internal.Transform.CopyNr}</code> in the filename: copy 0 reads
<code>part_0.csv</code>, copy 1 reads <code>part_1.csv</code>, and so on.</p>
+</div>
+<div class="paragraph">
+<p>A file in the list of more than one copy would be read by each of them, and
its rows output more than once. The transform doesn’t start reading then,
and the error names the file and the copies.</p>
+</div>
+<div class="paragraph">
+<p>Filenames from a previous transform are already divided over the copies, so
each copy reads the files it receives, whole.
+Turn on <strong>Running in parallel?</strong> to also cut large files into
chunks.</p>
+</div>
+<div class="paragraph">
+<p>With <strong>Running in parallel?</strong> the files of all copies are
planned together: a file that several copies list is read once, and a filename
that differs per copy, such as <code>${Internal.Transform.CopyNr}</code>, still
gets every file read.</p>
+</div>
+</div>
+</div>
+</div>
+</div>
</div>
<div class="sect1">
<h2 id="_buttons"><a class="link" href="#_buttons">Buttons</a></h2>
diff --git a/manual/next/pipeline/transforms/tokenreplacement.html
b/manual/next/pipeline/transforms/tokenreplacement.html
index 0dd90360ba..8f94b04bde 100644
--- a/manual/next/pipeline/transforms/tokenreplacement.html
+++ b/manual/next/pipeline/transforms/tokenreplacement.html
@@ -3176,7 +3176,7 @@ If not checked, Apache Hop will overwrite the file if it
exists.</p>
<div class="ulist">
<ul>
<li>
-<p>Should the curren date be included in the output filename?</p>
+<p>Should the current date be included in the output filename?</p>
</li>
</ul>
</div>
diff --git a/manual/next/pipeline/transforms/userdefinedjavaexpression.html
b/manual/next/pipeline/transforms/userdefinedjavaexpression.html
index b38b839e46..fffb83f163 100644
--- a/manual/next/pipeline/transforms/userdefinedjavaexpression.html
+++ b/manual/next/pipeline/transforms/userdefinedjavaexpression.html
@@ -3078,7 +3078,7 @@
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Transform
name</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Specify a
name for this transform. This must be unique wihin the pipeline.</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Specify a
name for this transform. This must be unique within the pipeline.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">New
field</p></td>
diff --git a/manual/next/pipeline/transforms/xmljoin.html
b/manual/next/pipeline/transforms/xmljoin.html
index 73809368dd..4e99d47804 100644
--- a/manual/next/pipeline/transforms/xmljoin.html
+++ b/manual/next/pipeline/transforms/xmljoin.html
@@ -3047,7 +3047,7 @@ When the complex join is enabled, a single ? character is
used as a placeholder.
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Complex
Join</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Flag to
enable the complex join syntax, using the placeholder in the XPath
Statment</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Flag to
enable the complex join syntax, using the placeholder in the XPath
Statement</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Join
Comparison Field</p></td>
@@ -3068,7 +3068,7 @@ The character encoding of the target XML is preserved
regardless of whether the
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Omit null
values from XML</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Whether
null values are removed fro the output. If this option is not selected, null
values are included in the XML output as an empty element, such as <abc/>.
+<td class="tableblock halign-left valign-top"><p class="tableblock">Whether
null values are removed from the output. If this option is not selected, null
values are included in the XML output as an empty element, such as <abc/>.
When this option is selected, the null values are completely omitted from the
output and no empty elements exist.
This is useful to save space in the output file (for high volume transactions)
or to use the empty tags for special conditions in the output.</p></td>
</tr>
diff --git a/manual/next/pipeline/transforms/yamlinput.html
b/manual/next/pipeline/transforms/yamlinput.html
index ab8d5d0f57..7d27a62791 100644
--- a/manual/next/pipeline/transforms/yamlinput.html
+++ b/manual/next/pipeline/transforms/yamlinput.html
@@ -3058,7 +3058,7 @@
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p
class="tableblock">Edit</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Moved the
selected line for editting.</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Moved the
selected line for editing.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Show
filename(s)</p></td>
diff --git a/manual/next/vfs/aws-s3-vfs.html b/manual/next/vfs/aws-s3-vfs.html
index 5c460af81a..41d42e9130 100644
--- a/manual/next/vfs/aws-s3-vfs.html
+++ b/manual/next/vfs/aws-s3-vfs.html
@@ -3052,6 +3052,30 @@ You can use the Tools/Edit config variables menu in Hop
GUI or you can use the h
</div>
</div>
<div class="sect1">
+<h2 id="_read_block_size"><a class="link" href="#_read_block_size">Read block
size</a></h2>
+<div class="sectionbody">
+<div class="paragraph">
+<p>Transforms that read a file in blocks, such as Text File Input fetching
ahead, download it in blocks of the <strong>Read block size</strong> of the S3
connection, 8MB when it’s left empty.
+Every block is downloaded with one request for exactly its bytes, several at a
time, so one file uses several connections.
+A request read to its end hands its connection back to the client, so the
connections are reused.</p>
+</div>
+<div class="paragraph">
+<p>Larger blocks make fewer requests per file and take more memory; smaller
blocks start the first rows sooner.</p>
+</div>
+</div>
+</div>
+<div class="sect1">
+<h2 id="_maximum_connections"><a class="link"
href="#_maximum_connections">Maximum connections</a></h2>
+<div class="sectionbody">
+<div class="paragraph">
+<p>The <strong>Maximum connections</strong> of an S3 connection is how many
HTTP connections to S3 can be open at the same time, 50 when it’s left
empty.
+One connection to S3 is often limited to a few MB/s, so reading many blocks at
the same time is how a large file is read fast, and every block in flight needs
a connection of its own.
+Text File Input running in parallel keeps up to copies × read-ahead blocks
requests in flight: 20 copies with 4 blocks each need 80 connections.
+With fewer connections allowed than that, the other requests wait for a free
one.</p>
+</div>
+</div>
+</div>
+<div class="sect1">
<h2 id="_usage_and_testing"><a class="link" href="#_usage_and_testing">Usage
and testing</a></h2>
<div class="sectionbody">
<div class="paragraph">
diff --git a/manual/next/vfs/google-cloud-storage-vfs.html
b/manual/next/vfs/google-cloud-storage-vfs.html
index af766a47d3..91548d77fc 100644
--- a/manual/next/vfs/google-cloud-storage-vfs.html
+++ b/manual/next/vfs/google-cloud-storage-vfs.html
@@ -3078,11 +3078,25 @@ The first use of <code>gs://</code> fails instead, with
an error that says why,
<td class="tableblock halign-left valign-top"><p class="tableblock">How long
to cache folder listing results (in seconds, default 5). If not set, falls back
to 10 seconds. Accepts variables.</p></td>
<td class="tableblock halign-left valign-top"><p
class="tableblock"><code>5</code></p></td>
</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Read block
size</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">How many
bytes one download request fetches when a file is read, like 8MB. Leave empty
for 8MB. Transforms that fetch ahead, such as Text File Input, download blocks
of this size, several at a time, each with one request. Accepts
variables.</p></td>
+<td class="tableblock halign-left valign-top"></td>
+</tr>
</tbody>
</table>
</div>
</div>
<div class="sect1">
+<h2 id="_read_block_size"><a class="link" href="#_read_block_size">Read block
size</a></h2>
+<div class="sectionbody">
+<div class="paragraph">
+<p>Files are downloaded in pieces of the <strong>Read block size</strong>
option, 8MB when it’s left empty, instead of the 2MB the Google Cloud
client uses on its own.
+Transforms that read a file in blocks, such as Text File Input fetching ahead,
download several blocks at a time, each with one request for exactly its
bytes.</p>
+</div>
+</div>
+</div>
+<div class="sect1">
<h2 id="_retries_timeouts_and_stalled_transfers"><a class="link"
href="#_retries_timeouts_and_stalled_transfers">Retries, timeouts and stalled
transfers</a></h2>
<div class="sectionbody">
<div class="paragraph">
diff --git a/manual/next/workflow/actions/cassandra-exec-cql.html
b/manual/next/workflow/actions/cassandra-exec-cql.html
index 1d418df44e..8dbed17d85 100644
--- a/manual/next/workflow/actions/cassandra-exec-cql.html
+++ b/manual/next/workflow/actions/cassandra-exec-cql.html
@@ -2944,7 +2944,7 @@
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
-<p>The <code>Cassandra Exec CQL</code> action executes one or more Cassandra
Quqery Languague (SQL) statements on a Cassandra cluster.</p>
+<p>The <code>Cassandra Exec CQL</code> action executes one or more Cassandra
Query Language (SQL) statements on a Cassandra cluster.</p>
</div>
<div class="paragraph">
<p>You can use it to create or drop key spaces, tables, indexes and so on.</p>
diff --git a/manual/next/workflow/actions/ftpdelete.html
b/manual/next/workflow/actions/ftpdelete.html
index b598d3d1b1..0d4841ae75 100644
--- a/manual/next/workflow/actions/ftpdelete.html
+++ b/manual/next/workflow/actions/ftpdelete.html
@@ -3030,7 +3030,7 @@ Existing workflows keep working and log a warning; new
ones should use Delete fi
<p>Authentication SSL</p>
</li>
<li>
-<p>Implicit SSL with crypted FTP connection</p>
+<p>Implicit SSL with encrypted FTP connection</p>
</li>
<li>
<p>Authentication TLS</p>
@@ -3039,7 +3039,7 @@ Existing workflows keep working and log a warning; new
ones should use Delete fi
<p>Implicit TLS</p>
</li>
<li>
-<p>Implicit TLS with crypted FTP connection</p>
+<p>Implicit TLS with encrypted FTP connection</p>
</li>
</ul>
</div></div></td>
diff --git a/manual/next/workflow/actions/ftpput.html
b/manual/next/workflow/actions/ftpput.html
index aa2cf4d660..0d5931cb3e 100644
--- a/manual/next/workflow/actions/ftpput.html
+++ b/manual/next/workflow/actions/ftpput.html
@@ -3069,7 +3069,7 @@ Select encoding that is valid for your server.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Test
folder</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Test if
the remote directory exists. This connects to the FTP server and attmepts to
access the specfied directory. If this directory doesn’t exist, the
upload fails.</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Test if
the remote directory exists. This connects to the FTP server and attempts to
access the specified directory. If this directory doesn’t exist, the
upload fails.</p></td>
</tr>
</tbody>
</table>
diff --git a/manual/next/workflow/actions/mail.html
b/manual/next/workflow/actions/mail.html
index 808a13013b..f4e1db336a 100644
--- a/manual/next/workflow/actions/mail.html
+++ b/manual/next/workflow/actions/mail.html
@@ -3201,7 +3201,7 @@ Every file in this list is marked with a file type and
you can select what type
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Zip files
to single archive?</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Enable to
have attachments achived in a zip file</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Enable to
have attachments archived in a zip file</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Name of
the zip archive</p></td>
diff --git a/manual/next/workflow/actions/xslt.html
b/manual/next/workflow/actions/xslt.html
index 3de4e11e8e..376aca5b96 100644
--- a/manual/next/workflow/actions/xslt.html
+++ b/manual/next/workflow/actions/xslt.html
@@ -2970,7 +2970,7 @@
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Get
filenames from previous result</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">When this
option is checked, the XML File name, XSL File name and Output File name are
read from the previous result set.
-When multilpe rows exists, it will loop.
+When multiple rows exists, it will loop.
At this time XML File name is the first field in the result set, XSL File name
the second fields and Output File name the third field.</p></td>
</tr>
<tr>
![https://hop.apache.org/img/social-card.png"](https://hop.apache.org/img/social-card.png"){kind=link}