This is an automated email from the ASF dual-hosted git repository.
cwylie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 8fd17fe fix a few typos in Kinesis doc (#11776)
8fd17fe is described below
commit 8fd17fe0af7a77d4a5385c56c9c0c5b15c9d923a
Author: Charles Smith <[email protected]>
AuthorDate: Wed Oct 6 19:43:20 2021 -0700
fix a few typos in Kinesis doc (#11776)
---
docs/development/extensions-core/kinesis-ingestion.md | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/docs/development/extensions-core/kinesis-ingestion.md
b/docs/development/extensions-core/kinesis-ingestion.md
index 2dc60ef..cbf2ffe 100644
--- a/docs/development/extensions-core/kinesis-ingestion.md
+++ b/docs/development/extensions-core/kinesis-ingestion.md
@@ -118,7 +118,7 @@ Where the file `supervisor-spec.json` contains a Kinesis
supervisor spec:
|Field|Description|Required|
|--------|-----------|---------|
-|`type`|The supervisor type, this should always be `kinesis`.|yes|
+|`type`|The supervisor type; this should always be `kinesis`.|yes|
|`spec`|Container object for the supervisor configuration.|yes|
|`dataSchema`|The schema that will be used by the Kinesis indexing task during
ingestion. See
[`dataSchema`](../../ingestion/ingestion-spec.md#dataschema).|yes|
|`ioConfig`|A KinesisSupervisorIOConfig object for configuring Kafka
connection and I/O-related settings for the supervisor and indexing task. See
[KinesisSupervisorIOConfig](#kinesissupervisorioconfig) below.|yes|
@@ -130,7 +130,7 @@ Where the file `supervisor-spec.json` contains a Kinesis
supervisor spec:
|Field|Type|Description|Required|
|-----|----|-----------|--------|
|`stream`|String|The Kinesis stream to read.|yes|
-|`inputFormat`|Object|[`inputFormat`](../../ingestion/data-formats.md#input-format)
to specify how to parse input data. See [the below
section](#specifying-data-format) for details about specifying the input
format.|yes|
+|`inputFormat`|Object|[`inputFormat`](../../ingestion/data-formats.md#input-format)
to specify how to parse input data. See [Specifying data
format](#specifying-data-format) for details about specifying the input
format.|yes|
|`endpoint`|String|The AWS Kinesis stream endpoint for a region. You can find
a list of endpoints
[here](http://docs.aws.amazon.com/general/latest/gr/rande.html#ak_region).|no
(default == kinesis.us-east-1.amazonaws.com)|
|`replicas`|Integer|The number of replica sets, where 1 means a single set of
tasks (no replication). Replica tasks will always be assigned to different
workers to provide resiliency against process failure.|no (default == 1)|
|`taskCount`|Integer|The maximum number of *reading* tasks in a *replica set*.
This means that the maximum number of reading tasks will be `taskCount *
replicas` and the total number of tasks (*reading* + *publishing*) will be
higher than this. See [Capacity Planning](#capacity-planning) below for more
details. The number of reading tasks will be less than `taskCount` if
`taskCount > {numKinesisShards}`.|no (default == 1)|
@@ -140,7 +140,7 @@ Where the file `supervisor-spec.json` contains a Kinesis
supervisor spec:
|`useEarliestSequenceNumber`|Boolean|If a supervisor is managing a dataSource
for the first time, it will obtain a set of starting sequence numbers from
Kinesis. This flag determines whether it retrieves the earliest or latest
sequence numbers in Kinesis. Under normal circumstances, subsequent tasks will
start from where the previous segments ended so this flag will only be used on
first run.|no (default == false)|
|`completionTimeout`|ISO8601 Period|The length of time to wait before
declaring a publishing task as failed and terminating it. If this is set too
low, your tasks may never publish. The publishing clock for a task begins
roughly after `taskDuration` elapses.|no (default == PT6H)|
|`lateMessageRejectionPeriod`|ISO8601 Period|Configure tasks to reject
messages with timestamps earlier than this period before the task was created;
for example if this is set to `PT1H` and the supervisor creates a task at
*2016-01-01T12:00Z*, messages with timestamps earlier than *2016-01-01T11:00Z*
will be dropped. This may help prevent concurrency issues if your data stream
has late messages and you have multiple pipelines that need to operate on the
same segments (e.g. a realtime an [...]
-|`earlyMessageRejectionPeriod`|ISO8601 Period|Configure tasks to reject
messages with timestamps later than this period after the task reached its
taskDuration; for example if this is set to `PT1H`, the taskDuration is set to
`PT1H` and the supervisor creates a task at *2016-01-01T12:00Z*, messages with
timestamps later than *2016-01-01T14:00Z* will be dropped. **Note:** Tasks
sometimes run past their task duration, for example, in cases of supervisor
failover. Setting earlyMessageReject [...]
+|`earlyMessageRejectionPeriod`|ISO8601 Period|Configure tasks to reject
messages with timestamps later than this period after the task reached its
taskDuration; for example if this is set to `PT1H`, the taskDuration is set to
`PT1H` and the supervisor creates a task at *2016-01-01T12:00Z*. Messages with
timestamps later than *2016-01-01T14:00Z* will be dropped. **Note:** Tasks
sometimes run past their task duration, for example, in cases of supervisor
failover. Setting `earlyMessageRejec [...]
|`recordsPerFetch`|Integer|The number of records to request per call to fetch
records from Kinesis. See [Determining fetch
settings](#determining-fetch-settings).|no (default == 4000)|
|`fetchDelayMillis`|Integer|Time in milliseconds to wait between subsequent
calls to fetch records from Kinesis. See [Determining fetch
settings](#determining-fetch-settings).|no (default == 0)|
|`awsAssumedRoleArn`|String|The AWS assumed role to use for additional
permissions.|no|
@@ -154,7 +154,7 @@ Where the file `supervisor-spec.json` contains a Kinesis
supervisor spec:
| Property | Description | Required |
| ------------- | ------------- | ------------- |
-| `enableTaskAutoScaler` | Enable or disable the auto scaler. When false or or
absent Druid disables the `autoScaler` even when `autoScalerConfig` is not
null| no (default == false) |
+| `enableTaskAutoScaler` | Enable or disable the auto scaler. When false or
absent, Druid disables the `autoScaler` even when `autoScalerConfig` is not
null.| no (default == false) |
| `taskCountMax` | Maximum number of Kinesis ingestion tasks. Must be greater
than or equal to `taskCountMin`. If greater than `{numKinesisShards}`, the
maximum number of reading tasks is `{numKinesisShards}` and `taskCountMax` is
ignored. | yes |
| `taskCountMin` | Minimum number of Kinesis ingestion tasks. When you enable
the auto scaler, Druid ignores the value of taskCount in `IOConfig` and
uses`taskCountMin` for the initial number of tasks to launch.| yes |
| `minTriggerScaleActionFrequencyMillis` | Minimum time interval between two
scale actions | no (default == 600000) |
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
