nicusX commented on code in PR #179: URL: https://github.com/apache/flink-connector-aws/pull/179#discussion_r1833225006
########## docs/content/docs/connectors/table/kinesis.md: ########## @@ -123,6 +158,430 @@ WITH ( Connector Options ----------------- +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 25%">Option</th> + <th class="text-center" style="width: 8%">Required</th> + <th class="text-center" style="width: 8%">Forwarded</th> + <th class="text-center" style="width: 7%">Default</th> + <th class="text-center" style="width: 10%">Type</th> + <th class="text-center" style="width: 42%">Description</th> + </tr> + <tr> + <th colspan="6" class="text-left" style="width: 100%">Common Options</th> + </tr> + </thead> + <tbody> + <tr> + <td><h5>connector</h5></td> + <td>required</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>Specify what connector to use. For Kinesis use <code>'kinesis'</code>.</td> + </tr> + <tr> + <td><h5>stream.arn</h5></td> + <td>required</td> + <td>yes</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>Name of the Kinesis data stream backing this table.</td> + </tr> + <tr> + <td><h5>format</h5></td> + <td>required</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The format used to deserialize and serialize Kinesis data stream records. See <a href="#data-type-mapping">Data Type Mapping</a> for details.</td> + </tr> + <tr> + <td><h5>aws.region</h5></td> + <td>required</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The AWS region where the stream is defined.</td> + </tr> + <tr> + <td><h5>aws.endpoint</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The AWS endpoint for Kinesis (derived from the AWS region setting if not set).</td> + </tr> + <tr> + <td><h5>aws.trust.all.certificates</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">false</td> + <td>Boolean</td> + <td>If true accepts all SSL certificates.</td> + </tr> + </tbody> + <thead> + <tr> + <th colspan="6" class="text-left" style="width: 100%">Authentication Options</th> + </tr> + </thead> + <tbody> + <tr> + <td><h5>aws.credentials.provider</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">AUTO</td> + <td>String</td> + <td>A credentials provider to use when authenticating against the Kinesis endpoint. See <a href="#authentication">Authentication</a> for details.</td> + </tr> + <tr> + <td><h5>aws.credentials.basic.accesskeyid</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The AWS access key ID to use when setting credentials provider type to BASIC.</td> + </tr> + <tr> + <td><h5>aws.credentials.basic.secretkey</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The AWS secret key to use when setting credentials provider type to BASIC.</td> + </tr> + <tr> + <td><h5>aws.credentials.profile.path</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>Optional configuration for profile path if credential provider type is set to be PROFILE.</td> + </tr> + <tr> + <td><h5>aws.credentials.profile.name</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>Optional configuration for profile name if credential provider type is set to be PROFILE.</td> + </tr> + <tr> + <td><h5>aws.credentials.role.arn</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The role ARN to use when credential provider type is set to ASSUME_ROLE or WEB_IDENTITY_TOKEN.</td> + </tr> + <tr> + <td><h5>aws.credentials.role.sessionName</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The role session name to use when credential provider type is set to ASSUME_ROLE or WEB_IDENTITY_TOKEN.</td> + </tr> + <tr> + <td><h5>aws.credentials.role.externalId</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The external ID to use when credential provider type is set to ASSUME_ROLE.</td> + </tr> + <tr> + <td><h5>aws.credentials.role.stsEndpoint</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The AWS endpoint for STS (derived from the AWS region setting if not set) to use when credential provider type is set to ASSUME_ROLE.</td> + </tr> + <tr> + <td><h5>aws.credentials.role.provider</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The credentials provider that provides credentials for assuming the role when credential provider type is set to ASSUME_ROLE. Roles can be nested, so this value can again be set to ASSUME_ROLE</td> + </tr> + <tr> + <td><h5>aws.credentials.webIdentityToken.file</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The absolute path to the web identity token file that should be used if provider type is set to WEB_IDENTITY_TOKEN.</td> + </tr> + <tr> + <td><h5>aws.credentials.custom.class</h5></td> + <td>required only if credential provider is set to CUSTOM</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The full path (in Java package notation) to the user provided + class to use if credential provider type is set to be CUSTOM e.g. org.user_company.auth.CustomAwsCredentialsProvider.</td> + </tr> + </tbody> + <thead> + <tr> + <th colspan="6" class="text-left" style="width: 100%">Source Options</th> + </tr> + </thead> + <tbody> + <tr> + <td><h5>source.init.position</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">LATEST</td> + <td>String</td> + <td>Initial position to be used when reading from the table. See <a href="#start-reading-position">Start Reading Position</a> for details.</td> + </tr> + <tr> + <td><h5>source.init.timestamp</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>The initial timestamp to start reading Kinesis stream from (when <code>scan.stream.initpos</code> is AT_TIMESTAMP). See <a href="#start-reading-position">Start Reading Position</a> for details.</td> + </tr> + <tr> + <td><h5>source.init.timestamp.format</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">yyyy-MM-dd'T'HH:mm:ss.SSSXXX</td> + <td>String</td> + <td>The date format of initial timestamp to start reading Kinesis stream from (when <code>scan.stream.initpos</code> is AT_TIMESTAMP). See <a href="#start-reading-position">Start Reading Position</a> for details.</td> + </tr> + <tr> + <td><h5>source.shard.discovery.interval</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">10 s</td> + <td>Duration</td> + <td>The interval between each attempt to discover new shards.</td> + </tr> + <tr> + <td><h5>source.reader.type</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">POLLING</td> + <td>String</td> + <td>The <code>ReaderType</code> to use for sources.</td> + </tr> + <tr> + <td><h5>source.shard.get-records.max-record-count</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">10000</td> + <td>Integer</td> + <td>Only applicable to POLLING <code>ReaderType</code>. The maximum number of records to try to get each time we fetch records from a AWS Kinesis shard.</td> + </tr> + <tr> + <td><h5>source.efo.consumer.name</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">(none)</td> + <td>String</td> + <td>Only applicable to EFO <code>ReaderType</code>. The name of the EFO consumer to register with KDS.</td> + </tr> + <tr> + <td><h5>source.efo.lifecycle</h5></td> + <td>optional</td> + <td>no</td> + <td style="word-wrap: break-word;">JOB_MANAGED</td> + <td>String</td> + <td>Only applicable to EFO <code>ReaderType</code>. Determine if the EFO consumer is managed by the Flink job <code>JOB_MANAGED|SELF_MANAGED</code>.</td> Review Comment: Sorry, I copied&pasted the deleted title from the other docs. It was "EFO Stream Consumer Lifecycle Management". But we can skip linking across docs for now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
