[
https://issues.apache.org/jira/browse/SPARK-27950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-27950:
-----------------------------------
Labels: pull-request-available (was: )
> Additional DynamoDB and CloudWatch config
> -----------------------------------------
>
> Key: SPARK-27950
> URL: https://issues.apache.org/jira/browse/SPARK-27950
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Affects Versions: 3.1.0
> Reporter: Eric S Meisel
> Priority: Minor
> Labels: pull-request-available
>
> I was researching getting Spark’s Kinesis integration running locally against
> {{localstack}}. We found this issue, and it creates a complication:
> [localstack/localstack#677|https://github.com/localstack/localstack/issues/677]
> Effectively, we need to be able to redirect calls for Kinesis, DynamoDB and
> Cloudwatch in order for the KCL to properly use the {{localstack}}
> infrastructure. We have successfully done this with the KCL (both 1.x and
> 2.x), but with Spark’s integration we are unable to configure DynamoDB and
> Cloudwatch’s endpoints:
> [https://github.com/apache/spark/blob/master/external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala#L162]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
