[ https://issues.apache.org/jira/browse/SPARK-27950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-27950: ------------------------------------ Assignee: (was: Apache Spark) > Additional DynamoDB and CloudWatch config > ----------------------------------------- > > Key: SPARK-27950 > URL: https://issues.apache.org/jira/browse/SPARK-27950 > Project: Spark > Issue Type: Improvement > Components: Structured Streaming > Affects Versions: 2.4.3 > Reporter: Eric S Meisel > Priority: Major > > I was researching getting Spark’s Kinesis integration running locally against > {{localstack}}. We found this issue, and it creates a complication: > [localstack/localstack#677|https://github.com/localstack/localstack/issues/677] > Effectively, we need to be able to redirect calls for Kinesis, DynamoDB and > Cloudwatch in order for the KCL to properly use the {{localstack}} > infrastructure. We have successfully done this with the KCL (both 1.x and > 2.x), but with Spark’s integration we are unable to configure DynamoDB and > Cloudwatch’s endpoints: > [https://github.com/apache/spark/blob/master/external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala#L162] -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org