Re: spark streaming and kinesis integration

Use case is , we want to read/write to kinesis streams using k8s Officially I could not find the connector or reader for kinesis from spark like it has for kafka. Checking here if anyone used kinesis and spark streaming combination ? On Thu, 6 Apr, 2023, 7:23 pm Mich Talebzadeh, wrote: > Hi

RE: spark streaming and kinesis integration

unsubscribe Regards, Kurt Jonske Senior Director Alvarez & Marsal Direct: 212 328 8532 Mobile: 312 560 5040 Email: kjon...@alvarezandmarsal.com www.alvarezandmarsal.com From: Mich Talebzadeh Sent: Thursday, April 06, 2023 11:45 AM To: Rajesh Katkar Cc:

Re: spark streaming and kinesis integration

Do you have a high level diagram of the proposed solution? In so far as I know k8s does not support spark structured streaming? Mich Talebzadeh, Lead Solutions Architect/Engineering Lead Palantir Technologies London United Kingdom view my Linkedin profile

Re: spark streaming and kinesis integration

Hi Rajesh, What is the use case for Kinesis here? I have not used it personally, Which use case it concerns https://aws.amazon.com/kinesis/ Can you use something else instead? HTH Mich Talebzadeh, Lead Solutions Architect/Engineering Lead Palantir Technologies London United Kingdom view

spark streaming and kinesis integration

Hi Spark Team, We need to read/write the kinesis streams using spark streaming. We checked the official documentation - https://spark.apache.org/docs/latest/streaming-kinesis-integration.html It does not mention kinesis connector. Alternative is - https://github.com/qubole/kinesis-sql which is