Re: Spark 2.2 Structured Streaming + Kinesis

[Gourav Sengupta]

Hi,

is the structured streaming kinesis connector available as an open source
package?



Thanks and Regards,
Gourav Sengupta

On Mon, Nov 13, 2017 at 11:30 PM Benjamin Kim  wrote:

> To add, we have a CDH 5.12 cluster with Spark 2.2 in our data center.
>
> On Mon, Nov 13, 2017 at 3:15 PM Benjamin Kim  wrote:
>
>> Does anyone know if there is a connector for AWS Kinesis that can be used
>> as a source for Structured Streaming?
>>
>> Thanks.
>>
>>

Re: Spark 2.2 Structured Streaming + Kinesis

[Benjamin Kim]

To add, we have a CDH 5.12 cluster with Spark 2.2 in our data center.

On Mon, Nov 13, 2017 at 3:15 PM Benjamin Kim  wrote:

> Does anyone know if there is a connector for AWS Kinesis that can be used
> as a source for Structured Streaming?
>
> Thanks.
>
>

Re: Spark 2.2 Structured Streaming + Kinesis

[Jules Damji]

You can use the Databricks to connect to Kinesis: 

https://databricks.com/blog/2017/08/09/apache-sparks-structured-streaming-with-amazon-kinesis-on-databricks.html

Cheers 
Jules 

Sent from my iPhone
Pardon the dumb thumb typos :)

> On Nov 13, 2017, at 3:15 PM, Benjamin Kim  wrote:
> 
> Does anyone know if there is a connector for AWS Kinesis that can be used as 
> a source for Structured Streaming?
> 
> Thanks.
> 

Spark 2.2 Structured Streaming + Kinesis

[Benjamin Kim]

Does anyone know if there is a connector for AWS Kinesis that can be used
as a source for Structured Streaming?

Thanks.