Re: [EXTERNAL] Re: Spark streaming

Thanks, I'll try it out. On Fri, 19 Aug 2022, 6:12 pm Saurabh Gulati, wrote: > You can also try out > https://debezium.io/documentation/reference/0.10/connectors/mysql.html > -- > *From:* Ajit Kumar Amit > *Sent:* 19 August 2022 14:30 > *To:* sandra sukumaran >

Re: [EXTERNAL] Re: Spark streaming

You can also try out https://debezium.io/documentation/reference/0.10/connectors/mysql.html From: Ajit Kumar Amit Sent: 19 August 2022 14:30 To: sandra sukumaran Cc: user@spark.apache.org Subject: [EXTERNAL] Re: Spark streaming Caution! This email originated

Re: [EXTERNAL] Re: Spark streaming - Data Ingestion

I am beginner with spark may , also know how to connect MySQL database with spark streaming Thanks and regards Akash P On Wed, 17 Aug, 2022, 8:28 pm Saurabh Gulati, wrote: > Another take: > >- Debezium > >to

Re: [EXTERNAL] Re: Spark streaming - Data Ingestion

The idea behind spark-streaming is to process change events as they occur, hence the suggestions above that require capturing change events using Debezium. But you can use jdbc drivers to connect Spark to relational databases On Wed, Aug 17, 2022 at 6:21 PM Akash Vellukai wrote: > I am

Re: [EXTERNAL] Re: Spark streaming - Data Ingestion

Another take: * Debezium to read Write Ahead logs(WAL) and send to Kafka * Kafka connect to write to cloud storage -> Hive * OR * Spark streaming to parse WAL -> Storage -> Hive Regards