thanks Roman. On Tue, Jun 27, 2023 at 3:15 PM Roman Schmitz <roman.schm...@gmail.com> wrote:
> Hi Avani, > > not sure what you mean by creating your own code. Of course you can write > an own Connector but especially in Source Connectors offset handling can > challenging depending on the system. > There are hundreds of different Connectors out there that can connect to > existing systems. If you find one that suits your needs that's probably a > better idea than starting from scratch. > Typically there are lots of configuration options and for quite simple > transformations you can use SMTs. To run Connectors, simply install some > Connect instance, have the jars for your connectors installed and > configure/deploy your Connector(s) using the Connect API. > > Kind regards, > Roman > > Am Di., 27. Juni 2023 um 11:26 Uhr schrieb Avani Panchal > <avani.panc...@adit.com.invalid>: > > > Hi Roman, > > > > Thanks for replying, I want to ask one question: can I create my own code > > to integrate data using Kafka connectors? > > > > Kind regards, > > Avani Panchal > > > > On Tue, Jun 27, 2023 at 12:29 PM Roman Schmitz <roman.schm...@gmail.com> > > wrote: > > > > > Hi Avani, > > > > > > totally agree - depending on the way you'd like to integrate your > source > > > that would be probably Debezium for CDC or JDBC source that can be used > > > either in a "full-load" as well as in an incremental way. > > > As a sink JDBC sink with the corresponding driver for the DB. Kindly > > have a > > > look at the Connect Documentation and some examples. > > > If you'd like to test with runnable Docker-based examples, you can > have a > > > look at this Docker Playground Repo: > > > > https://github.com/vdesabou/kafka-docker-playground/tree/master/connect > > > > > > Kind regards, > > > Roman > > > > > > Am Di., 27. Juni 2023 um 08:48 Uhr schrieb Avani Panchal > > > <avani.panc...@adit.com.invalid>: > > > > > > > Hi Sagar, > > > > > > > > Thank you for your email. > > > > > > > > Thanks & Regards, > > > > Avani Panchal > > > > > > > > > > > > On Tue, Jun 27, 2023 at 12:15 PM Sagar <sagarmeansoc...@gmail.com> > > > wrote: > > > > > > > > > Hi Avani, > > > > > > > > > > I already shared the documentation link for Kafka Connect. Let me > > share > > > > it > > > > > again: https://kafka.apache.org/documentation/#connect > > > > > > > > > > Regarding the connector documentation, you should be able to find > > them > > > by > > > > > just searching for JDBC source connector, JDBC sink connector and > > > > Debezium > > > > > connector for MySQL. > > > > > > > > > > Let me know if that works. > > > > > > > > > > Thanks! > > > > > Sagar. > > > > > > > > > > > > > > > > > > > > On Mon, Jun 26, 2023 at 2:10 PM Avani Panchal > > > > > <avani.panc...@adit.com.invalid> wrote: > > > > > > > > > > > Hi Sagar, > > > > > > > > > > > > Thank you for the information, you solved our confusion. > > > > > > I also saw lots of links for documentation on Kafka, but I am > > > confused > > > > > > which document I should use. > > > > > > So can you share the proper link from where I can read the > > documents. > > > > > > > > > > > > Thanks, > > > > > > Avani Panchal > > > > > > > > > > > > > > > > > > On Mon, Jun 26, 2023 at 1:48 PM Sagar <sagarmeansoc...@gmail.com > > > > > > wrote: > > > > > > > > > > > > > Hey Avani, > > > > > > > > > > > > > > Kafka Connect <https://kafka.apache.org/documentation/#connect > > > > is > > > > the > > > > > > > tool > > > > > > > to use when you want to stream data to/from Kafka via external > > > > systems. > > > > > > One > > > > > > > would typically configure connectors which allow streaming data > > > > to/from > > > > > > > Kafka. There are 2 types of connectors: > > > > > > > 1) Source Connectors: Which stream data from external systems > > like > > > > > > > databases etc to Kafka and > > > > > > > 2) Sink Connectors: Which stream data from Kafka to external > > > systems. > > > > > > > > > > > > > > Since you want to stream data from MySQL to SQL Server, with > > Kafka > > > > > > Connect > > > > > > > it would be a 2 step process: > > > > > > > > > > > > > > 1) Capture changes from MySQL to Kafka using connectors like > JDBC > > > > > source > > > > > > > connector or Debezium MySQL connector. > > > > > > > 2) Once the data is in Kafka, you can use JDBC sink connectors > to > > > > > stream > > > > > > > data from Kafka topics to the tables in SQL Server. > > > > > > > > > > > > > > Note that this is a very simplified view of how you can achieve > > > your > > > > > goal > > > > > > > of streaming changes from MySQL to SQL Server and I would > > recommend > > > > > > reading > > > > > > > the documentation of the individual connectors and the Kafka > > > Connect > > > > > > > framework to understand how to make it work for your usecase. > > > > > > > > > > > > > > Thanks for your interest on Apache Kafka! > > > > > > > > > > > > > > Thanks! > > > > > > > Sagar. > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 26, 2023 at 11:42 AM Avani Panchal > > > > > > > <avani.panc...@adit.com.invalid> wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > In my application I want to sync my client's data to my SQL > > > > server. > > > > > at > > > > > > > > client place the database is MYSQL. > > > > > > > > > > > > > > > > How can I achieve this using Kafka? I read a lot of documents > > > but I > > > > > > don't > > > > > > > > understand which setup I need and how I can achieve it. > > > > > > > > > > > > > > > > I was also wondering about "Book a demo with Kafka" but > didn't > > > find > > > > > it. > > > > > > > > > > > > > > > > Please help me. > > > > > > > > > > > > > > > > Thank you, > > > > > > > > Avani > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
