Hi,
I am trying to do POC for kafka CDC with database and ingest it into kafka.
I have been trying to make it work from past 5 days and had no luck. can
someone help me what is wrong in my approach.
Error Received:
*{"error_code":400,"message":"Connector configuration is invalid and
contains the following 1 error(s):\nUnable to connect: Communications link
failure\n\nThe last packet sent successfully to the server was 0
milliseconds ago. The driver has not received any packets from the
server.\nYou can also find the above list of errors at the endpoint
`/{connectorType}/config/validate`"*
I am able to connect mysql in docker container and also from my laptop
using sqlconnector with host as localhost and port 3306,I even tried with
IP address of database and no luck.
ALso tried various options of changing binding address for mysql and my
assumption is that connection is being made but somehow it is not
processing.
I can also see the request is sent to kafka connector where I can see the
same error in distributed.log
*Here are my code snippets.*
Jar used is debezium-connector-mysql-1.0.0.Final
Script file to call the rest api request.
#!/bin/bash
curl -X POST http://localhost:8083/connectors -H 'Content-Type:
application/json' -H 'Accept: application/json' -d
@./lms_dev_source_jdbc_config.json
JSON file to send request to connector.
{
"name": "mysql-connector-flattened",
"config": {
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"database.hostname":"localhost",
"database.port": "3306",
"database.user": "root",
"database.password": "Bismillah1!",
"database.server.id": "4",
"database.server.name": "recipes_database",
"database.history.kafka.bootstrap.servers": "localhost:9092",
"database.history.kafka.topic": "dbhistory.fullfillment" ,
"include.schema.changes": "true" ,
"transforms": "unwrap,changetopic",
"transforms.unwrap.type": "io.debezium.transforms.UnwrapFromEnvelope",
"transforms.changetopic.type":"org.apache.kafka.connect.transforms.RegexRouter",
"transforms.changetopic.regex":"(.*)",
"transforms.changetopic.replacement":"$1-smt"
}
}
DockerCompse File:
version: '2'
services:
db:
image: mysql:8.0
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_DATABASE: db
MYSQL_USER: debezium
MYSQL_PASSWORD: Bismillah1!
MYSQL_ROOT_PASSWORD: Bismillah1!
ports:
- 3306:3306
expose:
- 3306
kafka-cluster:
image: landoop/fast-data-dev:cp3.3.0
restart: always
environment:
ADV_HOST: 127.0.0.1 # Change to 192.168.99.100 if using
Docker Toolbox
RUNTESTS: 0 # Disable Running tests so the
cluster starts faster
ports:
- 2181:2181 # Zookeeper
- 3030:3030 # Landoop UI
- 8081-8083:8081-8083 # REST Proxy, Schema Registry, Kafka
Connect ports
- 9581-9585:9581-9585 # JMX Ports
- 9092:9092 # Kafka Broker
