jiazhai commented on a change in pull request #4666: Doc release 2.4.0
URL: https://github.com/apache/pulsar/pull/4666#discussion_r299912979
 
 

 ##########
 File path: site2/website/versioned_docs/version-2.4.0/io-cdc-canal.md
 ##########
 @@ -0,0 +1,175 @@
+---
+id: version-2.4.0-io-cdc-canal
+title: CDC Canal Connector
+sidebar_label: CDC Canal Connector
+original_id: io-cdc-canal
+---
+
+### Source Configuration Options
+
+The Configuration is mostly related to Canal task config.
+
+| Name | Required | Default | Description |
+|------|----------|---------|-------------|
+| `zkServers` | `false` | `127.0.0.1:2181` | `The address and port of the 
zookeeper . if canal server configured to cluster mode` |
+| `batchSize` | `true` | `5120` | `Take 5120 records from the canal server in 
batches` |
+| `username` | `false` | `` | `Canal server account, not MySQL` |
+| `password` | `false` | `` | `Canal server password, not MySQL` |
+| `cluster` | `false` | `false` | `Decide whether to open cluster mode based 
on canal server configuration, true: cluster mode, false: standalone mode` |
+| `singleHostname` | `false` | `127.0.0.1` | `The address of canal server` |
+| `singlePort` | `false` | `11111` | `The port of canal server` |
+
+
+### Configuration Example
+
+Here is a configuration Json example:
+
+```$json
+{
+    "zkServers": "127.0.0.1:2181",
+    "batchSize": "5120",
+    "destination": "example",
+    "username": "",
+    "password": "",
+    "cluster": false,
+    "singleHostname": "127.0.0.1",
+    "singlePort": "11111",
+}
+```
+You could also find the yaml example in this 
[file](https://github.com/apache/pulsar/blob/master/pulsar-io/canal/src/main/resources/canal-mysql-source-config.yaml),
 which has similar content below:
+
+```$yaml
+configs:
+    zkServers: "127.0.0.1:2181"
+    batchSize: "5120"
+    destination: "example"
+    username: ""
+    password: ""
+    cluster: false
+    singleHostname: "127.0.0.1"
+    singlePort: "11111"
+```
+
+### Usage example
+
+Here is a simple example to store MySQL change data using above example config.
+
+- Start a MySQL server
+
+```$bash
+docker pull mysql:5.7
+docker run -d -it --rm --name pulsar-mysql -p 3306:3306 -e 
MYSQL_ROOT_PASSWORD=canal -e MYSQL_USER=mysqluser -e MYSQL_PASSWORD=mysqlpw 
mysql:5.7
+```
+- Modify configuration files mysqld.cnf
+
+```
+[mysqld]
+pid-file    = /var/run/mysqld/mysqld.pid
+socket      = /var/run/mysqld/mysqld.sock
+datadir     = /var/lib/mysql
+#log-error  = /var/log/mysql/error.log
+# By default we only accept connections from localhost
+#bind-address   = 127.0.0.1
+# Disabling symbolic-links is recommended to prevent assorted security risks
+symbolic-links=0
+log-bin=mysql-bin
+binlog-format=ROW
+server_id=1
+```
+
+- Copy file to mysql server from local and restart mysql server
+```$bash
+docker cp mysqld.cnf pulsar-mysql:/etc/mysql/mysql.conf.d/
+docker restart pulsar-mysql
+```
+
+- Create test database in mysql server
+```$bash
+docker exec -it pulsar-mysql /bin/bash
+mysql -h 127.0.0.1 -uroot -pcanal -e 'create database test;'
+```
+
+- Start canal server and connect mysql server
+
+```
+docker pull canal/canal-server:v1.1.2
+docker run -d -it --link pulsar-mysql -e canal.auto.scan=false -e 
canal.destinations=test -e canal.instance.master.address=pulsar-mysql:3306 -e 
canal.instance.dbUsername=root -e canal.instance.dbPassword=canal -e 
canal.instance.connectionCharset=UTF-8 -e canal.instance.tsdb.enable=true -e 
canal.instance.gtidon=false --name=pulsar-canal-server -p 8000:8000 -p 
2222:2222 -p 11111:11111 -p 11112:11112 -m 4096m canal/canal-server:v1.1.2
+```
+
+- Start pulsar standalone
+
+```$bash
+docker pull apachepulsar/pulsar:2.3.0
+docker run -d -it --link pulsar-canal-server -p 6650:6650 -p 8080:8080 -v 
$PWD/data:/pulsar/data --name pulsar-standalone apachepulsar/pulsar:2.3.0 
bin/pulsar standalone
+```
+
+- Start pulsar-io in standalone
+
+- Config file canal-mysql-source-config.yaml
+
+```$yaml
+configs:
+    zkServers: ""
+    batchSize: "5120"
+    destination: "test"
+    username: ""
+    password: ""
+    cluster: false
+    singleHostname: "pulsar-canal-server"
+    singlePort: "11111"
+```
+- Consumer file pulsar-client.py for test
+```
+import pulsar
+
+client = pulsar.Client('pulsar://localhost:6650')
+consumer = client.subscribe('my-topic',
+                            subscription_name='my-sub')
+
+while True:
+    msg = consumer.receive()
+    print("Received message: '%s'" % msg.data())
+    consumer.acknowledge(msg)
+
+client.close()
+```
+
+- Copy config file and test file to pulsar server
+
+```$bash
+docker cp canal-mysql-source-config.yaml pulsar-standalone:/pulsar/conf/
+docker cp pulsar-client.py pulsar-standalone:/pulsar/
+```
+
+- Download canal connector and start canal connector
+```$bash
+docker exec -it pulsar-standalone /bin/bash
+wget 
http://apache.01link.hk/pulsar/pulsar-2.3.0/connectors/pulsar-io-canal-2.3.0.nar
 -P connectors
 
 Review comment:
   also here for 2.3.0. and the apache download link(http://apache.01link.hk) 
would be better replaced to a dynamic one?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to