官网:https://codechina.csdn.net/mirrors/alibaba/canal?utm_source=csdn_github_accelerator
下载地址:https://github.com/alibaba/canal/releases
参考部署地址:https://blog.csdn.net/qq_29116427/article/details/106498040
文档:https://github.com/alibaba/canal/wiki/QuickStart
Cannal单节点部署
Mysql配置
Mysql开启binlog写入功能,my.cnf 中配置如下:
[mysqld] log-bin=mysql-bin # 开启 binlog binlog-format=ROW # 选择 ROW 模式 server_id=1 # 配置 MySQL replaction 需要定义,不要和 canal 的 slaveId 重复 max_binlog_size=500m #每个binlog日志文件大小
创建mysql用户,并且赋权:
CREATE USER canal IDENTIFIED BY 'canal'; GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%'; -- GRANT ALL PRIVILEGES ON *.* TO 'canal'@'%' ; FLUSH PRIVILEGES;
重启mysql,查看binlog配置是否生效,如下表示生效:
mysql> show variables like 'binlog_format'; mysql> show variables like 'log_bin'; mysql> show master status;
Canal配置
下载canal.deployer-1.1.5.tar.gz 包
下载地址:https://github.com/alibaba/canal/releases
由于cannal解压,无根目录,故新建cannal_deployer目录,mkdir ./cannal_deployer,并将压缩包解压至改目录下:
Tar -zvxf ./ canal.deployer-1.1.5.tar.gz -C ../softwore/ cannal_deployer
进入cannal_deployer 目录:
Cd /root/software/cannal_deployer
修改canal配置文件
vi ./conf/example/instance.properties
修改内容如下:
# position info canal.instance.master.address=10.18.35.155:3309 # username/password canal.instance.dbUsername=root canal.instance.dbPassword=123456 # table regex *\..*表示监听所有数据库 canal.instance.filter.regex=.*\..* # mq config 表示生产数据到kafka时,创建的topic的名称 canal.mq.topic=canal_topic 修改canal.properties 文件 vi ./conf/canal.properties 修改内容如下: #默认为TCP,也就是你通过官方的example可以在终端查看数据,我们修改为kafka # tcp, kafka, rocketMQ, rabbitMQ canal.serverMode = kafka canal.destinations = example #kafka地址配置 kafka.bootstrap.servers = 10.18.35.155:9092,10.18.35.156:9092,10.18.35.157:9092 # 针对库名或者表名发送动态topic #canal.mq.dynamicTopic=mytest,.*,mytest.user,mytest\..*,.*\..*
Cannal配置文件说明:https://blog.csdn.net/weixin_35852328/article/details/87600871
启动cannal-server
./bin/startup.sh
查看日志:
tail ./logs/example/example.log
cannal测试:
查看当前kafka中topic
./kafka-topics.sh -zookeeper 192.168.0.20:2181 -list
查询canal创建的topic的消费情况
./kafka-console-consumer.sh --bootstrap-server 192.168.0.20:9092 --topic canal_topic --from-beginning