debezium 使用踩坑

在已经启动后的连接器配置中table.include.list 添加了一张已有数据的表,如何为该表做snapshot

> 开发环境  debezium版本是1.3.final 

如题,这里要介绍一个参数 “snapshot.new.tables” ,这个参数有点神奇,是被官方雪藏起来的,官方issue给的解释是 https://issues.redhat.com/browse/DBZ-1977

示例如下:

{
    "connector.class": "io.debezium.connector.mysql.MySqlConnector",
    "database.user": "debezium_mysql",
    "tasks.max": "1",
    "database.history.kafka.bootstrap.servers": "cdh04:9092,cdh05:9092,cdh06:9092",
    "database.history.kafka.topic": "ninth_studio_connector_history",
    "database.history.kafka.recovery.poll.interval.ms": "5000",
    "database.server.name": "ninth_studio_connector",
    "database.port": "3306",
    "tombstones.on.delete": "false",
    "snapshot.new.tables":"parallel",
    "database.hostname": "common.mysql.test.local",
    "database.password": "********",
    "database.serverTimezone":"UTC" ,
    "table.include.list": "ninth_studio.wehub_action_logs,ninth_studio.ab_py_assistant_binds",
    "database.include.list": "ninth_studio"
}

ps:1.3版本在配置了snapshot.mode = when_needed , snapshot.include.collection.list 后,偶尔会出现监听不到数据的情况,要更新配置之后(随便更新什么配置,主要是为了让connect重启,单纯的使用rest api 重启不会起作用)才能继续读取数据

时区问题

设置参考连接: https://my.oschina.net/dacoolbaby/blog/3096451

这位大佬分享了很多debezium的坑点,有很多可以借鉴的地方

原文地址:https://www.cnblogs.com/yangxusun9/p/14052156.html