使用canal分析binlog(一) 入门

canal介绍

canal是应阿里巴巴存在杭州和美国的双机房部署，存在跨机房同步的业务需求而提出的。早期，阿里巴巴B2B公司因为存在杭州和美国双机房部署，存在跨机房同步的业务需求。不过早期的数据库同步业务，主要是基于trigger的方式获取增量变更，不过从2010年开始，阿里系公司开始逐步的尝试基于数据库的日志解析，获取增量变更进行同步，由此衍生出了增量订阅&消费的业务，从此开启了一段新纪元。ps. 目前内部使用的同步，已经支持mysql5.x和oracle部分版本的日志解析。

基于日志增量订阅&消费支持的业务：
数据库镜像
数据库实时备份
多级索引 (卖家和买家各自分库索引)
search build
业务cache刷新
价格变化等重要业务消息

keyword：数据库同步，增量订阅&消费。

https://github.com/alibaba/canal/wiki

http://www.bubuko.com/infodetail-1268551.html

教程 http://blog.csdn.net/hackerwin7/article/details/37923607

http://liyonghui160com.iteye.com/blog/2176066

开启mysql binlog

http://blog.csdn.net/aitangyong/article/details/53114633

大数据量堆叠

http://blog.csdn.net/zhanlanmg/article/details/51213631

mysql多个slave同一serverId冲突

http://www.penglixun.com/tech/database/mysql_multi_slave_same_serverid.html

照着官方的例子修改canal.properties和example/instance.properties，server端启动起来并不难

在现有的spring boot项目的pom增加了canal引用后，项目无法启动

错误信息如下：

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Users/Administrator/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.5/log4j-slf4j-impl-2.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/Administrator/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
Exception in thread "main" java.lang.IllegalArgumentException: LoggerFactory is not a Logback LoggerContext but Logback is on the classpath. Either remove Logback or the competing implementation (class org.apache.logging.slf4j.Log4jLoggerFactory loaded from file:/C:/Users/Administrator/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.5/log4j-slf4j-impl-2.5.jar). If you are using WebLogic you will need to add 'org.slf4j' to prefer-application-packages in WEB-INF/weblogic.xml Object of class [org.apache.logging.slf4j.Log4jLoggerFactory] must be an instance of class ch.qos.logback.classic.LoggerContext
at org.springframework.util.Assert.isInstanceOf(Assert.java:346)
at org.springframework.boot.logging.logback.LogbackLoggingSystem.getLoggerContext(LogbackLoggingSystem.java:221)
at org.springframework.boot.logging.logback.LogbackLoggingSystem.getLogger(LogbackLoggingSystem.java:213)
at org.springframework.boot.logging.logback.LogbackLoggingSystem.beforeInitialize(LogbackLoggingSystem.java:98)
at org.springframework.boot.logging.LoggingApplicationListener.onApplicationStartedEvent(LoggingApplicationListener.java:215)
at org.springframework.boot.logging.LoggingApplicationListener.onApplicationEvent(LoggingApplicationListener.java:197)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:163)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:136)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:119)
at org.springframework.boot.context.event.EventPublishingRunListener.publishEvent(EventPublishingRunListener.java:111)
at org.springframework.boot.context.event.EventPublishingRunListener.started(EventPublishingRunListener.java:60)
at org.springframework.boot.SpringApplicationRunListeners.started(SpringApplicationRunListeners.java:48)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1191)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1180)
at com.paybox.sync.App.main(App.java:32)

错误产生原因：现有项目使用的日志组件是slf4j+log4j2，而canal使用的日志组件是slf4j+logback，log4j和logback冲突导致系统发生异常无法启动。

查看canal的依赖 http://mvnrepository.com/artifact/com.alibaba.otter/canal.common/1.0.22

解决方法：

修改pom文件

　　　　<dependency>
            <groupId>com.alibaba.otter</groupId>
            <artifactId>canal.client</artifactId>
            <version>${canal.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>ch.qos.logback</groupId>
                    <artifactId>logback-core</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>ch.qos.logback</groupId>
                    <artifactId>logback-classic</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.springframework</groupId>
                    <artifactId>spring</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

项目中原来已经配置过log4j2和slf4j，只要删除canal对logback的依赖即可。

而后又发现项目启动发生错误，因为canal还依赖spring2.5.6，现在项目使用的是spring boot4.2.5，我就直接把canal对spring的依赖给排除了，项目能正常运行，不过这样会导致什么还不清楚。