Springboot整合shardingsphere和druid进行读写分离

最近在使用springboot整合shardingsphere和druid实现mysql数据库读写分离时遇到了一些问题,特此记录一下。

依赖版本

  • Springboot 2.1.6.RElEASE
  • shardingsphere 4.1.1
  • druid 1.1.23

需要的依赖如下:

<dependency>
	<groupId>org.apache.shardingsphere</groupId>
	<artifactId>sharding-jdbc-spring-boot-starter</artifactId>
	<version>4.1.1</version>
</dependency>

<dependency>
	<groupId>com.alibaba</groupId>
	<artifactId>druid-spring-boot-starter</artifactId>
	<version>1.1.23</version>
</dependency>

yml文件配置

datasource配置

spring:
  datasource:
    type: com.alibaba.druid.pool.DruidDataSource
    driver-class-name: com.mysql.cj.jdbc.Driver
    url: jdbc:mysql://127.0.0.1:3306/test?characterEncoding=utf-8&allowMultiQueries=true&serverTimezone=Asia/Shanghai
    username: root
    password: root
    druid:
      async-init: true
      keep-alive: true
      filters: stat,wall,logback  # 必须配置项,否则sql监控页面没有内容
      initial-size: 5
      max-active: 50
      min-idle: 5
      max-wait: 6000
      validation-query: SELECT 'x'
      test-on-borrow: false
      test-on-return: false
      test-while-idle: true
      time-between-eviction-runs-millis: 60000
      min-evictable-idle-time-millis: 300000
      remove-abandoned: false
      log-abandoned: true
      filter:
        stat:
          enabled: true
          log-slow-sql: true
      web-stat-filter:
        enabled: true
        url-pattern: /*
        exclusions: '*.js,*.gif,*.jpg,*.bmp,*.png,*.css,*.ico,/druid/*'
      stat-view-servlet:
        enabled: true # 控制是否开启监控页面
        url-pattern: /druid/*
        reset-enable: false
		# ip白名单,默认是127.0.0.1,为空时表示所有的ip都可以访问,如果这里允许所有ip访问必须配置为空,否则只能127.0.0.1访问
        allow: 
		# ip黑名单,同理白名单
        deny:  
        login-username: druid  # 监控页面登陆用户名
        login-password: druid  # 监控页面登陆密码

读写分离的sharding配置

spring:
  shardingsphere:
    enabled: true # 是否启用sharding,不启用时使用datasource配置的数据源
    datasource:
      names: master,slave0 # 节点名称,多个时使用逗号隔开
      master:
        type: com.alibaba.druid.pool.DruidDataSource
        driver-class-name: com.mysql.cj.jdbc.Driver
        url: jdbc:mysql://10.18.121.222:3306/spider?characterEncoding=utf-8&allowMultiQueries=true&serverTimezone=Asia/Shanghai
        username: root
        password: 123456
        # 以下为druid配置,可以共用datasource中的druid配置,需要覆盖时再重新配置
        filters: stat,wall,logback
        initial-size: 2
        max-active: 45
        min-idle: 6
      slave0:
        type: com.alibaba.druid.pool.DruidDataSource
        driver-class-name: com.mysql.cj.jdbc.Driver
        url: jdbc:mysql://10.18.121.222:3307/spider?characterEncoding=utf-8&allowMultiQueries=true&serverTimezone=Asia/Shanghai
        username: root
        password: 123456
        # 以下为druid配置,可以共用datasource中的druid配置,需要覆盖时再重新配置
        filters: stat,wall,logback
        initial-size: 3
        max-active: 20
        min-idle: 8
    masterslave:
      name: ms
      master-data-source-name: master
      slave-data-source-names: slave0
    props:
      sql:
        show: true

注意:
sharding数据源中的druid配置项基本可以使用spring.datasource.druid中的配置项,但是在实际使用过程中发现,filters: stat,wall,logback这项配置必须重新给各个数据源配置,不然sql监控页面没有任何内容展示。

遇到的问题

通过以上两个步骤的配置,启动没有任何业务类的springboot项目,成功启动,并且访问druid监控页面也正常,在满怀激动的往正式项目中迁移后,发现项目启动失败,出现了下图所示的异常:

这是什么鬼?以为配置出错了,赶紧一顿检查检查配置,发现没有任何问题,在多次尝试无果后将配置还原发现又可以正常启动从而基本确定是sharding和druid的配置导致了项目启动失败。

从sharding官网的FAQ中发现如下解释:

根据官网的解释以及从网上查的一些资料来看,解决方案有两个:

  1. 去掉druid-spring-boot-starter,直接使用druid-xxx.jar来替代,这就不会出现两个数据源冲突的问题
  2. 仍然使用druid-spring-boot-starter,但是在springboot的启动类上exclude掉DruidDataSourceAutoConfigure这个类,忽略druid连接池的默认数据源配置(@SpringBootApplication(exclude = {DruidDataSourceAutoConfigure.class})

经过一番尝试,以上两种方案都可以解决启动报错的问题,但是使用上述两种方案,即使配置了打开druid监控页面的配置,访问监控页面时仍然是404,我的需求是要能监控数据库的,因此上述两种方案都不可行。

又要有监控页面,又要项目正常启动,一时陷入了僵局,后来在查找资料的过程中,发现可以通过手动创建数据源配置,并且将其指定为默认数据源就可以解决该问题,经过查到的资料以及公司大佬的支持,添加如下所示的配置类:

@Configuration
@EnableConfigurationProperties(JpaProperties.class)
public class DataSourceConfiguration {
	private final JpaProperties jpaProperties;

	private final Environment environment;

	public DataSourceConfiguration(JpaProperties jpaProperties, Environment environment) {
		this.jpaProperties = jpaProperties;
		this.environment = environment;
	}

	@Primary
	@Bean
	public DataSource dataSource() {
		String prefix = "spring.shardingsphere.datasource.";
		String each = this.getDataSourceNames(prefix).get(0);
		try {
			return this.getDataSource(prefix, each);
		} catch (final ReflectiveOperationException ex) {
			throw new ShardingSphereException("Can't find datasource type!", ex);
		}
	}

	@Primary
	@Bean
	public EntityManagerFactory entityManagerFactory() {
		HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
		vendorAdapter.setDatabase(Database.MYSQL);
		vendorAdapter.setGenerateDdl(true);
		vendorAdapter.setShowSql(true);
		LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
		factory.setJpaVendorAdapter(vendorAdapter);
		factory.setPersistenceUnitName("default");
		factory.setPackagesToScan("com.lzm.*");
		factory.setDataSource(this.dataSource());
		factory.setJpaPropertyMap(this.jpaProperties.getProperties());
		factory.afterPropertiesSet();
		return factory.getObject();
	}

	@Bean
	@Primary
	public EntityManager entityManager(EntityManagerFactory entityManagerFactory) {
		return SharedEntityManagerCreator.createSharedEntityManager(entityManagerFactory);
	}

	@Primary
	@Bean
	public PlatformTransactionManager transactionManager(EntityManagerFactory entityManagerFactory) {
		JpaTransactionManager txManager = new JpaTransactionManager();
		txManager.setEntityManagerFactory(entityManagerFactory);
		return txManager;
	}

	private List<String> getDataSourceNames(final String prefix) {
		StandardEnvironment standardEnv = (StandardEnvironment) this.environment;
		standardEnv.setIgnoreUnresolvableNestedPlaceholders(true);
		return null == standardEnv.getProperty(prefix + "name")
				? new InlineExpressionParser(standardEnv.getProperty(prefix + "names")).splitAndEvaluate()
				: Collections.singletonList(standardEnv.getProperty(prefix + "name"));
	}

	@SuppressWarnings("unchecked")
	private DataSource getDataSource(final String prefix, final String dataSourceName) throws ReflectiveOperationException {
		Map dataSourceProps = PropertyUtil.handle(this.environment, prefix + dataSourceName.trim(), Map.class);
		Preconditions.checkState(!dataSourceProps.isEmpty(), "Wrong datasource properties!");
		DataSource result = DataSourceUtil.getDataSource(dataSourceProps.get("type").toString(), dataSourceProps);
		DataSourcePropertiesSetterHolder.getDataSourcePropertiesSetterByType(dataSourceProps.get("type").toString())
				.ifPresent(dataSourcePropertiesSetter -> dataSourcePropertiesSetter.propertiesSet(this.environment, prefix, dataSourceName, result));
		return result;
	}
}

以上打码中根据sharding的配置手动创建数据源DataSource以及EntityManagerFactory等Bean,并且设置为默认加载的bean类型(@Primary),添加以上配置类后,重新启动项目,项目正常启动而且druid的监控页面也可以正常访问,此问题得到完美解决。具体源码可以参考码云

参考文档

1.JPA项目多数据源模式整合Sharding-jdbc实现数据脱敏
2.ardingSphere FAQ

原文地址:https://www.cnblogs.com/ybyn/p/13690752.html