基于Disruptor并发框架的分类任务并发

并发的场景

最近在编码中遇到的场景，我的程序需要处理不同类型的任务，场景要求如下：
1.同类任务串行、不同类任务并发。
2.高吞吐量。
3.任务类型动态增减。

思路

思路一：
最直接的想法，每有一个任务种类被新建，就创建对应的处理线程。
这样的思路问题在于线程数量不可控、创建、销毁线程开销大。不可取。

思路二：
比较常规的想法，所有任务共享线程池每有一个任务种类被创建，就新建一个队列，以保证同类任务串行。
这样的思路问题在于数据结构开销不可控，如果是任务种类繁多，但每种任务数量并不多的情况，那么建如此多的队列显得可笑。

于是我期望能够使用一个线程池、一个队列搞定这些事。

设计

代码实现

引入disruptor依赖：

<dependency>
    <groupId>com.lmax</groupId>
    <artifactId>disruptor</artifactId>
    <version>3.4.2</version>
</dependency>

Task接口

public interface Task<T> {
	public T exec();
}

TaskEvent类

import com.lmax.disruptor.EventFactory;

public class TaskEvent<T> {

	private Task<T> input;
	
	//用于标记一个并发分组
	private int partitionId;
	
	//disruptor的任务事件工场
	public static final EventFactory<TaskEvent> FACTORY = TaskEvent::new;

	public Task<T> getInput() {
		return input;
	}

	public void setInput(Task<T> input) {
		this.input = input;
	}

	public int getPartitionId() {
		return partitionId;
	}

	public void setPartitionId(int partitionId) {
		this.partitionId = partitionId;
	}

}

TaskHandler类

import com.lmax.disruptor.EventHandler;
import com.lmax.disruptor.LifecycleAware;

public class EventAdaptor<T> implements EventHandler<TaskEvent<T>>, LifecycleAware {
	
	//决定我处理那些任务
	private final int partitionId;

	public EventAdaptor(int partitionId) {
		super();
		this.partitionId = partitionId;
	}

	public void onEvent(TaskEvent<T> taskEvent, long arg1, boolean arg2) throws Exception {
		if(partitionId == taskEvent.getPartitionId()) {
			taskEvent.getInput().exec();
		}
	}

	@Override
	public void onShutdown() {
		Thread.currentThread().setName("handler-" + partitionId);
	}

	@Override
	public void onStart() {
		Thread.currentThread().setName("handler-" + partitionId);
	}

}

TaskService类

import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

import javax.annotation.PostConstruct;

import com.lmax.disruptor.BlockingWaitStrategy;
import com.lmax.disruptor.dsl.Disruptor;
import com.lmax.disruptor.dsl.ProducerType;

@Service
@Singleton
public class TaskService<V extends Task<T>, T> {

	private static final int BUFFER_SIZE = 1024;
	
	private static final int DEFAULT_POOL_SIZE = 5;
	
	private ThreadPoolExecutor executor;
	
	private Disruptor<TaskEvent>  disruptor;
	
	private Map<String,Integer> taskClassMapperPartition = new ConcurrentHashMap<>();
	
	private static final int PARALLEL_NUM = 5;
	
	private List<String> taskTypes = new ArrayList<>();
	
	@PostConstruct
	public void init() {
		
		//初始化处理器和线程池
		this.executor = new ThreadPoolExecutor(DEFAULT_POOL_SIZE, DEFAULT_POOL_SIZE, 15L, TimeUnit.SECONDS, new LinkedBlockingQueue<>());
		this.executor.prestartAllCoreThreads();
		this.disruptor = new Disruptor<>(TaskEvent.FACTORY, BUFFER_SIZE, executor, ProducerType.SINGLE, new BlockingWaitStrategy());
	
		EventAdaptor[] handlers = new EventAdaptor[PARALLEL_NUM];
		
		for(int i = 0; i < PARALLEL_NUM; i++) {
			handlers[i] = new EventAdaptor(i);
		}
		
		this.disruptor.handleEventsWith(handlers);
		this.disruptor.start();
		
		rePartition();
	}
	
	public void addTaskType(String type) {
		taskClassMapperPartition.put(type, taskTypes.size() % PARALLEL_NUM);
		taskTypes.add(type);
	}
	
	public void deleteTaskType(String type) {
		taskTypes.remove(type);
		taskClassMapperPartition.remove(type);
		rePartition();
	}

	private void rePartition() {
		for(int i = 0, length = taskTypes.size(); i < length; i++) {
			//给各任务处理器均衡发放任务
			taskClassMapperPartition.put(taskTypes.get(i), i % PARALLEL_NUM);
		}
	}
	
}

这里给出的代码是一套物业无得简单框架，你只要实现Task接口就可以了。