使用多线程优化复杂逻辑以及数据量多处理

在做一个数据分析的一个项目 有大量的数据分析处理,逻辑复杂,导致响应速度很慢。3分钟。。。。。。这怎么可能容忍话不多说,多线程搞起(这里就不说多线程优缺点,只是记录一下下次使用方便)

一开始我使用的 三个线程代码如下

 latch = new CountDownLatch(3);
        long startTime = System.currentTimeMillis();

        List<DetailDataDto> demandInputDtoPageInfo = listDetailDataByDate(conditionVo);

        class HeadIn implements Runnable {
            @Override
            public void run() {
                demandInputDtoPageInfo.parallelStream().forEach(s -> {
                    s.setCheckTime(basicInfoMapper.getCreateTime(s.getRequirementNO()));

                });

                latch.countDown();
            }
        }
        class AllIn implements Runnable {

            @Override
            public void run() {
                demandInputDtoPageInfo.parallelStream().forEach(s -> {
                    s.setEndTime(demandInputMapper.getEndTime(s.getRequirementNO()));
                });
                latch.countDown();
            }
        }
        class HeadNotIn implements Runnable {

            @Override
            public void run() {
                demandInputDtoPageInfo.parallelStream().forEach(s -> {
                    s.setDeployTime(basicInfoMapper.getDeployTime(s.getRequirementNO()));
                });
                latch.countDown();

            }
        }

 Thread myThread1 = new Thread(new HeadIn());
                    myThread1.start();
                    Thread myThread2 = new Thread(new AllIn());
                    myThread2.start();
                    Thread myThread3 = new Thread(new HeadNotIn());
                    myThread3.start();
                    latch.await();
//等待线程结束后执行最后逻辑 

这样有些优化 快来一分钟  但是还不能忍受 没事接着来 百度各种搜,完了 忘了原地址了 抱歉(…………)

于是就有下面的东西

   class Task implements Callable<List<DetailDataDto>> {

            private List<DetailDataDto> list;
            private int start;
            private int end;

            public Task(List<DetailDataDto> list, int start, int end) {
                this.list = list;
                this.start = start;
                this.end = end;
            }

            @Override
            public List<DetailDataDto> call() throws Exception {
                Object obj = null;
                List<DetailDataDto> retList = new ArrayList<DetailDataDto>();
                for (int i = start; i < end; i++) {
                    obj = list.get(i);
                    Thread myThread1 = new Thread(new HeadIn());
                    myThread1.start();
                    Thread myThread2 = new Thread(new AllIn());
                    myThread2.start();
                    Thread myThread3 = new Thread(new HeadNotIn());
                    myThread3.start();
                    latch.await();
// 我把三个多线程在这执行 
//这个可以填写自己的逻辑代码
                }
                //返回处理结果
                return retList;
            }
        }

        List<DetailDataDto> list = new ArrayList<DetailDataDto>(10000);
        int index = 0;
        ExecutorService ex = Executors.newFixedThreadPool(5);
        int dealSize = 2000;
        List<Future<List<DetailDataDto>>> futures = new ArrayList<>(5);
        //分配
        for (int i = 0; i <= 5; i++, index += dealSize) {
            int start = index;
            if (start >= list.size()) {
                break;
            }
            int end = start + dealSize;
            end = end > list.size() ? list.size() : end;
            futures.add(ex.submit(new Task(list, start, end)));
        }
        try {
            //处理
            List<Object> result = new ArrayList<>();
            for (Future<List<DetailDataDto>> future : futures) {
                //合并操作
                result.addAll(future.get());
            }
        } catch (Exception e) {
            e.printStackTrace();
        }

结果是很明显的 3秒  恩。。。这个还是可以接受,其他在硬件上优化  就是多线程分批跑数据 最后合并 在处理的过程。   记录的有些粗糙  不巧看到这篇贴子大神 勿喷。多谢

原文地址:https://www.cnblogs.com/blackCatFish/p/12572386.html