对纯for循环进行并行流和线程池优化简记

最近在项目中需要根据角色拿对应任务（activiti 中的内容），再根据任务拿工单集（任务跟工单为多对一的关系，所以还需过滤掉重复的工单），获得最终结果集的耗时较大，所以考虑从多线程方向优化。

这些操作在一个for循环里涉及两处数据库查询IO，但IO的阻塞不严重，不属于IO密集型的多线程设计条件，所以采用线程池，优化的效果不是很好（对于IO阻塞系数较小的应用优化，多线程的使用，CPU在线程非阻塞的状态下切换上下文，消耗很大，同时还得加上创建线程的开销）；采用concurrent包的Stream来进行并发流式的处理，优化效果还比较可观。

现针对三种不同的实现方式和执行效果通列如下：

原始for循环：

//同组测试数据，2038ms 结果集稳定

方法体：

List<WorkOrder> content = new ArrayList<WorkOrder>();
Map<String,Integer> widMap=new HashMap<String,Integer>();

for (String id : ids) {
List<Task> tasks = taskService.createTaskQuery()
.taskCandidateGroup(id).active().list();

//同组测试数据，2038ms 结果集稳定
for (Task task : tasks) {
ProcessInstance pi = runtimeService
.createProcessInstanceQuery()
.processInstanceId(task.getProcessInstanceId())
.singleResult();
String workOrderId = pi.getBusinessKey();
if(!widMap.containsKey(workOrderId)){
WorkOrder wo = workOrderService.findStartedWorkOrderAnd(
workOrderId, Constants.WORKORDER.CONFIGURED,
Constants.WORKORDER.INPROCESSED,
Constants.WORKORDER.UNFILLED);
if(wo!=null){
content.add(wo);
}
widMap.put(workOrderId, 1);
}
}

}

并行流：

//并行流，712ms 结果集稳定

方法体：

List<WorkOrder> content = new ArrayList<WorkOrder>();
Map<String,Integer> widMap=new HashMap<String,Integer>();

for (String id : ids) {
List<Task> tasks = taskService.createTaskQuery()
.taskCandidateGroup(id).active().list();

//并行流，712ms 结果集稳定
tasks.parallelStream().map(t->{
final String wid=runtimeService.createProcessInstanceQuery()
.processInstanceId(t.getProcessInstanceId())
.singleResult()
.getBusinessKey();
if(!widMap.containsKey(wid)){
WorkOrder wo = workOrderService.findStartedWorkOrderAnd(
wid, Constants.WORKORDER.CONFIGURED,
Constants.WORKORDER.INPROCESSED,
Constants.WORKORDER.UNFILLED);
if(wo!=null){
content.add(wo);
}
widMap.put(wid, 1);
}
return 1;
}
).count();

}

spring 下的异步线程池：

//线程池,2627ms ,结果集不稳定

方法体：

BlockingDeque<Future<WorkOrder>> resultContainer=new LinkedBlockingDeque<Future<WorkOrder>>();

Date start =new Date();
for (String id : ids) {
List<Task> tasks = taskService.createTaskQuery()
.taskCandidateGroup(id).active().list();
//线程池,2627ms ,结果集不稳定(最终结果可能与实际不符，存在漏数据，无数据现象)
int i=0;
for(Task task:tasks){
try {
resultContainer.put(workOrderAsynHander.gainWorkOrderByTask(task, id));
i++;
System.out.println("put successful,thread "+i+"executed");
} catch (InterruptedException e) {

e.printStackTrace();
}
}
int j=0;
while(!resultContainer.isEmpty()){
try {
Future<WorkOrder> wf=resultContainer.remove();
WorkOrder w= wf == null ? null : wf.get();
if(w!=null){
content.add(w);
}
j++;
System.out.println("get successful,thread "+j+"executed");
} catch (InterruptedException | ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}

异步的方法类：

import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.Future;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;


import org.activiti.engine.RuntimeService;
import org.activiti.engine.TaskService;
import org.activiti.engine.task.Task;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.annotation.Async;
import org.springframework.scheduling.annotation.AsyncResult;
import org.springframework.stereotype.Component;


@Component
public class WorkOrderAsynHander {
private static final ConcurrentMap<String,Object> workOrderIds=new ConcurrentHashMap<String,Object>();
private static Lock lock=new ReentrantLock();
@Autowired
private RuntimeService runtimeService;
@Autowired
private WorkOrderService workOrderService;

@Async("extraTaskAsynPool")
public Future<WorkOrder> gainWorkOrderByTask(Task task,String id){


String wid=runtimeService.createProcessInstanceQuery()
.processInstanceId(task.getProcessInstanceId())
.singleResult()
.getBusinessKey();
if(!workOrderIds.containsKey(wid)){
workOrderIds.put(wid, 1);
WorkOrder wo = workOrderService.findStartedWorkOrderAnd(
wid, Constants.WORKORDER.CONFIGURED,
Constants.WORKORDER.INPROCESSED,
Constants.WORKORDER.UNFILLED);
return new AsyncResult<WorkOrder>(wo);
}


return null;
}
}

线程池定义：

import java.util.concurrent.Executor;
import java.util.concurrent.ThreadPoolExecutor;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

@Configuration  
@EnableAsync  
public class TaskExecutorPool {

@Value("${extraTaskAsynPool.corePoolSize}")
private int corePoolSize;
@Value("${extraTaskAsynPool.maxPoolSize}")
private int maxPoolSize;
@Value("${extraTaskAsynPool.queueCapacity}")
private int queueCapacity;
@Value("${extraTaskAsynPool.keepAliveSeconds}")
private int keepAliveSeconds;

@Bean
public Executor extraTaskAsynPool(){
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();  
        executor.setCorePoolSize(corePoolSize);  
        executor.setMaxPoolSize(maxPoolSize);  
        executor.setQueueCapacity(queueCapacity);  
        executor.setKeepAliveSeconds(keepAliveSeconds);  
        executor.setThreadNamePrefix("extra_");  
  
        // rejection-policy：当pool已经达到max size的时候，处理新任务策略  
        // CALLER_RUNS：不在新线程中执行任务，而是由调用者所在的线程来执行  
        executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());  
        executor.initialize();  
        return executor; 
}
}

spring boot  下的异步启动配置：

@SpringBootApplication
@EnableScheduling
@EnableAsync
public class MESApplication {
public static void main(String[] args) {
SpringApplication.run(MESApplication.class, args);
}

}