【开发过程的随笔总结,欢迎您的点评,可以做到更加严谨】
前段时间因为写一个业务,对mysql的插入量非常大,在idea开发环境,使得idea一直爆出outofmemry提示,并导致old 区一直涨。使用Mongodb 作为存储后,不仅插入速度大大提升,idea会卡死的顽疾也解决了。
Mongodb 能够非常简单的整合进项目
1 添加相应database的权限用户
2 mongoFactory构造mogotemplate链接对象
3 进行mongo的增删改查操作
- Mongo 的用户管理(授权操作需要有用户)
请参考下面mongo的用户管理
http://www.cnblogs.com/billerbird/p/5648428.html
下面是我的整合过程
- Mongo 与Spring的集成 配置文件bean.xml
<!-- mongodb -->
<bean id="mongodb" class="org.springframework.data.mongodb.core.MongoFactoryBean">
<property name="host" value="${mongodb.host}" />
<property name="port" value="${mongodb.port}" />
</bean>
<bean id="credentials" class="org.springframework.data.authentication.UserCredentials">
<constructor-arg name="username" value="${mongodb.username}"/>
<constructor-arg name="password" value="${mongodb.password}"/>
</bean>
<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate">
<constructor-arg name="mongo" ref="mongodb" />
<constructor-arg name="databaseName" value="${mongodb.databaseName}" />
<constructor-arg name="userCredentials" ref="credentials" />
</bean>
- mongotemplate的增删该查过程
- mongotemplate 增
@Autowired
private MongoTemplate mongoTemplate;
public void saveBatch(List<APIRequest> apiRequests) {
mongoTemplate.insert(apiRequests, APIConstant.API_REQUEST_COLLECTION);
}
- mongotemplate 查
@Autowired
private MongoTemplate mongoTemplate;
public List<APIRequest> getPricingRequest(String startDate, String endDate, String partnerId) {
Date start = DateUtil.parse(startDate,"yyyy-MM-dd");
Date end = DateUtil.parse(endDate,"yyyy-MM-dd");
Query query = query(where("partnerId").is(partnerId))
.addCriteria(where("requestType").is("precisePricing"))
.addCriteria(where("requestDate").gt(start).lte(end));
List<APIRequest> apiRequests = mongoTemplate.find(query, APIRequest.class, APIConstant.API_REQUEST_COLLECTION);
System.out.println("MongoDB的数据" + apiRequests.size());
return apiRequests;
}
- mongotemplate 分页
@Autowired
private MongoTemplate mongoTemplate;
/**分页查询 shopping数据**/
public List<APIRequest> getAllAPIRequest(String startDate, String endDate, String partnerId, Integer skip, Integer limit) {
Date start = DateUtil.parse(startDate,"yyyy-MM-dd");
Date end = DateUtil.parse(endDate,"yyyy-MM-dd");
Query query = query(where("partnerId").is(partnerId))
.addCriteria(where("requestType").is("shopping"))
.addCriteria(where("requestDate").gt(start).lte(end)).skip(skip).limit(limit);
List<APIRequest> apiRequests = mongoTemplate.find(query, APIRequest.class, APIConstant.API_REQUEST_COLLECTION);
System.out.println("MongoDB的数据" + apiRequests.size());
return apiRequests;
}
- mongotemplate group
@Autowired
private MongoTemplate mongoTemplate;
/**group 查询不能超过20000**/
public List<APIRequest> getAllAPIRequestByGroup(String startDate, String endDate, String partnerId) {
Date start = DateUtil.parse(startDate,"yyyy-MM-dd");
Date end = DateUtil.parse(endDate,"yyyy-MM-dd");
String keys = "origin;destination;journeyType";
GroupBy groupBy = GroupBy.key(keys.split(";")).initialDocument("{requestCount:0}").reduceFunction("function(curr, result){result.requestCount+=curr.requestCount;}");
Criteria criteria = where("partnerId").is(partnerId).and("requestType").is("shopping").and("requestDate").gt(start).lte(end);
GroupByResults<APIRequest> rst = mongoTemplate.group(criteria,APIConstant.API_REQUEST_COLLECTION,groupBy, APIRequest.class);
System.out.println("have result");
return null;
}
- mongotemplate mapReduce
@Autowired
private MongoTemplate mongoTemplate;
/**map reduce**/
public List<APIRequest> getShoppingByMapReduce(String startDate, String endDate, String partnerId) {
Date start = DateUtil.parse(startDate,"yyyy-MM-dd");
Date end = DateUtil.parse(endDate,"yyyy-MM-dd");
String keys = "origin;destination;journeyType";
GroupBy groupBy = GroupBy.key(keys.split(";")).initialDocument("{requestCount:0}").reduceFunction("function(curr, result){result.requestCount+=curr.requestCount;}");
String mapFuntion = "function() {
" +
"emit(
" +
"{"origin":this.origin,"destination":this.destination,"journeyType":this.journeyType},
" +
" this.requestCount);
" +
"}";
String reductFunction = "function(key, values) {
" +
" return Array.sum(values);
" +
"}";
Criteria criteria = where("partnerId").is(partnerId).and("requestType").is("shopping").and("requestDate").gt(start).lte(end);
Query query = new Query(criteria);
MapReduceOptions mrO=new MapReduceOptions();
mrO.outputCollection("map_reduce_result");
MapReduceResults<APIRequest> mapReduce = mongoTemplate.mapReduce(query,APIConstant.API_REQUEST_COLLECTION,mapFuntion,reductFunction,mrO,APIRequest.class);
List<DBObject> person_results = mongoTemplate.execute("map_reduce_result",new CollectionCallback<List<DBObject> >() {
public List<DBObject> doInCollection(DBCollection collection)
throws MongoException, DataAccessException {
DBCursor dbCursor=collection.find();
return dbCursor.toArray();
}
});
List<APIRequest> apiRequests = new ArrayList<>();
for (DBObject dbObject : person_results) {
APIRequest apiRequest = new APIRequest();
apiRequest.setRequestCount(((Double)dbObject.get("value")).intValue());
apiRequest.setOrigin((String) ((DBObject)dbObject.get("_id")).get("origin"));
apiRequest.setDestination((String) ((DBObject)dbObject.get("_id")).get("destination"));
apiRequest.setJourneyType((String) ((DBObject)dbObject.get("_id")).get("journeyType"));
apiRequests.add(apiRequest);
}
System.out.println("map reduce的结果:"+person_results.size() +" 转换后的结果:"+apiRequests.size());
return apiRequests;
}
- mongotemplate aggregate
aggregate 欢迎您贴代码
结束