每日nginx的日志切分及爬虫分析

ref:

good article

http://1399.us/html/2010/04/19/73.html

http://www.blogjava.net/frankiegao123/archive/2010/06/01/322490.html

张宴slide:http://www.slideshare.net/pigso/nginx

cut-log.sh脚本如下

#!/bin/bash

## 零点执行该脚本

## Nginx 日志文件所在的目录

LOGS_PATH=/usr/local/nginx/logs

## 获取昨天的 yyyy-MM-dd

YESTERDAY=$(date -d "yesterday" +%Y-%m-%d)

## 移动文件

mv ${LOGS_PATH}/access.log ${LOGS_PATH}/access_${YESTERDAY}.log

## 向 Nginx 主进程发送 USR1 信号。USR1 信号是重新打开日志文件

kill -USR1 $(cat /usr/local/nginx/logs/nginx.pid)

进入root用户，给脚本赋予权限

chrom 777 /bin/bash /home/mlzboy/my/idea2/deploy/nginx/cut-log.sh

su root

crontab -e

16 17 * * * /bin/bash /home/mlzboy/my/idea2/deploy/nginx/cut-log.sh

extra learing

http://www.ibm.com/developerworks/cn/linux/l-job-scheduling.html

http://www.trucy.org/blog/eric/archives/2008/02/linux-cron-howto.html

linux 百度/Google(谷歌) 蜘蛛爬行 分析命令

http://1399.us/html/2010/04/19/7.html