<pre name="code" class="html"> 10.168.102.19 - - [22/Sep/2016:20:35:11 +0800] "POST /api/client/asset HTTP/1.1" 200 430 0.047 121.43.145.64
10.168.102.19 - - [22/Sep/2016:20:37:11 +0800] "POST /api/common/getdate HTTP/1.1" 200 171 0.049 121.43.145.64
80.82.78.38 - - [22/Sep/2016:20:37:47 +0800] "GET /cache/global/img/gs.gif HTTP/1.1" 404 - 0.000 -
10.168.102.19 - - [22/Sep/2016:20:37:51 +0800] "POST /api/common/getdate HTTP/1.1" 200 171 0.073 121.43.146.114
10.168.102.19 - - [22/Sep/2016:20:37:51 +0800] "POST /api/notice/page HTTP/1.1" 200 2339 0.092 121.43.146.114
jrhapt11:/usr/local/apache-tomcat-7.0.55_8082/logs> echo '80.82.78.38 - - [22/Sep/2016:20:37:47 +0800] "GET /cache/global/img/gs.gif HTTP/1.1" 404 - 0.000 -' >>localhost_access_log.2016-09-22.txt
jrhapt11:/usr/local/apache-tomcat-7.0.55_8082/logs>
导致 logstash 挂掉:
ArgumentError: comparison of String with 5 failed
>= at org/jruby/RubyComparable.java:155
>= at org/jruby/RubyString.java:1853
output_func at (eval):115
output_batch at /usr/local/logstash-2.3.4/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:293
each at org/jruby/RubyArray.java:1613
inject at org/jruby/RubyEnumerable.java:852
output_batch at /usr/local/logstash-2.3.4/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:287
worker_loop at /usr/local/logstash-2.3.4/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:232
start_workers at /usr/local/logstash-2.3.4/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:201
此时这个表达式匹配不上:
match => [
"message" , "s*%{IPORHOST:clientip}s+-s+-s+[%{HTTPDATE:time}]s+"%{WORD:verb}s+(?<api>(S+))?.*s+HTTP/%{NUMBER:httpversion}"s+%{NUMBER:http_status_code}s+%{NUMBER:bytes}s+(%{BASE16FLOAT:request_time})s+%{IPORHOST:remoteip}",
"message" ,"s*%{IPORHOST:clientip}s+-s+-s+[%{HTTPDATE:time}]s+"%{WORD:verb}s+(?<api>(S+))s+HTTP/%{NUMBER:httpversion}"s+%{NUMBER:http_status_code}s+%{NUMBER:bytes}s+(%{BASE16FLOAT:request_time})s+%{IPORHOST:remoteip}",
"message" ,"s*%{IPORHOST:clientip}s+-s+-s+[%{HTTPDATE:time}]s+"%{WORD:verb}s+(?<api>(S+))s+HTTP/%{NUMBER:httpversion}"s+%{NUMBER:http_status_code}s+-s+(%{BASE16FLOAT:request_time})s+%{IPORHOST:remoteip}"
需要补上一条:
s*%{IPORHOST:clientip}s+-s+-s+[%{HTTPDATE:time}]s+"%{WORD:verb}s+(?<api>(S+))s+HTTP/%{NUMBER:httpversion}"s+%{NUMBER:http_status_code}s+-s+(%{BASE16FLOAT:request_time})s+(%{IPORHOST:remoteip}|-)
此时正常:
"@version" => "1",
"@timestamp" => "2016-09-22T12:37:47.000Z",
"path" => "/data01/applog_backup/zjzc_log/zj-api-access02.2016-09-23",
"host" => "dr-mysql01.zjcap.com",
"type" => "zj_api_access",
"clientip" => "80.82.78.38",
"time" => "22/Sep/2016:20:37:47 +0800",
"verb" => "GET",
"api" => "/cache/global/img/gs.gif",
"httpversion" => "1.1",
"http_status_code" => "404",
"response_time" => 0.0,
"messager" => "zj_api_access- 80.82.78.38 - - [22/Sep/2016:20:37:47 +0800] "GET /cache/global/img/gs.gif HTTP/1.1" 404 - 0.000 -"
}