在外理日志时，除了访问日志外，还要处理运行时日志，该日志大都用程序写的，比如log4j。运行时日志跟访问日志最大的不同是，运行时日志是多行，也就是说，连续的多行才能表达一个意思。
在filter中,加入以下代码:
filter {
multiline { }
}
如果能按多行处理，那么把他们拆分到字段就很容易了。
字段属性：
对于multiline插件来说，有三个设置比较重要：negate , pattern 和 what
negate:类型是boolean默认为false
pattern：
必须设置，并且没有默认值，类型为string,要匹配下则表达式
what:
必须设置，并且没有默认值，可以为previous(之前的)或next
下面看看这个例子：
# cat logstash_multiline_shipper.conf
input {
file {
path => "/apps/logstash/conf/test/c.out"
type => "runtimelog"
codec => multiline {
pattern => "^["
negate => true
what => "previous"
}
start_position => "beginning"
sincedb_path => "/apps/logstash/logs/sincedb-access"
ignore_older =>0
}
}
output {
stdout{
codec => rubydebug
}
}
说明：区配以"["开头的行，如果不是，那肯定是属于前一行的
测试数据如下:
[16-04-12 03:40:01 DEBUG] model.MappingNode:- ['/store/shopclass'] matched over. [16-04-12 03:40:02 DEBUG] impl.JdbcEntityInserter:- from product_category product_category where product_category.PARENT_ID is null and product_category.STATUS = ? and product_category.DEALER_ID is null order by product_category.ORDERS asc [16-04-12 03:40:03 DEBUG] model.MappingNode:- ['/store/shopclass'] matched over. [16-04-12 03:40:04 DEBUG] model.MappingNode:- ['/store/shopclass'] matched over. [16-04-12 03:40:05 DEBUG] impl.JdbcEntityInserter:- from product_category product_category where product_category.PARENT_ID is null and product_category.STATUS = ? and product_category.DEALER_ID is null order by product_category.ORDERS desc [16-04-12 03:40:06 DEBUG] impl.JdbcEntityInserter:- from product_category product_category where product_category.PARENT_ID is null and product_category.STATUS = ? and product_category.DEALER_ID is null order by product_category.ORDERS asc [16-04-12 03:40:07 DEBUG] model.MappingNode:- ['/store/shopclass'] matched over.
启动logstash：
# ./../bin/logstash -f logstash_multiline_shipper.conf
Sending Logstash's logs to /apps/logstash/logs which is now configured via log4j2.properties
[2016-12-09T15:16:59,173][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2016-12-09T15:16:59,192][INFO ][logstash.pipeline ] Pipeline main started
[2016-12-09T15:16:59,263][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9601}
加入测试数据到被监控的log后，查看输出：
# ./../bin/logstash -f logstash_multiline_shipper.conf
Sending Logstash's logs to /apps/logstash/logs which is now configured via log4j2.properties
[2016-12-09T15:16:59,173][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2016-12-09T15:16:59,192][INFO ][logstash.pipeline ] Pipeline main started
[2016-12-09T15:16:59,263][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9601}
{
"path" => "/apps/logstash/conf/test/c.out",
"@timestamp" => 2016-12-09T07:21:15.403Z,
"@version" => "1",
"host" => "ofs1",
"message" => "# ./../bin/logstash -f logstash_multiline_shipper.conf
Sending Logstash's logs to /apps/logstash/logs which is now configured via log4j2.properties",
"type" => "runtimelog",
"tags" => [
[0] "multiline"
]
}
{
"path" => "/apps/logstash/conf/test/c.out",
"@timestamp" => 2016-12-09T07:21:15.409Z,
"@version" => "1",
"host" => "ofs1",
"message" => "[2016-12-09T15:16:59,173][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}",
"type" => "runtimelog",
"tags" => []
}
{
"path" => "/apps/logstash/conf/test/c.out",
"@timestamp" => 2016-12-09T07:21:15.410Z,
"@version" => "1",
"host" => "ofs1",
"message" => "[2016-12-09T15:16:59,192][INFO ][logstash.pipeline ] Pipeline main started",
"type" => "runtimelog",
"tags" => []
}