简介：
ELK Stack 安装文档，这次都使用最新版本(5.2.2)、RPM 包的方式搭建 ELK Stack。
下载地址:
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.2.2.rpm
https://artifacts.elastic.co/downloads/logstash/logstash-5.2.2.rpm
https://artifacts.elastic.co/downloads/kibana/kibana-5.2.2-x86_64.rpm
jre: http://javadl.oracle.com/webapps/download/AutoDL?BundleId=216423
jdk: http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
# 如果只是需要 java 环境，那么安装 jre 即可，如还需编译 java 包就需要安装 jdk 了。
# 运行 Elasticsearch jre 即可
一、安装
shell > rpm -e elasticsearch Stopping elasticsearch service... OK warning: /etc/sysconfig/elasticsearch saved as /etc/sysconfig/elasticsearch.rpmsave warning: /etc/init.d/elasticsearch saved as /etc/init.d/elasticsearch.rpmsave warning: /etc/elasticsearch/elasticsearch.yml saved as /etc/elasticsearch/elasticsearch.yml.rpmsave Deleting log directory... OK Deleting plugins directory... OK shell > rpm -e kibana Stopping kibana service... OK warning: /opt/kibana/config/kibana.yml saved as /opt/kibana/config/kibana.yml.rpmsave
# 我之前使用 2.4.1 版本，也是通过 rpm 安装的，需要先卸载。
shell > rm -rf /etc/sysconfig/elasticsearch.rpmsave shell > rm -rf /etc/init.d/elasticsearch.rpmsave shell > rm -rf /etc/elasticsearch/elasticsearch.yml.rpmsave shell > rm -rf /opt/kibana/config/kibana.yml.rpmsave
# 可以看到卸载的时候，这些文件没有被删除，强迫症的我是不允许这些文件存在的。
shell > cd /usr/local/src; rpm -ivh elasticsearch-5.2.2.rpm logstash-5.2.2.rpm kibana-5.2.2-x86_64.rpm
# 由于是测试一下新版本，所以都装在了一台服务器上。
二、配置
1、Elasticsearch
shell > grep -vP '^#|^$' /etc/elasticsearch/elasticsearch.yml # 集群名称 cluster.name: elk # 节点名称 node.name: node-1 # 数据路径 path.data: /data/elast/data # 日志路径 path.logs: /data/elast/logs # 启动时锁住内存，防止数据被交换到 SWAP bootstrap.memory_lock: true # 监听地址 network.host: 0.0.0.0 # 与其余节点通信地址 network.publish_host: 10.127.174.217 # 开启 HTTP 协议 http.port: 9200 # 解决启动报错 bootstrap.system_call_filter: false shell > mkdir -p /data/elast/{data,logs} shell > chown -R elasticsearch.elasticsearch /data/elast
# 创建数据、日志目录
2、Logstash
shell > vim /etc/logstash/conf.d/for_elk.conf # 输入插件，这里从 redis 中读取数据 input { redis { data_type => "list" key => "for_elk" host => "10.217.79.61" port => 6379 threads => 10 } } # 过滤插件，按需切割日志、加减字段等 filter { mutate { split => ["message", "|"] add_field => {"clientip" => "%{message[0]}"} add_field => {"localtime" => "%{message[1]}"} add_field => {"api" => "%{message[2]}"} add_field => {"request_all" => "%{message[3]}"} add_field => {"http_code" => "%{message[4]}"} add_field => {"request_body" => "%{message[6]}"} add_field => {"request_time" => "%{message[11]}"} } date { match => ["localtime", "dd/MMM/yyyy:HH:mm:ss Z"] } geoip { source => "clientip" fields => ["city_name", "latitude", "longitude"] } kv { source => "request_body" field_split => "&" remove_field => "host" remove_field => "path" remove_field => "message" remove_field => "request_all" remove_field => "request_body" } mutate { convert => [ "id", "integer", "cid", "integer", "tid", "integer", "vid", "integer", "version", "float", "http_code", "integer", "request_time", "float" ] } } # 输出插件 output { elasticsearch { hosts => ["10.127.174.217:9200"] index => "logstash-%{+YYYY.MM.dd}" template_overwrite => true } # stdout { # codec => rubydebug # } }
# 可以测试能否从 redis 拿到数据，然后在做 filter，最后测试能否写入 elasticsearch
3、Kibana
shell > /etc/kibana/kibana.yml
# Kibana 其实不用修改，暂时采用默认配置即可
三、启动
1、Elasticsearch
shell > /etc/init.d/elasticsearch start
2、Logstash
shell > /usr/share/logstash/bin/logstash --path.settings /etc/logstash > /dev/null &
3、Kibana
shell > /etc/init.d/kibana start
四、访问
# http://x.x.x.x:5601 即可，根据 index 建立索引，嗯 确实比 K4 漂亮
五、插件安装
1、Elasticsearch head (从 5.0 起，该插件以一个单独的服务运行)
shell > cd /usr/local shell > git clone git://github.com/mobz/elasticsearch-head.git shell > cd elasticsearch-head shell > npm install shell > vim Gruntfile.js connect: { server: { options: { hostname: '0.0.0.0', port: 9100, base: '.', keepalive: true } } }
# 默认只监听 127.0.0.1，所以要加上 hostname: '0.0.0.0'
shell > ./node_modules/grunt/bin/grunt server > /dev/null & shell > vim /etc/elasticsearch/elasticsearch.yml # head plugin http.cors.enabled: true http.cors.allow-origin: "*"
# elasticsearch 5.x 需要设置该参数，否则无法 head 无法连接 es
# 你可能注意到 es 集群状态为 yellow，不要慌...
# 那是因为副本不可用，因为只有一个 es 节点，而副本不能在本机，不碍事 ！
2、IK Analysis for Elasticsearch
shell > wget http://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.5.0/binaries/apache-maven-3.5.0-bin.tar.gz shell > tar zxf apache-maven-3.5.0-bin.tar.gz -C /usr/local shell > echo -e ' export JAVA_HOME=/usr/java/default' >> /etc/profile && source /etc/profile shell > wget https://github.com/medcl/elasticsearch-analysis-ik/archive/v5.2.2.zip shell > unzip v5.2.2.zip shell > cd elasticsearch-analysis-ik-5.2.2 shell > /usr/local/apache-maven-3.5.0/bin/mvn package shell > unzip target/releases/elasticsearch-analysis-ik-5.2.2.zip -d /usr/share/elasticsearch/plugins/ik shell > /usr/share/elasticsearch/bin/elasticsearch-plugin list ik shell > /etc/init.d/elasticsearch restart
附件：
1、Elasticsearch 启动报错
> bootstrap.memory_lock: true 参数导致
memory locking requested for elasticsearch process but memory is not locked
解决方法:
shell > vim /etc/security/limits.conf # allow user 'elasticsearch' mlockall elasticsearch soft memlock unlimited elasticsearch hard memlock unlimited
> CentOS 6.x 不支持 CONFIG_SECCOMP 导致
[2017-03-01T12:00:53,986][WARN ][o.e.b.JNANatives ] unable to install syscall filter: java.lang.UnsupportedOperationException: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed at org.elasticsearch.bootstrap.SystemCallFilter.linuxImpl(SystemCallFilter.java:363) ~[elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.bootstrap.SystemCallFilter.init(SystemCallFilter.java:638) ~[elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.bootstrap.JNANatives.tryInstallSystemCallFilter(JNANatives.java:215) [elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.bootstrap.Natives.tryInstallSystemCallFilter(Natives.java:99) [elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:110) [elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:203) [elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) [elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) [elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) [elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) [elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) [elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.cli.Command.main(Command.java:88) [elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) [elasticsearch-5.2.2.jar:5.2.2] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) [elasticsearch-5.2.2.jar:5.2.2] bootstrap checks failed system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
解决方法：
shell > vim /etc/elasticsearch/elasticsearch.yml
bootstrap.system_call_filter: falses
> /etc/security/limits.d/90-nproc.conf 默认参数过低导致启动失败
[2017-07-06T14:57:47,840][ERROR][o.e.b.Bootstrap ] [node01] node validation exception bootstrap checks failed max number of threads [1024] for user [elasticsearch] is too low, increase to at least [2048]
解决方法：
shell > vim /etc/security/limits.d/90-nproc.conf * soft nproc 2048 root soft nproc unlimited # 将原 1024 改为 2048