环境
kibana6.4.3
logstash6.4.3
es6.4.3
日志收集服务器内存8g
前景
elk 服务器搭建好的前几天服务都正常
大概一周以后出现kibana 连接不上，登录服务器发现只有 kibana 挂掉，es 和 logstash 都正常运行，排查日志发现：
<--- Last few GCs ---> [7232:0x28c9b10] 1349425042 ms: Mark-sweep 1360.5 (1431.3) -> 1360.5 (1431.3) MB, 1475.4 / 0.0 ms allocation failure GC in old space requested [7232:0x28c9b10] 1349426671 ms: Mark-sweep 1360.5 (1431.3) -> 1360.5 (1431.3) MB, 1628.7 / 0.1 ms last resort GC in old space requested [7232:0x28c9b10] 1349428162 ms: Mark-sweep 1360.5 (1431.3) -> 1360.5 (1431.3) MB, 1490.7 / 0.0 ms last resort GC in old space requested <--- JS stacktrace ---> ==== JS stack trace ========================================= Security context: 0x2e54d3625879 <JSObject> 2: /* anonymous */(aka /* anonymous */) [/opt/kibana-6.4.3/src/server/status/lib/cgroup.js:70] [bytecode=0x1d79566d9321 offset=20](this=0x754e6a822d1 <undefined>,controller=0x31e957539041 <String[4]: pids>) 3: arguments adaptor frame: 3->1 4: forEach(this=0x31e957539061 <JSArray[1]>) 5: /* anonymous */(aka /* anonymous */) [/opt/kibana-6.4.3/src/server/status/lib/cgroup.js:69] [b... FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory 1: node::Abort() [./../node/bin/node] 2: 0x8c21ec [./../node/bin/node] 3: v8::Utils::ReportOOMFailure(char const*, bool) [./../node/bin/node] 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [./../node/bin/node] 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [./../node/bin/node] 6: v8::internal::HashTable<v8::internal::StringTable, v8::internal::StringTableShape>::NewInternal(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [./../node/bin/node] 7: v8::internal::HashTable<v8::internal::StringTable, v8::internal::StringTableShape>::New(v8::internal::Isolate*, int, v8::internal::PretenureFlag, v8::internal::MinimumCapacity) [./../node/bin/node] 8: v8::internal::HashTable<v8::internal::StringTable, v8::internal::StringTableShape>::EnsureCapacity(v8::internal::Handle<v8::internal::StringTable>, int, v8::internal::PretenureFlag) [./../node/bin/node] 9: v8::internal::StringTable::LookupString(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>) [./../node/bin/node] 10: v8::internal::LookupIterator::PropertyOrElement(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, bool*, v8::internal::LookupIterator::Configuration) [./../node/bin/node] 11: v8::internal::Runtime_SetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [./../node/bin/node] 12: 0x22a4d09842fd
发现是前端 js 报的内存 oom 异常，通过网上资料发现node.js 的默认内存大小为1.4g
Node 中通过 JavaScript 使用内存时只能使用部分内存（64位系统下约为1.4 GB，32位系统下约为0.7 GB），这就是我们编译项目时为什么会出现内存泄露了，因为前端项目如果非常的庞大，webpack 编译时就会占用很多的系统资源，如果超出了V8对 Node 默认的内存限制大小就会出现node内存溢出的报错:CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
进入 kibana/bin 目录（也可以设置在 node 的环境变量里，我是加在 kibana 文件中）
vi kibana
添加内存空间的环境变量（以下 size 为1g，根据实际情况修改）
NODE_OPTIONS="--max-old-space-size=12288"
https://segmentfault.com/a/1190000017972349
https://banyudu.com/posts/dynamic-change-nodejs-memory-limit.3c34c2
https://forum.search-guard.com/t/kibana-7-6-memory-issue-with-sg/1814