最近发现kunshan这家的生产环境进程mycat cpu 800%然后mycat基本上就假死了，无法查询了。
后来回忆部署这家生产环境的时候，因为服务器内存大于等于64G 然后mycat 就启动不了，然后配置了
<property name="useOffHeapForMerge">0</property> 然后就可以启动了
这就是上面mycat cpu 800%的原因
这种方法能够解决了笔者启动报错问题，但是禁用堆外内存牺牲性能，并没有从根本上解决问题
后来找了网上一个文章，说的是将 wrapper.java.additional.5=-XX:MaxDirectMemorySize=2G改为8G加大MaxDirectMemorySize 内存，再次启动，问题得到解决。
https://blog.csdn.net/u013716179/article/details/89886452
以前在进行Mycat调优的时候设置过Mycat的useOffHeapForMerge参数，发现在大数据量查询聚合的时候启用堆外内存对于查询性能提升非常明显，但是最近在新的生产环境部署Mycat启动的时候总是报错java.lang.NumberFormatException: Size must be specified as bytes (b), kibibytes (k), mebibytes (m), gibibytes (g), tebibytes (t), or pebibytes§. E.g. 50b, 100k, or 250m.
Failed to parse byte string: -1315333734B…类似这种，如下
2019-05-06 16:26:27.082 ERROR [WrapperSimpleAppMain] (io.mycat.MycatStartup.main(MycatStartup.java:63)) - 2019-05-06 16:26:27 startup error
java.lang.NumberFormatException: Size must be specified as bytes (b), kibibytes (k), mebibytes (m), gibibytes (g), tebibytes (t), or pebibytes(p). E.g. 50b, 100k, or 250m.
Failed to parse byte string: -1315333734B
at io.mycat.memory.unsafe.utils.JavaUtils.byteStringAs(JavaUtils.java:223) ~[Mycat-server-1.6.7.1-release.jar:?]
at io.mycat.memory.unsafe.utils.JavaUtils.byteStringAsBytes(JavaUtils.java:234) ~[Mycat-server-1.6.7.1-release.jar:?]
at io.mycat.memory.unsafe.utils.MycatPropertyConf.byteStringAsBytes(MycatPropertyConf.java:92) ~[Mycat-server-1.6.7.1-release.jar:?]
at io.mycat.memory.unsafe.utils.MycatPropertyConf.getSizeAsBytes(MycatPropertyConf.java:50) ~[Mycat-server-1.6.7.1-release.jar:?]
at io.mycat.memory.unsafe.memory.mm.MemoryManager.<init>(MemoryManager.java:30) ~[Mycat-server-1.6.7.1-release.jar:?]
at io.mycat.memory.unsafe.memory.mm.ResultMergeMemoryManager.<init>(ResultMergeMemoryManager.java:15) ~[Mycat-server-1.6.7.1-release.jar:?]
at io.mycat.memory.MyCatMemory.<init>(MyCatMemory.java:125) ~[Mycat-server-1.6.7.1-release.jar:?]
at io.mycat.MycatServer.startup(MycatServer.java:388) ~[Mycat-server-1.6.7.1-release.jar:?]
at io.mycat.MycatStartup.main(MycatStartup.java:58) ~[Mycat-server-1.6.7.1-release.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_161]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_161]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_161]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_161]
at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:240) ~[wrapper.jar:3.2.3]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
对于此异常，笔者上网查询过解决方案，几乎全部是将useOffHeapForMerge置0
<property name="useOffHeapForMerge">1</property>
1
确实，这种方法能够解决了笔者启动报错问题，但是禁用堆外内存牺牲性能，并没有从根本上解决问题
至此，只能去github拉取mycat源码看下问题究竟出在哪里，直接源码里搜异常堆栈信息，找到包异常的地方在JavaUtils.java类的byteStringAs方法中
/**
* Convert a passed byte string (e.g. 50b, 100kb, or 250mb) to the given. If no suffix is
* provided, a direct conversion to the provided unit is attempted.
*/
public static long byteStringAs(String str, ByteUnit unit) {
String lower = str.toLowerCase().trim();
try {
Matcher m = Pattern.compile("([0-9]+)([a-z]+)?").matcher(lower);
Matcher fractionMatcher = Pattern.compile("([0-9]+\.[0-9]+)([a-z]+)?").matcher(lower);
if (m.matches()) {
long val = Long.parseLong(m.group(1));
String suffix = m.group(2);
// Check for invalid suffixes
if (suffix != null && !byteSuffixes.containsKey(suffix)) {
throw new NumberFormatException("Invalid suffix: "" + suffix + """);
}
// If suffix is valid use that, otherwise none was provided and use the default passed
return unit.convertFrom(val, suffix != null ? byteSuffixes.get(suffix) : unit);
} else if (fractionMatcher.matches()) {
throw new NumberFormatException("Fractional values are not supported. Input was: "
+ fractionMatcher.group(1));
} else {
throw new NumberFormatException("Failed to parse byte string: " + str);
}
} catch (NumberFormatException e) {
String byteError = "Size must be specified as bytes (b), " +
"kibibytes (k), mebibytes (m), gibibytes (g), tebibytes (t), or pebibytes(p). " +
"E.g. 50b, 100k, or 250m.";
throw new NumberFormatException(byteError + "
" + e.getMessage());
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
首先我们可以看到，str解析异常会有三种情况，我们报的是最后一种即else里面的Failed to parse byte string，那么看看日志我们传进来str是’-1315333734B’，首先会在方法的第一部分转小写，其次前两个正则匹配整数或者小数+字母，前两种均没匹配上，出现最后一种情况原因只能是字符串开头非数字，且是一个负数，那为什么会出现负数呢，我们接着看str如何生成的，进入MycatMemory类
public MyCatMemory(SystemConfig system,long totalNetWorkBufferSize) throws NoSuchFieldException, IllegalAccessException {
this.system = system;
LOGGER.info("useOffHeapForMerge = " + system.getUseOffHeapForMerge());
LOGGER.info("memoryPageSize = " + system.getMemoryPageSize());
LOGGER.info("spillsFileBufferSize = " + system.getSpillsFileBufferSize());
LOGGER.info("useStreamOutput = " + system.getUseStreamOutput());
LOGGER.info("systemReserveMemorySize = " + system.getSystemReserveMemorySize());
LOGGER.info("totalNetWorkBufferSize = " + JavaUtils.bytesToString2(totalNetWorkBufferSize));
LOGGER.info("dataNodeSortedTempDir = " + system.getDataNodeSortedTempDir());
this.conf = new MycatPropertyConf();
numCores = Runtime.getRuntime().availableProcessors();
this.systemReserveBufferSize = JavaUtils.
byteStringAsBytes(system.getSystemReserveMemorySize());
this.memoryPageSize = JavaUtils.
byteStringAsBytes(system.getMemoryPageSize());
this.spillsFileBufferSize = JavaUtils.
byteStringAsBytes(system.getSpillsFileBufferSize());
/**
* 目前merge，order by ，limit 没有使用On Heap内存
*/
long maxOnHeapMemory = (Platform.getMaxHeapMemory()-systemReserveBufferSize);
assert maxOnHeapMemory > 0;
resultSetBufferSize =
(long)((Platform.getMaxDirectMemory()-2*totalNetWorkBufferSize)*DIRECT_SAFETY_FRACTION);
assert resultSetBufferSize > 0;
/**
* mycat.merge.memory.offHeap.enabled
* mycat.buffer.pageSize
* mycat.memory.offHeap.size
* mycat.merge.file.buffer
* mycat.direct.output.result
* mycat.local.dir
*/
if(system.getUseOffHeapForMerge()== 1){
conf.set("mycat.memory.offHeap.enabled","true");
}else{
conf.set("mycat.memory.offHeap.enabled","false");
}
if(system.getUseStreamOutput() == 1){
conf.set("mycat.stream.output.result","true");
}else{
conf.set("mycat.stream.output.result","false");
}
if(system.getMemoryPageSize() != null){
conf.set("mycat.buffer.pageSize",system.getMemoryPageSize());
}else{
conf.set("mycat.buffer.pageSize","32k");
}
if(system.getSpillsFileBufferSize() != null){
conf.set("mycat.merge.file.buffer",system.getSpillsFileBufferSize());
}else{
conf.set("mycat.merge.file.buffer","32k");
}
conf.set("mycat.pointer.array.len","1k")
.set("mycat.memory.offHeap.size", JavaUtils.bytesToString2(resultSetBufferSize));
LOGGER.info("mycat.memory.offHeap.size: " +
JavaUtils.bytesToString2(resultSetBufferSize));
resultMergeMemoryManager =
new ResultMergeMemoryManager(conf,numCores,maxOnHeapMemory);
serializerManager = new SerializerManager();
blockManager = new DataNodeDiskManager(conf,true,serializerManager);
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
可以看到，是根据Platform.getMaxDirectMemory()、totalNetWorkBufferSize、DIRECT_SAFETY_FRACTION三个参数程程的
resultSetBufferSize =
(long)((Platform.getMaxDirectMemory()-2*totalNetWorkBufferSize)*DIRECT_SAFETY_FRACTION);
1
2
而产生负数的原因无外乎前面值太小，或者后面值太大，而Platform.getMaxDirectMemory()取得值是MAX_DIRECT_MEMORY，这是wrapper.conf种的一个参数
#********************************************************************
# Wrapper Properties
#********************************************************************
# Java Application
wrapper.java.command=java
wrapper.working.dir=..
# Java Main class. This class must implement the WrapperListener interface
# or guarantee that the WrapperManager class is initialized. Helper
# classes are provided to do this for you. See the Integration section
# of the documentation for details.
wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp
set.default.REPO_DIR=lib
set.APP_BASE=.
# Java Classpath (include wrapper.jar) Add class path elements as
# needed starting from 1
wrapper.java.classpath.1=lib/wrapper.jar
wrapper.java.classpath.2=conf
wrapper.java.classpath.3=%REPO_DIR%/*
# Java Library Path (location of Wrapper.DLL or libwrapper.so)
wrapper.java.library.path.1=lib
# Java Additional Parameters
#wrapper.java.additional.1=
wrapper.java.additional.1=-DMYCAT_HOME=.
wrapper.java.additional.2=-server
wrapper.java.additional.3=-XX:MaxPermSize=64M
wrapper.java.additional.4=-XX:+AggressiveOpts
wrapper.java.additional.5=-XX:MaxDirectMemorySize=2G
wrapper.java.additional.6=-Dcom.sun.management.jmxremote
wrapper.java.additional.7=-Dcom.sun.management.jmxremote.port=1984
wrapper.java.additional.8=-Dcom.sun.management.jmxremote.authenticate=false
wrapper.java.additional.9=-Dcom.sun.management.jmxremote.ssl=false
wrapper.java.additional.10=-Xmx4G
wrapper.java.additional.11=-Xms1G
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
这里笔者将wrapper.java.additional.5=-XX:MaxDirectMemorySize=2G改为8G加大MaxDirectMemorySize内存，再次启动，问题得到解决。
————————————————
版权声明：本文为CSDN博主「张老七没脾气」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/u013716179/article/details/89886452