redis 大批量数据插入导致MISCONF Redis is configured to save RDB snapshots的解决

PS:之前写过一遍,那个方法没有彻底解决,现找到真正的解决方法

环境:redis 3.2.100 windows版(注意!!!这是关键),win10,redis客户端spring boot 2.0.7,以及配对的spring data redis

某功能会频繁地大批量地往redis写入数据,数据量大概10秒内超过10000条

插入时可能会报错,报错时间点不定,大概都是运行一段时间后报错,错误信息如下:

nested exception is io.lettuce.core.RedisCommandExecutionException: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.

同时在redis的log找到如下报错:

=== REDIS BUG REPORT START: Cut & paste starting from here ===
Redis version: 3.2.100
[18808] 15 Mar 11:12:49.805 # --- EXCEPTION_ACCESS_VIOLATION
[18808] 15 Mar 11:12:49.805 # --- STACK TRACE
redis-server.exe!LogStackTrace(c:
elease
edissrcwin32_interopwin32_stacktrace.cpp:95)(0x0014E250, 0x0014FF90, 0x00000000, 0x4013A7F8)
redis-server.exe!UnhandledExceptiontHandler(c:
elease
edissrcwin32_interopwin32_stacktrace.cpp:185)(0x00000000, 0x00000000, 0x00000000, 0x38D470E0)
KERNELBASE.dll!UnhandledExceptionFilter(c:
elease
edissrcwin32_interopwin32_stacktrace.cpp:185)(0x401555B4, 0x38E3C15C, 0x00000000, 0x00000000)
ntdll.dll!memset(c:
elease
edissrcwin32_interopwin32_stacktrace.cpp:185)(0x0014EEB0, 0x00000000, 0x40140E48, 0x0014EEB0)
ntdll.dll!_C_specific_handler(c:
elease
edissrcwin32_interopwin32_stacktrace.cpp:185)(0x00000000, 0x0014E3C0, 0x0014E9C0, 0x00000000)
ntdll.dll!_chkstk(c:
elease
edissrcwin32_interopwin32_stacktrace.cpp:185)(0x0014E3C0, 0x0014E9C0, 0x3623AB10, 0x00000000)
ntdll.dll!RtlWalkFrameChain(c:
elease
edissrcwin32_interopwin32_stacktrace.cpp:185)(0x00000000, 0x511B7D55, 0x0014F220, 0x6506FC19)
ntdll.dll!KiUserExceptionDispatcher(c:
elease
edissrcwin32_interopwin32_stacktrace.cpp:185)(0x40094F01, 0x0014F1F0, 0x400BFA57, 0x0014F2E0)
redis-server.exe!dictSdsHash(c:
elease
edissrcserver.c:496)(0x0014F1F0, 0x400BFA57, 0x0014F2E0, 0x00000001)
redis-server.exe!dictFind(c:
elease
edissrcdict.c:517)(0x0014F220, 0x00000001, 0x0000002B, 0x00000001)
redis-server.exe!getExpire(c:
elease
edissrcdb.c:871)(0x00000001, 0x0014F2E0, 0x0014F2E0, 0x03C07040)
redis-server.exe!rdbSaveRio(c:
elease
edissrc
db.c:814)(0x40167210, 0x00DD0000, 0x00000005, 0x011D2754)
redis-server.exe!rdbSave(c:
elease
edissrc
db.c:884)(0x00DD0000, 0x00DD0000, 0x5C8B2352, 0x00000005)
redis-server.exe!QForkChildInit(c:
elease
edissrcwin32_interopwin32_qfork.cpp:337)(0x00000005, 0x00000000, 0x004BEF40, 0x00000005)
redis-server.exe!QForkStartup(c:
elease
edissrcwin32_interopwin32_qfork.cpp:515)(0x00000006, 0x00000000, 0x00000000, 0x004A5550)
redis-server.exe!main(c:
elease
edissrcwin32_interopwin32_qfork.cpp:1240)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
redis-server.exe!__tmainCRTStartup(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
KERNEL32.DLL!BaseThreadInitThunk(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
ntdll.dll!RtlUserThreadStart(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
ntdll.dll!RtlUserThreadStart(f:ddvctoolscrtcrtw32startupcrt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
[18808] 15 Mar 11:12:50.398 # --- INFO OUTPUT
[5660] 15 Mar 11:12:55.080 # fork operation failed
[5660] 15 Mar 11:12:55.226 # Background saving terminated by signal 1
[5660] 15 Mar 11:12:55.353 * 10000 changes in 60 seconds. Saving...

网上能找到很多解决方法(具体执行baidu),一种主流的方法是redis配置stop-writes-on-bgsave-error设为false,但这种方法只是让出错不停止,错误还是存在的,所以不想用这种方法

另外原因大概也清楚:就是redis写入数据会先写到内存,然后每隔一段时间会把内存数据写入rdb(一个文件数据库),以实现不会因为关机内存数据丢失就没了数据。问题是在写入rdb时出错

但不知道写入rdb出错原因在哪,网上很多说是内存不够,我看了内存占用也很低,只用了200+m

此问题直接原因应该是写入rdb时出错,但根本原因在于redis本身有bug

redis只有linux版,而windows版是微软自己做的,而且也停止更新,大概因此有bug也是正常吧

解决方法是redis(服务端)使用3.0.504,此版本已解决此bug

原文地址:https://www.cnblogs.com/cannel/p/11078096.html