模拟在线更换硬盘
-
- 踢盘;
- 确定故障磁盘;
- 剔出磁盘组z;
- 重平衡完毕;
- 更换;
- 存储节点上从qlink中剔除故障磁盘;
- 将坏盘从操作系统detach掉;
- 点亮故障硬盘槽位指示灯;
- 更换;
- attach上新盘;
- 添加新盘到原有qlink位置;
- 加回磁盘组;
- 计算节点确定原有故障盘正常;
- 加回磁盘组;
-
- ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
- 1.确定损坏磁盘在ASM磁盘组中的磁盘名
- set
line 1000
- col
name for a30;
- col
failgroup for a50;
- col
path for a50;
- select
b.name,a.name,a.failgroup,a.path from v$asm_disk a,v$asm_diskgroup b where
a.group_number = b.group_number order by 2;
-
- 确定磁盘组信息,磁盘路径,把插到的坏的磁盘组那一行全部复制出来:
- NAME NAME FAILGROUP PATH
- SSDDG SSDDG_0057 SDDDGSTO3
/dev/qdata/mpath-1s03.3263.01.P3B00S21
-
- 2.从ASM中删除该ssd盘
- alter
diskgroup <dgname> drop disk <disk_name> force;
-
- alter
diskgroup ssddg drop disk ssddg_0057 force;
-
- 3.重平衡数据
- alter
diskgroup <dgname> rebalance power 11;
-
- alter
diskgroup ssddg rebalance power 11;
-
- 4.确认重平衡进度
- asmcmd
lsop 该命令返回无结果,则可进行下一步
- crsctl
status res -t //确认集群状态
-
-
- 5.存储节点上qlink中删除该盘
- qdatamgr
qlink del_lun -p 3261 -t 1 -b /dev/qdisk/Pxxx
- qdatamgr
qlink show -t //确认存储节点的target
-
- qdatamgr
qlink show -t
- qdatamgr
qlink del_lun -p 3263 -t 1 -b /dev/qdisk/P3B00S21
- qdatamgr
qlink show -t
-
- 计算节点查看该盘的状态:
- qdatamgr
qlink show -c | grep -P -A7 P3B00S21
-
- 6.将坏盘从操作系统detach
- qdatamgr
media detach_disk -s Pxxxx
-
- qdatamgr
media detach_disk -s P3B00S21
-
-
- 7.将磁盘槽位灯点亮,确定磁盘的物理位置,更换磁盘
- qdatamgr
media lighton -s Pxxxx
-
- qdatamgr
qlink show -t
- qdatamgr
media lighton -s P3B00S21
-
-
- 8.新的SSD重新插入
- 9.初始化新的磁盘
- qdatamgr
media attach_disk -s Pxxxx
-
- qdatamgr
media attach_disk -s P3B00S21
-
-
- 10.qlink中添加新盘
- qdatamgr
qlink add_lun -p 3261 -t 1 -b /dev/qdisk/Pxxxx
-
- qdatamgr
qlink add_lun -p 3263 -t 1 -b /dev/qdisk/P3B00S21
-
-
- 11.qlink中确认加载的新盘
- qdatamgr
qlink show -c | grep -P -A7 Pxxxx
- qdatamgr
qlink load -r //计算节点重新load
-
- qdatamgr
qlink show -c | grep -P -A7 P3B00S21
-
-
- 12.ASM中重新加入新盘
- alter
diskgroup ssddg add failgroup datasto1 disk
'/dev/qdata/mpath-1s01.3261.01.P0B00S02' force;
-
- alter
diskgroup ssddg add failgroup ssddgsto3 disk
'/dev/qdata/mpath-1s03.3263.01.P3B00S21' force;
- alter
diskgroup ssddg rebalance power 11;
-
- select
b.name,a.name,a.failgroup,a.path from v$asm_disk a,v$asm_diskgroup b where
a.group_number = b.group_number order by 2;
-
-
- +++++++++++++++++++++++++++++++++++++++++++++++++++++==
-
- 确定磁盘位置:
- set line 1000
- col name for
a30;
- col failgroup
for a50;
- col path for
a50;
- select
b.name,a.name,a.failgroup,a.path from v$asm_disk a,v$asm_diskgroup b where
a.group_number = b.group_number order by 2;
-
-
-
-
-
-
- 删除磁盘后,查看asm磁盘组的状态:
- su – grid
- asmcmd
lsdg
- asmcmd
lsop
- 节点2上的磁盘组状态:
- 重平衡数据:
-
-
-
-
-
- 节点2:
-
- 到对应的存储节点确定磁盘位置:
- qdatamgr
media show
- 从存储节点上删除该磁盘:
-
- ++++++++++++++++++++++ 额外信息
++++++++++++++++++++++++++++
- 当存储节点删除掉硬盘以后,从计算节点查看:
-
- 当存储节点添加完硬盘,假如到原来的qlink中后,从计算节点查看,qlink会自动挂载:
- +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
- 将坏盘从操作系统detach掉:
-
- 将硬盘槽位灯点亮,确定磁盘位置,更换磁盘:
-
-
- 更换上新的磁盘后,硬盘槽位灯恢复正常;
-
-
-
- 将新的盘attach上:
-
-
-
-
- 将新的磁盘添加到qlink中:
-
-
-
- 将磁盘加到磁盘组:
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- 相关日志输出:
-
-
-
-
-
-
-
-
- ++++++++++++++++++++++++++++++
- 在计算节点2上操作更换硬盘的操作,grid报的完整日志信息如下:
- Thu Nov 30
11:53:00 2017
- SQL> alter
diskgroup ssddg drop disk ssddg_0027 force
- Thu Nov 30
11:53:00 2017
- NOTE:
GroupBlock outside rolling migration privileged region
- NOTE:
requesting all-instance membership refresh for group=4
- NOTE: process
_user86851_+asm2 (86851) initiating offline of disk 27.3915936961 (SSDDG_0027)
with mask 0x7e in group 4 (SSDDG) without client assisting
- NOTE:
initiating PST update: grp 4 (SSDDG), dsk = 27/0xe96874c1, mask = 0x6a, op =
clear
- Thu Nov 30
11:53:00 2017
- GMON updating
disk modes for group 4 at 26 for pid 33, osid 86851
- Thu Nov 30
11:53:00 2017
- NOTE: PST
update grp = 4 completed successfully
- NOTE:
initiating PST update: grp 4 (SSDDG), dsk = 27/0xe96874c1, mask = 0x7e, op =
clear
- Thu Nov 30
11:53:00 2017
- GMON updating
disk modes for group 4 at 27 for pid 33, osid 86851
- Thu Nov 30
11:53:00 2017
- NOTE: cache
closing disk 27 of grp 4: SSDDG_0027
- Thu Nov 30
11:53:00 2017
- NOTE: PST
update grp = 4 completed successfully
- Thu Nov 30
11:53:00 2017
- GMON updating
for reconfiguration, group 4 at 28 for pid 33, osid 86851
- Thu Nov 30
11:53:00 2017
- NOTE: cache
closing disk 27 of grp 4: (not open) SSDDG_0027
- Thu Nov 30
11:53:00 2017
- NOTE: group 4
PST updated.
- Thu Nov 30
11:53:00 2017
- NOTE:
membership refresh pending for group 4/0x605884d1 (SSDDG)
- Thu Nov 30
11:53:00 2017
- GMON querying
group 4 at 29 for pid 22, osid 21402
- Thu Nov 30
11:53:00 2017
- NOTE: cache
closing disk 27 of grp 4: (not open) _DROPPED_0027_SSDDG
- Thu Nov 30
11:53:00 2017
- SUCCESS:
refreshed membership for 4/0x605884d1 (SSDDG)
- Thu Nov 30
11:53:00 2017
- SUCCESS:
alter diskgroup ssddg drop disk ssddg_0027 force
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
- Thu Nov 30
11:53:01 2017
- NOTE:
starting rebalance of group 4/0x605884d1 (SSDDG) at power 1
- Starting
background process ARB0
- Thu Nov 30
11:53:01 2017
- ARB0 started
with pid=37, OS id=88936
- NOTE:
assigning ARB0 to group 4/0x605884d1 (SSDDG) with 1 parallel I/O
- Thu Nov 30
11:53:44 2017
- SQL> alter
diskgroup ssddg rebalance power 11
- Thu Nov 30
11:53:44 2017
- NOTE: cache
closing disk 27 of grp 4: (not open) _DROPPED_0027_SSDDG
- Thu Nov 30
11:53:44 2017
- NOTE:
GroupBlock outside rolling migration privileged region
- Thu Nov 30
11:53:44 2017
- NOTE:
stopping process ARB0
- Thu Nov 30
11:53:46 2017
- NOTE:
rebalance interrupted for group 4/0x605884d1 (SSDDG)
- Thu Nov 30
11:53:46 2017
- NOTE:
requesting all-instance membership refresh for group=4
- Thu Nov 30
11:53:46 2017
- GMON updating
for reconfiguration, group 4 at 31 for pid 33, osid 86851
- Thu Nov 30
11:53:46 2017
- NOTE: cache
closing disk 27 of grp 4: (not open) _DROPPED_0027_SSDDG
- Thu Nov 30
11:53:46 2017
- NOTE: group 4
PST updated.
- Thu Nov 30
11:53:46 2017
- NOTE:
membership refresh pending for group 4/0x605884d1 (SSDDG)
- Thu Nov 30
11:53:46 2017
- GMON querying
group 4 at 32 for pid 22, osid 21402
- Thu Nov 30
11:53:46 2017
- NOTE: cache
closing disk 27 of grp 4: (not open) _DROPPED_0027_SSDDG
- Thu Nov 30
11:53:46 2017
- SUCCESS:
refreshed membership for 4/0x605884d1 (SSDDG)
- Thu Nov 30
11:53:46 2017
- SUCCESS:
alter diskgroup ssddg rebalance power 11
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
- Thu Nov 30
11:53:49 2017
- NOTE:
starting rebalance of group 4/0x605884d1 (SSDDG) at power 11
- Starting
background process ARB0
- Thu Nov 30
11:53:49 2017
- ARB0 started
with pid=37, OS id=89644
- NOTE:
assigning ARB0 to group 4/0x605884d1 (SSDDG) with 11 parallel I/Os
- Thu Nov 30
11:56:56 2017
- NOTE:
GroupBlock outside rolling migration privileged region
- NOTE:
requesting all-instance membership refresh for group=4
- Thu Nov 30
11:56:57 2017
- GMON updating
for reconfiguration, group 4 at 33 for pid 38, osid 91860
- Thu Nov 30
11:56:57 2017
- NOTE: cache
closing disk 27 of grp 4: (not open) _DROPPED_0027_SSDDG
- Thu Nov 30
11:56:57 2017
- NOTE: group 4
PST updated.
- SUCCESS: grp
4 disk _DROPPED_0027_SSDDG going offline
- Thu Nov 30
11:56:57 2017
- GMON updating
for reconfiguration, group 4 at 34 for pid 38, osid 91860
- Thu Nov 30
11:56:57 2017
- NOTE: cache
closing disk 27 of grp 4: (not open) _DROPPED_0027_SSDDG
- Thu Nov 30
11:56:57 2017
- NOTE: group 4
PST updated.
- Thu Nov 30
11:56:57 2017
- NOTE:
membership refresh pending for group 4/0x605884d1 (SSDDG)
- Thu Nov 30
11:56:57 2017
- GMON querying
group 4 at 35 for pid 22, osid 21402
- GMON querying
group 4 at 36 for pid 22, osid 21402
- Thu Nov 30
11:56:57 2017
- NOTE: Disk
_DROPPED_0027_SSDDG in mode 0x0 marked for de-assignment
- SUCCESS:
refreshed membership for 4/0x605884d1 (SSDDG)
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
- Thu Nov 30
11:57:25 2017
- NOTE:
stopping process ARB0
- Thu Nov 30
11:57:26 2017
- SUCCESS:
rebalance completed for group 4/0x605884d1 (SSDDG)
- Thu Nov 30
12:16:24 2017
- SQL> alter
diskgroup ssddg add failgroup ssddgsto2 disk
'/dev/qdata/mpath-1s02.3262.01.P0B00S10' force
- Thu Nov 30
12:16:24 2017
- NOTE:
GroupBlock outside rolling migration privileged region
- NOTE:
Assigning number (4,27) to disk (/dev/qdata/mpath-1s02.3262.01.P0B00S10)
- NOTE:
requesting all-instance membership refresh for group=4
- NOTE: Disk 27
in group 4 is assigned fgnum=4
- NOTE:
discarding redo for group 4 disk 27
- NOTE:
initializing header (replicated) on grp 4 disk SSDDG_0027
- NOTE:
initializing header on grp 4 disk SSDDG_0027
- NOTE:
requesting all-instance disk validation for group=4
- Thu Nov 30
12:16:25 2017
- NOTE:
skipping rediscovery for group 4/0x605884d1 (SSDDG) on local instance.
- Thu Nov 30
12:16:25 2017
- NOTE:
requesting all-instance disk validation for group=4
- Thu Nov 30
12:16:25 2017
- NOTE:
skipping rediscovery for group 4/0x605884d1 (SSDDG) on local instance.
- Thu Nov 30
12:16:25 2017
- GMON updating
for reconfiguration, group 4 at 38 for pid 33, osid 86851
- Thu Nov 30
12:16:25 2017
- NOTE: group 4
PST updated.
- Thu Nov 30
12:16:25 2017
- NOTE:
membership refresh pending for group 4/0x605884d1 (SSDDG)
- Thu Nov 30
12:16:25 2017
- GMON querying
group 4 at 39 for pid 22, osid 21402
- NOTE: cache
opening disk 27 of grp 4: SSDDG_0027
path:/dev/qdata/mpath-1s02.3262.01.P0B00S10
- Thu Nov 30
12:16:27 2017
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
- Thu Nov 30
12:16:27 2017
- GMON querying
group 4 at 40 for pid 22, osid 21402
- Thu Nov 30
12:16:27 2017
- SUCCESS:
refreshed membership for 4/0x605884d1 (SSDDG)
- Thu Nov 30
12:16:27 2017
- SUCCESS:
alter diskgroup ssddg add failgroup ssddgsto2 disk
'/dev/qdata/mpath-1s02.3262.01.P0B00S10' force
- Thu Nov 30
12:16:27 2017
- NOTE:
starting rebalance of group 4/0x605884d1 (SSDDG) at power 1
- Starting
background process ARB0
- Thu Nov 30
12:16:27 2017
- ARB0 started
with pid=36, OS id=104046
- NOTE:
assigning ARB0 to group 4/0x605884d1 (SSDDG) with 1 parallel I/O
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
- Thu Nov 30
12:17:39 2017
- SQL> alter
diskgroup ssddg rebalance power 11
- Thu Nov 30
12:17:39 2017
- NOTE:
GroupBlock outside rolling migration privileged region
- Thu Nov 30
12:17:39 2017
- NOTE:
stopping process ARB0
- Thu Nov 30
12:17:39 2017
- NOTE:
rebalance interrupted for group 4/0x605884d1 (SSDDG)
- Thu Nov 30
12:17:39 2017
- NOTE:
requesting all-instance membership refresh for group=4
- Thu Nov 30
12:17:39 2017
- GMON updating
for reconfiguration, group 4 at 42 for pid 33, osid 86851
- Thu Nov 30
12:17:39 2017
- NOTE: group 4
PST updated.
- Thu Nov 30
12:17:39 2017
- NOTE:
membership refresh pending for group 4/0x605884d1 (SSDDG)
- Thu Nov 30
12:17:39 2017
- GMON querying
group 4 at 43 for pid 22, osid 21402
- Thu Nov 30
12:17:39 2017
- SUCCESS:
refreshed membership for 4/0x605884d1 (SSDDG)
- Thu Nov 30
12:17:39 2017
- SUCCESS:
alter diskgroup ssddg rebalance power 11
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
- Thu Nov 30
12:17:42 2017
- NOTE:
starting rebalance of group 4/0x605884d1 (SSDDG) at power 11
- Starting
background process ARB0
- Thu Nov 30
12:17:42 2017
- ARB0 started
with pid=36, OS id=104813
- NOTE:
assigning ARB0 to group 4/0x605884d1 (SSDDG) with 11 parallel I/Os
- Thu Nov 30
12:21:55 2017
- NOTE:
GroupBlock outside rolling migration privileged region
- NOTE:
requesting all-instance membership refresh for group=4
- Thu Nov 30
12:21:57 2017
- GMON updating
for reconfiguration, group 4 at 44 for pid 37, osid 108281
- Thu Nov 30
12:21:57 2017
- NOTE: group 4
PST updated.
- Thu Nov 30
12:21:57 2017
- NOTE:
membership refresh pending for group 4/0x605884d1 (SSDDG)
- Thu Nov 30
12:21:57 2017
- GMON querying
group 4 at 45 for pid 22, osid 21402
- Thu Nov 30
12:21:57 2017
- SUCCESS:
refreshed membership for 4/0x605884d1 (SSDDG)
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
-
-
-
- Thu Nov 30
12:24:10 2017
- NOTE:
stopping process ARB0
- Thu Nov 30
12:24:13 2017
- SUCCESS:
rebalance completed for group 4/0x605884d1 (SSDDG)
-
- ++++++++++++++++++++++++++
- 节点1上报的完整日志信息:
- Thu Nov 30
11:52:59 2017
- NOTE:
updating disk modes to 0x15 from 0x7f for disk 27 (SSDDG_0027) in group 4
(SSDDG): lflags 0x4
- NOTE:
updating disk modes to 0x1 from 0x15 for disk 27 (SSDDG_0027) in group 4
(SSDDG): lflags 0x4
- Thu Nov 30
11:52:59 2017
- NOTE: cache
closing disk 27 of grp 4: SSDDG_0027
- Thu Nov 30
11:52:59 2017
- NOTE: cache
closing disk 27 of grp 4: (not open) SSDDG_0027
- Thu Nov 30
11:52:59 2017
- NOTE:
membership refresh pending for group 4/0x60589fce (SSDDG)
- Thu Nov 30
11:52:59 2017
- GMON querying
group 4 at 34 for pid 22, osid 21900
- Thu Nov 30
11:52:59 2017
- NOTE: cache
closing disk 27 of grp 4: (not open) SSDDG_0027
- Thu Nov 30
11:52:59 2017
- SUCCESS:
refreshed membership for 4/0x60589fce (SSDDG)
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
- Thu Nov 30
11:53:46 2017
- NOTE: cache
closing disk 27 of grp 4: (not open) _DROPPED_0027_SSDDG
- Thu Nov 30
11:53:46 2017
- NOTE:
membership refresh pending for group 4/0x60589fce (SSDDG)
- Thu Nov 30
11:53:46 2017
- GMON querying
group 4 at 35 for pid 22, osid 21900
- Thu Nov 30
11:53:46 2017
- NOTE: cache
closing disk 27 of grp 4: (not open) _DROPPED_0027_SSDDG
- Thu Nov 30
11:53:46 2017
- SUCCESS:
refreshed membership for 4/0x60589fce (SSDDG)
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
-
-
- Thu Nov 30
11:56:56 2017
- NOTE: cache
closing disk 27 of grp 4: (not open) _DROPPED_0027_SSDDG
- Thu Nov 30
11:56:57 2017
- NOTE:
membership refresh pending for group 4/0x60589fce (SSDDG)
- Thu Nov 30
11:56:57 2017
- GMON querying
group 4 at 36 for pid 22, osid 21900
- GMON querying
group 4 at 37 for pid 22, osid 21900
- Thu Nov 30
11:56:57 2017
- NOTE: Disk
_DROPPED_0027_SSDDG in mode 0x0 marked for de-assignment
- SUCCESS:
refreshed membership for 4/0x60589fce (SSDDG)
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
- Thu Nov 30
12:16:24 2017
- NOTE: disk
validation pending for 1 disk in group 4/0x60589fce (SSDDG)
- NOTE:
completed disk validation for 4/0x60589fce (SSDDG)
- NOTE: disk
validation pending for 1 disk in group 4/0x60589fce (SSDDG)
- NOTE:
Assigning number (4,27) to disk (/dev/qdata/mpath-1s02.3262.01.P0B00S10)
- NOTE:
completed disk validation for 4/0x60589fce (SSDDG)
- NOTE:
membership refresh pending for group 4/0x60589fce (SSDDG)
- Thu Nov 30
12:16:24 2017
- GMON querying
group 4 at 38 for pid 22, osid 21900
- NOTE: cache
opening disk 27 of grp 4: SSDDG_0027
path:/dev/qdata/mpath-1s02.3262.01.P0B00S10
- GMON querying
group 4 at 39 for pid 22, osid 21900
- Thu Nov 30
12:16:24 2017
- SUCCESS:
refreshed membership for 4/0x60589fce (SSDDG)
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
- Thu Nov 30
12:17:38 2017
- NOTE:
membership refresh pending for group 4/0x60589fce (SSDDG)
- Thu Nov 30
12:17:38 2017
- GMON querying
group 4 at 40 for pid 22, osid 21900
- Thu Nov 30
12:17:39 2017
- SUCCESS:
refreshed membership for 4/0x60589fce (SSDDG)
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
- Thu Nov 30
12:21:56 2017
- NOTE:
membership refresh pending for group 4/0x60589fce (SSDDG)
- Thu Nov 30
12:21:56 2017
- GMON querying
group 4 at 41 for pid 22, osid 21900
- Thu Nov 30
12:21:56 2017
- SUCCESS:
refreshed membership for 4/0x60589fce (SSDDG)
- NOTE:
Attempting voting file refresh on diskgroup SSDDG
- NOTE: Refresh
completed on diskgroup SSDDG. No voting file found.
- +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
原文地址:https://www.cnblogs.com/cnhyk/p/12125206.html