http://levipereira.wordpress.

com/2013/10/10/how-to-movemigrate-ocr-voting-disk-a
sm-spfile-and-pwdfile-and-mgmtdb-to-a-different-diskgroup-on-asm-12-1-0-1/
https://community.oracle.com/message/11127649
./crsctl check crs --> check crs status
asmcmd
lsdg

--> Voting_files atribute --> check which disk says this as Y

./crsctl query css votedisk --> check currently to which diskgroup voting disk
belongs too.
Please stop CRS on both nodes using crsctl stop crs and then start CRS in exclus
ive mode on any of one node.
crsctl stop crs
./crsctl start crs -excl
ps -ef |grep -i d.bin
./crsctl replace votedisk +DATADG2 --> if this diskgroup is normal so 3 copies
would be made as redundancy is normal. if external onle on copy. The previosu d
isk woudl be deleted automaticaly. in case you want to keep voting disk in previ
ous disk you would have to use ./crsctl add css votedisk +DATADG2 command to do
it instead of replace.
From above output, we can see that now VOTING disk is moved into three disks out
of 4 ASM disks since redundancy of ASM diskgroup is normal. now stop crs and st
art on both nodes. We've to use force option to shutdown CRS.
./crsctl stop crs -f
ps -ef |grep -i d.bin
./crsctl start crs
./crsctl query css votedisk

Example of replacingVoting devices into from one ASM diskgroup to another.

Now we'll create second ASM diskgroup DATADG2 using remaining disks with normal
redundancy.
[oracle@rac1 ~]$ asmcmd
ASMCMD> lsdg
State
Type
Rebal Sector Block
AU Total_MB Free_MB Req_mir_free_
MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N
512 4096 1048576
10228
9071
0
9071
0
Y DATADG1/
MOUNTED NORMAL N
512 4096 1048576
20456
20169
51
14
7527
0
N DATADG2/
Since Voting disk is already located in DATADG1 and we want to move it to DATADG
2 while CRS is up, we get below error.
[root@rac1 bin]# ./crsctl query css votedisk
## STATE
File Universal Id
File Name Disk group
-- ----------------------------- --------1. ONLINE 43887170ea864f1cbf3c4008f21fc78a (ORCL:DISK1) [DATADG1]
Located 1 voting disk(s).
[root@rac1 bin]# ./crsctl add css votedisk +DATADG2

CRS-4671: This command is not supported for ASM diskgroups.

CRS-4000: Command Add failed, or completed with errors.
[root@rac1 bin]# ./crsctl replace votedisk +DATADG2
CRS-4602: Failed 27 to add voting file 6ac9fe19e3444fcfbf827a035155bd18.
CRS-4602: Failed 27 to add voting file a4ffb5d52ad14f7ebf563726fbe15761.
CRS-4602: Failed 27 to add voting file 92b962c4090c4f7abfd3184f721796cc.
Failure 27 with Cluster Synchronization Services while deleting voting disk.
Failed to replace voting disk group with +DATADG2.
CRS-4000: Command Replace failed, or completed with errors.
[root@rac1 bin]# ./crsctl query css votedisk
## STATE
File Universal Id
File Name Disk group
-- ----------------------------- --------1. ONLINE 43887170ea864f1cbf3c4008f21fc78a (ORCL:DISK1) [DATADG1]
Located 1 voting disk(s).
Please stop CRS on both nodes using crsctl stop crs and then start CRS in exclus
ive mode on any of one node.
[root@rac1 bin]# ./crsctl start crs -excl
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'rac1'
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
CRS-2676: Start of 'ora.drivers.acfs' on 'rac1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
[root@rac1 bin]# ps -ef |grep -i d.bin
root
8102
1 2 10:55 ?
00:00:04 /u01/app/11.2.0/grid/bin/ohasd.b
in exclusive
oracle
8223
1 1 10:55 ?
00:00:02 /u01/app/11.2.0/grid/bin/oraagen
t.bin
oracle
8235
1 0 10:55 ?
00:00:00 /u01/app/11.2.0/grid/bin/mdnsd.b
in
oracle
8258
1 1 10:55 ?
00:00:01 /u01/app/11.2.0/grid/bin/gpnpd.b
in
root
8268
1 0 10:55 ?
00:00:00 /u01/app/11.2.0/grid/bin/cssdmon
itor
oracle
8270
1 1 10:55 ?
00:00:02 /u01/app/11.2.0/grid/bin/gipcd.b
in
root
8290
1 0 10:55 ?
00:00:01 /u01/app/11.2.0/grid/bin/cssdage
nt
oracle
8314
1 2 10:55 ?
00:00:03 /u01/app/11.2.0/grid/bin/ocssd.b
in -X

root
agent.bin
root
bin
root
n reboot
root

8367

1 1 10:56 ?

00:00:02 /u01/app/11.2.0/grid/bin/oraroot

8381

1 0 10:56 ?

00:00:00 /u01/app/11.2.0/grid/bin/octssd.

8675

1 1 10:56 ?

00:00:01 /u01/app/11.2.0/grid/bin/crsd.bi

8840 6677 0 10:58 pts/2

00:00:00 grep -i d.bin

[root@rac1 bin]# ./crsctl replace votedisk +DATADG2

Successful addition of voting disk ae7a31fc941d4fa6bf813b58cbf4ae6d.
Successful addition of voting disk 9a7cd52bf7e24fe4bfac1939a2433a2a.
Successful addition of voting disk 75459ba3e9134f0abfa54468ac51610c.
Successful deletion of voting disk 43887170ea864f1cbf3c4008f21fc78a.
Successfully replaced voting disk group with +DATADG2.
CRS-4266: Voting file(s) successfully replaced
[root@rac1 bin]# ./crsctl query css votedisk
## STATE
File Universal Id
File Name Disk group
-- ----------------------------- --------1. ONLINE ae7a31fc941d4fa6bf813b58cbf4ae6d (ORCL:DISK3) [DATADG2]
2. ONLINE 9a7cd52bf7e24fe4bfac1939a2433a2a (ORCL:DISK4) [DATADG2]
3. ONLINE 75459ba3e9134f0abfa54468ac51610c (ORCL:DISK5) [DATADG2]
Located 3 voting disk(s).
From above output, we can see that now VOTING disk is moved into three disks out
of 4 ASM disks since redundancy of ASM diskgroup is normal. now stop crs and st
art on both nodes. We've to use force option to shutdown CRS.
[root@rac1 bin]# ./crsctl stop crs
CRS-2796: The command may not proceed when Cluster Ready Services is not running
CRS-4687: Shutdown command has completed with errors.
CRS-4000: Command Stop failed, or completed with errors.
[root@rac1 bin]# ps -ef |grep -i d.bin
root
8102
1 1 10:55 ?
00:00:06 /u01/app/11.2.0/grid/bin/ohasd.b
in exclusive
oracle
8223
1 1 10:55 ?
00:00:06 /u01/app/11.2.0/grid/bin/oraagen
t.bin
oracle
8235
1 0 10:55 ?
00:00:00 /u01/app/11.2.0/grid/bin/mdnsd.b
in
oracle
8258
1 0 10:55 ?
00:00:02 /u01/app/11.2.0/grid/bin/gpnpd.b
in
root
8268
1 0 10:55 ?
00:00:01 /u01/app/11.2.0/grid/bin/cssdmon
itor
oracle
8270
1 0 10:55 ?
00:00:03 /u01/app/11.2.0/grid/bin/gipcd.b
in
root
8290
1 0 10:55 ?
00:00:01 /u01/app/11.2.0/grid/bin/cssdage
nt
oracle
8314
1 1 10:55 ?
00:00:07 /u01/app/11.2.0/grid/bin/ocssd.b
in -X
root
8367
1 1 10:56 ?
00:00:05 /u01/app/11.2.0/grid/bin/oraroot
agent.bin
root
8381
1 0 10:56 ?
00:00:01 /u01/app/11.2.0/grid/bin/octssd.
bin
root
8675
1 0 10:56 ?
00:00:01 /u01/app/11.2.0/grid/bin/crsd.bi
n reboot
root
9331 6677 0 11:03 pts/2
00:00:00 grep -i d.bin
[root@rac1 bin]# ./crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resourc
es on 'rac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'

CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac1'
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac1' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ra
c1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@rac1 bin]# ps -ef |grep -i d.bin
root
9500 6677 0 11:04 pts/2
00:00:00 grep -i d.bin
[root@rac1 bin]# ./crsctl query css votedisk
## STATE
File Universal Id
File Name Disk group
-- ----------------------------- --------1. ONLINE 44b55c5576154fc2bf3e54f08099a0e9 (ORCL:DISK3) [DATADG2]
2. ONLINE 0fcd8b592b604faabf9120ffef2a8e18 (ORCL:DISK5) [DATADG2]
3. ONLINE 00dc165a6d5c4ffebf4ba43496477d13 (ORCL:DISK6) [DATADG2]
Located 3 voting disk(s).
[root@rac1 bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[root@rac2 bin]# ./crsctl query css votedisk
## STATE
File Universal Id
File Name Disk group
-- ----------------------------- --------1. ONLINE 44b55c5576154fc2bf3e54f08099a0e9 (ORCL:DISK3) [DATADG2]
2. ONLINE 0fcd8b592b604faabf9120ffef2a8e18 (ORCL:DISK5) [DATADG2]
3. ONLINE 00dc165a6d5c4ffebf4ba43496477d13 (ORCL:DISK6) [DATADG2]
Located 3 voting disk(s).
[root@rac2 bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
We've to perform same steps for OCR replacement like stopping CRS and starting i
n exclusive mode. Since we added/removed copy of OCR, we need not to performs st
op/start(exclusive) operations.