Multiple OCRs and vote disks on ASM in Oracle 11gR2

Edited: 3-August-2012

Oracle Clusterware used to provide installation option for multiple locations for the OCR and vote disks prior to 11gR2. In Oracle 11gR2 ASM disk groups are used to store the OCR and vote disks. While we can use the mirroring capabilities of ASM disk groups we still can manually add additional OCR disks in Oracle 11gR2 on ASM disk groups to provide redundancy and High Availability. Here in this post we will look at adding additional OCR disks to new ASM disks groups and moving the vote disk to a disk group with high redundancy. Initially we have Oracle GI installed on a disk group DATA with external redundancy. We created a disk group dgdup1 high redundancy with 5 disks and a disk group dgdup2 with external redundancy.
Make sure that Oracle GI is running

[root@raclinux2 ~]# cd /u01/app/11.2.0.2/grid [root@raclinux2 grid]# cd bin [root@raclinux2 bin]# ./crsctl check cluster -all ************************************************************** raclinux1: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** raclinux2: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online ************************************************************** [root@raclinux2 bin]#

Vote disks
Lets move the vote disk to a high redundancy group.

[root@raclinux2 bin]# [root@raclinux2 bin]# ./crsctl query css votedisk ## STATE -- ----File Universal Id File Name Disk group --------- ---------

-----------------

1. ONLINE 7b7f9d9ae9484f2cbf7e62c164aa221b (/dev/oracleasm/disks/DISK1) [DATA] Located 1 voting disk(s). [root@raclinux2 bin]# ./crsctl replace votedisk +dgdup1 Successful addition of voting disk 786515d3e2cc4fa7bfc5fd0b6e87cfeb. Successful addition of voting disk afe5c0e3da484f62bfcc36b1c0eb4aa4. Successful addition of voting disk 1ec83edcf92b4f55bfdc9711c49e0ddd. Successful addition of voting disk 97278bb3f36f4f84bf5965682efe87d2. Successful addition of voting disk 46021fd53e394fb8bf0f476f1fa210dc. Successful deletion of voting disk 7b7f9d9ae9484f2cbf7e62c164aa221b. Successfully replaced voting disk group with +dgdup1. CRS-4266: Voting file(s) successfully replaced

[root@raclinux2 bin]# ./crsctl query css votedisk ## STATE -- ----File Universal Id File Name Disk group --------- ---------

-----------------

1. ONLINE 786515d3e2cc4fa7bfc5fd0b6e87cfeb (/dev/oracleasm/disks/DISK11) [DGDUP1] 2. ONLINE afe5c0e3da484f62bfcc36b1c0eb4aa4 (/dev/oracleasm/disks/DISK12) [DGDUP1] 3. ONLINE 1ec83edcf92b4f55bfdc9711c49e0ddd (/dev/oracleasm/disks/DISK13) [DGDUP1] 4. ONLINE 97278bb3f36f4f84bf5965682efe87d2 (/dev/oracleasm/disks/DISK14) [DGDUP1] 5. ONLINE 46021fd53e394fb8bf0f476f1fa210dc (/dev/oracleasm/disks/DISK15) [DGDUP1] Located 5 voting disk(s). [root@raclinux2 bin]# [root@raclinux2 bin]# ./crsctl add css votedisk +data CRS-4671: This command is not supported for ASM diskgroups. CRS-4000: Command Add failed, or completed with errors. [root@raclinux2 bin]# ./crsctl add css votedisk /u03/vote_acfs.dsk CRS-4258: Addition and deletion of voting files are not allowed because the voting files are on ASM [root@raclinux2 bin]#

As seen from above we can move a vote disk to a high redundancy disk group dgdup1 and benefit from the High Redundancy of the group. We cannot use the crsctl add css votedisk to add a vote disk on ASM disk group or ACFS file system.

OCR disks

While we can benefit from the redundancy that the disk group storing the OCR provides we still can manually add OCR to different disk groups.

[root@raclinux2 bin]# cat /etc/oracle/ocr.loc #Device/file getting replaced by device +dgdup2 ocrconfig_loc=+DATA local_only=false [root@raclinux2 bin]# ./ocrcheck Status of Oracle Cluster Registry is as follows : Version : : : 3 262120 3380 258740

Total space (kbytes) Used space (kbytes)

Available space (kbytes) : ID

: 1332773503 : +DATA

Device/File Name

Device/File integrity check succeeded Device/File not configured Device/File not configured Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check succeeded

[root@raclinux2 bin]# [root@raclinux2 bin]# ./ocrconfig -h Name: ocrconfig - Configuration tool for Oracle Cluster/Local Registry. Synopsis: ocrconfig [option] option: [-local] -export - Export OCR/OLR contents to a file [-local] -import [-local] -upgrade [ []] - Upgrade OCR from previous version -downgrade [-version ] - Downgrade OCR to the specified version [-local] -backuploc - Configure OCR/OLR backup location - Import OCR/OLR contents from a file

[-local] -showbackup [auto|manual] - Show OCR/OLR backup information [-local] -manualbackup [-local] -restore - Perform OCR/OLR backup

- Restore OCR/OLR from physical backup

-replace -replacement - Replace a OCR device/file <filename1> with <filename2> -add -delete - Add a new OCR device/file - Remove a OCR device/file

-overwrite

- Overwrite OCR configuration on disk

-repair -add | -delete | -replace -replacement - Repair OCR configuration on the local node -help Note: * A log file will be created in $ORACLE_HOME/log//client/ocrconfig_.log. Please ensure you have file creation privileges in the above directory before running this tool. * Only -local -showbackup [manual] is supported. * Use option '-local' to indicate that the operation is to be performed on the Oracle Local Registry. [root@raclinux2 bin]# [root@raclinux2 bin]# [root@raclinux2 bin]# ./ocrconfig -add +dgdup2 [root@raclinux2 bin]# ./ocrconfig -add +dgdup1 [root@raclinux2 bin]# ./ocrcheck Status of Oracle Cluster Registry is as follows : Version : : : 3 262120 3380 258740 - Print out this help information

Total space (kbytes) Used space (kbytes)

Available space (kbytes) : ID

: 1332773503

Device/File Name

+DATA

Device/File integrity check succeeded Device/File Name : +dgdup2

Device/File integrity check succeeded Device/File Name : +dgdup1

Device/File integrity check succeeded Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check succeeded [root@raclinux2 bin]# [root@raclinux2 bin]# cat /etc/oracle/ocr.loc #Device/file getting replaced by device +dgdup1 ocrconfig_loc=+DATA ocrmirrorconfig_loc=+dgdup2 ocrconfig_loc3=+dgdup1 local_only=false[root@raclinux2 bin]#

Summary: While we benefit from the redundancy a disk group provides we can add multiple OCR on a different disk groups. We can move the VOTE disk to a disk group with high redundancy. To be precise vote disk in ASM has the following specifics.

No file in ASM spans a disk group including a vote file corresponding to the vote disks. Difference is that vote file is not mirrored and stripped like any other files (data, redo, control, OCR etc) and has a fixed assignment to a failure group in a disk. Vote disk can be accessed even if ASM is not up whereas another file cannot be accessed if ASM is not up and running. You cannot mix vote files on ASM and vote files on non-ASM. Instead you can dedicate either one ASM DG for all you cluster vote disk files or you can use nonASM but never both except for initial migration to 11.2 when previous versions were on non-ASM storage. For vote files you need a special number of failure groups in a disk groups as indicated below 1. For External redundancy DG you need 1 failure group for one Vote disk file 2. For Normal redundancy DG you need 3 failure groups for 3 vote disk file 3. For High redundancy DG you need 5 failure groups for 5 vote disk files