How to Move OCR, Voting Disk File, ASM SPILE to a New Diskgroup ( 12.1.0.2 )

OCR, Voting Disk file, ASM spfile  and MGMTDB were installed onto +DATA group when Grid Infrastructure 12c was installed.

MGMTDB has been relocated on to a dedicated diskgroup +GIMR as per my another post “How to Move 12c Grid Infrastructure Management Repository ( GIMR ) to Another Diskgroup“.

This post demonstrates how to move OCR, Voting Disk file and ASM spfile onto a dedicated diskgroup OCR.

  1. create a new diskgroup +OCR

    As per section “Create New Disks and Create New Diskgroup FRA” for creating a diskgroup.

[root@racnode1 bin]# fdisk /dev/sdi

Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xf9ee9a74.
Command (m for help): n
Partition type:
 p primary (0 primary, 0 extended, 4 free)
 e extended
Select (default p):
Using default response p
Partition number (1-4, default 1):
First sector (2048-2097151, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-2097151, default 2097151):
Using default value 2097151
Partition 1 of type Linux and of size 1023 MiB is set
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

[root@racnode1 bin]# oracleasm listdisks

ASM_DISK1
ASM_DISK2
ASM_DISK3
ASM_DISK4
ASM_DISK5
ASM_DISK6
ASM_DISK7

[root@racnode1 bin]# oracleasm createdisk ASM_OCR_VOTE /dev/sdi1
Writing disk header: done
Instantiating disk: done
[root@racnode1 bin]# oracleasm listdisks
ASM_DISK1
ASM_DISK2
ASM_DISK3
ASM_DISK4
ASM_DISK5
ASM_DISK6
ASM_DISK7
ASM_OCR_VOTE

SQL> CREATE DISKGROUP OCR EXTERNAL REDUNDANCY DISK '/dev/oracleasm/disks/ASM_OCR_VOTE';

Diskgroup created.

SQL>

2. Ensure that the new diskgroup is mounted on all cluster nodes

One node1, +OCR diskgroup has just been created and mounted.

SQL> select name, state, type from v$asm_diskgroup;

NAME STATE TYPE
------------------------------ ----------- ------
GIMR MOUNTED EXTERN
OCR MOUNTED EXTERN
FRA MOUNTED EXTERN
DATA MOUNTED EXTERN

SQL> !asmcmd lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 15357 6082 0 6082 0 Y DATA/
MOUNTED EXTERN N 512 4096 1048576 10238 9914 0 9914 0 N FRA/
MOUNTED EXTERN N 512 4096 1048576 5119 855 0 855 0 N GIMR/
MOUNTED EXTERN N 512 4096 1048576 1023 973 0 973 0 N OCR/

One node2, manually mounted +OCR diskgroup.

SQL> select name, state, type from v$asm_diskgroup;

NAME STATE TYPE
------------------------------ ----------- ------
GIMR MOUNTED EXTERN
OCR DISMOUNTED
FRA MOUNTED EXTERN
DATA MOUNTED EXTERN

SQL> !asmcmd lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 15357 6082 0 6082 0 Y DATA/
MOUNTED EXTERN N 512 4096 1048576 10238 9914 0 9914 0 N FRA/
MOUNTED EXTERN N 512 4096 1048576 5119 855 0 855 0 N GIMR/

SQL> alter diskgroup OCR mount;

Diskgroup altered.

SQL> select name, state, type from v$asm_diskgroup;

NAME STATE TYPE
------------------------------ ----------- ------
GIMR MOUNTED EXTERN
OCR MOUNTED EXTERN
FRA MOUNTED EXTERN
DATA MOUNTED EXTERN

SQL> !asmcmd lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 15357 6082 0 6082 0 Y DATA/
MOUNTED EXTERN N 512 4096 1048576 10238 9914 0 9914 0 N FRA/
MOUNTED EXTERN N 512 4096 1048576 5119 855 0 855 0 N GIMR/
MOUNTED EXTERN N 512 4096 1048576 1023 930 0 930 0 N OCR/

3. Relocate OCR from +DATA to +OCR diskgroup

Check the current OCR location:

[root@racnode1 bin]# pwd
/u01/app/12.1.0/grid/bin
[root@racnode1 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
 Version : 4
 Total space (kbytes) : 409568
 Used space (kbytes) : 1604
 Available space (kbytes) : 407964
 ID : 675911095
 Device/File Name : +DATA
 Device/File integrity check succeeded

Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured

Cluster registry integrity check succeeded
Logical corruption check succeeded

Add +OCR datagroup as  another OCR location besides +DATA diskgroup

[root@racnode1 bin]# ./ocrconfig -add +OCR
PROT-30: The Oracle Cluster Registry location to be added is not usable
PROC-8: Cannot perform cluster registry operation because one of the parameters is invalid.
ORA-15056: additional error message
ORA-17502: ksfdcre:4 Failed to create file +OCR.255.1
ORA-15221: ASM operation requires compatible.asm of 11.1.0.0.0 or higher
ORA-06512: at line 4

Check current diskgroup attribute compatibility :

SQL> col COMPATIBILITY format a14
SQL> col DATABASE_COMPATIBILITY format a15
SQL> select NAME,COMPATIBILITY,DATABASE_COMPATIBILITY,VOTING_FILES,CON_ID from v$asm_diskgroup;

NAME COMPATIBILITY DATABASE_COMPAT V CON_ID
------------------------------ -------------- --------------- - ----------
GIMR 10.1.0.0.0 10.1.0.0.0 N 0
OCR 10.1.0.0.0 10.1.0.0.0 N 0
FRA 10.1.0.0.0 10.1.0.0.0 N 0
DATA 12.1.0.0.0 10.1.0.0.0 Y 0

SQL> alter diskgroup OCR set attribute 'compatible.asm'='11.2';
Diskgroup altered.

SQL> select NAME,COMPATIBILITY,DATABASE_COMPATIBILITY,VOTING_FILES,CON_ID from v$asm_diskgroup;

NAME COMPATIBILITY DATABASE_COMPAT V CON_ID
------------------------------ -------------- --------------- - ----------
GIMR 10.1.0.0.0 10.1.0.0.0 N 0
OCR 11.2.0.0.0 10.1.0.0.0 N 0
FRA 10.1.0.0.0 10.1.0.0.0 N 0
DATA 12.1.0.0.0 10.1.0.0.0 Y 0

Add +OCR datagroup as  another OCR location again, so there are two OCR locations now.

[root@racnode1 bin]# ./ocrconfig -add +OCR

[root@racnode1 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
 Version : 4
 Total space (kbytes) : 409568
 Used space (kbytes) : 1604
 Available space (kbytes) : 407964
 ID : 675911095
 Device/File Name : +DATA
 Device/File integrity check succeeded
 Device/File Name : +OCR
 Device/File integrity check succeeded

 Device/File not configured

 Device/File not configured

 Device/File not configured

 Cluster registry integrity check succeeded

 Logical corruption check succeeded

Delete the old OCR location in +DATA diskgroup:

[root@racnode1 bin]# ./ocrconfig -delete +DATA
[root@racnode1 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
 Version : 4
 Total space (kbytes) : 409568
 Used space (kbytes) : 1604
 Available space (kbytes) : 407964
 ID : 675911095
 Device/File Name : +OCR
 Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

4. Relocate voting disks to a new ASM diskgroup

Check current votedisk status:

[root@racnode1 bin]# ./crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
 1. ONLINE caa686735a414f12bf0b008227dd1be6 (/dev/oracleasm/disks/ASM_DISK1) [DATA]
Located 1 voting disk(s).

Relocate current voting disk onto +OCR :

[root@racnode1 bin]# ./crsctl replace votedisk +OCR
Successful addition of voting disk efa833df31254f77bf9b19ce921a9c3a.
Successful deletion of voting disk caa686735a414f12bf0b008227dd1be6.
Successfully replaced voting disk group with +OCR.
CRS-4266: Voting file(s) successfully replaced
[root@racnode1 bin]#

[root@racnode1 bin]# ./crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
 1. ONLINE efa833df31254f77bf9b19ce921a9c3a (/dev/oracleasm/disks/ASM_OCR_VOTE) [OCR]
Located 1 voting disk(s).
[root@racnode1 bin]#

5. Change ASM SPFILE location from +DATA to +OCR Diskgroup.

Login as SYSASM using SQLPLUS into ASM instance on one node and run below commands:

[grid@racnode1 ~]$ sqlplus / as sysasm
SQL>
SQL> show parameter pfile

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string +DATA/racnode-cluster/ASMPARAMETERFILE/registry.253.90399542 7

SQL> create pfile='/tmp/tmp.asm.init' from spfile;

File created.

SQL> create spfile='+OCR' from pfile='/tmp/tmp.asm.init';

File created.

SQL> ! cat /tmp/tmp.asm.init
+ASM1.__oracle_base='/u01/app/grid'#ORACLE_BASE set from in memory value
+ASM2.__oracle_base='/u01/app/grid'#ORACLE_BASE set from in memory value
+ASM1.asm_diskgroups='FRA','GIMR','OCR'#Manual Mount
+ASM2.asm_diskgroups='FRA','GIMR','OCR'#Manual Mount
*.asm_diskstring='/dev/oracleasm/disks'
*.asm_power_limit=1
*.large_pool_size=12M
*.remote_login_passwordfile='EXCLUSIVE'

SQL> ! ls -ltr /dev/oracleasm/disks
total 0
brw-rw---- 1 oracle oinstall 8, 65 Mar 25 22:11 ASM_DISK4
brw-rw---- 1 oracle oinstall 8, 97 Mar 25 22:12 ASM_DISK6
brw-rw---- 1 oracle oinstall 8, 49 Mar 25 22:58 ASM_DISK3
brw-rw---- 1 oracle oinstall 8, 33 Mar 25 22:58 ASM_DISK2
brw-rw---- 1 oracle oinstall 8, 129 Mar 25 23:16 ASM_OCR_VOTE
brw-rw---- 1 oracle oinstall 8, 113 Mar 25 23:16 ASM_DISK7
brw-rw---- 1 oracle oinstall 8, 81 Mar 25 23:16 ASM_DISK5
brw-rw---- 1 oracle oinstall 8, 17 Mar 25 23:16 ASM_DISK1

Now GPNPTOOL will get updated with new ASM SPFILE location.

That can be verified by below command :

[grid@racnode1 ~]$ which gpnptool
/u01/app/12.1.0/grid/bin/gpnptool
[grid@racnode1 ~]$ /u01/app/12.1.0/grid/bin/gpnptool get

Warning: some command line parameters were defaulted. Resulting command line:
 /u01/app/12.1.0/grid/bin/gpnptool.bin get -o-

<?xml version="1.0" encoding="UTF-8"?><gpnp:GPnP-Profile Version="1.0" xmlns="http://www.grid-pnp.org/2005/11/gpnp-profile" xmlns:gpnp="http://www.grid-pnp.org/2005/11/gpnp-profile" xmlns:orcl="http://www.oracle.com/gpnp/2005/11/gpnp-profile" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.grid-pnp.org/2005/11/gpnp-profile gpnp-profile.xsd" ProfileSequence="5" ClusterUId="fb397bfca4aa7fd4bf8f11b9334bc794" ClusterName="racnode-cluster" PALocation=""><gpnp:Network-Profile><gpnp:HostNetwork id="gen" HostName="*"><gpnp:Network id="net1" IP="192.168.78.0" Adapter="enp0s3" Use="public"/><gpnp:Network id="net2" IP="172.16.100.0" Adapter="enp0s8" Use="cluster_interconnect"/></gpnp:HostNetwork></gpnp:Network-Profile><orcl:CSS-Profile id="css" DiscoveryString="+asm" LeaseDuration="400"/><orcl:ASM-Profile id="asm" DiscoveryString="/dev/oracleasm/disks" SPFile="+OCR/racnode-cluster/ASMPARAMETERFILE/registry.253.907456701" Mode="legacy"/><ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#"><ds:SignedInfo><ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/><ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/><ds:Reference URI=""><ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"> <InclusiveNamespaces xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="gpnp orcl xsi"/></ds:Transform></ds:Transforms><ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/><ds:DigestValue>FkEXhPGopCO4x41Vwi7l1x777Pw=</ds:DigestValue></ds:Reference></ds:SignedInfo><ds:SignatureValue>lRRcwNSU5dBRxVeDbZCezWL5r1SGmhfnk/zl5NzqTxLNB3ctalsb9np9cBAH2HW/O7S539L3teKOHV/oszc1BmTeN+fuMM080Y1Qpw4+jhlnUgUZhrLr+Sroyb7HihB18+OFWe45mfJPFnVzqIeiLJ9hOOVNp1e5Q7cjAZHcyqg=</ds:SignatureValue></ds:Signature></gpnp:GPnP-Profile>
Success.
[grid@racnode1 ~]$

Restarted all cluster node :

[root@racnode1 bin]# ./crsctl check cluster -all
**************************************************************
racnode1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
racnode2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[root@racnode1 bin]# ./crsctl stop cluster -all
CRS-2673: Attempting to stop 'ora.crsd' on 'racnode1'
CRS-2673: Attempting to stop 'ora.crsd' on 'racnode2'
......
......
......
CRS-2677: Stop of 'ora.cssd' on 'racnode1' succeeded
[root@racnode1 bin]#


[root@racnode1 bin]# ./crsctl start cluster -all
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racnode1'
CRS-2672: Attempting to start 'ora.evmd' on 'racnode1'
CRS-2672: Attempting to start 'ora.evmd' on 'racnode2'
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racnode2'
CRS-2676: Start of 'ora.cssdmonitor' on 'racnode2' succeeded
CRS-2676: Start of 'ora.cssdmonitor' on 'racnode1' succeeded
......
......
......
......
CRS-2672: Attempting to start 'ora.crsd' on 'racnode2'
CRS-2676: Start of 'ora.crsd' on 'racnode1' succeeded
CRS-2676: Start of 'ora.crsd' on 'racnode2' succeeded


[root@racnode1 bin]# ./crsctl check cluster -all
**************************************************************
racnode1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
racnode2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************

Check spfile has been relocated on to +OCR diskgroup:

SQL> show parameter pfile

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string +OCR/racnode-cluster/ASMPARAMETERFILE/registry.253.907456701

 

Advertisements
This entry was posted in 12c new features, GI and RAC and tagged , , , , , , , , , , . Bookmark the permalink.

2 Responses to How to Move OCR, Voting Disk File, ASM SPILE to a New Diskgroup ( 12.1.0.2 )

  1. Gupta says:

    Very Nice Document. Thank you very much.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s