11Gr2RAC使用备份恢复ocr和votedisk

来源:互联网 发布:炉石盒子 mac 编辑:程序博客网 时间:2024/06/08 13:38

                              11Gr2RAC使用备份恢复ocr和votedisk
                                          


环境: 2节点rac
软件版本:oracle 11.2.0.4
说明:使用备份恢复ocr和votedisk,在生产库中,系统是自动备份的。
1)检查环境,准备工作
[root@rac2 bin]# ./crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       rac1                                         
               ONLINE  ONLINE       rac2                                         
ora.FRA.dg
               ONLINE  ONLINE       rac1                                         
               ONLINE  ONLINE       rac2                                         
ora.LISTENER.lsnr
               ONLINE  ONLINE       rac1                                         
               ONLINE  ONLINE       rac2                                         
ora.asm
               ONLINE  ONLINE       rac1                     Started             
               ONLINE  ONLINE       rac2                     Started             
ora.gsd
               OFFLINE OFFLINE      rac1                                         
               OFFLINE OFFLINE      rac2                                         
ora.net1.network
               ONLINE  ONLINE       rac1                                         
               ONLINE  ONLINE       rac2                                         
ora.ons
               ONLINE  ONLINE       rac1                                         
               ONLINE  ONLINE       rac2                                         
ora.registry.acfs
               ONLINE  ONLINE       rac1                                         
               ONLINE  ONLINE       rac2                                         
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac2                                         
ora.cvu
      1        ONLINE  ONLINE       rac2                                         
ora.oc4j
      1        ONLINE  ONLINE       rac2                                         
ora.orcl.db
      1        ONLINE  ONLINE       rac1                     Open                
      2        ONLINE  ONLINE       rac2                     Open                
ora.rac1.vip
      1        ONLINE  ONLINE       rac1                                         
ora.rac2.vip
      1        ONLINE  ONLINE       rac2                                         
ora.scan1.vip
      1        ONLINE  ONLINE       rac2                                         
[root@rac2 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3112
         Available space (kbytes) :     259008
         ID                       :  257970053
         Device/File Name         : +CRSVOTEDISK
                                    Device/File integrity check succeeded




                                    Device/File not configured




                                    Device/File not configured




                                    Device/File not configured




                                    Device/File not configured




         Cluster registry integrity check succeeded




         Logical corruption check succeeded




[root@rac2 bin]# ./crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   1c33b77017d34f94bf17db020e40edc0 (/dev/raw/raw1) [CRSVOTEDISK]
 2. ONLINE   1f6c69dab3344fa6bf605ab2a7601948 (/dev/raw/raw2) [CRSVOTEDISK]
 3. ONLINE   8273e16873b94ff3bf213b5597027cd1 (/dev/raw/raw3) [CRSVOTEDISK]
Located 3 voting disk(s).
资源正常,手工备份一下ocr
[root@rac2 bin]# ./ocrconfig -manualbackup




rac2     2016/05/19 03:41:38     /u01/app/11.2.0/grid/cdata/rac-cluster/backup_20160519_034138.ocr




rac2     2016/05/16 07:35:30     /u01/app/11.2.0/grid/cdata/rac-cluster/backup_20160516_073530.ocr
[root@rac2 bin]# ./ocrconfig -showbackup
PROT-24: Auto backups for the Oracle Cluster Registry are not available




rac2     2016/05/19 03:41:38     /u01/app/11.2.0/grid/cdata/rac-cluster/backup_20160519_034138.ocr




rac2     2016/05/16 07:35:30     /u01/app/11.2.0/grid/cdata/rac-cluster/backup_20160516_073530.ocr


创建一份参数文件,因为asm实例的启动参数放在ocr上,可以用show parameter spfile查看
[grid@rac2 ~]$ sqlplus / as sysasm




SQL*Plus: Release 11.2.0.4.0 Production on Thu May 19 03:45:45 2016




Copyright (c) 1982, 2013, Oracle.  All rights reserved.








Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options




SQL> create pfile='/tmp/pfile1.ora' from spfile; 




File created.
使用dd命令破坏磁盘
[root@rac2 bin]# dd if=/dev/zero of=/dev/raw/raw3 bs=1024 count=1000
[root@rac2 bin]# dd if=/dev/zero of=/dev/raw/raw2 bs=1024 count=1000
[root@rac2 bin]# dd if=/dev/zero of=/dev/raw/raw1 bs=1024 count=1000
关闭两节点
[root@rac2 bin]# ./crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac2'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac2'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac2'
CRS-2673: Attempting to stop 'ora.cvu' on 'rac2'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac2'
CRS-2673: Attempting to stop 'ora.orcl.db' on 'rac2'
CRS-2673: Attempting to stop 'ora.oc4j' on 'rac2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac2'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac2'
CRS-2677: Stop of 'ora.cvu' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cvu' on 'rac1'
CRS-2676: Start of 'ora.cvu' on 'rac1' succeeded
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac2'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.rac2.vip' on 'rac2'
CRS-2677: Stop of 'ora.registry.acfs' on 'rac2' succeeded
CRS-2677: Stop of 'ora.rac2.vip' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.rac2.vip' on 'rac1'
CRS-2677: Stop of 'ora.scan1.vip' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac1'
CRS-2676: Start of 'ora.rac2.vip' on 'rac1' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac1'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac1' succeeded
CRS-2677: Stop of 'ora.orcl.db' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac2'
CRS-2673: Attempting to stop 'ora.FRA.dg' on 'rac2'
CRS-2677: Stop of 'ora.FRA.dg' on 'rac2' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac2'
CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded
CRS-2677: Stop of 'ora.oc4j' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.oc4j' on 'rac1'
CRS-2676: Start of 'ora.oc4j' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac2'
CRS-2677: Stop of 'ora.ons' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac2'
CRS-2677: Stop of 'ora.net1.network' on 'rac2' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac2' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac2'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac2'
CRS-2673: Attempting to stop 'ora.asm' on 'rac2'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac2'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac2'
CRS-2677: Stop of 'ora.evmd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac2'
CRS-2677: Stop of 'ora.ctssd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac2'
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac2' succeeded
CRS-2677: Stop of 'ora.cssd' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'rac2'
CRS-2677: Stop of 'ora.crf' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac2'
CRS-2677: Stop of 'ora.gipcd' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac2'
CRS-2677: Stop of 'ora.gpnpd' on 'rac2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
由于要使用独占模式所以将rac2关机了,或者停止ohas,下面操作在节点1上完成(也可以使用节点2)
[root@rac1 ~]# /u01/app/11.2.0/grid/bin/crsctl start crs
CRS-4640: Oracle High Availability Services is already active
CRS-4000: Command Start failed, or completed with errors.
查看ocr状态
[grid@rac1 rac1]$ ocrcheck
PROT-602: Failed to retrieve data from the cluster registry
PROC-26: Error while accessing the physical storage
ORA-15077: could not locate ASM instance serving a required diskgroup
ORA-29701: unable to connect to Cluster Synchronization Service
查看votedisk
[root@rac1 bin]# ./crsctl query css votedisk
Located 0 voting disk(s).
强制停止集群,以exclusive模式启动
[root@rac1 bin]# ./crsctl stop crs -f
CRS-2797: Shutdown is already in progress for 'rac1', waiting for it to complete
CRS-2797: Shutdown is already in progress for 'rac1', waiting for it to complete
CRS-4133: Oracle High Availability Services has been stopped.
[root@rac1 bin]# ./crsctl start crs -excl -nocrs
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'rac1'
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
CRS-2676: Start of 'ora.drivers.acfs' on 'rac1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2679: Attempting to clean 'ora.asm' on 'rac1'
CRS-2681: Clean of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
打开asm实例创建新磁盘组OCRA
[grid@rac1 ~]$ sqlplus / as sysasm


SQL*Plus: Release 11.2.0.4.0 Production on Thu May 19 08:13:42 2016


Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
SQL> shutdown immediate;
ORA-15100: invalid or missing diskgroup name


ORA-15100: invalid or missing diskgroup name


ASM instance shutdown
SQL> startup pfile='/tmp/pfile.ora';  
ASM instance started


SQL> startup pfile='/tmp/pfile.ora';    #pfile文件注意修改asm_diskstring,我的磁盘搜索路径是/dev/raw/*
ASM instance started




Total System Global Area 1135747072 bytes
Fixed Size                  2260728 bytes
Variable Size            1108320520 bytes
ASM Cache                  25165824 bytes
ORA-15032: not all alterations performed
ORA-15017: diskgroup "OCR" cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "OCR" #这个报错忽略
SQL>  create diskgroup OCRA normal redundancy disk '/dev/raw/raw1','/dev/raw/raw2', '/dev/raw/raw3'
  2  attribute 'compatible.asm'='11.2.0.0.0', 'compatible.rdbms'='11.2.0.0.0';




Diskgroup created.




SQL> create spfile='+OCRA' from pfile='/tmp/pfile.ora';




File created.
SQL> exit 
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
[grid@rac1 ~]$ exit
logout
[root@rac1 bin]# ./ocrconfig -restore /u01/app/
11.2.0/       grid/         oraInventory/ 
[root@rac1 bin]# ./ocrconfig -restore /u01/app/11.2.0/grid/cdata/rac-cluster/backup_20160519_073859.ocr 
PROT-35: The configured OCR locations are not accessible.
[root@rac1 bin]# vi /etc/oracle/ocr.loc
ocrconfig_loc=+OCR
local_only=FALSE
/*
/tmp/pfile.ora内容
+ASM1.__oracle_base='/u01/app/grid'#ORACLE_BASE set from in memory value
+ASM1.asm_diskgroups='DATA'#Manual Mount
+ASM2.asm_diskgroups='DATA'#Manual Mount
*.asm_diskstring='/dev/raw/*'
*.asm_power_limit=1
*.diagnostic_dest='/u01/app/grid'
*.instance_type='asm'
*.large_pool_size=12M
*.remote_login_passwordfile='EXCLUSIVE'
*/


使用备份复原ocr
[root@rac1 bin]# ./ocrconfig -restore /u01/app/
11.2.0/       grid/         oraInventory/ 
[root@rac1 bin]# ./ocrconfig -restore /u01/app/11.2.0/grid/cdata/rac-cluster/backup_20160519_073859.ocr 
PROT-35: The configured OCR locations are not accessible.
[root@rac1 bin]# vi /etc/oracle/ocr.loc
ocrconfig_loc=+OCR
local_only=FALSE
将+OCR改为+OCRA
[root@rac1 bin]# ./ocrconfig -restore /u01/app/11.2.0/grid/cdata/rac-cluster/backup_20160519_073859.ocr 
[root@rac1 bin]# ./crsctl replace votedisk +OCRA
Successful addition of voting disk 050efccd74244f05bfa6bf3978895a85.
Successful addition of voting disk 42b911116d5a4fe1bfc26382d48d16da.
Successful addition of voting disk 5d6a633e8f594f8dbff3bdd795671613.
Successfully replaced voting disk group with +OCRA.
CRS-4266: Voting file(s) successfully replaced
[root@rac1 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3408
         Available space (kbytes) :     258712
         ID                       : 1954953290
         Device/File Name         :      +OCRA
                                    Device/File integrity check succeeded




                                    Device/File not configured




                                    Device/File not configured




                                    Device/File not configured




                                    Device/File not configured




         Cluster registry integrity check succeeded




         Logical corruption check succeeded




[root@rac1 bin]# ./crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   050efccd74244f05bfa6bf3978895a85 (/dev/raw/raw1) [OCRA]
 2. ONLINE   42b911116d5a4fe1bfc26382d48d16da (/dev/raw/raw2) [OCRA]
 3. ONLINE   5d6a633e8f594f8dbff3bdd795671613 (/dev/raw/raw3) [OCRA]
Located 3 voting disk(s).
退出独占模式
[root@rac1 bin]# ./crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@rac1 bin]# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@rac1 bin]# su - grid
[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[grid@rac1 ~]$ crsctl query css votedisk
[grid@rac1 ~]$ crsctl query css votedisk
[grid@rac1 ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3408
         Available space (kbytes) :     258712
         ID                       : 1954953290
         Device/File Name         :      +OCRA
                                    Device/File integrity check succeeded




                                    Device/File not configured




                                    Device/File not configured




                                    Device/File not configured




                                    Device/File not configured




         Cluster registry integrity check succeeded




         Logical corruption check bypassed due to non-privileged user




[grid@rac1 ~]$ crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   050efccd74244f05bfa6bf3978895a85 (/dev/raw/raw1) [OCRA]
 2. ONLINE   42b911116d5a4fe1bfc26382d48d16da (/dev/raw/raw2) [OCRA]
 3. ONLINE   5d6a633e8f594f8dbff3bdd795671613 (/dev/raw/raw3) [OCRA]
Located 3 voting disk(s).
启动另一个节点
[grid@rac1 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       rac1                                         
               ONLINE  ONLINE       rac2                                         
ora.FRA.dg
               ONLINE  ONLINE       rac1                                         
               ONLINE  ONLINE       rac2                                         
ora.LISTENER.lsnr
               ONLINE  ONLINE       rac1                                         
               ONLINE  ONLINE       rac2                                         
ora.OCR.dg
               ONLINE  OFFLINE      rac1                                         
               OFFLINE OFFLINE      rac2                                         
ora.asm
               ONLINE  ONLINE       rac1                     Started             
               ONLINE  ONLINE       rac2                     Started             
ora.gsd
               OFFLINE OFFLINE      rac1                                         
               OFFLINE OFFLINE      rac2                                         
ora.net1.network
               ONLINE  ONLINE       rac1                                         
               ONLINE  ONLINE       rac2                                         
ora.ons
               ONLINE  ONLINE       rac1                                         
               ONLINE  ONLINE       rac2                                         
ora.registry.acfs
               ONLINE  ONLINE       rac1                                         
               ONLINE  ONLINE       rac2                                         
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac1                                         
ora.cvu
      1        ONLINE  ONLINE       rac1                                         
ora.oc4j
      1        ONLINE  ONLINE       rac1                                         
ora.orcl.db
      1        ONLINE  ONLINE       rac1                     Open                
      2        ONLINE  ONLINE       rac2                     Open                
ora.orcl.ssss.svc
      1        ONLINE  ONLINE       rac1                                         
      2        ONLINE  ONLINE       rac2                                         
ora.rac1.vip
      1        ONLINE  ONLINE       rac1                                         
ora.rac2.vip
      1        ONLINE  ONLINE       rac2                                         
ora.scan1.vip
      1        ONLINE  ONLINE       rac1

0 0