GRID downgrade.

Here is the steps to downgrade a grid.

In this example.I listed the downgrade from Patch description:  "FA DATABASE BUNDLE PATCH 12.1.0.2.160906 (24511435)" to Patch description:  "FA DATABASE BUNDLE PATCH: 11.2.0.4.160811 (24479697)"

Step1: Make sure no ASM rebalance running

SQL> select * from gv$asm_operation;

If ASM rebalance is still running, and cannot complete in short time, you can stop it manually using:

SQL> alter diskgroup <ASM_DISKGROUP_NAME> rebalance power 0;

Step2: Before proceeding with the downgrade identify the "OCR-node"

This is usually the node on which the first rootupgrade.sh was run during the upgrade. Look in the cdata folder inside GI HOME and locate the backup ocr of the form ocrold_version.

[root@slcc20db02 ~]# ps -ef|grep css
root     330509 216921  0 23:00 pts/1    00:00:00 grep css
root     380702      1  0 Jan20 ?        00:03:59 /u01/app/12.1.0/grid_160907/bin/cssdmonitor
root     384331      1  0 Jan20 ?        00:04:00 /u01/app/12.1.0/grid_160907/bin/cssdagent
oracle   384636      1  0 Jan20 ?        00:14:04 /u01/app/12.1.0/grid_160907/bin/ocssd.bin
[root@slcc20db01 ~]# dcli -g dbs_group -l root 'ls -lrt /u01/app/12.1.0/grid_160907/cdata/ocr11.2.0.4.0'
slcc20db01: -rw------- 1 root root 170479 Jan 20 07:20 /u01/app/12.1.0/grid_160907/cdata/ocr11.2.0.4.0
slcc20db02: ls: cannot access /u01/app/12.1.0/grid_160907/cdata/ocr11.2.0.4.0: No such file or directory

In this example, slcc20db01  is the OCR-node.

Do CRS downgrade in parallel on all other nodes, except the OCR-node:

[root@slcc20db02 grid_patch]# cd /u01/app/12.1.0/grid_160907

[root@slcc20db02 grid_160907]# cd crs/install/
[root@slcc20db02 install]#  time ./rootcrs.pl -downgrade -force -dstcrshome /u01/app/11.2.0.4/grid_patch
Using configuration parameter file: ./crsconfig_params
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.crsd' on 'slcc20db02'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.LOCAL_ZDTUPGP.lsnr' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.zdtupgf.db' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.zdtupgi.db' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.zdtupgp.db' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.zupgods.db' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.DBFS_DG.dg' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.oc4j' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.cvu' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'slcc20db02'
CRS-2677: Stop of 'ora.cvu' on 'slcc20db02' succeeded
CRS-2672: Attempting to start 'ora.cvu' on 'slcc20db01'
CRS-2677: Stop of 'ora.LOCAL_ZDTUPGP.lsnr' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.DBFS_DG.dg' on 'slcc20db02' succeeded
CRS-2673: Attempting to stop 'ora.LISTENER_ZDT11G.lsnr' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.LISTENER_ZDTOVMIDM.lsnr' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.LOCAL_ZDTUPGF.lsnr' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.LOCAL_ZDTUPGI.lsnr' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.LOCAL_ZUPGODS.lsnr' on 'slcc20db02'
CRS-2676: Start of 'ora.cvu' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.zupgods.db' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.LISTENER_ZDT11G.lsnr' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.LISTENER_ZDTOVMIDM.lsnr' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.LOCAL_ZDTUPGF.lsnr' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.LOCAL_ZDTUPGI.lsnr' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.LOCAL_ZUPGODS.lsnr' on 'slcc20db02' succeeded
CRS-2673: Attempting to stop 'ora.slcc20db02.vip' on 'slcc20db02'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'slcc20db02' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'slcc20db02'
CRS-2677: Stop of 'ora.zdtupgf.db' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.zdtupgi.db' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.zdtupgp.db' on 'slcc20db02' succeeded
CRS-2673: Attempting to stop 'ora.RECO_SLCC20.dg' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.DATA_SLCC20.dg' on 'slcc20db02'
CRS-2677: Stop of 'ora.DATA_SLCC20.dg' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.RECO_SLCC20.dg' on 'slcc20db02' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'slcc20db02'
CRS-2677: Stop of 'ora.asm' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'slcc20db02' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'slcc20db01'
CRS-2677: Stop of 'ora.slcc20db02.vip' on 'slcc20db02' succeeded
CRS-2672: Attempting to start 'ora.slcc20db02.vip' on 'slcc20db01'
CRS-2677: Stop of 'ora.oc4j' on 'slcc20db02' succeeded
CRS-2672: Attempting to start 'ora.oc4j' on 'slcc20db01'
CRS-2676: Start of 'ora.scan1.vip' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'slcc20db01'
CRS-2676: Start of 'ora.slcc20db02.vip' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.oc4j' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'slcc20db02'
CRS-2677: Stop of 'ora.ons' on 'slcc20db02' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'slcc20db02'
CRS-2677: Stop of 'ora.net1.network' on 'slcc20db02' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'slcc20db02' has completed
CRS-2677: Stop of 'ora.crsd' on 'slcc20db02' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'slcc20db02'
CRS-2677: Stop of 'ora.storage' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'slcc20db02' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.ctssd' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.evmd' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.asm' on 'slcc20db02'
CRS-2677: Stop of 'ora.mdnsd' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.crf' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.evmd' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.asm' on 'slcc20db02' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'slcc20db02'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'slcc20db02' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'slcc20db02'
CRS-2677: Stop of 'ora.cssd' on 'slcc20db02' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'slcc20db02'
CRS-2673: Attempting to stop 'ora.diskmon' on 'slcc20db02'
CRS-2677: Stop of 'ora.gipcd' on 'slcc20db02' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'slcc20db02' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'slcc20db02' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2017/01/21 23:05:14 CLSRSC-4006: Removing Oracle Trace File Analyzer (TFA) Collector.

2017/01/21 23:05:43 CLSRSC-4007: Successfully removed Oracle Trace File Analyzer (TFA) Collector.

2017/01/21 23:05:43 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.

2017/01/21 23:06:10 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.

Successfully downgraded Oracle Clusterware stack on this node

real    1m55.469s
user    0m39.253s
sys     0m5.947s
[root@slcc20db02 install]# ps -ef|grep css
root     162622 216921  0 23:06 pts/1    00:00:00 grep css

Step4: Do CRS downgrade on the OCR-node, which is the last DB node
Once all the non-OCR Nodes are downgraded, run the downgrade on the OCR-Node with the lastnode option.
Note: When downgrading to pre-11.2, as root user run root.sh from pre-upgrade clusterware home ifrootcrs.pl asks so - read the screen output carefully.

time ./rootcrs.pl -downgrade -force -dstcrshome /u01/app/11.2.0.4/grid_patch -lastnode
Using configuration parameter file: ./crsconfig_params
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.crsd' on 'slcc20db01'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.zupgods.oggservzupgods.svc' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.LOCAL_ZDTUPGF.lsnr' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.zdtupgp.db' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.zdtupgi.db' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.zdtupgf.ggconzdtupgf.svc' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.slcc20db02.vip' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.oc4j' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.mgmtdb' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.cvu' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.LOCAL_ZDTUPGP.lsnr' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.LOCAL_ZDTUPGI.lsnr' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.LISTENER_ZDT11G.lsnr' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.LOCAL_ZUPGODS.lsnr' on 'slcc20db01'
CRS-2677: Stop of 'ora.cvu' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.zupgods.oggservzupgods.svc' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.zupgods.db' on 'slcc20db01'
CRS-2677: Stop of 'ora.zdtupgf.ggconzdtupgf.svc' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.zdtupgf.db' on 'slcc20db01'
CRS-2677: Stop of 'ora.LOCAL_ZDTUPGF.lsnr' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.LOCAL_ZUPGODS.lsnr' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.LOCAL_ZDTUPGP.lsnr' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.LOCAL_ZDTUPGI.lsnr' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.LISTENER_ZDT11G.lsnr' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.scan2.vip' on 'slcc20db01'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'slcc20db01'
CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.scan3.vip' on 'slcc20db01'
CRS-2677: Stop of 'ora.slcc20db02.vip' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.slcc20db01.vip' on 'slcc20db01'
CRS-2677: Stop of 'ora.zupgods.db' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.zdtupgf.db' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.zdtupgp.db' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.zdtupgi.db' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.DATA_SLCC20.dg' on 'slcc20db01'
CRS-2677: Stop of 'ora.DATA_SLCC20.dg' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.scan3.vip' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.mgmtdb' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.MGMTLSNR' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.DBFS_DG.dg' on 'slcc20db01'
CRS-2677: Stop of 'ora.scan2.vip' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.DBFS_DG.dg' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.RECO_SLCC20.dg' on 'slcc20db01'
CRS-2677: Stop of 'ora.MGMTLSNR' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.oc4j' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.RECO_SLCC20.dg' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'slcc20db01'
CRS-2677: Stop of 'ora.asm' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.slcc20db01.vip' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'slcc20db01'
CRS-2677: Stop of 'ora.ons' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'slcc20db01'
CRS-2677: Stop of 'ora.net1.network' on 'slcc20db01' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'slcc20db01' has completed
CRS-2677: Stop of 'ora.crsd' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'slcc20db01'
CRS-2677: Stop of 'ora.storage' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.ctssd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.evmd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.asm' on 'slcc20db01'
CRS-2677: Stop of 'ora.drivers.acfs' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.evmd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.crf' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.asm' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'slcc20db01'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'slcc20db01'
CRS-2677: Stop of 'ora.cssd' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.diskmon' on 'slcc20db01'
CRS-2677: Stop of 'ora.gipcd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'slcc20db01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'slcc20db01' has completed
CRS-4133: Oracle High Availability Services has been stopped.

ASM downgrade operation succeeded

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.crsd' on 'slcc20db01'
CRS-2677: Stop of 'ora.crsd' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'slcc20db01'
CRS-2677: Stop of 'ora.storage' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.ctssd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.evmd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.asm' on 'slcc20db01'
CRS-2677: Stop of 'ora.drivers.acfs' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.evmd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.crf' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.asm' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'slcc20db01'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'slcc20db01'
CRS-2677: Stop of 'ora.cssd' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.diskmon' on 'slcc20db01'
CRS-2677: Stop of 'ora.gipcd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'slcc20db01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'slcc20db01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'slcc20db01'
CRS-2677: Stop of 'ora.drivers.acfs' on 'slcc20db01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'slcc20db01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'slcc20db01'
CRS-2672: Attempting to start 'ora.evmd' on 'slcc20db01'
CRS-2676: Start of 'ora.mdnsd' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.evmd' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'slcc20db01'
CRS-2676: Start of 'ora.gpnpd' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'slcc20db01'
CRS-2676: Start of 'ora.gipcd' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'slcc20db01'
CRS-2676: Start of 'ora.cssdmonitor' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'slcc20db01'
CRS-2672: Attempting to start 'ora.diskmon' on 'slcc20db01'
CRS-2676: Start of 'ora.diskmon' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.cssd' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'slcc20db01'
CRS-2672: Attempting to start 'ora.ctssd' on 'slcc20db01'
CRS-2676: Start of 'ora.ctssd' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'slcc20db01'
CRS-2676: Start of 'ora.asm' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'slcc20db01'
CRS-2676: Start of 'ora.storage' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'slcc20db01'
CRS-2676: Start of 'ora.crf' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'slcc20db01'
CRS-2676: Start of 'ora.crsd' on 'slcc20db01' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-6017: Processing resource auto-start for servers: slcc20db01
CRS-6016: Resource auto-start has completed for server slcc20db01
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
Starting the Grid Infrastructure Management Repository database
Using PFILE='/u01/app/12.1.0/grid_160907/dbs/init-dropmgmtdb.ora'
Command output:
SQL*Plus: Release 12.1.0.2.0 Production on Sat Jan 21 23:12:59 2017 Copyright (c) 1982, 2014, Oracle. All rights reserved. Connected to an idle instance. SQL> ORACLE instance started. Total System Global Area 788529152 bytes Fixed Size 2929352 bytes Variable Size 314576184 bytes Database Buffers 465567744 bytes Redo Buffers 5455872 bytes Database mounted. SQL> Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production With the Partitioning, Automatic Storage Management and Advanced Analytics options
Starting the Grid Infrastructure Management Repository database succeeded
Creating new pfile for the Management Repository database

SQL*Plus: Release 12.1.0.2.0 Production on Sat Jan 21 23:13:10 2017

Copyright (c) 1982, 2014, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management and Advanced Analytics options

SQL>
File created.

SQL> Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management and Advanced Analytics options
Succeeded in creating PFILE '/u01/app/12.1.0/grid_160907/dbs/init-dropmgmtdbSAVED.ora'
Dropping the Grid Infrastructure Management Repository database
dropdb.pl exiting ...
Dropping the Grid Infrastructure Management Repository database succeeded
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.crsd' on 'slcc20db01'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.DBFS_DG.dg' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.DATA_SLCC20.dg' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.RECO_SLCC20.dg' on 'slcc20db01'
CRS-2677: Stop of 'ora.DBFS_DG.dg' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.RECO_SLCC20.dg' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.DATA_SLCC20.dg' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'slcc20db01'
CRS-2677: Stop of 'ora.asm' on 'slcc20db01' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'slcc20db01' has completed
CRS-2677: Stop of 'ora.crsd' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.evmd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.storage' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'slcc20db01'
CRS-2677: Stop of 'ora.storage' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'slcc20db01'
CRS-2677: Stop of 'ora.drivers.acfs' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.evmd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.asm' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'slcc20db01'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'slcc20db01'
CRS-2677: Stop of 'ora.cssd' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.crf' on 'slcc20db01'
CRS-2677: Stop of 'ora.crf' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'slcc20db01'
CRS-2677: Stop of 'ora.diskmon' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'slcc20db01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'slcc20db01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.evmd' on 'slcc20db01'
CRS-2672: Attempting to start 'ora.mdnsd' on 'slcc20db01'
CRS-2676: Start of 'ora.mdnsd' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.evmd' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'slcc20db01'
CRS-2676: Start of 'ora.gpnpd' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'slcc20db01'
CRS-2672: Attempting to start 'ora.gipcd' on 'slcc20db01'
CRS-2676: Start of 'ora.cssdmonitor' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.gipcd' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'slcc20db01'
CRS-2672: Attempting to start 'ora.diskmon' on 'slcc20db01'
CRS-2676: Start of 'ora.diskmon' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.cssd' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'slcc20db01'
CRS-2672: Attempting to start 'ora.ctssd' on 'slcc20db01'
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'slcc20db01'
CRS-2676: Start of 'ora.crf' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.ctssd' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'slcc20db01'
CRS-2676: Start of 'ora.asm' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'slcc20db01'
CRS-2676: Start of 'ora.storage' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'slcc20db01'
CRS-2676: Start of 'ora.crsd' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.crsd' on 'slcc20db01'
CRS-2677: Stop of 'ora.crsd' on 'slcc20db01' succeeded
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.storage' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'slcc20db01'
CRS-2677: Stop of 'ora.storage' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.ctssd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.evmd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.asm' on 'slcc20db01'
CRS-2677: Stop of 'ora.mdnsd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.crf' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.evmd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.asm' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'slcc20db01'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'slcc20db01'
CRS-2677: Stop of 'ora.cssd' on 'slcc20db01' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'slcc20db01'
CRS-2673: Attempting to stop 'ora.diskmon' on 'slcc20db01'
CRS-2677: Stop of 'ora.gipcd' on 'slcc20db01' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'slcc20db01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'slcc20db01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd' on 'slcc20db01'
CRS-2676: Start of 'ora.mdnsd' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'slcc20db01'
CRS-2676: Start of 'ora.gpnpd' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'slcc20db01'
CRS-2672: Attempting to start 'ora.gipcd' on 'slcc20db01'
CRS-2676: Start of 'ora.cssdmonitor' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.gipcd' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'slcc20db01'
CRS-2672: Attempting to start 'ora.diskmon' on 'slcc20db01'
CRS-2676: Start of 'ora.diskmon' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.cssd' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'slcc20db01'
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'slcc20db01'
CRS-2672: Attempting to start 'ora.ctssd' on 'slcc20db01'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'slcc20db01'
CRS-2674: Start of 'ora.drivers.acfs' on 'slcc20db01' failed
CRS-2676: Start of 'ora.ctssd' on 'slcc20db01' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'slcc20db01' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'slcc20db01'
CRS-2676: Start of 'ora.asm' on 'slcc20db01' succeeded
2017/01/21 23:16:22 CLSRSC-338: Successfully downgraded OCR to version 11.2.0.4.0

CRS-2672: Attempting to start 'ora.crsd' on 'slcc20db01'
CRS-2676: Start of 'ora.crsd' on 'slcc20db01' succeeded
2017/01/21 23:16:59 CLSRSC-4006: Removing Oracle Trace File Analyzer (TFA) Collector.

2017/01/21 23:17:16 CLSRSC-4007: Successfully removed Oracle Trace File Analyzer (TFA) Collector.

2017/01/21 23:17:16 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.

2017/01/21 23:17:43 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.

Successfully downgraded Oracle Clusterware stack on this node
Run '/u01/app/11.2.0.4/grid_patch/bin/crsctl start crs' on all nodes to complete downgrade

real    9m33.315s
user    0m45.637s
sys     0m8.648s
You have new mail in /var/spool/mail/root
[root@slcc20db01 install]# ps -ef|grep css
root     315261 388352  0 23:22 pts/0    00:00:00 grep css
You have new mail in /var/spool/mail/root
[root@slcc20db01 install]# /u01/app/11.2.0.4/grid_patch/bin/crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@slcc20db01 install]# ps -ef|grep css
root     369714      1  1 23:22 ?        00:00:00 /u01/app/11.2.0.4/grid_patch/bin/cssdmonitor
root     369978      1  1 23:22 ?        00:00:00 /u01/app/11.2.0.4/grid_patch/bin/cssdagent
oracle   370239      1  2 23:22 ?        00:00:00 /u01/app/11.2.0.4/grid_patch/bin/ocssd.bin
root     375344 388352  0 23:22 pts/0    00:00:00 grep css

Steps 5: check the results:

[oracle@slcc20db01 grid_patch]$ ./OPatch/opatch lsinv|grep -i descript
ARU platform description:: Linux x86-64
Patch description:  "FA DATABASE BUNDLE PATCH: 11.2.0.4.160811 (24479697)"
Patch description:  "OCW Patch Set Update : 11.2.0.4.160719 (23054319)"
Patch description:  "ACFS Patch Set Update : 11.2.0.4.160419 (22502505)"